> However, it also seems plausible that you could build an abstract representation of the world through studying a vast amount of human language and that'll be a good approximation of the real-world too and furthermore it seems possible that reasoning about that abstract representation can take place in the depths of the layers of a large transformer.
While I agree this is possible, I don't see why you'd think it's likely. I would instead say that I think it's unlikely.
Human communication relies on many assumptions of a shared model of the world that are rarely if ever discussed explicitly, and without which certain concepts or at least phrases become ambiguous or hard to understand.
GP argument seems to be about "thinking" when restricted to knowledge through language, and "possible" is not the same as "likely" or "unlikely" — you are not really disagreeing, since either means "possible".
GP said plausible, which does mean likely. It's possible that there's a teapot in orbit around Jupiter, but it's not plausible. And GP is specifically saying that by studying human language output, you could plausibly learn about the world that have birth to the internal models that language is used to exteriorize.
If we are really nitpicking, they said it's plausible you could build an abstract representation of the world by studying language-based data, but that it's possible it could be made to effectively reason too.
Anyway, it seems to me we are generally all in agreement (in this thread, at least), but are now being really picky about... language :)
While I agree this is possible, I don't see why you'd think it's likely. I would instead say that I think it's unlikely.
Human communication relies on many assumptions of a shared model of the world that are rarely if ever discussed explicitly, and without which certain concepts or at least phrases become ambiguous or hard to understand.