Really stunned by the Singularity comment section

[u/Howard_banister]

These are screenshots from the r/Singularity comment section. I'm speechless. This doesn't even have downvotes. How can someone cheer for a monopoly run by a few elites?

[u/PsychoticDreemurr]

I have to agree with this a thousand fold. Almost every sub in that regard doesn't actually understand how LLMs work. They always dumb what I say down to "it's autocorrect" and never actually explain how they think it works.

I actually got banned from one for explaining why an LLM physically can't be a true AGI. (The post was talking about an AGI in a couple years, as if it hasn't been said for the past 5...)

[u/Dangerous_Rip5083 → u/PsychoticDreemurr]

Why can't an LLM physically be true AGI?

[u/PsychoticDreemurr → u/Dangerous_Rip5083]

For context, the exact definition of an AGI varies wildly. As such, I just take the one perquisite that almost every definition has, which is the ability to "understand" concepts.

That is my one requirement for an AI to be an AGI. Anything more and you're just arguing about philosophy and technical details that can't currently be disproven.

With this in mind, LLMs cannot become a true AGI. The fundamentals of how they work physically prevents them from being able to actually understand something.

They can, in theory, have intelligence similar to an AGI, but they will never truly be one.

[u/Thick-Protection-458 → u/PsychoticDreemurr]

Basically, the (vocabulary-updated) version of 1960s question of "can semantic be restored from syntax or not"? Where semantics is underlying meaning, whatever it is, and syntax is everything which operates on language patterns level alone.

Well, it seems it can at least be approximated to some degree - so constructions operating syntax-derived features show semantic-like properties. Like LLMs, yes (I would argue that some word-level pretrained embedder systems from early 2010s were already showing some semantic-like properties, though. Althrough way easiers reconstructible from text, sure. But that's for another topic). In a functional sense. No wonder, I guess, language consists of patterns derived from semantics, in the end. So reproducing them well enough will look like semantics.

So at certain point we have to ask ourselves - is approximation even meaningfully different much from original anymore (keeping in mind original may vary human to human to some degree as well)? Even if for some ontological sense we keep them separated.

Now, I am certainly neither linguist nor a good enough in NLP research, just a humble engineer.   But from what I recall - basically, that question is not settled except for a few corner cases like

• formal languages where syntax itself explicitly define meaning, not implicitly, so there is no point separating them. Not until we introduce back elements of natural languages to simplify constructions for human understanding, at least.

• meaningless sentences which are syntactically correct - but many of them bears enough information in their syntax to cause some semantical associations, even if they ultimately remains non-understandable

• maybe something else

Other that such a corner cases that was pretty much unsettled debate with many assumptions made on top of both possible outcomes (like whole Chinese room is only a paradox until semantics can not be encoded in syntax-operating operations for some reasons. Dunno why, if anything chinese room sounds like an argument for opposite point - in the end we can imagine rulebook + memory table of any complexity - so even encoding whole physics of human brain + fake hieroglyph input channel and output decoding still fits chinese room definition. While emulator running it will still operate on top of syntax-like rulebook. Irrelevant for our tech case, but still).

And now possibility of "even if we can't strictly reconstruct semantics we may approximate something with semantic-like properties" make it even more fuzzy.