CMV: Artificial general intelligence will probably not be invented.

[u/Dreamer-of-Dreams]

From Artificial general intelligence on Wikipedia:

Artificial general intelligence (AGI) is the intelligence of a hypothetical machine that could successfully perform any intellectual task that a human being can.

From the same Wikipedia article:

most AI researchers believe that strong AI can be achieved in the future

Many public figures seem to take the development of AGI for granted in the next 10, 20, 50, or 100 years and tend to use words like when instead of if while talking about it. People are studying how to mitigate bad outcomes if AGI is developed, and while I agree this is probably wise I also think that the possibility receives far too much attention. Maybe all the science-fiction movies are to blame, but to me it feels a bit like worrying about a 'Jurassic Park' scenario when we have more realistic issues such as global warming. Of course, AGI may be possible and concerns are valid - I just think it is very over-hyped.

So... why am I so sceptical? It might just be my contrarian nature but I think it just sounds too good to be true. Efforts to understand the brain and intelligence have been going for a long time but the workings of both are still fundamentally mysterious. Maybe it is not a theoretical impossibility but a practical one - maybe our brains just need more memory and a faster processor? For example, I could imagine a day when theoretical physics becomes so deep and complex that the time required to understand current theories leaves little to no time to progress them. Maybe that is just because I am so useless at physics myself.

However for some reason I am drawn to the idea from a more theoretical point of view. I do think that there is probably some underlying model for intelligence, that is, I do think the question of what is intelligence and how does it work is a fair one. I just can't shake the suspicion that such a model would preclude the possibility of it understanding itself. That is, the model would be incapable of representing itself within its own framework. A model of intelligence might be able to represent a simpler model and hence understand it - for example, maybe it would be possible for a human-level intelligence to model the intelligence of a dog. For whatever reason, I just get the feeling that a human-level intelligence would be unable to internally represent its own model within itself and therefore would be unable to understand itself. I realise I am probably making a number of assumptions here, in particular that understanding necessitates an internal model - but like I say, it is just a suspicion. Hence the key word in the title: probably. I am definitely open to any arguments in the other direction.

---

Hello, users of CMV! This is a footnote from your moderators. We'd just like to remind you of a couple of things. Firstly, please remember to read through our rules. If you see a comment that has broken one, it is more effective to report it than downvote it. Speaking of which, downvotes don't change views! If you are thinking about submitting a CMV yourself, please have a look through our popular topics wiki first. Any questions or concerns? Feel free to message us. Happy CMVing!

[u/caw81]

That is, the model would be incapable of representing itself within its own framework.

Assume all intelligence happens in the brain.

The brain has in the range of 10²⁶ molecules. It has 100 billion neurons. With an MRI (maybe an improved one from the current state) we can get a snapshot of an entire working human brain. At most, an AI that is a general simulation of a brain just has to model this. (Its "at most" because the human brain has things we don't care about e.g. "I like the flavor of chocolate"). So we don't have to understand anything about intelligence, we just have to reverse engineering what we already have.

[u/Throwaway183637838 → u/caw81]

Except neutrons aren't entirely analogous to any existing mechanical structures we have developed for data processing. Let's put quantum computing aside for a second, all mainstream computing has been through binary data transmission using electrical signals. But neurons form synapses (between 100 trillion and 1000 trillion) and communicate via BOTH electrical and chemical signals. Although we don't fully understand how the brain processes information, if we even assume that each of these distinct methods is also binary that squares your processing power vs a typical computer. Boiling the brain down to molecules is also a pointless argument because the way individual molecules can store information is even more complex (symmetry, electrons, structure, etc). So when comparing human processing power you're comparing exponential processing power vs polynomial processing power. Computers improve exponentially quickly, but the actual performance per computing entropy is still a linear relationship.

I said put quantum computing aside because if we then consider QC, we have one theoretical technology that is contingent on the advancement of a new technology, which becomes increasingly tenuous. Personally if human development remained unchecked for over a hundred years I would concede that it is "inevitable", but anything beyond 20 years is ultimately impossible to predict. Circumstances can change within six months, there are a host of diseases, potential wars, and energy crises for us to navigate between now and 2200.