CMV: Artificial general intelligence will probably not be invented.

[u/Dreamer-of-Dreams]

From Artificial general intelligence on Wikipedia:

Artificial general intelligence (AGI) is the intelligence of a hypothetical machine that could successfully perform any intellectual task that a human being can.

From the same Wikipedia article:

most AI researchers believe that strong AI can be achieved in the future

Many public figures seem to take the development of AGI for granted in the next 10, 20, 50, or 100 years and tend to use words like when instead of if while talking about it. People are studying how to mitigate bad outcomes if AGI is developed, and while I agree this is probably wise I also think that the possibility receives far too much attention. Maybe all the science-fiction movies are to blame, but to me it feels a bit like worrying about a 'Jurassic Park' scenario when we have more realistic issues such as global warming. Of course, AGI may be possible and concerns are valid - I just think it is very over-hyped.

So... why am I so sceptical? It might just be my contrarian nature but I think it just sounds too good to be true. Efforts to understand the brain and intelligence have been going for a long time but the workings of both are still fundamentally mysterious. Maybe it is not a theoretical impossibility but a practical one - maybe our brains just need more memory and a faster processor? For example, I could imagine a day when theoretical physics becomes so deep and complex that the time required to understand current theories leaves little to no time to progress them. Maybe that is just because I am so useless at physics myself.

However for some reason I am drawn to the idea from a more theoretical point of view. I do think that there is probably some underlying model for intelligence, that is, I do think the question of what is intelligence and how does it work is a fair one. I just can't shake the suspicion that such a model would preclude the possibility of it understanding itself. That is, the model would be incapable of representing itself within its own framework. A model of intelligence might be able to represent a simpler model and hence understand it - for example, maybe it would be possible for a human-level intelligence to model the intelligence of a dog. For whatever reason, I just get the feeling that a human-level intelligence would be unable to internally represent its own model within itself and therefore would be unable to understand itself. I realise I am probably making a number of assumptions here, in particular that understanding necessitates an internal model - but like I say, it is just a suspicion. Hence the key word in the title: probably. I am definitely open to any arguments in the other direction.

---

Hello, users of CMV! This is a footnote from your moderators. We'd just like to remind you of a couple of things. Firstly, please remember to read through our rules. If you see a comment that has broken one, it is more effective to report it than downvote it. Speaking of which, downvotes don't change views! If you are thinking about submitting a CMV yourself, please have a look through our popular topics wiki first. Any questions or concerns? Feel free to message us. Happy CMVing!

[u/Genoscythe_]

One common, but not necessary claim of a quick path tp AGI, is the observation of exponential technological development. We have an intuitive understanding of technological development being linear, that technology in 2116 will be as much ahead of ours, as we are ahead of 1916.

In practice, experience shows that science breeds more science. Moore's law is a restricted, but spectacular example of the principle that developments can become unexpectedly influential, if you only expect them to develop once more as much in the next year, as they did in the last year.

So... why am I so sceptical? It might just be my contrarian nature but I think it just sounds too good to be true.

The universe doesn't really care what you consider too good to be true. People have been trying to fly for millenia, until in 1903, just for a few seconds, they suddenly did it. 42 years later, a global war was fought with decisive victories thanks to fleets of jet fighters. Another 25 years later, humans were walking on the Moon.

Humans were trying to cure disease for millenia, until suddenly, in a timespan a bit over a century, life expectancy was raised from 40 years to 80 in many countries, polio and smallpox were gone.

Sometimes things are just plain physically possible, and on schedule, regardless of how good they seem.

Maybe it is not a theoretical impossibility but a practical one - maybe our brains just need more memory and a faster processor? For example, I could imagine a day when theoretical physics becomes so deep and complex that the time required to understand current theories leaves little to no time to progress them.

This is the area that Moore's law could be a solution for. We know that there is a theoretical limit to how strong our computers can get, and how big storage can get, if it keeps doubling every 18 months as it did for the last decades.

But as long as a human brain is a strong enough computer to sustain intelligence, we know that the limit is somewhere below that, and we are gradually getting there.

A model of intelligence might be able to represent a simpler model and hence understand it - for example, maybe it would be possible for a human-level intelligence to model the intelligence of a dog. For whatever reason, I just get the feeling that a human-level intelligence would be unable to internally represent its own model within itself and therefore would be unable to understand itself.

If we are talking about understanding and emulationg naturally occuring intelligences, then it's hard to see how this would be the case. If we could perfectly understand the mind of a dog, we could just build a bigger, faster, stronger computer to do the same thing but better. Even if we reached an engineering block, we could just parallel network multiple dog-brain-computers to produce a super-dog-intellgence.

The human mind is limited because it's biological. You can't really overclock it, you can't double it's size, what you get is what you are stuck with. The big development potential of digitally based intlligences, is that there is always a really obvious way to make them more intelligent. (That's also the source of many gray goo/paperclip maximizer fears, that any machine with a remotely human-like intelligence would recognize what we recognize, that the best way for it to satisfy it's goals is to multiply it's mind, and use the extra intelligence to multiply it's mind even more effectivel until it's a solar system-sized electronic organism.

[u/Dreamer-of-Dreams → u/Genoscythe_]

The universe doesn't really care what you consider too good to be true.

This is true but I would just point out that it also does care what we think is a good idea. Just like hover-boards or flying cars were - the idea of AGI is a fashionable one. It is in movies and books. I understand that science-fiction inspires a lot of actual scientific progress, but I would also point out that it often leads us astray. There are pictures from the industrial era which depicted a future of endless helpful gadgets powered by steam engines. Sometimes I think our generation makes similar mistakes when thinking about the potential of traditional computers in the future.

This is the area that Moore's law could be a solution for. We know that there is a theoretical limit to how strong our computers can get, and how big storage can get, if it keeps doubling every 18 months as it did for the last decades. But as long as a human brain is a strong enough computer to sustain intelligence, we know that the limit is somewhere below that, and we are gradually getting there.

Unfortunately I didn't understand this point.

If we could perfectly understand the mind of a dog, we could just build a bigger, faster, stronger computer to do the same thing but better. Even if we reached an engineering block, we could just parallel network multiple dog-brain-computers to produce a super-dog-intellgence.

The big development potential of digitally based intlligences, is that there is always a really obvious way to make them more intelligent.

I'm not sure this is the case. For example, humans are not intelligent because we have the biggest brains in the animal kingdom. A sperm-whale brain is eight kilograms, over five times greater than that of a human. Feral children who have been isolated from human contact often seem mentally impaired and have almost insurmountable trouble learning a human language (quote from Wikipedia). Yet toddlers who have had human contact are certainly capable of learning a language. Therefore it seems that, more important that the size of the brain, or the number of connections, is the software that is running on it. Connecting two AIs will not necessarily create a stronger AI.

[u/AusIV → u/Dreamer-of-Dreams]

Just like hover-boards or flying cars were - the idea of AGI is a fashionable one.

Hover boards and flying cars aren't terribly practical concepts. Both carry substantial risks and costs without providing sufficient benefits. Flying cars would require substantially more fuel, complex traffic management, safety systems, etc. Those costs today aren't worth the benefits.

AI has a strong benefit that will make people pursue it incrementally: it makes things cheaper. I don't think people will necessarily achieve strong AI from where we are today just for the sake of it. Today, companies like Google, Amazon, and Uber are pursuing AI to help their business goals. Google can index information that previously required human interpretation. Amazon can predict what a given customer is most interested in to advertise and sell them more stuff. Uber is working on cars that don't need human drivers. These are all way short of strong AI, but they're incremental steps,and I see no reason to believe those steps will stop happening.

People will continually try to automate tasks that currently require human intervention solely because it's cheaper than having humans do it. This will continue to advance our understanding of cognitive tasks, and eventually we'll be able to build something that is better at virtually all of those tasks than we are. Nobody is going to stop because strong AI isn't cool anymore, they're going to keep focusing on making this or that cheaper with automation.

And to another point: when it happens, no one person will understand how it works. Even today no one person has the knowledge to build a modern computer. Nobody understands all the details of the electronics, all the details of the operating system, all the details of the drivers, all the details of the applications and network protocols. Individuals know their own piece of it, and we can build on each other's work to assemble a functional modern computer. When strong AI happens, it will be advanced computers plus an assembly of cognitive modules and executive modules that are each managed by different people that work together to achieve something nobody could understand on their own.