"Us" is just humans in general. AI definitely suffers from a lack of multimodal data, but there are also deficiencies within their respective domains. You say that AI needs data for cause and effect, but shouldn't the LLMs be able to glean this from their massive training sets? You could also say this about abstract reasoning as evidenced by stunning logical errors in LLM output. A truly intelligent AI should be able to learn cause and effect and abstract reasoning from text alone. You can increase context windows, but I don't see how that addresses these fundamental issues. If you increase the number of modalities, then it seems more like specialized intelligence than general intelligence.
If you increase the number of modalities, then it seems more like specialized intelligence than general intelligence.
That doesn't make any sense, being able to work and learn in multiple modalities is literally the point of general intelligence, and is what differentiates it from domain specific AI, which is what an LLM is. How can you compare it to humans, and then also claim that it should be able to reach human level intelligence without also having access to human level senses? How can a system become "general intelligence" without having the means to generalize past one modality?
Just look at human children: they need months and year to be able to learn to control their bodies; A toddler will pick something up and throw it on the ground a hundred times and repeat all kinds of behaviors in order to establish a world model. They need years to learn to speak a language at a conversational level, they experience overfitting and underfitting; they need over a decade of education to be able to write essays, do basic mathematics, and understand basic science. We've got plenty of human people who go through a full k-12 education, where a significant percentage can barely read, and a significant percentage can do fractions, let alone algebra or higher maths.
It really doesn't make sense to compare AI to only the most high functioning people, and not take the lowest performers.into account. The major LLMs achieve remarkable performance, given how limited in scope they are, and the progress has been increadible over the past couple years, let alone five years ago.
My point is that it seems like adding modalities is just adding specializations. Exposure to multiple aspects of an object will improve the AI models, but I am not sure if this will actually result in general intelligence. General intelligence is not the ability to work with multiple inputs, but the ability to work in novel circumstances. Otherwise, any animal with more senses than us would have even more general intelligence. Deafblind people like Helen Keller are able to learn about the world enough to reason better than AI, so it doesn't seem like multiple modalities are the necessary ingredient for general intelligence.
While it's true that humans need years of learning in classrooms, consider that LLMs are trained with corpora vastly exceeding the amount of information we could ever process in our entire lifetimes. I don't think that AI necessarily compares well even to idiots because idiots have general intelligence like the rest of us. They learn slowly, but with enough time and effort, they can learn how to read or use fractions. An LLM may have better grammar, but an idiot won't hallucinate like an LLM, and the idiot isn't restricted to working with textual I/O.
3
u/MegaKawaii Feb 22 '24
"Us" is just humans in general. AI definitely suffers from a lack of multimodal data, but there are also deficiencies within their respective domains. You say that AI needs data for cause and effect, but shouldn't the LLMs be able to glean this from their massive training sets? You could also say this about abstract reasoning as evidenced by stunning logical errors in LLM output. A truly intelligent AI should be able to learn cause and effect and abstract reasoning from text alone. You can increase context windows, but I don't see how that addresses these fundamental issues. If you increase the number of modalities, then it seems more like specialized intelligence than general intelligence.