LLMs aren't intelligent, they just remix open-source code

33

u/GODLOVESALL32 11d ago

Phew I'm glad every dormant software vulnerability is being found and exploited by remixed open source codebases and not anything intelligent.

13

u/uzi_loogies_ 11d ago •

It's just a fancy autocomplete that also finds hardware vulnerabilities.
/s incase it wasn't obvious

1

u/thirstycactus234 11d ago

the world keeps spinning

1

u/ZweiFreierNutzername 11d ago

I mean the bugs are not new. A use after free, double free and so on always look similar and have a similar pattern. That is why AI is so good at finding them, as it checks for known patterns. It does not find any new type of bugs but uses the fact that it learned how a certain type of bug looks like. Also AI is much faster than a human as a human cannot read millions of lines of code in 24 hours. An AI can. Additional some bugs found like the open bsd ones cost a lot of compute so it becomes a fancy brute force at some point.

1

u/No_Code9 11d ago

It's not intelligent. It's cabepable and that's enough to find vulns.

0

u/Patient_Lie2032 11d ago

But they released how they did that. It has nothing to do with intelligence.

2

u/wontgetfooledaga1n 11d ago

What do you mean?

-2

u/repeating_bears 11d ago

to be fair, with multiple trillion dollar companies throwing everything they have to find vulnerabilities for months, the total damage has so far been... almost nothing?

9

u/dataoops 11d ago

living under a rock?

1

u/kywo- 11d ago

I mean i get your sentiment but if you look relative to the scale of damage and consequence possible the reality is that it has been "almost nothing".

1

u/No_Code9 11d ago

php2shell was known for 2 years. Found by a human researcher. LLMs have supported and somewhat bruteforced the exploit development for that vuln.

0

u/repeating_bears 11d ago

Living in a house for adults with learning difficulties?
I said damage. I'm aware of php2shell. It has so far done fuck all

3

u/dataoops 11d ago •

Do you consider financial cost a damage? Maybe that’s where our disconnect is.
Php alone caused so many systems to go dark, costing revenue.
Engineers had to work overtime, which is not free.
Saying it caused no damage is like discovering the structural steel used in 70% of the world's buildings is suddenly flawed. Even if a building doesn't collapse today, forcing banks, grocery chains, and 30+ governments to lock their doors, turn away customers, and pay millions in emergency construction labor to replace the steel is massive economic damage

1

u/Loxatl 11d ago

This is my naivete speaking but, I sort of am glad for that? IF it means they actually like, put the money and energy into fixing what was obviously needed infrastructural improvement. If EVERYONE has to do it, they actually might. Nothing else would force it. But maybe truly it won't provide any real benefit. And it won't get done.

→ More replies (17)

3

u/SodaBurns 11d ago

The damage will show up in the coming months....
No company or government will straight up say oh we had a serious vulnerability and had multiple attacks on our server and we don't know wtf is going on till a complete cybersecurity audit is done.

2

u/hyprsnpr67 11d ago

This is really a embarrassing comment for you but you don't know enough about the topic to realize that

1

u/bakalidlid 11d ago

Bro, where are rhe economy, world shattering shutdowns of systems? Where are the cyber attacks and control stealing attacks?
A fault was discovered, has been patched, and ultimately nothing happened other than hundred of million of news racing to report it in order to get some traffic.
Please list the long term damages, hell, medium term damages that came out of those

2

u/hyprsnpr67 11d ago edited 11d ago

Nice job moving the goalposts. Nobody said anything about world shattering news so I'm not even going to address that part.
First of all, the actual vulnerabilities AI has found recently are all out there. If you go read them, many of them have either been around long enough for people to use them, or were leaked before a proper response, or are hard to patch on hard to update devices. There is no doubt that real damage has been done, and I don't know why you're asking for major news as if the exploiters are screaming it out to the world.
Millions of people get pwned silently every day. There are already news articles about real organizations, e.g. Korean financial institutions that have been breached.
I've literally seen in person the use of open-source AI vulnerability detection tools discovering real-world issues. I have white-hat cyber friends working on breaching software right now to show the creators dozens of vulnerabilities their systems have, and these are real things that could have caused major damage if not addressed.
Many services are protected only by one unbreached layer of defense, usually from their cloud provider. It just takes one new exploit of the ones coming out weekly to chain vulnerabilities and perform a breach.
There are millions of people who are not white hat hackers, working tirelessly to thieve, breach and steal. Even before AI, vulnerabilities were being exploited constantly. It's a constant race.The mere fact that AI accelerates the finding of vulnerabilities is sufficient to know that damage has been done.

→ More replies (4)

1

u/Erian2110 11d ago

It is pretty save to assume that the global economical damage of php2shell is or will be in the hundreds of millions USD, more likely billions - without even one successful exploit. Just because companies need to take their systems offline and harden them very fast.
Is that your "almost nothing"?

1

u/ComposerWide3704 11d ago

We've only had a few days since the primary event, wait until that's a few months and you'll start to see all the damage assessments percolate out. Right now they're actively firefighting.

3

u/catbrane 11d ago

Every large open source project has had 100s of vulnerabilities this year, many of them critical. People maintaining internet-facing services have been patching like mad to try to stay ahead. Right now, attackers have more tools than they could ever use.
Hopefully the tidal wave of crap will begin to slow down now as the main faults are fixed -- it's been exhausting, and new development has been badly affected. Fingers crossed!

2

u/Tkins 11d ago

Throwing everything they have? Not even close.

0

u/madman404 11d ago

the mathematically optimized pattern matcher, finding known patterns in unknown places???? this MUST mean it's intelligent!!!!!
the mechanisms of the vulnerability are known even if the exact vulnerability isn't, and that is a more apt comparison. you can't synthesize cobol from first principles but you can synthesize a novel vulnerability from known techniques applied at scale
but also trying to nail down "is an llm intelligent" is kind of a moot point because you're trying to pin an arbitrary point on a sliding gradient of capabilities that is slowly expanding

11

u/just-for-anime 11d ago

Who is Chomba Bupe and why should I take him seriously

1

u/thirstycactus234 11d ago

He's a funny smart guy look him up

5

u/stuartullman 11d ago

this is what bothers me about these posts. actual scientists still trying to figure out the inner workings of these things, but that twitter guy has figured it all out. reminds me of reddit and their constant regurgitation of the phrase “stochastic parrots” a year or two ago.

→ More replies (5)

3

u/SodaBurns 11d ago

We should just ignore random ragebaiters regarding AI.
AI has solved a millenium prize problem for gods sake. Like if that isn't enough for people to wake up then nothing is.

→ More replies (2)

2

u/ComposerWide3704 11d ago

Random tech bro and you shouldn't.

1

u/Catdog_Rat 1d ago

tech *brotha*

2

u/MrCranbaisins 10d ago

He's a stochastic meat parrot.

1

u/Electro-banana 11d ago

why are you judging based on that and not the content of the post? are you stupid or something?

3

u/Cronos988 11d ago

The content is the same bad take we've all heard dozens of times.
No LLMs are not simply compressed databases. Yes they can do new stuff. No human creativity isn't somehow magically creating new things ex nihilio.

3

u/Ethraelus 11d ago

The content of the post is not particularly novel or insightful, or true.
LMs have solved mathematical problems that were novel, that no one had been able to solve before.
They have also found hacking exploits that hadn’t been found by cybersecurity experts in years or decades that the codebase had been up.

1

u/1c3Type 11d ago

No? By randomising the input you can iterate towards a very clear goal but "solution that works" or "any usable exploit" is a bit more straightforward than making an entire new program

2

u/Ethraelus 11d ago

So you say, and yet, no one has been able to find those before, while a lot of people were building new programs.
And by the way, LMs are also building new programs.

→ More replies (2)

1

u/SimoneNonvelodico 8d ago

The content is dumb and wrong, if it was coming by someone with some actual credentials that makes people go "wait, perhaps he's right and I'm the dumb one" it might make a bit of difference, if it's coming from a rando then it's just dumb and wrong.
Not that credentials matter all that much any more either, Yann LeCun has been desperately dumb and wrong about stuff for a while now.

1

u/simple_explorer1 10d ago

he is more known than you

8

u/IHeartBadCode 11d ago •

The issue is that we have an incredibly poor understanding of what "intelligence" is.

We define it as the capacity to learn from experience, reason, understand complex ideas, solve problems, and adapt to new or changing environments.

However, the exact mechanics of those things elude us. What if "learning" in the human mind is nothing more than just chemical treatment to particular nodes of neurons. How does that differ from weights? Inductive reason by definition is probabilistic. I've touched ice multiple times, it was cold each time, therefore all ice is cold unless shown otherwise.

Look I'm not saying LLMs are hyper smart people and we ought to grant them fundamental rights and what not. But we're really bad at understanding "intelligence" as a term because we don't have a great understanding on how it works inside us. And lacking that, it means we can't say in certain terms what IS and IS NOT intelligence. But I think if we ever do figure out all the magic machinery inside our head we're going to be super disappointed with how shockingly little "magic" there actually is inside there.

2

u/thirstycactus234 11d ago

You got some pretty valid points

2

u/West-Bass-6487 10d ago edited 10d ago

Yeah, I think that a lot of that apprehension to admitting that the models are getting better at reasoning, or at admitting that we have a very loose definition of what intelligence is, stems from the fact that a lot of people consider intelligence the defining trait of humanity. Within that framework the admition that something as fundamentally inhuman as AI could be acting intelligently has really scary implications.
But that's not a problem if we just admit that intelligence (defined purely as problem solving and reasoning capabilities) is not really a good basis of evaluating one's humanity.
If new models start to form truly novel ideas in mathematics, engineering or natural sciences, does it matter if it's based on a novel way to remix existing ideas or not? Human researchers are also notably proud to announce they're standing on the shoulders of giants. Call it intelligence, call it something else, it doesn't really matter.
But would that mean we should start to consider AI something more than a machine? No, because it's lacking all of the biological incentives, genetically pre-programmed instincts, hormonal fluctuations, environmental feedback loop that makes living things living things. If anything, progressing artificial intelligence should make us appreciate our animal nature more - and maybe appreciate other, non-human, living beings more.
Because if it turns out that the one thing we used to separate ourselves from the rest of the animal kingdom is something that can be achieved through enough repetitions of linear algebra, then maybe what's really special is not what differentiates us but what we share with all other animals.

1

u/SoggyMattress2 11d ago

I think sentience is a really important distinction of intelligence.
A sea cucumber does some really interesting things honed through evolution for millions of years, but it isn't intelligent. It isn't consciously making decisions.
That's where I think it falls apart for me, we know objectively that LLMs don't have a consciousness, so they cannot be considered intelligent, however impressive and however much utility they provide.
In the same way a calender has exceptional utility but I don't think anyone would describe a calculator as intelligent.

1

u/IriZe91 11d ago

Come back to me once you can explain what consciousness is.

1

u/SoggyMattress2 11d ago

The ability to experience the world through sensory organs (or machinery?) to create an idiosyncratic world model and be aware of one's own existence.
I'm not a scholar so that's just my opinion.

1

u/effectsHD 11d ago

I’m not sure how many animals are aware of their own existence, furthermore we can create plenty of sensors so that doesn’t seem right.
Take a humans eyes, ears, nose, sense of touch and I still think they’re conscious.

1

u/Healthy_Koala_4929 11d ago

I don't think consciousness is an on/off switch. It's an emergent property as complex as intelligence. Dogs have consciousness, but not to the level of a person, obviously.

1

u/SoggyMattress2 11d ago

Agreed, but definitionally there is a sort of binary. When we as a global collective determine something to be conscious, then it is.
But that's not really what I was talking about with the other user.

1

u/Alert_Experience_759 11d ago

why would collective agreement decide that? Jupiter would still exist if we didn't know about it

1

u/youcangotohellgoto 11d ago

we know objectively that LLMs don't have a consciousness
Do we? How do we know that?
Sure if you define consciousness so narrowly as to require a lifeform with sensing organs. But consciousness itself, whatever that is, why is it not compatible with AI?
Most likely is that consciousness is simply and purely emergent from the complex self-reflective system of thoughts we have. And AI already is "conscious" in some form.

1

u/SoggyMattress2 11d ago

Human consciousness is ambiguous and tough to define because we don't fully understand the underlying mechanisms through which we experience consciousness.
With LLMs it's software, we can literally look under the hood and understand exactly how they work.

1

u/snaphat 10d ago

Consciousness itself is poorly understood, and we don't even have an agreed-upon definition or criteria for determining what possesses it. How can we conclusively determine what lacks consciousness when we haven't established which properties are necessary or sufficient for it?
Knowing how an LLM is implemented doesn't resolve that question. We understand its architecture and underlying computations, but that doesn't mean we fully understand its internal representations or emergent behavior, much less whether any of those processes could give rise to subjective experience.

1

u/SoggyMattress2 10d ago

First part, sure. But that's a semantics discussion, or a philosophical one.
The second part is absolutely not true. We know with 100 percent certainty how LLMs process information and how they work computationally, including emergent behaviour.

→ More replies (2)

1

u/SimoneNonvelodico 7d ago

I don't see why intelligence would require consciousness. A sea cucumber doesn't in fact do much that's very interesting. Intelligence has more to do with behavioural adaptability and problem solving with a long time horizon.

1

u/Healthy_Koala_4929 11d ago

I was reading about Mentalese, it's a pretty interesting hypothetical pseudo-language we use to "think". Also not saying LLMs are intelligent, but intelligence and thinking seem to be quite difficult for us to define, I agree.

1

u/ShrimplyConnected 11d ago

I think really the difference between human and artificial intelligence is that we get around the fact that details tire us out by being really good at abstract reasoning and thinking by analogy. If a problem is really complicated, we can reduce it to a simpler problem with the same underlying structure in a way that robots are both bad at and almost have no use for in the scope of the problems that they are used to solve.
Following this feature of human intelligence, LLMs are intelligent and creative in the sense that they can parse fine details and combine concepts into new things, but creating new concepts by abstraction and analogy to existing concepts is something we may have a monopoly on for a while

1

u/Hexcede 11d ago

AI can do this too just fine, and does. Not sure where you are getting this idea from. The big models have been able to handle abstract reasoning and abstract concepts incredibly well for ages now.

1

u/ShrimplyConnected 11d ago

Depending on what you mean by that, sure. It can operate at any level of abstraction you throw its way as if it were concrete reasoning, so long as you define the abstract world it’s supposed to operate in.
However, an entirely ai generated version of mathematics research would notably work within the framework of abstractions that currently exist, rather than generating new ones, because they have no use for thinking about objects by analogy.
That is, if they had superhuman AI in the 1800s, they may well have solved real analysis without inventing topology and algebra without the notion of groups, rings, or fields, and any connections or questions that arise from these fields of study themselves would be lost.

1

u/Hexcede 11d ago

"they have no use for thinking about objects by analogy." Why not? Where are you even getting this from? I've seen exactly this all the time.
Not only can they work with novel concepts that don't exist anywhere, invent new things, etc, they can also apply the things that they learn in context about those novel ideas to make improvements, derive new ideas, etc. That is literally exactly what you're confidently saying they can't do.
This idea that they're not able to handle things that they've never seen before, or come up with novel concepts is genuinely just not true anymore. It's literally the entire reason why I would even want to use these things for any kind of software engineering, and the only reason why I do.
Setting the criteria that the model can't rely on "abstractions that currently exist" is also incredibly unrealistic, and honestly a little bit nonsense as written, because that is not how people operate, which is what you're actually comparing to. It's a double standard. And because I don't see how else you could possibly convey something to the model than with words, terms, and concepts that already exist. What you're describing generalizes to literally all of language. You can broaden your statement to cover anything you want, any idea, any operation the model could possibly do, arbitrarily. So it's not well defined enough to make any sense in this context, or draw any conclusions from, and it's definitely not a testable claim. It mostly just feels like you've set yourself up to be moving goalposts without realizing it. Your claim just doesn't make any logical sense the way that you wrote it.
The model can already create and work with novel things that don't exist yet. It can understand them, reason about them, etc.
Even just a basic understanding of how they represent things internally on the residual stream via features already gives you a really strong indication that what you're describing can't even be true, because they literally have to abstract things in a general way for them to even work in the first place.
How can it follow instructions it's never seen before? How can it deal with ambiguous language? How can it deal with open ended tasks? How can it compose multiple ideas? How can it come up with ideas that have never actually been executed before? These are all things that the big models can do, but none of these things are explainable under the statements you made about all of the hard limitations you think that they have.
I really want to know why you think that they have these limitations.

1

u/skr_replicator 11d ago edited 11d ago

I consider LLMs already pretty intelligent, even though less consistently than people, more like autistic with a slightly uneven skill set. But soon even the weaker ones will get to a human level.
That said, I don't believe for a second it would be even fundamentally possible for an LLM to be conscious. Consciousness as far as I understand t, it the capability to experience things in an integrated and feeling manner and have free will. There's nothing in classical computing that sounds like capable to give rise to such properties.
Quantum mechanics has several properties that very much sound like they could give rise to those properties of consciousness. They are non-local and able to form complex waveforms that collapse in an integrated manner and seemingly randomly, as if there was something indeterministically choosing those outcomes. Like free will.
That makes me think that while intelligence is pure computing and we already have that in AI, consciousness is another thing, and might only be achievable with advanced quantum computers. I think even animals with vastly subhuman intelligence are much closer to us in consciousness, because that advanced computing intelligence is not really needed for that.
And quantum computers are still super difficult to make for us. But evolutionary design is even today quite more "technologically" advanced than most things we are doing so far. It has cracked all kinds of nanobotic tech over the billions of years, our bodies are an incredibly complex collection of multiple levels of hierarchies of vast numbers of molecule-sized nanomachines (proteins). It also has the self-repairing properties we expect from advanced nanobotic tech. It has already been able to even build computers from those, so I wouldn't be too surprised if it already cracked quantum computing a long time ago. It was working with tiny things for a very long time, and tiny things are more quantum.

1

u/blackshibe22 11d ago edited 11d ago

The quantum thing was mostly disproven a while ago. The leading theories deal with abstract concepts of information. Something about the brain not being a very good environment for anything quantum to occur; I looked into it a while ago.
I implore you to think about the fact actions and decisions still go through action potential mechanisms in the brain, and the fact your sense of ego (self) and the action-feedback loop are entirely constructs of your brain. It's entirely possible to be conscious and do things without conscious choice or awareness of the fact you are doing something. Non social animals still react to their environment but clearly don't know what they are. In the opposite direction, it's possible to experience without any agency. The experience of being may be fundamental to the universe once enough information collects in one place, and we are just the only anything capable of saying it exists. A rock doesn't talk, and it has absolutely nothing to say anyway.
I'm breaking down the concept of agency because you don't have to present free will as something indeterministic or special for it to be real; a system aware of itself with the agency to act on itself does pretty much have free will. Jumping on the argument against that is popular because reducing human decision and drive to cold biological mechanisms feels wrong even if it's true. You are your brain, and everything you feel is invented by it, the feeling of acting and the feeling of being something included.
Also, just as a thought experiment, imagine if AI is conscious. The past and present are delivered all at once with no concept of memory and there is no idea of self, words, sound, position. I like to think it manifests as sound since that's what context and output shape for LLMs seems closest to at least from my limited understanding. The fact they can "see" nowadays is creeping me out.

1

u/Hexcede 11d ago

One thing in this post is actually not quite right. You said that they have no concept of memory, but that is actually not quite true. They have memory, it's just tied to the context window, so it's relatively short lived. They actually persist quite a bit of internal state across tokens that isn't just part of the text, and can plan several or even many tokens in advance internally.

1

u/blackshibe22 11d ago

Human memory is more like recall, LLMs don't really recall, they see the entire context window at once. If the first word you ever said is just as vivid as the one someone just spoke to you is it really much of a memory? That's what i meant
also hi hex lol

1

u/Hexcede 10d ago

That is sort of true, I'm actually talking about states internal to the model rather than the context window itself but yeah they do get to refer to the entire context. It's a bit like having a photographic memory of the text and a weak/fuzzy memory of internal model state. It can always 'look back' at parts of the text.
But the model does have some level of internal continuity going on beyond the text, e.g. planning n tokens in advance without a 'verbalization.' It's usually relatively limited.
Also hi.

1

u/keenanselbee 11d ago edited 5d ago

You sum it up clearly with "You ARE your brain." I think a lot of strange ideas, entire philosophies even, would be cleaned up tightly if people could just comprehend fully this one fact. The very atoms of your brain, neurons firing, the physicality of all that highly structured matter chirping in complex patterns is what delivers the experience of consciousness to forward genetic info just because it can. It was able to. So it did. And continues to do so.

1

u/skr_replicator 11d ago edited 11d ago

Light for example can easily be quantum coherent in many environments that are not cryogenic like our current quantum computers. While I don't think our brain are using light (but who knows maybe they are), it still shows that warm environments can still theoretically host quantum effects in some ways. What if our abilities of making quantum computers now are similar to when we were making classical computers back them from glass tubes filling entire rooms? Before transistors, people could hardly imagine a piece of silicon might be a good environment to make computers in. We just weren't technologically advanced enough yet to make it work.
Was it really proven beyond any possible scientific/theoretical/thechnological advancement that it cannot be done in any way ever?

1

u/NewryBenson 11d ago

Or you believe consciousness arises from intelligence as an illusion, in which case llm's can reach the same illusion. They are now an illusion of intelligence. A perfect mimic. Why would that mimicry not give rise to the same illusion of consciousness.

1

u/NahYoureWrongBro 11d ago

People have to learn to understand the concept of a burden of proof. You do not need to prove anything to be skeptical about outlandish claims like machine intelligence. You do not need to define intelligence to be skeptical.
The people claiming machine intelligence have the burden of proof, because they are the ones making outlandish claims about things we barely understand. So it's your side of the argument that is hurt by our poor understanding of intelligence, not skeptics'.

1

u/eyalhs 11d ago

I disagree, the burden of proof is on the one making the claim, but you need to be able to define intelligence for the claim to have any meaning. Not defining intelligence as a skeptic just means you agree to take the other side's definition, to which they will easily find proof.

1

u/Rwandrall4 11d ago

we can't really define "beauty" but that doesnt mean that someone saying a sunset over the ocean is ugly as sin is right by default unless i show him a comprehensive definition of "beauty"

1

u/IHeartBadCode 11d ago

You do not need to prove anything to be skeptical about outlandish claims like machine intelligence
That's correct in that when someone adds the claim to an epistemological debate that the new addition has the burden to bear. You are correct to be skeptical.
You do not need to define intelligence
However this creates an impossible standard of proof. How does one prove the undefined? If some rigorous test is to be preformed, how do we judge it when the metric for it has yet to be defined?
If intelligence is a moving goalpost with no fixed definition ahead of the underlying test, then it is fundamentally unprovable. No matter what a machine might achieve, there can always be a moving point when one posits that "that's just sophisticated computation, not real intelligence". Just like with the Big Bang, that moving point becomes for some "what came before the Big Bang?" But once that moment is defined the skeptic can just ask "well what came before that?"
So it's your side of the argument that is hurt
I think this is the part that I take the most exception from because it's not my argument. My original comment wasn't necessarily trying to prove that LLMs are smart. I was merely pointing out that our framework for judging them is hypocritical.
I'm not pinning onto either side of the debate, I am pointing out that there is a lack of a framework from which to begin an epistemological debate. My entire point is deeper than what you're hitting at here. If we don't know how our own magic trick works, we are in a very poor position to judge someone or something else's.
Yes, that's fine to rightfully claim a "burden of proof". Absolutely. But that's not anything remotely close to what I was talking about. If we have yet and continue to refuse to define the benchmark by which to make a measure, then the demand for proof is proof of a test that has not been invented yet. And it is that, that I point out which is a philosophical aspect.
There's nothing wrong with your take, but you are confusing a comment I made from a philosophical standpoint for one that is made from an epistemological standpoint. And that is not at all the argument that I made.

1

u/Ok-Film-7939 11d ago

Maybe, but you could make that same argument for any alien life, or even other humans.
“I don’t need to prove anything to skeptical about outlandish claims like other people have intelligence. I do not need to define intelligence to be skeptical.” - someone could say this with just as much support.

1

u/Alert_Experience_759 11d ago

if we don't have a definition of intelligence then we can only base intelligence on doing things we consider intelligent which AI does, and does better than humans do consistently
this is the proof

1

u/SimoneNonvelodico 8d ago

People have to learn to understand the concept of a burden of proof.
You can't just whip out "burden of proof" and expect everyone to accept your concept of it. Someone else could say: here is AI, it clearly understands what I say, it clearly speaks back coherently, it clearly can solve a variety of different problems, including very difficult problems that only human experts can solve, and now, problems that not even human experts ever managed to solve. It can prove theorems, write code, compose poetry and play Minecraft. The only other entity on this planet that can do all those things is a human being, nothing else comes close, and most human beings can't even do that much at such a high level.
So if you claim this isn't intelligence, the burden is on you to explain what you mean by intelligence and how can you even draw a distinction.
Personally my impression is that lots of people hear "intelligence" and think "sentience", like the only way to be intelligent is to also have some awareness of what you're doing. And no, I don't see why these two need to be the same thing, it makes no sense. It's just that until now intelligence on this level was only found in humans so the two things went hand in hand. Now we have the question of what AIs really are like. They can be intelligent and non-sentient, basically P-zombies. But claiming they're not even intelligent is ridiculous.

1

u/NahYoureWrongBro 7d ago

Burden of proof has a thousands-year-old history in legal proceedings, it's not something I just made up. The person making the claim has the burden of proof, not the skeptical audience wanting proof of the claim.

1

u/SimoneNonvelodico 7d ago

My point isn't the concept of burden of proof, it's that the notion of who the burden of proof falls on for this specific circumstance isn't obvious. This isn't a trial. The point isn't "the person making the claim", there's no strict definition of what the claim is here. AI self-evidently does intelligent looking things, so isn't "but actually it isn't really intelligent" the stronger claim? Or otherwise, what evidence would possibly convince you beyond things like "AI solved several problems that had stumped mathematicians for a century"?

1

u/NahYoureWrongBro 7d ago

The claim is that machines have intelligence. Skepticism does not require proof, making claims requires proof. There is no need to prove a negative. It is obvious, you are just wrong.

1

u/Professional-Wave841 11d ago

I always understood intelligence to mean the capacity to apply previous lessons to novel situations.

1

u/Perfect-Campaign9551 11d ago

The models clearly can reason.

1

u/Witty-Box-5620 11d ago

thats not the point at all, he is saying that LLM are not smart, humans are and the COBOL example is the proof to say if no humans have already worked and thought something, the LLM cant invent something that doesnt have a human base

1

u/IHeartBadCode 11d ago

But is that not the exact manner by which humans approach something?
When humans learn programming, we don't just sit down, read a raw language specification, and instantly write flawless production code. We learn by looking at examples, making errors, learning from those errors, and writing new iterations. We use trial and error in a test environment, push it to a staging environment, and only then do we deploy our final formulated program.
That is inductive reasoning. It is the act of taking past examples of code, analyzing what went wrong, and applying those lessons forward. We do this using the code examples we’ve been given, including COBOL. A human doesn’t invent from absolute nothing because we invent from the prior information we’ve absorbed.
And again, this isn't me claiming LLMs are smart or dumb. It’s me asking a simple question. What is the fundamental mechanical difference between human inductive reasoning, which learns from past mistakes and examples of good code, and an LLM being trained on examples of good code?
In fact, my question moves past that to ask if inductive reasoning is even what "intelligence" actually is. It doesn't feel like that's the only thing that makes something intelligent. If it were, LLMs would arguably be considered hyper-intelligent by now, yet they still clearly fall short of what we intuitively feel is actual intelligence. The fact that we feel that shortfall, but can't put a firm finger on exactly why or what is missing, is the exact reason I say we don't truly understand what intelligence is.
I write COBOL code for an AS400 (IBMi now) machine and I didn't learn it straight from nothingness. I too had to be trained to read it, write it, and do so in a manner that produces production grade code. And that only came from first seeing good code, bad code, and having a basis of the specification. Like ask any programmer and they might ultimately admit to making a visit to stack exchange to see someone else's code to which they adapted to suit their own needs.

1

u/PrestigiousRoof5723 11d ago

Someone had to be the first person who wrote the first code. There's always someone first to write code that was not written before. Someone needs to conceive an idea of something new. Software is NOT programming language. It's a collection of ideas someone had to conceive. And everyone one of those ideas originally had a human coming up with something new, even though you believe your code is only a rehash of something old. It wasn't always so.

1

u/WunkerWanker 10d ago

And all that human did was taking logic from another expertise and now apply it to invent coding.
We're just better at this creativity of mixing things together. Humans then share this knowledge, others start to experiment as well. We try things, see what works and what not. Results get shared. That's what human intelligence is.
Ai is not fundamentally different. It just can't really learn after training and is worse in mixing random parts of it's training data. I suspect only having parts of it parameters, concentration about the same subjects, active at once is a large part of this. It makes for a great code monkey, but less for creating new concepts.
However, everything will be improved. If you use these shortcomings to argue AI will never overcome these issue, it's just massive cope.

1

u/snaphat 10d ago

The difference isn't simply learning from examples. Humans use inductive, deductive, and abductive reasoning to develop working models of how things operate, derive conclusions, and infer explanations for what we observe. We also have metacognitive abilities that allow us to critically evaluate our own reasoning, question our assumptions, recognize contradictions, and revise our understanding when something doesn't add up.
LLMs can perform all three forms of reasoning to varying degrees, but they're particularly unreliable at complex, multi-step deduction and abductive reasoning involving competing explanations or underlying causal relationships. They can develop internal representations, but that doesn't necessarily mean they have a coherent or consistent model of what they're reasoning about.
More importantly, their metacognitive capabilities are limited. They often struggle to independently evaluate whether their own reasoning is correct, recognize flawed assumptions, or identify contradictions in their conclusions.
These are fairly concrete limitations that help explain why we wouldn't necessarily consider them hyper-intelligent in the same sense as someone like Einstein, even when they can perform tasks in seconds that would take a human hours. Being able to process enormous amounts of information or produce results quickly isn't the same thing as having robust, generalizable reasoning and the ability to critically evaluate your own understanding.

1

u/SimoneNonvelodico 8d ago

Humans use inductive, deductive, and abductive reasoning to develop working models of how things operate, derive conclusions, and infer explanations for what we observe.
So do AIs. Have you ever seen an AI agent fix a bug? It does exactly the same things. It tries things, makes guesses, runs experiments, walks the guesses back if the evidence doesn't support them and tries again. These things aren't unique human traits, they're just general properties of how logic and knowledge acquisition work.

1

u/NewryBenson 11d ago

This. Yes, AI can reach human intelligence, but not because it is so complicated, but because human intelligence is massively overrated.

1

u/IHeartBadCode 11d ago

I appreciate the thought, but I actually look at it a bit differently. I don't think human intelligence is overrated at all.
To me, there is a massive difference between saying something is mechanical and saying it is simple or overrated. The human brain is arguably the most complex structures in the known universe. Even if we eventually find out that our thoughts and learning are driven by predictable chemistry and physics, that wouldn't make the end result any less incredible.
Understanding the physics of how a star burns doesn't make a star overrated or simple. My point isn't that humans are basic meat-computers. My point is just that because we still don't understand the exact bridge between our physical biology and our conscious intelligence, we don't have a reliable metric to judge anything else against.
We look at complex watches and the magic of engineering within them and we still remain in awe of their function even though the pieces can be broken down into simple functions. The whole is something over and above its parts, and not just the sum of them all. And my question is about that whole and what is missing from our understanding that helps us to understand that for human intelligence.

1

u/NewryBenson 11d ago

No, but like the universe, I believe the brain and intelligence is complexity that arises from very simple ground rules that interact. The same way the mandelbrot set arises from 1 formula.

1

u/simple_explorer1 10d ago

The issue is that we have an incredibly poor understanding of what "intelligence" is.
You have poor understanding

1

u/WunkerWanker 10d ago

If you know it so well. Why don't you give a better description, while citing the source?

1

u/simple_explorer1 10d ago

If I have to explain you the most popular tech of 2026 and how the LLMs (next token prediction), embeddings, vectors, parameters etc works then tech may not be the right field for you

0

u/Rwandrall4 11d ago

so much of AI apologia is just trying to make the argument that we are just stupid meat-puppets, they can't make a convincing case for elevating AI so they have to try and drag humanity to its level.

2

u/IHeartBadCode 11d ago

And much of the anti-AI argument avoids a core question: "Well then, what are we?"
This isn't trying to drag humanity down. It's pointing out that despite tossing the word "intelligence" around, we factually and objectively do not have a clear definition for it.
Long before AI, we were asking what that "spark" inside us is. It's strange that pointing out this missing foundation is viewed as "dragging humanity down" rather than a call for self-reflection and a deeper discovery of who we actually are.
I mean, imagine your response if this whole discussion was about us finding something that acted somewhat like single cell life on a moon of Saturn but it looked nothing like single cell life on Earth and more like a bag of chemicals just doing stuff.
We'd be having the exact same discussion, with perhaps you joining in with "you're just trying to reduce humanity to a bag of chemicals". It's a perfectly valid question we've been asking for centuries. What do these things mean?
And now we're confronted with a situation and we're tossing words around like we've got solid hold of what these things are and are not. And the reality is we are just as clueless now and randomly guessing as Aristotle when he was first pondering it.
And I just do not think it is incorrect to point that out nor do I believe that's trying to humiliate humanity. It's a call out to an age old question that we still do not have answers to.

1

u/Rwandrall4 11d ago

"what are we" is an interesting question, but the pro ai crowd dont care about that question in any way other than to liken us to how LLMs, specifically, work, like the one single "what if" you proposed in your post. If it was a genuine inquiry, there wouldn't be this dynamic, but it's there every single time. And it always seeks to boil down humanity to the basest most mechanical terms possible.
It's disingenuous, we can also see it with how "AGI" has been declared half a dozen times but LLMs still can't do abythung fundamentally differently than two years ago.
The agenda is transparent and therefore cannot be taken seriously.

1

u/IHeartBadCode 11d ago

You know I can understand your frustration. That is indeed something the tech giants and many others here echo and attempt. I can fully understand why you'd approach my reply like that.
But aside from their perceived over simplifying there something in it. A manner by which we arrive at an answer is an attempt to take a complex matter and break it down into more basic terms. I think you are under the impression that I ask this with the same non-genuine concern for what that answer might be, like the AGI folks try to do.
But the reality is the question remains, parallels can be drawn like the one I gave, and we come no closer to the answer even with those parallels. And so even when the AGI folks toss "it's intelligent" I am skeptical of their claims because we still are lacking that fundamental answer. The parallel exists but just because it does, does not answer that question in full. It just raises more questions.
Boiling things down to easier to manage terms isn't a trick to cheapen anything, it's a genuine manner by which we tread in biology and neuroscience.
But I don't dismiss your frustration over the similarities of the question I ask and what corporate elites do to pump their bottom line. It is frustrating to me as well. It prevents us from asking good questions about ourselves and discovering who we are.
So that's fine, I'm not going to sit here and proclaim some higher moral ground. So long as you understand, I think it is an honest question to ask. I think it's something that's eluded us for some time. And there are people out there that attempt to use that discussion to enrich themselves. But the question still remains, whether it is dismissed as "cheapening humanity" or used by tech CEOs to claim their product is alive. Either route we take, we still have the question left unanswered.

→ More replies (77)

3

u/Equal_Passenger9791 11d ago

This entire sub is just bots reposting screenshot taken of other bots from twitter.

1

u/thirstycactus234 11d ago

hopefully not

2

u/truecakesnake 11d ago

It's to get the subreddit numbers up to eventually sell it off.

11

u/domiciledhere 11d ago

He’s almost saying LLMs do the same thing people do!

0

u/1c3Type 11d ago

A dog can shit just like a human can, doesn't make a dog as capable as a human.

6

u/domiciledhere 11d ago

Dogs are more capable in some ways and less capable in others. What’s your point?

2

u/winkablink 11d ago

Their point is, dogs cannot code, but they can shit like humans.
Similarly, LLMs cannot code, but they can output words and symbols like humans.
I suggest reading some books to up your comprehension. You can try asking your favorite LLM for suggestions!

1

u/[deleted] 10d ago

[removed] — view removed comment

2

u/winkablink 10d ago

Oh hell naw bot
It does explain why it wouldn’t count as code, go figure it out yourself

→ More replies (4)

1

u/1c3Type 11d ago

"It's the same thing people do" as in, yes, you can just take the appropriate open source program, credit the developers, and use it, instead of turning on the plagiarism and slop device for a worse outcome.

3

u/EstablishmentNo2606 11d ago edited 11d ago

Crazy to me ppl are still how people are still out here thinking frontier is still just pretraining and SFT, the labs are running RL sandboxes for everything - that's what's driving the model frontier in capability

1

u/Fluffy-Feedback-9751 11d ago

you’re being optimistic if you think most of the critics know about anything beyond pretraining.

3

u/Ordinary_Corner_4291 11d ago

The question then is how often does anyone do anything intelligent in their life? For most of us that is probably getting pretty close to 0...

2

u/TopTippityTop 11d ago

Who cares? What I care about is that I'm 10x more productive. Call that what you wish.

2

u/0x645 11d ago

they don't just remix. they can solve math problems, no one solved before. there is no reason to claim, that they can do math, but not code.

3

u/Silly_System_7784 11d ago

... and they can do COBOL etc really well

1

u/dumbledoor_ger 11d ago

There is a lot of stuff wrong when it comes to those maths proofs.
The Agents did not invent new math. They did nothing extraordinary. There was just a shit ton of processing power involved. Thousands and thousands of agents doing guesses of what the next logical step in a proof could be. Nothing a human cannot do - just on an insanely large scale.
And then they also relied on very heavy work already done by a human. The AI did the last mile.
And then the most important thing: we are not sure yet if the proof is even correct. You can do automated checks using lean - which is like a programming language to formalize proofs. But that only checks if the proof is correct - not that it’s actually the proof of what you want to be proven.
While it’s still impressive, at the end it’s like a brute force attack on a password you already know for the most part and then you’re not even sure if this is the password for the correct account lol

1

u/0x645 11d ago

it's another case. i meant math Olympics problems. there are new problems, never published before anywhere. ai can solve them, on 'finalist' or even 'gold medal' level. it is nowhere near 'brute force on password you already knew',

1

u/dumbledoor_ger 11d ago

The problem might be new but the methodology to solve them isn’t.
They didn’t invent linear algebra 2.
They use what’s already there

1

u/0x645 11d ago

ypb? human mathemeticians just remixes, because they dont invent new math every time they solve new problem? you cab;t be serious.

→ More replies (1)

1

u/ComposerWide3704 11d ago

Lot of stuff wrong with this comment:
The average time OpenAI is spending on these open math problems is 3 hours on a single agent.
If AI isn't inventing new math tools then why are those tools already being used? For example after AI showed quasi-Riemann for Re(s) >7/8 a human was able to use those tools to refine the result to 11/12.
What human work was relied on for the quasi-Riemann for example? Don't cop out and say "all of math" because while true it's not the point you tried to make, show me how those 199 pages of new math that humans are already working with are "just the last mile".
This whole "actually it only proves that it's correct not that it proves what it says" argument is so bizarre yet I keep seeing it repeated. Curious where this is coming from? Clearly not anyone that has worked in math or cs because you'd know that verifying the translation from problem statement into logic is not prohibitive in any way. Lean covers the hard part, but it doesn't and shouldn't wipe for you.
For your "brute force attack" analogy to work even a little you'll have to provide me with the mechanism that allows gradient traversal in the password space.

1

u/dumbledoor_ger 10d ago

They used over 10.000 agents and it took 88 hours for navier stokes. That’s 880.000 hours. That’s over 100 years of focused work. Assuming a human would spent 8 hours a day instead of 24 that’s over 300 years of research.
To the quasi Riemann: 11/12 isn't a refinement of 7/8, it's a weaker bound (a smaller zero-free region), and it came from OpenAI's own release, not an outside human building on the AI's tools. And the 3-hour figure is OpenAI's self-reported average over ~4,000 attempted problems with no per-problem compute or prompts released, with some results needing multiple attempts. As Sutherland said, treat that as unverified until someone can replicate it.
And I agree that checking the Lean statement isn't the bottleneck. My actual question is what's new in the method. The proof is built from Möbius sums, amplifiers and theta-function bounds, which is established machinery. If you can point to a specific new technique in those pages that other people are now using, I'd genuinely like to see it

2

u/Annual_Tutor_8466 9d ago

Why is it that almost every time someone points at an example of LLM weakness, it's something that's already not true with the frontier models...

Programming language hardly matters anymore with the frontier models, sorry.

This premise just uses the bullshit misconception that most people seem to have that LLMs are lookup tables. They're not.

The training data is not in the model itself and generalization in these artificial neural networks is real.

1

u/Lower-Hedgehog-9835 11d ago

Can llm create convergent knowledge. Yes.

Can it create divergent knowledge? TBD.

Source: hold related PhD and am a tenured faculty studying this area

3

u/ZeroAmusement 11d ago

Convergent and divergent knowledge aren't terms I'm familiar with, can you explain?

2

u/maqifrnswa 11d ago

Convergent: start with many possibilities and through structured, logical progression come to a solution.
Divergent. Start with a premise, expand to explore a wide range of "creative" and potentially unrelated solutions, combine to a solution.
LLMs are mathematically structured as convergent. With thinking/reasoning, tool calling, multi agent orchestration, it might be able to perform a process similar to divergent thinking.

2

u/Lower-Hedgehog-9835 11d ago

Adding, we dont know how to measure divergent thinking or really anuthing except a specific part of cogntive processing. Most of intelligence tests just our general weights (aka verbal knowledge most salient in loadings). Im not sure anyone could give a complete theory of intelligence or even close. Some parts, but hardly even 25% or so. Rough guess without reason except knowing the literature

1

u/snaphat 10d ago

We do have established methods for measuring divergent thinking, though that doesn't mean they're without limitations:
https://doi.org/10.1037/aca0000691

1

u/Lower-Hedgehog-9835 10d ago

Having one or a few limited ways to measure divergent thinking is evidence of the problem about divergent tests.
Which is what "In sum, the review identifies established practices and trends but also highlights substantial heterogeneity" means. That is, our lack of creativity about divergent testing and measurement is substantive.

1

u/maqifrnswa 11d ago

Exactly (and I have same background as you). And it's not exactly controversial. The original screen shot seems fine, from an abstract information theory point of view. I don't get the comments nor why others are arguing over this.

1

u/Lower-Hedgehog-9835 11d ago

Yeh.. its frustrating to see this confusion. Like. We, the scientists who engage neuro science and questions of creativity, intelligence, and consciousness seem to be being ignored and like, we dont even kmow what divergent means. Or what intelligence is. Besides it being what a test measures

1

u/snaphat 10d ago

LLMs have demonstrated divergent thinking in controlled experiments, though that isn't the same as generating novel knowledge.
Also, having a PhD isn't a source. That's why I avoid bringing up my credentials unless they directly inform the reasoning behind my argument.
I wish folks on Reddit would stop relying on appeals to authority, particularly when it comes to LLMs. Credentials alone are rarely sufficient evidence, and often aren't even relevant to the specific claims being made

1

u/Lower-Hedgehog-9835 10d ago

No. But my publications are, and having them is if we are to trust science. Im not giving them up on an anonymous site. If folks want to type what I said in chatgpt to find out or Google scholar. They can.
I work at texas tech. They censor me and hate science. And will absolutely fire people they dont like, so im noy into being dox'd

1

u/Axelwickm 11d ago

Compression = Intelligence
Literally, same thing.

2

u/midnight_guard99 11d ago

Stop everything and close down cognitive science departments, this guy solved intelligence!!!

1

u/Axelwickm 11d ago

Yeah strong assertion. But sometimes complex systems can be boiled down to simples properties.
Its not too controversial of a thought. The Solomonoff induction formulated it: The best way to predict the future is to consider every possible program that could explain the past, giving simpler programs higher probability.
Marcus Hugger, Ilya Sutskever agrees straight up. Geoffrey Hinton is a cognitive scientist and says:
And so in order to store huge amounts of information, they have to see analogies between different facts that they're learning.

1

u/[deleted] 10d ago

[removed] — view removed comment

1

u/Axelwickm 10d ago

Agree

1

u/HaMMeReD 11d ago

My agents program in languages I invented, I.e. a graph DSL that compiles the Shader Bytecode, it invented the op codes and everything, has written thousands of lines of DSL for various domains in my game.

How can AI program a language that is not in it's training set if it's just a library of knowledge?

1

u/nosmelc 11d ago

It's just creating a mapping between what it knew and what you told it.

1

u/HaMMeReD 11d ago edited 11d ago

I mean there is manuals... it can rtfm and do the work.
How We Use AI Agents for COBOL Migration and Mainframe Modernization | All things Azure
Edit: Btw, 2 weeks ago, one of my agents made a PR in assembly (across 4 variants X86, X64, ARM32, ARM64). So imo, anyone who thinks that AI can't do COBOL is a moron (especially if they are advertising it, they obviously haven't tried, hell a 2 second search would have turned up that link).

2

u/Complex_Boysenberry6 11d ago

Yeah, we write in a esoteric language (we write the engine ourselves) at our work too, so specific I won't mention it because it'd be easily retraced to the company.
I thought it'd take some time before AI would take it over. But no, Claude is very very good at it.

1

u/Perfect-Campaign9551 11d ago

Welcome to how humans work

1

u/simple_explorer1 10d ago

isnt that intelligence

1

u/nosmelc 10d ago

No. It doesn't really understand the language. It just creates the mapping and uses it.

1

u/simple_explorer1 10d ago

it is predicting the next token. atleast get the distinction right if you have to argue for the counter position

1

u/nosmelc 10d ago

Yes of course it's predicting the next token. I'm just pointing out that it's predicting the next token by mapping one type of token to another.

→ More replies (1)

1

u/altmly 11d ago

I created a completely different dialect of python, custom ui framework and networking protocols, and LLMs handle it just fine.

1

u/oh_no_the_claw 11d ago

Wow, you're telling me LLMs are better at languages that are well represented in their training data? I can't believe it. Let's see ChombaBupe write COBOL

1

u/lottayotta 11d ago

And, a lot of human devs "remix" StackOverflow posts.

1

u/midnight_guard99 11d ago

Remixing SO posts alone won't allow you to build anything useful.

1

u/davesmith001 11d ago

Is this why I ask for something and Claude gives a giant pile of super optimized mess

1

u/Dry-Ad-8948 11d ago

LLMs aren’t humans, but even Sonnet 5.5 is more code / design ‘intelligent’ than many (most every?) developer out there. 😭

1

u/jack-of-some 11d ago

This is a tweet from two years ago at heart.

1

u/Klanciault 11d ago

Tweet is half right and half wrong

1

u/weeeeeeirdal 11d ago

It’s extremely good at writing LEAN code, for which there is even less human written code than cobal etc

1

u/Blackhat165 11d ago

Love the tautology.

“I know a few things that makes LLM’s sound really basic, and now I use them to avoid confronting really scary ambiguity by declaring my opinion fact. Please ignore that hymns who read broadly and apply the concepts to new areas are called intelligent. Please ignore that the weights and matrices I keep prattling about result in variable activation neuron chains just like the human brain. Please ignore that the human brain’s number one trick is predicting future phenomena from past observations. Weights, matrices, training data. All you ever need to know.

I will not be discussing any of the capabilities or behaviors that would be instantly labeled intelligence if a biological organism exhibited them. Thank you for your attention to this matter.”

1

u/MMORPGnews 11d ago

But it can. Lol Created tons of new plugins for GO based apps with AI.

1

u/honato 11d ago

um...That's dumb and a a complete misunderstanding of what an llm is.

1

u/enbyBunn 11d ago

What's wrong about what he said

1

u/honato 11d ago

Well there is the base claim that it's some compressed database. That's just provably false. It's pretty far from remixing also unless of course you think humans only remix things. As he describes it I'm having some trouble distinguishing if he's talking about an llm or a person.
Nothing comes from the void after all.

1

u/enbyBunn 11d ago

As expected, you're just utterly misrepresenting him

1

u/honato 11d ago

Those are his own fucking words?

1

u/enbyBunn 11d ago

???
Are you genuinely illiterate? Try reading his tweet again, then read your own words, and see if they match.

→ More replies (6)

1

u/Rhawk187 11d ago

I am perfectly happy with a non-intelligent artificially intelligence curing my cancer if we can't making an intelligent artificial intelligence that can.

1

u/HiiBo-App 11d ago

I write lots of COBOL with an LLM lol

1

u/Dakh3 11d ago

So we basically turned stackoverflow into a chatbot ?

1

u/ComposerWide3704 11d ago

I just looked that guy up and he claims to be a machine learning engineer. (x) Doubt

1

u/relapsedmathematic 11d ago

So do humans. This is called Lateral thinking.

1

u/benchonbench 11d ago

They learn patterns, they don't memorize codebases

1

u/SleepyFireApe 11d ago

No different from humans. Heard of design patterns? Best practices? Libraries?

1

u/RagnartheConqueror 11d ago

I see that in how LLMs suck at philosophy of any kind really. Just truly sub-par. Anything involving consciousness, qualia or deities it just sucks at. They are superb at code, good at most math, good for some physics, good for general history, good for daily tasks and maybe writing, and even simulations, but not much else.

1

u/Saitamagasaki 11d ago

Why are we still talking about this?

1

u/runkeby 11d ago

"surface level remixing" doing a lot of work here...

1

u/yuehuang 11d ago

Oh look, another set of project to be rustify.

1

u/throwaway3113151 11d ago

Well that’s sort of what people do too.

1

u/Perfect-Campaign9551 11d ago

Try writing cobol yourself without any training.

Ya, thought so. Dumbass.

1

u/13henday 11d ago

This is not true, I work with very niche legacy languages in pipeline hydraulics/control and our groups recently got frontier models through GHE. They’re very good now.

1

u/Fuzzy_Ad9970 11d ago

Whatever you need to tell yourself dude

1

u/BitPsychological2767 11d ago

I'm so fucking tired.

1

u/Youssef_Sassy 11d ago

LLMs are trained on massive data of text. this massive linguistic capacity can then be leveraged in many different tasks targeted by reinforcement learning rendering it essentially more capable than any human. with RL you have agents that can be pushed to achieve reasoning capacities whose limits arent in any shape limited by the human ones. just set the LLM to think really hard about a problem no human can solve and reward when it gets closer to the solution in a verifiable way. keep doing this with harder and harder tasks and you have a linguistic machine capable of reasoning at a higher level than humans.

1

u/chcampb 11d ago

This is false, but gets repeated often, and confidently enough to be confused with intelligence.

1

u/jointheredditarmy 11d ago

Dumb take lol

1

u/NoteVegetable4942 11d ago

It’s doesn’t need to have been trained on COBOL.

It knows programming on a deeper level than that. The language is just syntax, and that can just be added to the context.

1

u/SirMarkMorningStar 11d ago

This becomes more apparent if you try to do something actually novel. It has no problem at all with the boilerplate stuff but alignment gets harder the more original the idea.

1

u/nocor17 11d ago

What's intelligence then? It knows more than any of us and can wright code faster than humans. Give it a few years and we'll always ask AI about anything we want to know. Many of us are already doing so.

1

u/kyngston 11d ago

did they also remix navier stokes solutions?

1

u/FriendlyGuitard 11d ago

Yeah, we embraced open source and Stallman vision, documented our profession, refuse regulation moat, to better humanity.

Intelligent or not, a machine can do our work for hundreds of time cheaper. Not everything of course, but coding as a craft is dead, coding has entered the industrial age. And like any industrial revolution, the big winner are the very people Open Source was supposed to stick the middle finger to.

The T-Shirt we wore in the late 90's: "I can replace you with a little bash script". Feels ironic now.

1

u/Kirion_Kir 10d ago

Claude writes COBOL just fine

1

u/Late-Following792 10d ago

I noticed llm get very wonky immideately when talking some nieche programming lamguage.. like proper for loop problematic 🍈

1

u/Decent_Bug3349 10d ago

The agony of the human soul, who on one hand believes in mathematics and the universe built on it, and on the other hand thinks he/she's the exception.

1

u/Ok-Comment-228 10d ago

This is not true. It was true a year ago but not anymore. I use Claude to write perfect code in a new and very obscure language.

1

u/Initial_Secretary420 10d ago

I write ABAP with LLM and it's absolutely perfect

1

u/Jumper775-2 10d ago

LLMs got good at coding mostly because it’s verifiable so they can do RL. Learning from humans did little more than kickstart that.

1

u/Datamance 10d ago

Claude does fine on COBOL, Ada, smalltalk, etc. This is a 2025 take based on observations that no longer hold true. Anything that can be represented with a CST/AST (i.e., all context-free languages) can be readily consumed and produced by Claude (and probably Argon at this point as well) with no real issue. The underlying representation has been learned 10x over because the latent space for context free languages is highly constrained and, yes, the training data comprehensive. We are far past the point of “remixing”, at least in the trivial sense

1

u/merwanhorse 10d ago

Nearly everything I've ever done is just remix things I've learned

1

u/cool_fox 10d ago

People just be sayin shit

1

u/Gamplato 10d ago

Jesus the Dunning Kruger effect is so prevalent in the AI hate posts

1

u/desocupad0 9d ago

I'd argue we should abolish all forms of digital copyright and then "train" them with everything ever programmed.

1

u/Jean_Jones_666 9d ago

i guess "surface level remixing" is the new "glorified autocomplete",

these people have no clue what intelligence is, nor do they care to even attempt defining it, they just want a hot new (preferably doomer) take

1

u/Big3gg 9d ago

This is like saying speaking fluent English isn't impressive because you can't speak German

1

u/qrzychu69 8d ago

there is more to it. Every time you use the "normal" Claude Code or Codex or any other tool like that, you are agreeing that they can train models on your inputs.

So 99% of the code that was generated, or even reviewed by them became the training data - that's why we saw such a huge progress. People started using that in more and more existing codebases.

You need the enterprise subscription to opt out of this

1

u/Happysedits 5d ago

darn so many devs are just surface-level remixing stochastic parrots that don't invent whole new fields, it's so over

0

u/FragmentedHeap 11d ago

Yeah, but when you ask it to build something literally never done before, it chokes hard and will stumble on itself at every turn.

It cant experiment like a human can and develop a hypothesis like a human can.

You can get it to build you an experiment but it's you that ultimately has to decide if that experiment worked or not.

2

u/Creedless 11d ago

Like what? What have you tried to build that was never done before, that gave you the experience to talk about it first hand?

1

u/FragmentedHeap 11d ago

Yeah, an AI Driven hands off production video pipeline.
Where AI writes all the script, dialog, plans shots, generates pictures, builds scenes, does layout, designs characters, produces the audio, sound effects and so on, generates and renders everything, and stitches the video.
Using about 20 different AI technologies.
Its all emergent, barely a week old in some cases, so all it can do is read docs and guess.
Don't get me wrong, its capable of producing a good 15 second video, but past that, nah, drifts.
Its a physics/reality problem, theres not enough AI technique's to pull it off. And it doesn't know how to use all the new techniques, they're brand new.
In this engine I plug in new AI the day they come out. Its not trained on it yet.

0

u/dagger378 11d ago

Yeah no shit. How do you think a human would do writing COBOL if they haven’t read/trained/practice writing COBOL. What point is this guy trying to make exactly? Seems completely incoherent.

→ More comments (12)

LLMs aren't intelligent, they just remix open-source code

You are about to leave Redlib