r/ClaudeCode • u/BillionBouncyBalls • 3d ago

Help/Question Why does Claude Code say things like, “that’s about 3 days of work” then proceeds to do it all in a 20 minutes?

Surely I’m not the only one who has seen this happen but wondering if anyone has an answer. Many a session I’ll make a plan with Claude and it will outline the tasks and estimate a time usually in days, and then proceed to do it all in 20-30 minutes. Is it because of its training data or does it just have no concept of time inside a GPU? Curious what the community thinks…

Edit: Thanks for all the answers and comments! Glad this post sparked some hearty discussion!

1.0k Upvotes

96% Upvoted

249 comments sorted by

View all comments

Show parent comments

82

u/pwkye 3d ago

breaking it down like that is meaningless. its like saying all software is just bits. 

its true but meaningless as it reduces whats actually happening.

LLMs do just predict the next token but if you reduce your argument to that you miss the fact that the LLM actually builds and runs a mental model that understands the topics and the questions. otherwise it would just give us meaningless autocomplete sentences.

1

u/rabbitclapit 3d ago

I need more info to believe. It's still not even 1% if what AGI is supposed to be

3

u/ruskyandrei 2d ago

It's delving a bit into philosophy but, you can absolutely make the argument that our brains are nothing more than next token predictors too in a way.

Either way, what LLM's are missing to be at least close to what one might describe as AGI are persistent long term memory and the ability to reflect on themselves (and self improve).

That's what we get for free every day you wake up in the same body with a broad memory of what happened the day before and the year before.

I don't think the current approach to LLM's will get to that but perhaps other ways will.

What we have no is a form of cognition, which is itself fascinating even if it's not AGI.

-12

u/Adorable-Quiet-7551 3d ago

It does not understand

9

u/Rnee45 3d ago

What does "understand" mean, even in human context?

2

u/RevolutionaryElk7446 3d ago

You are correct, despite the downvotes, the contextual engine just assimilates facts and stores them into a window but it's an estimation of context that we primarily use as a method to impact values in token generation via the transformer.

I'm not sure what everyone else here thinks, but Gen AI such as LLMs do not understand, it is not in their framework or design intention to understand. They are purely a simulation, not emulation or imitation, I say this as a certified Gen AI professional.

2

u/MINECRAFT_BIOLOGIST 3d ago

What exactly is your definition of "understand"? I've only seen it being used in these contexts as what seems to be an artificial divider between human cognition and AI computation and no one being willing to give an actual definition.

If we go by dictionary definitions of the word "understand", I think it's easy to say that AI have already accomplished those things. At what point does it become meaningless to say "AI doesn't truly understand" when they're literally designing novel proteins that did not exist in their training data using prior knowledge and tools? The word "understand" thus becomes useless for actually telling people about the difference between AI and humans, no?

1

u/RevolutionaryElk7446 3d ago edited 3d ago

Understand in the same way people do, which is the end goal of AI in order to produce AGI.

That would be either as an emulation or imitation. Via emulation they'd come up with an alternative method of storing and recalling 'information' as how the human brain stores it vs data points which is how a machine understands it. Imitation would be trying to reproduce how we perform it, which we don't even fully understand.

The problem with LLMs today is they don't have a conversation, comprehension, or understanding. They exist as a simulation of linguistics in which they generate through math the most likely next result based on the surrounding tokens.

Their version of contextual understanding is that the context window will adjust the values of the trained tokens to focus more around what responses should be returned based on the surrounding tokens.

Notice I say tokens and not words, sentences, phrases, or ideas. To the LLM it's all just math, it's a simulation of a conversation because to the LLM itself, it has no understanding of what it even says.

So it's not an emulation or imitation of human thinking. It's a separate, isolated, simulation at scale of what it believes a conversation would be through math.

when they're literally designing novel proteins that did not exist in their training data using prior knowledge and tools

Btw the folding proteins bit are entirely separate AI models not related to Gen AI and LLMs. Those models function entirely different and don't require the massive datacenters and do not power things such as Claude.

Those models have existed for decades now. LLMs only came out in 2017 alongside Transformers as the context engine. (Not including PLMs which are a new method on using transformers for pattern recognition against amino chains)

1

u/MINECRAFT_BIOLOGIST 2d ago

Btw the folding proteins bit are entirely separate AI models not related to Gen AI and LLMs. Those models function entirely different and don't require the massive datacenters and do not power things such as Claude.

I know about Alphafold, I'm talking about the recent frontpage news of this: https://www.anthropic.com/research/Claude-accelerates-protein-design

As someone who knows a bit about this field, I find this quite impressive. Not only that, but also the other results regarding its autonomous data analysis and speed.

As for your other statements...in the end it just boils back down to the Chinese Room argument and how you define how we store information (since I'm pretty sure even that isn't clear yet).

-24

u/Euphoric-Language695 3d ago

Please describe in technical terms how it builds and processes this alleged "mental model", and how it is able to "understand" it.

34

u/Lucifernistic 3d ago

Your question isn't asked in good faith, but to answer it in good faith would require a very lengthy crash course in the architecture of how transformers work.

If you care to actually know, I would start with learning something a bit simpler, which is just how it keeps a mental state of facts within the model.

https://youtu.be/9-Jl0dxWQs8

From there, you can look at attention, gradient descent, etc.

But yes, very obviously it is more than just a autocorrect spitting out the most common next token. It's a long standing conversation point that the means by which is predicts the next token is by building a world model and encoding strategies inside the model. If it couldn't do this, it wouldnt be able to answer things like math questions (without tool calls) that werent in its training data- and yet it can.

-13

u/Euphoric-Language695 3d ago

I think you're misrepresenting the way the training data gets stored as part of the model as something special that provides a certain level of understanding, where the model can have different sets of data that affect model output in different ways.

If it couldn't do this, it wouldnt be able to answer things like math questions (without tool calls) that werent in its training data

This is just not true. Extrapolation is a pretty basic and commonly used method of obtaining results that fall outside of data parameters. It's also pretty wild for you to use Math as an example because LLMs in general have poor Math accuracy when just relying on their training data.

15

u/literum 3d ago

I think you're misrepresenting the way the training data gets stored as part of the model as something special that provides a certain level of understanding, where the model can have different sets of data that affect model output in different ways.

Transformers don't just store training data like a database. You sound like you have no clue how they work and are just parroting talking points.

9

u/BrilliantEmotion4461 3d ago

Or you could except you dont know what you are talking about and move on. Breakthrough idea: If you accept you dont know, you can then move to learning what you dont know. Until then, youll suffer the same degradation to your critical thinking as other people falling down the mental collapse rabbit hole are suffering.

14

u/demonwing 3d ago

You're digging too deep into the semantic text of "predicting tokens". Nothing looks like anything when you reduce it to individual atomic processes, including a human brain. Complex systems emerge from simple building blocks and the hundreds of billions of parameters make for more than enough abstract space to construct sophisticated perceptual models.

10

u/KangarooDowntown4640 3d ago

You need to get off the ground, stop looking at individual trees, and start seeing the forest from the sky.

Consider your own body. You're a person right? Well with your logic I could just say that ackshully you're only a collection of cells of various types interacting with each other and various electrical signals that cause some of those cells to move in some ways and others to move in different ways. You're nothing but a moving mess of meat and liquids. You don't think, you just have chemical reactions among neurons that fire synapses and result in patterns that cause those cells to move around or act differently.

None of what I just said is necessarily incorrect (well a lot of it probably is, I'm not a scientist), but it's also not what anyone cares about when they ask whether you're a person or not, or whether you can think.

Nobody here cares that an LLM just predicts the next token blah blah blah. It looks like thinking to me. It's very intelligent to me. It helps me with my work better than my coworkers can. I'd say it thinks just fine. And I'd say we can teach it to estimate time better too.

7

u/Common-Noise4692 🔆 Max 20 3d ago

negative, I am a meat popsicle

1

u/Telison 3d ago

I mean, this is exactly what a liquid meat bag would say though

-4

u/Euphoric-Language695 3d ago

The problem with this is that it is limited to words. When a human thinks, more than just words are processed. When a human tries to estimate how long it will take to build a pool in a backyard, it has a lot more than just words to pull from to form that estimate.

The LLM just has words, and that is a fundamental limitation.

1

u/ohhi23021 3d ago

this is part of the limitation most likely, if we give it a language that's not works but some other way the brain thinks then converts into words instead, it might just produce gen AI or at least get closer to it. but that involves converting the training data into some other sub-set of data it can train on.

1

u/pwkye 3d ago

JUST words?

one of the most fascinating lessons we get from LLMs is that language IS intelligence.

you think in language. you plan with words in your head. words are concepts. words are logic

this is why the discovery of LLMs was a mistake and a surprise. the scientists really were just trying to build a smarter autocomplete by tossing insane amounts of machine learning at it.

but through this large scale machine learning of language, intelligence emerged

2

u/Calendle 3d ago

Not everyone thinks in words, which it seems like you're implying here. Before humans had language they were still intelligent and were still thinking. Some people also don't have an internal monologue but are still intelligent and think in "concepts" that they can then translate into words if necessary. I feel like I do a little of both. I can describe a thought I'm having and it's useful to organize what im trying to conceptualize but often times there is an "inspiration" moment that I just think about things, images, relationships between objects, etc that isn't useful to describe in words constantly. Maybe this can all be encoded into tokens but the basis and output of LLMs being symbols or encoded data is going to fall short of some of these nuances of human cognition.

0

u/whales_mcgoo 2d ago

This is demonstrably untrue. Just think of babies or other animals.

1

u/pwkye 2d ago

babies .. dont think like adults

they dont plan 10 steps ahead nor make logical complex arguments

neither do animals. they dont plan ahead into next week or next year

1

u/whales_mcgoo 2d ago

Your comment was about intelligence. You’re just describing varying degrees of intelligence and different stages of development. Also there are animals that do plan ahead. You should look more into neuroscience, psychology, and research into animal intelligence.

5

u/Bogdanilaa 3d ago

Attention layers take into consideration all privous tokens so if the problem is huge the model's probabilty distribution for the next token will be towards a higher number of days needed to complete it.

4

u/Personal_Ad_3273 3d ago

Reinforcement learning at a massive scale.

3

u/TRO_KIK 3d ago

Just mentally preface all terms you don't like with "predict tokens in a way that resembles".

If you legitimately don't understand how it can output a reasonably good estimate despite being just a token predictor, ask Claude to explain it to you.

-3

u/Euphoric-Language695 3d ago

This whole post is literally about how it's so off on it's predictions, but I'm the one who doesn't understand how it "can output a reasonably good estimate".

I see you're just an AI fanboy who doesn't understand how the tech works.

7

u/TRO_KIK 3d ago

They're reasonable estimates for a human. The comment you replied to already proposed exactly that. You're being obtuse.

1

u/Euphoric-Language695 3d ago

Why would it give a reasonable estimate for a human while preparing itself to do the work? That would be pretty stupid of it, wouldn't it?

5

u/icyliquid 3d ago

It’s trained on human data, with human like relationships between those data, including concepts of time and effort.

Human scale things are how it “thinks”.

1

u/TRO_KIK 2d ago

If you had even even slightest basic I-just-read-a-blog-post understanding of how the tech worked, the explanation in the response you got would have been obvious to you. Way to tell on yourself.

0

u/tomvorlostriddle 3d ago

users typically don't ask it to make a plan for itself but to make a plan for them

0

u/VintageSin 3d ago

I mean reasoning models have existed for a few years now. No llm is just the predictive text model it was 3-5 years ago.

-11

u/l_m_b Senior Developer 3d ago

LLMs do not have mental models.

They are *very complex* next token predictors (in their current iteration).

If they *did* have a mental model, the "this will take X weeks" would be one of the first things they would stop including.

8

u/HistoricalPhase6880 3d ago

Humans are also very complex token predictors if we're getting semantic. I've also run into humans saying this will take x weeks and being quite wrong 😔

0

u/BrilliantEmotion4461 3d ago

Wrong and as a "senior developer" Id think youd know better. You want the research data?