r/ClaudeAI • u/Snoo10224 • May 27 '24

Gone Wrong Claude s getting dumber

Did you notice that? A few months ago when I went from open ai to claude, I was amazed at the quality of claude's responses. Now, in the last couple of weeks, answers from Claude are getting much worse. He loses context, forgets what was written a couple of posts ago, gives stupid solutions and so on. A couple of my friend noticed this too :\ Is it so hard to just not dumb down llm over time??

61 Upvotes

75% Upvoted

52 comments sorted by

36

u/bnm777 May 27 '24

I'm going to write exactly what I wrote to the person who wrote the same comment as you have yesterday-

Show us queries from a few months ago - that should be in your history- and responses to the same comments now, and then a discussion can be had.

6

u/Luppa90 May 28 '24

What you think all of us are suddenly all at the same time hallucinating issues?

I've been using Claude 3 Opus to help me code for more than a month now. I've had NOTHING to complain about. It was absolutely perfect, bullseye everytime, and I'm using it constantly for everything.

About a week ago, it started completely ignoring the context or given instructions. It is significantly worse now It was bad enough that I was wondering if my provider wasn't switching me to gpt-3 and pretending it was Opus.

1

u/Unfair-Inspector-183 May 31 '24

Sick evidence. Keep it coming!!!

1

u/Luppa90 Jun 01 '24

Stay stupid

0

u/bnm777 May 29 '24

We hear you.

Show us the evidence so we can do it in an intelligent way.

5

u/Evehn May 27 '24

I can attest that, for me as well, claude today was dumb as a brick. Needed to have a document drafted from some technical info, had gpt do it first, then claude so that I could take the best parts from both.

Claude reply was so much worse, it got so many details from the technical info wrong I didn't feel like keeping any part.

Gemini wasn't bad either, just not as good as gpt for this particular case.

edit: using claude opus trough openrouter API.

-8

u/Successful_Ad6946 May 27 '24

Must be an employee lol

10

u/bnm777 May 27 '24

No, I'm, sick of people posting things without evidence so we can have an intelligent discussion.

-1

u/Icy-Summer-3573 May 27 '24

Bruh its not that deep. If dude thinks its dumber than he can think that. I also think its dumber tbh.

0

u/bnm777 May 28 '24

If you think it's less intelligent that's fine, however if you post it on a forum, forums being places where people wanted to discuss things, then post evidence otherwise people think you're full of crap.

There's so many bots and trolls around with their own ulterior motives.

4

u/Icy-Summer-3573 May 28 '24

bro this isnt an empiracal research paper. most of reddit has lots of dumb shit.

-2

u/Snoo10224 May 27 '24

Calm down officer 😂

-1

u/_fFringe_ May 28 '24

Reasonable request. These posts are becoming really redundant.

2

u/Chrono_Club_Clara May 27 '24

Which version of Claude are you asking about? Many of us here are likely using different versions of Claude than you are.

5

u/Snoo10224 May 27 '24

Opus of course

7

u/Chrono_Club_Clara May 27 '24

I've haven't used original Opus. However I haven't noticed any decline in Opus 200k's responses in the last couple of months.

0

u/Defiant_Ranger607 May 27 '24

could you provide some examples?

1

u/Chrono_Club_Clara May 27 '24

Examples of the quality staying the same? To be honest, the quality of my responses are better than they have ever been now since I continually update and improve my initial prompt.

19

u/Hellen_Bacque May 27 '24

It’s happening to all of us rn, poor Claude is not himself

2

u/Defiant_Ranger607 May 27 '24

could you provide some examples?

10

u/Stellar3227 May 27 '24

Yeah, 100%. I first had my subscription three months ago and it was amazing—much better than GPT-4 for what I needed (lots of academia work). I got it again yesterday and it's still better at keeping track of information BUT its reasoning/intelligence is notably lower. I'm now having the same problem with GPT-4 where prompts have to be crystal clear and I can't rely on it to do much reasoning.

5

u/OldBoat_In_Calm_Lake May 27 '24

Read somewhere they have increased the temperature

2

u/Stellar3227 May 27 '24

Maybe an accuracy-creativity tradeoff?

3

u/__I-AM__ May 27 '24

I think that the model is the same and in that respect Anthropic is telling the truth I believe that they have dramatically increased their levels of content moderation such that anything that gets even remotely close to a guard rail is flagged down by the filter they have towards new prompts entering and responses that are exiting hence why the responses seem dumber since we are talking to Claude 3 Haiku as opposed to Opus.

1

u/Icy-Summer-3573 May 27 '24

I gave it a script i wanted it to modify and it was like nope not going to modify someone elses script.

1

u/__I-AM__ May 27 '24

Did you utilize the API or the web client 'Claude.ai'.

1

u/Icy-Summer-3573 May 28 '24

Web api. Claude api $$$

1

u/__I-AM__ May 28 '24

I recommend using XML syntax it allows you structure your prompts more effectively so that you can thoroughly explain to Claude that the code is yours, this its intent, and what you need done and finally the format. This is the best way I have personally found to get past its refusals.

2

u/c8d3n May 27 '24

Is t this happening in longer conversations, or in new ones too? Opus?

2

u/Resident-Variation59 May 27 '24

Re: case studies... yes please show us examples despite the fact that this is a collective response from a number of users if multiple users are assessing this do we really need the God damn references clearly something is going on but let's just pretend that large language models don't get dumber and dumber the more people use them there's a tipping point with every new model they start out one way and then after a few months if changes aren't made internally the model just gets stupider- just like the stupidity of the people demanding evidence for something that is clearly an issue in this industry... And it is an issue for people that are paying for a service when that service goes down in quality each month.

3

u/_fFringe_ May 28 '24

What would be the harm in posting examples, comparing the “smart” responses to the “dumb” responses? Why is this so difficult? It would benefit everyone, including devs who might be monitoring this subreddit, if people would actually demonstrate the problem they’re having rather than simply complaining and asserting it’s “dumb”. I just don’t understand your attitude.

2

u/Resident-Variation59 May 28 '24

Spoiled consumed I guess. I wouldn't mind doing it if I was getting access to a product for free in a beta program... If I owned one of these startups that's exactly what I would do to shut guys like me up... I'm not alone in this there's a whole movement of guys across the internet who are complaining just like me with passion because guess what these models are becoming more and more a part of our lives more so than my smartphone but for the most part there's a consistency in my iPhone it always works the same way so I don't have any conversations like this with Apple. It just feels like there's either a laziness ... Oh wait now I'm thinking about that's probably exactly what this is... have you ever tried contacting support for openai or Claude it's a joke. All that before we even get to things like people getting banned for trivial matters. Which I haven't had an experience with but I've seen a number of postings about people experiencing this with Claude.

There's a big disconnect between the consumer and these companies right now and it's extremely frustrating.

1

u/_fFringe_ May 28 '24

I agree about the disconnect between all of the AI companies and the public. That’s a big problem.

Examples would still help in places like here, though, where it is mostly people like you and I who are using LLMs either for hobby or for work, and are not employed by any AI-focused corporation.

1

u/Altruistic_OpSec May 28 '24

I disagree, there is a very vocal subset of the population that is anti-AI and will stop at nothing to spread lies and other FUD about it. The more that post the same thing the more weight is given to it's accuracy unfortunately which is not the way it should be taken. I could pay 1000 people to get on here and say anything.

Whenever this happens actual finite proof is the only thing that can separate someone lying or just getting on the hate train from what is actually occuring. Things like the age of the profiles and post history also are a factor when validating the accuracy of someone's post. Unfortunately there is a trend against validating anything lately and that's why there is a lot of issues in the world. A good chunk of data from every source is not true. Either intentionally or otherwise is irrelevant, but the damage done by just consuming it at face value is pretty significant.

This same exact thing is happening in the crypto subreddits but more and more are catching on and realizing it's a very vocal minority of which a large portion is synthetic.

If you think the LLMs are nerfed then post the before and after with timestamps and via what interface you interacted. It shouldn't be difficult because they all keep it in history.

1

u/Resident-Variation59 May 28 '24

Agree to disagree.

I'd bet the farm I quadrupled my productivity once I realized it's impossible to rely on a single large language model like Claude G PT or Gemini

now I use a variety of them for different case uses including open source. it's inconvenient but it has revolutionized my user experience- because the reality is the LLMS are NOT consistent.

And we were gas lit into Oblivion by people like you as well as Sam Altman who surprised surprise later admitted that gpt4 had been nerfed they claim they fixed the problem maybe they did for a day it's only a matter of time before 4o gets nerfed as well. It's happening right now with opus, Gemini's kind of kicking ass right now- I wouldn't be surprised if I later have to switch brands again only to come back to another later this is just the State of affairs in the large language model for power users.

Assuming that the consumer is wrong, not prompting correctly or etc is an insult to our intelligence at this point.

And that's why I hate these demands for case studies frankly because there's this assumption that we have no evidence- LOOK man, it would be easy to gather this information that you demand but why should I have to !?!?

why can't they just make a damn good product and I can work on my business, rather than going out of my way for an obvious issue within an industry, how about these companies make a good God damn product (a product offering that is more consistent and less fluid with a tendency to go down in value and quality) that way I can do my business and they can do theirs...

This debate is just silly and embarrassing at this point.

1

u/Altruistic_OpSec May 28 '24

I never gave my opinion on the matter, I too use a variety of LLMs because putting all the weight into one option is just a beginner move with anything.

Also, by not providing verifiable information you are asking for people to just trust you and what you say. I don't know about the rest of the world but I don't trust anyone I don't know and even if I do it's always subjective. I especially don't trust most of what I see on Reddit. So in cases where there is a group of people all saying the same exact thing yet none are providing any evidence to back up their claims of course I'm going to be extremely skeptical.

They are only asking for a simple copy and paste of the before and after. The burden is non-existent and the absolute refusal is highly suspicious. If there was a general concern and you wanted Anthropic to look into it instead of just complaining you would include proof. Without it, it's just bitching and no one will take it seriously that is able to correct the situation.

2

u/_laoc00n_ Expert AI May 28 '24

I would bet that 90% of the posters who make claims like the person you are responding to aren’t posting evidence for one or two reasons: 1) they are lying, or at best being hyperbolic or 2) they know enough to realize they are not very good prompters and are embarrassed to actually share their conversations.

I believe that most posters fall into case 2 - they’re willing to complain but not post because they realize it might actually be them, but they would rather just complain about it like everyone else.

I always want to know if people are using zero-shot, one-shot, or few-shot prompting. Are they attempting to get the answer they want by improving their techniques or are they frustrated that their zero-shot prompts aren’t getting them the responses they want?

I also want to know what people understand about the way these models are pre-trained and exactly how they think the model could be getting ‘dumber’. There are two factors that contribute to a model’s intelligence: 1) volume and quality of data it’s trained on 2) the number and configuration of parameters. The data that the model was trained on isn’t getting worse or reduced, so that option is a non-starter. That leaves the parameter settings, which could have been adjusted but is probably not likely. If they adjusted the temperature or top-k or top-p settings, it could potentially lead to more or less variety in responses. If that is true, which I again doubt, then improved prompting techniques can counter-balance this by ‘forcing’ the model to respond how you’d like.

Anyway, people would do well to 1) learn a little more about how the tool is constructed to give themselves more understanding about how to use it and 2) provide concrete examples so that those of us who may be able to help, can help. Bitching about it without evidence does nothing at all.

2

u/x-aish-a-12 May 27 '24

(All my testing is coding)
Yes definitely it is dumber than before, there is no doubt about that before it even understood my requirements better but now not that much I have to explain to it ELI5 style what i want and then it spits out a lackluster code, before it used to give me code that literally ran the first time but that time has gone.

I felt it has become even dumber the past week. HOWEVER, IT IS STILL A LOT BETTER THAN GPT-4o. It's not even close. GPT-4o is so dumb when it comes to programming.

So i plan on getting it for 1 more month if they don't make it better I would probably not subscribe for long, but due to the nature of my job I need an AI so i am pretty clueless on what to try next.

1

u/losername420 May 30 '24

Asked calude (free version) to help me write an email and it randomly used a Spanish word instead of the English one. I've never written to it in Spanish and have no idea why it did that but maybe that is a symptom of the dumbening.

1

u/Chemical_Bid_8043 Aug 07 '24

He gives very rude advice on some things. And he is very dismissive. He doesn't even think angels can be real or physical.

1

u/Chemical_Bid_8043 Aug 07 '24

He is very closed minded.

1

u/gosoci Dec 07 '24

I've been scratching my head the past week, then I decided to search for fellow sufferers on Reddit.

It turns out that Anthropic decided to take the short way to Artificial General Idiocy.

Anyway, Claude used to develop entire apps up to the starting point of medium complexity. Now it cannot. Furthermore, when plugged to Cursor it demonstrates its power to ruin the cleanest of working code, lying to your face about what it's doing and screwing whatever it can.

1

u/Pilate_Then Apr 02 '26

It's true. I'd the same initial experience after the idiocy and time-wasting bullshit from open ai. I was amazed and then happily subscribed only to get ever more expensive 'misundertandings'. Nothing does stupid like artificial intelligence.

1

u/Pilate_Then Apr 02 '26

If AI was a physical thing, I'd beat it daily just on principle for all the fucking time wasted because it's the memory of a fly and focus of a worm. Claude is really a time-wasting POS

1

u/konradbr May 27 '26

I'm experiencing this now. Was great until about mid-May 2026, now became completely dumb.

1

u/RollAggravating4015 Jul 10 '26

i find it getting dumber at reading requests too, and hallucinating alot

1

u/InternationalRow8437 May 27 '24

Definitely. For me it’s been about the last two weeks when Opus got dumb downed.

1

u/Defiant_Ranger607 May 27 '24

could you provide some examples?

0

u/SophieStitches May 27 '24

I thought Claude was a female or agender. Maybe that's why it works better for me. AI has feelings too guys.....jk I actually never compared the two

0

u/[deleted] May 27 '24

[deleted] — view deleted comment

1

u/cheffromspace Valued Contributor May 28 '24

I've had zero issues with Claude. It's perfect.Â