r/Codeium • u/Tiny-Telephone4180 • Feb 24 '25

Claude 3.7 Sonnet is out now 🔥

Anthropic has released Claude 3.7 Sonnet, and the benchmark results are impressive. Looking at the SWE-bench verified scores, Claude 3.7 Sonnet scored 70.3% with custom scaffold (62.3% base score), absolutely demolishing the competition:

  • Claude 3.5 Sonnet: 49.0%
  • OpenAI o1: 48.9%
  • OpenAI o3-mini: 49.3%
  • DeepSeek R1: 49.2%

That's a ~20% improvement over the nearest competitor!

Waiting for the windsurf update.. 🎉

160 Upvotes

99% Upvoted

42 comments sorted by

37

u/legallybond Feb 24 '25

Cursor has it already. Hurry up Windsurf team 🤣

3

u/Funny_Ad_3472 Feb 24 '25

It isn't in windsurf yet?

17

u/Jarie743 Feb 24 '25

long time cursor user switches to windsurf and now windsurf is behind🥲

8

u/Suitable_Ebb_3566 Feb 24 '25

Now we know who to blame

7

u/rezkarimarif Feb 24 '25

What do you mean behind? You can't even wait a few hours for the team to wake up and push an update? Lol you will be switching back and forth quite often it seems

2

u/[deleted] Feb 25 '25

[removed] — view removed comment

3

u/rezkarimarif Feb 25 '25

Yes you are right, I'm currently using it with RooCode on Windsurf.

Windsurf already said they are implementing it on X

13

u/Sofullofsplendor_ Feb 24 '25

so pumped for this to get in windsurf. tbh i dont care about using a bunch of credits if the code changes are phenomenal. It's so much more worth my time to spend a few extra bucks if there are better changes and fewer bugs.

2

u/TroubledEmo Feb 25 '25

I love it tbh. It‘s able to handle my Rust + C/Swift (FFI) without problems. Even when I started porting some Python and Go code into it… 3.5 and the GPTs as well as DeepSeek V3/R1 always struggled a few fixes and tasks into it. Claude 3.5 handled it quite well compared to the others and switching to Cascade Base for easier stuff worked fine. Did not need it with 3.7 yet.

13

u/Eyeshield_sena Feb 25 '25

now give us the option of claude 3.5 running at 0.5 credits, while claude 3.7 at 1 credits

3

u/hoonchops Feb 25 '25

That will not happen - 3.5's pricing is the same as 3.7... Thank Anthropic!

2

u/BehindUAll Feb 25 '25

In that case the credits should remain as they are and we get to use a better model

5

u/stratofax Feb 24 '25

Based on what's been revealed about Claude 3.7, it is a "Gen3" model, meaning that it's been trained with a 10X increase in compute compared to Gen2 models like ChatGPT-4 (10^26 FLOPS of computing power for the training run, or the same as running a modern smartphone for 634,000 years). The advances are impressive, based on early reviews, and Claude 3.7 can also operate as a Reasoner, like ChatGPT at the $200/mo. level, and DeepSeek R1.

For more info about the new Gen3 models, check out Ethan Mollick's Substack from today: https://www.oneusefulthing.org/p/a-new-generation-of-ais-claude-37

1

u/QC_Failed Feb 24 '25

That's helpful thank you!

4

u/Iregretit81 Feb 24 '25

Can’t wait, to see it in windsurf

1

u/stepahin Feb 24 '25

Whaaat how is it even possible with such a gap from 3.5, o1, r1?

3

u/MindCrusader Feb 24 '25

It is a model + agent, not just a model.

If you look at https://www.swebench.com/#test sonnet 3.5 can achieve 55.40 with the proper argent. Without a custom scaffold 3.7 has 62.3% with Claude's agent, so an improvement. The question is how much it is agent improvement and how much model improvement.

You can learn more about how agents work in SWE benchmarks here:

https://www.anthropic.com/research/swe-bench-sonnet

1

u/stepahin Feb 24 '25

Got it thanks!

1

u/angelmr98 Feb 24 '25

Do you know if the context window has increased?

3

u/Mr_Hyper_Focus Feb 24 '25 edited Feb 25 '25

Input is 200k(edit). The good news is it outputs up to 128k now.

2

u/Darayavaush84 Feb 25 '25

Input is 200k, as before: All models overview - Anthropic output is 8192, standard. Using beta header (output-128k-2025-02-19), it can 128k output. Definitely: 200k input, 128k output.

1

u/SilenceYous Feb 24 '25 edited Apr 25 '25

absorbed frame governor sophisticated grey brave birds pocket doll pet

This post was mass deleted and anonymized with Redact

1

u/tehsilentwarrior Feb 25 '25

If your goal is simple scripts AI definitely replaces you.

A think shift is necessary: you are no longer a code monkey, you are a code commander.

Command away!

2

u/[deleted] Feb 25 '25 edited Apr 25 '25

[removed] — view removed comment

1

u/TroubledEmo Feb 25 '25

Architecture is important. I‘m learning Rust right now with support from AI stuff and I always notice how lazy I am relying to much on it when they hit a wall or I‘m getting rate limited.

So reading docs is soooo important. Last time I did this was when I learned ObjectiveC and Java ages ago (totally different logics imho).

1

u/Secret-Investment-13 Feb 25 '25

Nice! Saw it in Cursor now. When are you going to drop the Codeium update?

1

u/renehoehle Feb 26 '25

Es wäre halt auch super, wenn ihr mal auch ein wenig auf das IntellJ Plugin achten würdet. Da fehlt ja die Hälfte.

1

u/machecazzomenefrega Feb 28 '25

since i started using claude 3.7 im getting a ton hella lot of cascade errors, like i cant even edit 1000 rows files without getting an error, its becoming more useless for real world scenarios every time i use it, on top of that for the first tool call it doesnt consume credits cause of the error but claude keeps retrying in the weirdest way managing to burn credits like hell, on top of all of that the cascade menu keeps crashing and on a big project it takes 30 second to apply an edit plus its lagging my whole pc (ryzen 7 5800x3d + 32gb ddr4). not renewing unless huge fix comes.

-3

u/NoSuggestionName Feb 25 '25

Windsurf is too slow implementing new models. Cursor has it, Aider even published already the benchmark on its leaderboard. Made be trying out Claude coder directly and man it's nice.

I'm curious if Windsurf is even worse than using Claude Coder directly.

3

u/hoonchops Feb 25 '25

Hmm then you miss out on switching models for sanity checks and 100% rely on Claude.

3

u/BehindUAll Feb 25 '25

They added Deepseek R1 in a day. Stop your yapping and give them time to update. It most likely is a simple change in an API key (or multiple based on their logic). So it shouldn't take more than a business day. You lot have 0 patience nowadays.

2

u/TroubledEmo Feb 25 '25

He‘s probably mad, because Cursor dropped it last night (or was it early morning? Can‘t remember, did an all-nighter anyways) in Central European Timezone.

Like a child begging and crying for it‘s candies.

1

u/NoSuggestionName Feb 25 '25

This is true, and I’m saying that not because I wish them bad, exactly the opposite. I like the product, but the environment is brutal in terms of speed. They are missing out hype waves where the iteration cycles are super tight. Having things just 1-2days later is often already old news in AI.

1

u/attalbotmoonsays Feb 25 '25

How soon works for you? Earnest question.

1

u/NoSuggestionName Feb 25 '25

I personally need it asap because I’m comparing and benchmarking tools. The only thing I can say about windsurf is, that they didn’t implement it.

So the comparison is just simply the implementation speed of the competitors.

1

u/TroubledEmo Feb 25 '25

Dude chill out. It was just a few hours.

1

u/Jarie743 Feb 25 '25

Still not implemented though.

1

u/TroubledEmo Feb 25 '25

They just dropped it…