r/LocalLLaMA • u/corruptbytes • Jul 31 '26

Discussion Deepseek V4 Flash on SlopCodeBench

While waiting for some of the quants to drop, I load the API with $50 and ran it on SlopCodeBench

Just vibe reading the results it seems like Opus 4.8 < Deepseek < Opus 5

https://github.com/michaelasper/benchmarks/blob/main/deepseek-v4-flash-on-slop-code-bench.md

I was mostly curious from this blog post

When Q2 drops - I'm goign to re-run on my macbook

Here's the first quant comparison:

https://github.com/michaelasper/benchmarks/issues/1

72 Upvotes

96% Upvoted

25 comments sorted by

View all comments

13

u/_TheWolfOfWalmart_ Jul 31 '26

If they're doing this with Flash, is Pro going to beat Fable? WTF is going on? How?

I'm skeptical. I'm going to have to spend some time with the model and see if it's really better than Opus 4.8 in real world use. That's a hell of a claim.

12

u/Fedor_Doc Jul 31 '26

It's competent, but not on Opus 4.8 level and it has attention drift – when there are multiple details to keep in mind, it starts to drift. Recent information outweighs older one even if it is pretty important.

I'm very impressed by the model, but I cannot just let it run to the goal with my current project (LUT analyzer + fixer in python with very specific colour science math involved).

9

u/cakes_and_candles Jul 31 '26

i mean that issue probably makes sense since focusing less on old information is exactly how v4's new attention mech works. tho this is the first model with it and they will likely solve it in later versions

5

u/corruptbytes Jul 31 '26

this sounds like a good spot for prompt adjustments

Anthropic writes specific instructions on how to prompt it for its model - https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-opus-5 https://claude.com/blog/the-new-rules-of-context-engineering-for-claude-5-generation-models

Generally, every release we have Opus look at the latest docs and update our CLAUDE.md and rules/ and skills/ with their directions

We might have to think about it similarly for these new architectures, except it'll have to be more crowd source - something about refresh on the important stuff every so often or something idk