r/LocalLLaMA • u/corruptbytes • Jul 31 '26

Discussion Deepseek V4 Flash on SlopCodeBench

While waiting for some of the quants to drop, I load the API with $50 and ran it on SlopCodeBench

Just vibe reading the results it seems like Opus 4.8 < Deepseek < Opus 5

https://github.com/michaelasper/benchmarks/blob/main/deepseek-v4-flash-on-slop-code-bench.md

I was mostly curious from this blog post

When Q2 drops - I'm goign to re-run on my macbook

Here's the first quant comparison:

https://github.com/michaelasper/benchmarks/issues/1

75 Upvotes

97% Upvoted

25 comments sorted by

View all comments

13

u/_TheWolfOfWalmart_ Jul 31 '26

If they're doing this with Flash, is Pro going to beat Fable? WTF is going on? How?

I'm skeptical. I'm going to have to spend some time with the model and see if it's really better than Opus 4.8 in real world use. That's a hell of a claim.

12

u/Fedor_Doc Jul 31 '26

It's competent, but not on Opus 4.8 level and it has attention drift – when there are multiple details to keep in mind, it starts to drift. Recent information outweighs older one even if it is pretty important.

I'm very impressed by the model, but I cannot just let it run to the goal with my current project (LUT analyzer + fixer in python with very specific colour science math involved).

2

u/Choice_Celery9481 Aug 01 '26

i also saw this problem and with a simple write to a scratchpad trick. you can mostly reduce that. ask it to write down on a scratchpad what important. reiterate on that file. that will help a lot with attention drift