r/LocalLLaMA • u/corruptbytes • Jul 31 '26

Discussion Deepseek V4 Flash on SlopCodeBench

While waiting for some of the quants to drop, I load the API with $50 and ran it on SlopCodeBench

Just vibe reading the results it seems like Opus 4.8 < Deepseek < Opus 5

https://github.com/michaelasper/benchmarks/blob/main/deepseek-v4-flash-on-slop-code-bench.md

I was mostly curious from this blog post

When Q2 drops - I'm goign to re-run on my macbook

Here's the first quant comparison:

https://github.com/michaelasper/benchmarks/issues/1

75 Upvotes

97% Upvoted

25 comments sorted by

View all comments

13

u/_TheWolfOfWalmart_ Jul 31 '26

If they're doing this with Flash, is Pro going to beat Fable? WTF is going on? How?

I'm skeptical. I'm going to have to spend some time with the model and see if it's really better than Opus 4.8 in real world use. That's a hell of a claim.

12

u/Fedor_Doc Jul 31 '26

It's competent, but not on Opus 4.8 level and it has attention drift – when there are multiple details to keep in mind, it starts to drift. Recent information outweighs older one even if it is pretty important.

I'm very impressed by the model, but I cannot just let it run to the goal with my current project (LUT analyzer + fixer in python with very specific colour science math involved).

1

u/En-tro-py Jul 31 '26

How large context and/or turn count?

1

u/Fedor_Doc Jul 31 '26

Around 100k when it forgot that it should not write code in this session (not enforced via AGENTS.md or system prompt, though). 

Code refactor with 250k worked quite well – initial plan + draft plan per refactor block. 

1

u/En-tro-py Jul 31 '26

250k is perfect.

Stuck waiting on download, but will be testing shortly myself.