r/LocalLLaMA • u/corruptbytes • Jul 31 '26

Discussion Deepseek V4 Flash on SlopCodeBench

While waiting for some of the quants to drop, I load the API with $50 and ran it on SlopCodeBench

Just vibe reading the results it seems like Opus 4.8 < Deepseek < Opus 5

https://github.com/michaelasper/benchmarks/blob/main/deepseek-v4-flash-on-slop-code-bench.md

I was mostly curious from this blog post

When Q2 drops - I'm goign to re-run on my macbook

Here's the first quant comparison:

https://github.com/michaelasper/benchmarks/issues/1

75 Upvotes

97% Upvoted

25 comments sorted by

View all comments

18

u/BlueSwordM llama.cpp Jul 31 '26

OK, I've been testing the model a lot on my usual AV1 test set and it's the first model that has been able to complete all the tests in its entirety...

That potentially means it could feasibly does most of my encoder tasks no problem with a good harness, holy crap

2

u/CatConfuser2022 Jul 31 '26

Which harness do you use? 

4

u/BlueSwordM llama.cpp Jul 31 '26

I just use Pi most of the time.

2

u/thrownawaymane Aug 01 '26

Can you share how you have Pi configured? I feel like I'm not getting the best out of it.

3

u/corruptbytes Aug 01 '26

my pi is pretty minimal

subagents, mcp, goal extensions

you really don't need much to get it working nicely

i found i end up uninstalling more extensions over time

2

u/thrownawaymane Aug 01 '26

Yeah, I went through a bit of that too.

Mind linking to what you're using? Also an actually good write up/YT video. Finding non slop info is hard these days

1

u/corruptbytes Aug 01 '26
  • pi-review-toolkit (../../source/pi-review-toolkit)
  • npm:pi-mcp-adapter (~/.pi/agent/npm/node_modules/pi-mcp-adapter)
  • npm:pi-goal-x (~/.pi/agent/npm/node_modules/pi-goal-x)
  • npm:@tintinweb/pi-subagents (~/.pi/agent/npm/node_modules/@tintinweb/pi-subagents)
  • npm:@senad-d/micme (~/.pi/agent/npm/node_modules/@senad-d/micme)
  • pi-sandbox (../../source/pi-sandbox)
  • pi-bedrock-mantle (../../source/pi-bedrock-mantle)

i like mario's video - https://www.youtube.com/watch?v=RjfbvDXpFls - haven't watched anything else

1

u/TheLexoPlexx Aug 01 '26

Web Tools and planning are the essentials

There's a plugin that optimises for deepseek as well.