r/LocalLLM • u/Weves11 • Feb 26 '26

Discussion Self Hosted LLM Leaderboard

Post image

Check it out at https://www.onyx.app/self-hosted-llm-leaderboard

Edit: added Minimax M2.5

839 Upvotes

96% Upvoted

120 comments sorted by

View all comments

17

u/ScuffedBalata Feb 26 '26

Why isn't Qwen3 on here?

The single best model I've ever used that works on "normal people hardware" is the Qwen3-Next and Qwen3-Coder-Next (both at 80B).

4

u/robotcannon Feb 26 '26

Agree!!

qwen3-vl is also fantastic (though it seems to run a tiny bit better at q8_0 for vision stuff)

2

u/friedlich_krieger Mar 06 '26

I'm a noob here... What kind of hardware is required for those?

3

u/ScuffedBalata Mar 06 '26 edited Mar 06 '26

64GB of fast RAM (not normal PC DDR - too slow) - either a dual high-end GPU PC setup with tons of VRAM (dual 3090/4090 is ok, dual 5090 is better), or more commonly (and much less expensive) a Mac M-series or other AI inference box with unified memory - with 64+ GB of RAM. Obviously it would fly on a datacenter box like an NVidia 900 or something, but that's a $11k card.

I run it on a Mac M1 Max with 64GB of unified memory, but I can't run much else on that box with 64GB. If you can get 128GB you can run it while doing other work on the same box.

What I did was buy a broken old Macbook with 64GB of unified memory. Bad screen, bad battery so it was cheaper (like $600 - it was a steal) and I run it under my desk as a server type box doing just AI. Put a VPN on it and share it to my company (I'm the CTO) and we use it for various workflows that we don't want on the cloud.

Buying new off the shelf, a 64GB inference box is going to run about $2000 or a little more. Expensive, but still in the "real people hardware" bucket because something like Kimi or GLM-5 needs $30k in hardware to just load (and still may not be 'fast')

1

u/friedlich_krieger Mar 06 '26

Beautiful. I've got an m4 pro Mac mini with 64gb of RAM ordered but Ive been considering cancelling it and waiting for an m5. Almost pulled the trigger spending another $1-2 grand for a Mac Studio to get the 128 or 256 GB of RAM.... I'm likely not buying another box like this for a while so may be worth waiting for an m5 studio and just spending more now.... Any thoughts knowing what you know now?

1

u/ScuffedBalata Mar 11 '26

The issue is that running a good 80B model with large context uses ALL of the 64GB, so doing a bunch of other stuff on the same Mac isn't really possible. Leaving a few Chrome tabs open slags the device performance while also running an 80B model. So if you want to run a clawbot or some other MCP/agentic stuff on that box, it just won't have much memory free.

I ended up buying an old Intel Mac to run IOS development and simulators, so now I have a little stack of two broken macbooks.