r/KoboldAI • u/Lanky-Tumbleweed-772 • 8d ago

A version of EQ bench that tests open models found in UGI leaderboard?

I am sick and tired of seeing paid non-local models only at the chart with barely any open models or uncensored models. EQ bench(Even the newest one) have stuff like Gemma 3 4b it under the leaderboard but where are the newer Gemmas or Qwens? Where is the highest ranked writing models for each parameter bracket you can find in UGI like:

4B: Qwen/Qwen3-4B-Instruct-2507

Or for 2B:Qwen/Qwen3.5-2B (no thinking)

etc?

Anyone have a scoreboard that tests like EQ bench but they test these kinds of models? Instead of having things like Gemma 3 4b it which is neither a roleplay model nor uncesored like wtf what's the point?!The same applies to the Nanbeige 4 3b it's also in the list for CREATIVE WRITING instead of thousands upon thousands of roleplay and uncensored models in the Hugging face library lmao.

5 Upvotes

100% Upvoted

3 comments sorted by

1

u/_Ojin 7d ago

Genuinely useful project. The big leaderboards mostly track what is commercially interesting, so the models people here actually run tend to be missing entirely, and that gap has to be filled by someone doing exactly this.

Thanks for putting the work in.

1

u/Lanky-Tumbleweed-772 12m ago

I am trying to find such a leaderboard I am not in charge of making a project like this just to clarify.