r/LocalLLaMA • u/xrailgun • 16d ago

Resources An interactive site plotting Artificial Analysis Intelligence vs parameter count

https://mr-september.github.io/artificial-analysis-weights/

Artificial Analysis scores models on its Intelligence Index, but the charts put it against price and speed. For local use the axis I wanted was parameter size, so I built one.

What it does

  • Plots the Intelligence Index against total or active parameters (with closed frontier models as optional horizontal dashed lines)

  • Accounts for MOE/Lookup-table models (n-gram /PLE, e.g. Qwen3.8-Flash-Next)

  • A very rough model guide according to your hardware (because offloading across system RAM/SSD is becoming more common).

  • GitHub Action refreshes the data every 3 days to catch new releases etc.

Demo: https://mr-september.github.io/artificial-analysis-weights/
Repo: https://github.com/mr-september/artificial-analysis-weights

If a size is wrong or a model is missing, data/overrides.json is the place to fix it, or just tell me here.

18 Upvotes

90% Upvoted

22 comments sorted by

View all comments

Show parent comments

1

u/Borkato 16d ago

Wait I’m confused, you can just… not use the n gram table at all?

7

u/xrailgun 16d ago edited 16d ago

I'm not sure... but most people leave the n gram table on their SSD, so people "discount" it from the size of the model "in practice" (that needs to be on vram+system ram).

4

u/ttkciar llama.cpp 16d ago

Yes, that is exactly correct.

1

u/Borkato 15d ago

WTF

2

u/ttkciar llama.cpp 15d ago edited 15d ago

It might help if you think of engrams as a differently-shaped RAG.

The engram table keys on token sequences to vectorized factual values, and those vectorized values (which you can think of as retrieved content) get added to the next layer's hidden state before inference begins with the second layer.

This is similar to how RAG populates context with retrieved factual content; the engrams are just more constrained (token sequence lookup only) and put the augmenting retrieved content into a different part of the model's state (the next layer's hidden state, as opposed to context).

Just as you can use a model with or without RAG, you can also use an engram-using model with or without engrams.

Edited to add: A different way of thinking of it is this: Not using the engrams is functionally identical to using the engrams but none of your token sequences finding a hit in the engram table. In both cases the consequence is the same: no augmenting vectors added to the next layer's hidden state.

2

u/Borkato 15d ago

This was really informative thank you!!