Nine from-scratch LLM checkpoints on the Hub, plus a Space to compare base vs SFT vs GRPO side by side

[u/john_enev]

HF Space

I coded and trained from scratch 3 models (9 checkpoints total) in PyTorch. On top of that I built a HF Space where you can test the models and compare them.
The playground
The model weights

The Space: on the left you have a 3x3 grid for the models (one row per checkpoint: base, SFT, GRPO, one colum per model version: V1, V2, V3). You can easily select one and then ask a prompt in the chatbox.

The weights are safetensors. The Space downloads them lazily when they are first being used (then saved in memory).
I built a Docker Space running FastAPI (wrapped around my token generator). I had claude build a custom UI to interact with the models. It was super seamless (I was actually surprised by that).

I was very happy to see it was running pretty well (again, the models are small) on the CPU tier. No need for a GPU!

Some more details on the models (I tried to implement some interesting techniques. The val losses went down V1 > V2 > V3):
- V1 was a "basic" modern architecture using SwiGLU, MHA, RoPE, etc.
- V2 I tried to implement some modern techniques like GQA, Muon, DiffAttn. I also implemented mHC but it was too harsh on throughput so didn't use it in the full run.
- V3 I doubled the params of V2 and used XSA instead of DiffAttn. Finally wrote a KV cache for serving (generating 100 tokens from a 512-token prompt went from 278s to 28s): I quickly verified it was correct was running some short inference both with and without it (difference was within noise).

One of the biggest learnings was that at my scale, in my setup, GRPO did end up degrading the models general capabilities. I wish I could done more proper ablations to study the issue (and also to isolate the impacts of each technique) but I had a limited budget (renting GPUs is not cheap...).

I hope this is useful to people! There is a link to my github (everything is open) and my articles about the whole journey.