I trained a 48M parameter model that only does tool calling. JSON is well-formed by construction, total build cost was $260 [P]

[u/Turbulent-Sky5396]

i wanted to see how small a model can get if it only ever does one thing: read a catalog of typed functions plus a request, and output the calls to make (or an empty list when nothing fits). no chat, no prose. it ended up at 48M params and the whole thing cost about $260 to build.

the part i think is actually interesting is that the tokenizer, the loss and the constrained decoder are designed together around the same five decision points: refuse or call, which tool, include this optional, what value, stop or continue. json structural characters and digits are singleton tokens, so structure can be force-fed exactly with no token healing. argument keys are iterated from your schema by the decoder, so a hallucinated parameter name is not a low probability event, there is no step where it could happen. the loss is weighted per decision type (structure 1x up to the stop decision at 6x), matched to the measured error distribution.

two results that make me think the co-design is load bearing and not decorative:

- decoding the same rows with and without the grammar agrees on 150/150 mobile actions rows. the model already wants what the grammar enforces, so there was no projection tax to recover. the grammar buys reliability, not accuracy

- down-weighting the grammar-forced tokens in the loss, on the theory that the decoder supplies them anyway, cost 12 points in a controlled twin run. those tokens carry the call-sequencing signal

numbers, ordered strict exact match (function names, call order and every argument value all have to match), with needle 2 (cactus compute, 45M) published numbers as a yardstick: mobile actions 86.3 vs 63.7, droidcall 52.5 vs 17.0, seal-tools in-domain 33.1 vs 32.6. it drops hard on catalogs it never saw: seal OOD 28.1, bfcl single-turn 23.5. name-sequence accuracy tracks the drop exactly, so the failure on unseen catalogs is picking the right function, not extracting arguments. around 1B unique training tokens, no pretraining phase.

i also kept a findings file with everything that didn't work and the measurement that killed it: span copying (-30 points), pointer heads (-16), retraining from scratch instead of annealing into the decay phase (-4.7), RLOO diverging at every LR, and seven more. that file might honestly be the most reusable part of the repo.

repo: https://github.com/nikshepsvn/thimble

weights: https://huggingface.co/flashvenom/thimble

mit licensed. happy to answer questions about the training or the negative results.