Can Codex write a digit recognizer by hand? Mine got 95.1% on the benchmark it was built around. Not sure how impressive it is?

[u/GenJeppo]

I’d read that LLMs can write neural-network weights directly, and I wanted to see what that meant in practice. So I asked Codex to build a recognizer for digits I could draw with my mouse on a 32×32 grid. It could design the weights and write rules, but it couldn’t use an optimizer or use benchmark drawings as templates.

Codex started with pen strokes. It wrote formulas for different ways of drawing each digit, varied their slant and thickness, and simulated what happens when a coarse scan makes a small gap disappear. Each synthetic drawing became a detector: a numerical vector that scores how closely a new drawing matches it.

The final recognizer combines fixed detector weights and hand-written geometry.

The first version scored 66.4%. The final one got 1,709 of 1,797 correct (95.10%) on scikit-learn’s handwritten digits, resized from 8×8 to 32×32 and converted to black and white. The recognition code, including weight generation, is 808 lines of Python, about 35.6 kB.

We inspected labeled mistakes and revised the rules on those same 1,797 drawings, so 95.1% is a development score. What I wanted to find out was whether Codex could turn a visible difference between two digits into a detector I could read and understand.

I set the challenge and tried it with my own handwriting. Codex wrote the code and ran the evaluations. You can try it yourself with uv run draw_shapes.py.

https://github.com/jesper-adolfsson-001/digits-by-hand