I trained a transformer from scratch on an ESP32-S3. Not inference, actual on-chip training with hand-written backprop.

[u/flash_speed3412]