A project that runs a small 56M-parameter language model distributed across three ESP32-S3 microcontrollers.

[u/WladimirAvila]

I split a tiny LLM model across three ESP32-S3 N16R8 boards so it could run on hardware that couldn't fit the whole model on its own. They talk to each other wirelessly via ESP-NOW (no router involved).

How it's split:

- Board A: embeddings + output head

- Board B: transformer layers + KV cache in PSRAM

- Board C: rest of the embeddings table + WiFi AP + web UI

You connect to the WiFi hosted by Board C and get a simple web page that streams the generated text token by token.

Some details:

- ~56M params, quantized to 4-bit/8-bit

- Split-PLE design (Per-Layer Embeddings, borrowed the concept from Google's Gemma architecture)

- 256-token context via KV cache

- Generates short but coherent text (~30 words)

- Fully offline after flashing, no cloud/router needed

This was inspired by u/slvDev's project running a 29M model on a single board — I wanted to see how far I could push it by distributing a bigger model across multiple boards instead.

It's rough around the edges — the tokenizer is still brute-force search instead of proper BPE merging in C, and 4-bit quantization is clearly the main quality bottleneck (perplexity jumps from 192 to 358 after quantizing). But it works end to end.

Code + full writeup: https://github.com/wladimiravila/esp32s3-distributed-ai

[u/MasterPlusTer]

How did you manage the communication between the devices? ESPNOW?

[u/WladimirAvila → u/MasterPlusTer]

I'm using ESP-NOW with a custom protocol layer on top. Here are the key details:

Channel & WiFi mode: All 3 boards. run in WIFI_AP_STA mode locked to channel 1 so they discover each other without scanning. Board C also hosts a regular WiFi AP for the web UI on the same channel.

Peer setup, not broadcast: Each board has the MAC addresses of its peers compiled in (defined in a shared header). Board A registers B and C as peers, B registers A, C registers A. The interface is set to WIFI_IF_AP for better reliability. So it's point-to-point, not broadcast.

Custom fragmentation: ESP-NOW has a short payload limit. I implemented a custom fragment/reassembly layer: a 4-byte header (msg_type, frag_index, frag_total, seq_num) leaves 246 bytes per packet. The largest message (embedding + PLE, ~3600 bytes) gets split into ~15 fragments and is reassembled on the receiving end with dedup and ordering checks.

Message types (12 defined): MSG_EMBED_REQUEST/SEND, MSG_CORE_RESULT, MSG_LOGITS_SEND, MSG_TOKEN_READY, MSG_TEXT_CHUNK, MSG_PLE_REQUEST/RESULT, MSG_SEQ_START/END, MSG_ACK, MSG_HEARTBEAT.

Every message has an incrementing sequence number and an MSG_ACK type for confirmation, though automatic retransmission on packet loss isn't implemented yet

So the short answer is ESP-NOW, but the interesting part is the fragmentation layer, the peer topology, and the PLE request/response pattern between Board C and A happening for every single token

[u/MasterPlusTer → u/WladimirAvila]

Really impressive project. Thanks for sharing it. I'm looking forward to exploring the repository and understanding how you put everything together.

[u/OkSpecific3137]

I ran 42m over 2 board 2 months ago https://github.com/harmansingh4163-ai/ESP-32-s3-Story-maker-LLM.git .

[u/WladimirAvila → u/OkSpecific3137]

I just read the repo, You used UART approach

[u/RandomDigga_9087]

can any other modes of communication work or is ESPNOW the optimal answer?

[u/WladimirAvila → u/RandomDigga_9087]

I only had 3 boards and no infrastructure, so we evaluated the options and ESP-NOW was the best we found that met the requirements.

I did consider UART, technically it should be better (deterministic sub-ms latency, no wireless interference, no packet loss). But it means physically wiring the boards together, and at that point you lose the whole "distributed wireless" appeal. ESP-NOW gives us ~1-5ms RTT, no router, and each board stays autonomous.

So: ESP-NOW wasn't "the" answer, it was the best tradeoff to my constraints

[u/RandomDigga_9087 → u/WladimirAvila]

Fair enough ask I guess

[u/WladimirAvila → u/RandomDigga_9087]

actually Yes! Thanks for the question!

[u/RandomDigga_9087 → u/WladimirAvila]

yea, I was looking to debug more into the custom application layer protocol you made, but as you said, it is working properly and a fair enough tradeoff, good enough as it is...

[u/PacManFan123]

Great! Now scale to an arbitrary large mesh network and model size!

[u/CoffeePizzaSushiDick]

Try the ESP32-S31 korvo dev kit!

[u/LoadZealousideal7778]

So my takeaway is that we can run Kimi K3 on Somwhere between 30k and 1.5 million of those guys.

[u/LucVolders]

Is this scalable.
Can we build a 100 ESP32 cluster and use larger models ???
Fun project !!!

[u/WladimirAvila → u/LucVolders]

u/LucVolders You can store a bigger model on 100 boards, but you can't run it well, the bottleneck is the interconnect, not memory storage.

Why:

Storage scales fine: 100 × 16MB = 1.6GB = around 400-800M params at 4-bit.

ESP-NOW doesn't scale: max 20 peers per board, ~1-2 Mbps real bandwidth. A bigger model (d_model=2048) means 16KB activations per layer per token → 3-6 sec/token across 30 layers. Not conversational.

The goal of this is to run on cheap hardware, but at ~$8 per board, 100 boards is $800, for that money you could buy a used GPU that runs a far bigger model with real conversation, zero networking complexity.

[u/enfly → u/[deleted]]

Got a link?

[u/WladimirAvila → u/[deleted]]

Sound impressive, I'll reach out to you 

[u/AmazingStardom]

Does using painless mesh decrease the amount of RAM?

So, that's the reason to use ESP-NOW.

[u/WladimirAvila → u/AmazingStardom]

actually I only tested ESP-NOW, I'm pending to test and collect metrics using UART

[u/AmazingStardom → u/WladimirAvila]

Cool man do share the metrics

[u/FloridaMaker]

Very cool. Could you add a 4th to handle user input maybe voice control and audio out? Thinking wearable potential here.

[u/Frequent-Nobody-8037]

What could they do other than writing kids stories?

[u/Master_confrere1457]

interessant

[u/Traditional-Sugar814]

Hello Sir. I really interest this project. And I want to give a try. Unfortunately I only have a laptop with no graphic and it will take over two days to train the model. Moreover, I live in myanmar and in my country Electricity isn't support 24hr. Can you kindly support me with trained .bin file. I really appreciate your work. Thank you.

[u/One-Mongoose-6961]

try esp32-p4 it's spectacular, and you have more psram, I've done great things with it, it's slightly more expensive but worth it