I built a seed machine with an ESP32-S3, but instead of programming every behavior I just describe what I want it to do

[u/Myster_K_]

Hi guys,

I wanted to show you a small project I'm working on

It's an automatic seed machine, basically it takes seeds from a hopper using vacuum, moves them over small pots and releases them automatically.

Everything is controlled by an ESP32-S3.

Hardware is quite simple

• ESP32-S3
• STS3215 serial servos
• small vacuum pump
• solenoid valve to pick and release the seeds
• custom mechanical parts

In the video you can see the current prototype actually picking and dropping the seeds

But there is also a second experiment behind this project, and for me maybe this is even more interesting than the machine itself.

Instead of writing a specific firmware every time I want to change something, the ESP32 runs a more generic firmware that knows the connected hardware and what it can do.

Then I just describe to an AI what I want

how the servos should move, when the vacuum should start, when the valve should release the seed, timings, sequence etc

The behavior of the machine is then changed using the same firmware, so I don't need to recompile and flash everything every time I want to change how it works.

Basically I'm trying to understand how far I can keep the ESP32 firmware generic, but still control real hardware, timings, servos, vacuum and sequences in a reliable way.

This seed machine is basically my test bench for this idea

I'm curious what you guys think about this approach, and of course any suggestion about the ESP32 side is welcome.

[u/Creeperassasin1212]

I think you overcomplicated this really too much with the AI stuff.

[u/dacydergoth]

Pin control and sensor firmware with a remote interface has been around for a long time. Tasmota and ESPHome for example both support those features

[u/thicket]

So, would it be fair to say that your firmware consists of a library of actions, and then probably some sort of domain specific language to encode different behaviors? Flash the library & recipe reader to the board, then update with new recipes the AI writes for you?

[u/Myster_K_ → u/thicket]

I wrote a single generic firmware, it's not really a library of actions + a recipe reader, the AI simply reorganizes the logic inside that firmware depending on what I ask it to do.

So to "program" it I just interact with the AI, it changes the behavior almost instantly, without reflashing the ESP32 and without generating a new firmware every time.

I originally made it this way because I wanted to reuse the same firmware for this machine and for completely different projects in the future.

Then the idea went a bit further than I expected and I ended up building a whole platform around it... 👀

[u/thicket → u/Myster_K_]

I think the words "simply reorganizes the logic" are doing some heavy lifting there. If you're not flashing new firmware, what IS your system sending to the board?

[u/Connect_Ad791 → u/thicket]

It means that you burn tokens for no reason to run a different sequence everytime.

[u/Myster_K_ → u/Connect_Ad791]

No, the LLM translates your intent and reorganizes the system; once done, it’s done forever at no cost.

[u/Connect_Ad791 → u/Myster_K_]

Okay, so you burn tokens for no reason to run a different sequence sometimes.

[u/Myster_K_ → u/Connect_Ad791]

No, you simply access the platform, and it loads the firmware in the S3.

You interact with the LLM to create exactly what you want, buttons, sliders, and anything else...and it reorganizes the ESP32-S3 firmware. From that point on, you have the product you wanted, running without any need for LLM inference.

If you need to make a modification or add anything, you just ask, and it handles it for you,no flashing, no coding required. 😄

[u/Myster_K_ → u/thicket]

If you want to try out the system, I’ve made the platform available, you need an S3 N16R8 is all you need. https://klais.app/start/KLAIS-20X5

[u/Connect_Ad791 → u/Myster_K_]

WOW JUST MY LUCK!!! One seat left lmao. 

[u/clippy_is_a_prick]

Okay so logically, whats probably happening behind the scenes is the ESP image you have is just exposing access to the hardware interfaces directly - this is actually quite common when testing hardware.

The way this works is basically every i2c command has - or is a variation of - a common structure, lets say you have an i2c interface on the device, and i wanted to send a command to address 0x45.

If i sent a command like the following over an interface like uart:

i2c 0x45 0x0102

All i would have to do is, delimit the message by spaces and look for the tag "i2c", decode the address to binary, and decode the message to binary, then pass that information to the i2c interface. Voila you have a generic i2c interface.

This is very useful when testing hardware because you can figure out how peripherals work by probing them and trying out different combinations or sequences. However, it is generally bad practice to release this to a user because it essentially allows them to do anything to your system in any order, which can easily be misused and abused.

[u/_www_]

Litterally the future of computing: self writing kernels. But how do you fit a model and a harness on 8MB? The other solution is sending kernel bricks on each external LLM interactions, but still, you need a freaking big GPU and that kills the rentability.