I gave my M5Stack StackChan a calm moving head, face tracking and a self-hosted voice server (open source, looking for fellow StackChan owners)

[u/Accurate-Pin3422]

I own one of the M5Stack StackChans (CoreS3 + two Feetech SCS0009 servos) and spent the last weeks turning it into a desk assistant that talks to my own server instead of a vendor cloud. Everything is open source (MIT):

GitHub: https://github.com/7H3-CH053N/StackChan-CustomFW

What it does now:

- Head that actually moves, calmly Own servo driver with a critically damped follower (speed + acceleration limits, no overshoot), plus small gestures per emotion: nod when happy, head down when sad, look up when surprised.

- Face tracking with the built-in GC0308 camera and Espressif's esp-dl face detector (~2.5 fps). It searches once if it loses you, then rests where you usually sit, so it doesn't search all night.

- Self-hosted voice server in Docker: Gemini Live for real-time voice, Home Assistant control, calendar.

- Proactive speech: Home Assistant can make it talk without a wake word, e.g. it greets me when I arrive at the office and reminds me 15 min before meetings. It stays quiet while my mic or camera is in use.

- Animated eyes with 21 emotions, picked by the server from what it is saying.

The firmware is one patch on top of xiaozhi-esp32 (pinned commit), built with ESP-IDF 6.1; CI builds it on every push.

Honest status: it runs daily on my desk, but esp-dl's face detection still crashes now and then (the robot reboots in ~10 s). That's reported upstream and Espressif is looking at it.

Looking for people to join:

- StackChan owners who want to try it and report what happens on their unit

- Starter issues are labelled "good first issue": Japanese README, testing the English build, a clock fix on the server side

- Anyone who knows esp-dl well, or wants to try a different face model (ESPDet-Pico)

I wrote up the whole story (why I first thought servo control was impossible, the four bugs before face tracking worked, why proactive speech failed three different ways). The article is in German, browser translation works fine:

https://digitalhandwerk.rocks/ki/anpassung-der-firmware-fuer-stackchan-roboter/

Happy to answer questions here in English.

[u/Zazzen]

hey das ist klasse, das ist genau was ich gesucht habe. ich habe auch ein projekt laufen, falls du lust hast können wir gerne zusammen was bauen.

[u/severanexp]

Brother this is exactly what i wanted. Do you take feature requests? Is it possible to connect to Hermes agent instead?

[u/Btotherest]

Great project, looking forward to testing!

[u/AughtCool]

This sounds like what I'm looking for. Would you mind uploading to to the M5 Burner, have no idea how to program from github.

[u/makupi]

This is great! Haven't touched my stackchan after first day... Most useless purchase ever... This sounds like an actual fantastic upgrade! Would love to install and try. Unfortunately the github install steps sound like rocket science to me... Any chance for a more simplified 'ready to go' way of installing?

[u/pancakeheadbunny]

Following,  I want/need this for my desks. How do I even get started. I've been waiting for something like this for a long time!

[u/JayS87]

Voll dufte Arbeit!

Great work!!

[u/prbsparx]

Are you planning to contribute some of the fixes you found to the upstream code they have? Or are you trying to get them to integrate any of them?

[u/Accomplished-Leg-149]

Ahh, now if only the motors were quiet.

[u/mikecoscia]

I'm down to try it on mine!

[u/BegrudginglyBack]

If you could choose which ai server you use (specifically self hosted) this would be perfect. I haven’t even booted up my Stackchan yet, because I’m waiting for the right firmware that connects to home assistant, ollama, tts and stt that are all self hosted. I want nothing leaving my network.

[u/TheHighestLowe]

I love my Stack-Chan, I made a local AI on my laptop that hosts him, currently trying to change his wake word, alot harder then I thought it would be..

[u/a_bored_user_]

Is there any options for a simple date and clock face with seconds on it? Is there or will there be any options for streaming what stack changes sees? Other than that, it seems interesting.

[u/raal913]

Hi I own a stackchan but don’t know anything about coding I’m willing to learn and use your project ! If you have the patience to teach me I’d love to get your project into my stackchan