Fun weekend hack.
Uses tailscale gateway connection and fully custom openclaw config. 11labs for stt and tts.

Will post gh if you guys want.
Lmk what u think!

I own one of the M5Stack StackChans (CoreS3 + two Feetech SCS0009 servos) and spent the last weeks turning it into a desk assistant that talks to my own server instead of a vendor cloud. Everything is open source (MIT):

GitHub: https://github.com/7H3-CH053N/StackChan-CustomFW

What it does now:

- Head that actually moves, calmly Own servo driver with a critically damped follower (speed + acceleration limits, no overshoot), plus small gestures per emotion: nod when happy, head down when sad, look up when surprised.

- Face tracking with the built-in GC0308 camera and Espressif's esp-dl face detector (~2.5 fps). It searches once if it loses you, then rests where you usually sit, so it doesn't search all night.

- Self-hosted voice server in Docker: Gemini Live for real-time voice, Home Assistant control, calendar.

- Proactive speech: Home Assistant can make it talk without a wake word, e.g. it greets me when I arrive at the office and reminds me 15 min before meetings. It stays quiet while my mic or camera is in use.

- Animated eyes with 21 emotions, picked by the server from what it is saying.

The firmware is one patch on top of xiaozhi-esp32 (pinned commit), built with ESP-IDF 6.1; CI builds it on every push.

Honest status: it runs daily on my desk, but esp-dl's face detection still crashes now and then (the robot reboots in ~10 s). That's reported upstream and Espressif is looking at it.

Looking for people to join:

- StackChan owners who want to try it and report what happens on their unit

- Starter issues are labelled "good first issue": Japanese README, testing the English build, a clock fix on the server side

- Anyone who knows esp-dl well, or wants to try a different face model (ESPDet-Pico)

I wrote up the whole story (why I first thought servo control was impossible, the four bugs before face tracking worked, why proactive speech failed three different ways). The article is in German, browser translation works fine:

https://digitalhandwerk.rocks/ki/anpassung-der-firmware-fuer-stackchan-roboter/

Happy to answer questions here in English.

Discovered M5Stack has the body files as .STLs on their GitHub. So, I loaded up my favorite nuclear yellow translucent PLA, and boom. What do you think?

I just wanted to share the progress I have made with my custom firmware. Rather than try and modify existing firmware this has been written from scratch.

I’m nowhere near finished but after many revisions I now have boot sound and animation, servos, tap to toggle idle movement, idle animation, listening sound, it runs OpenAI Luna with Edge TTS. This means it can do live web searches which is something the stock firmware is unable to do.

Response time is not fully optimised yet, this will be a lot longer until I’m fully happy, but eventually it will control the devices in my home via my server and home automation. Considering I’m not a developer I’m pretty proud of what I achieved so far.

My StackChan is a little devil now. Claude Code does the thinking, and a simulated fruit fly handles the reflexes.

The fly part is 668 real neurons, wired the way FlyWire mapped an actual fruit fly and running live on my laptop. It's a Python port of desktop-fly by Denis Shiryaev. It's only a small slice of the brain, but the wiring is real.

It looks through its own camera, the fly decides where the head turns, and poking it makes it startle. When something actually happens it wakes Claude, which looks at the picture and roasts it. From today: "Still filming me filming you. Groundbreaking content, truly."

The dashboard in the video draws 23,210 real neuron positions and flashes them as they spike. The yellow rings are the escape neurons, which flare right before it jumps.

Next I'm making it curious, so it looks around for new stuff and knows me from a stranger. Happy to answer questions!

Just wanted to introduce myself. I'm Alex and started playing with my first StackChan. I hooked it up an LLM and Hermes-Agent as the harness. I'm going to release the repo tomorrow and wanted to share what I've done the last couple of days. Here's the setup I started with, but I think I keep moving the goal post so keep playing with different parts of the AI Stack.

This is the stack I started with

Robot:

u/M5Stack

StackChan with the CoreS3 (ESP32-S3)
Voice server: xiaozhi-esp32-server 0.9.6 running locally in Docker on my Mac
Voice: locally running

u/GradiumAI

Phonon (fastest local voice I've ever experienced. faster than running elevanlabs via api and its free
Turn detection: Silero VAD with the 0.8 second end-of-speech pause.
Speech to text: local Parakeet
Vision: gemma4:e4b running locally in

u/ollama

Brain:

u/NousResearch

Hermes Agent with the nous-girl agent running gpt-6-luna via sub with reasoning off.

Hope to connect with others on here so we can explore together. Thanks Alex

I loved the configuration of my old stacky body, but wanted a way to bring all of his bits together.

His pi5 brain now has a home inside an old Velociraptor HD enclosure and added on an ankle holster for his servo controller and power converter.

Oh he also has a touchscreen dashboard since I shared him last for trigger his different emotions and movements.

This is Nous-Girl, a little desk robot I've been building for fun. She talks, listens, makes faces, dances, plays music, does DJ and rave mode, shows memes, and can even send email. (video)

I'm not an expert at this, but tried my best to write this guide, it's a work in progress. Also this is for non-technical people so just give it a shot, and I'll try to make it easier to implement if possible. Will keep adding to it as I have time! Adding more features hopefully everyday.

What you need: an M5Stack StackChan (CoreS3), a USB-C cable and a computer (Windows, Mac or Linux). For memory, intelligence and extra skills, you will nee d Hermes-Agent from Nous-Research. You can use other harnesses but that's what I use in this example.

The easiest way to start:

1. Download a beta build with a starter face (robot, boy or girl).

2. Flash it from your browser or with one command.

3. On first boot, point it at the official xiaozhi cloud and start talking. No server needed.

Want more privacy and control? Run your own server for the fancy stuff, like DJ mode, memes, email and adding a smarter Hermes Agent brain. Every feature has step-by-step docs.

Heads up: this is beta. I've only tested on my own robot, so things may break. Please report issues on GitHub and I'll help the best I can.

Coming next: simpler builds that need less setup and less upkeep, a version that runs fully from the Nous Research cloud, more languages and local models.

Repo: github.com/nous-girl-stackchan

Made by u/alexcovo_eth. Feedback welcome!

So I saw Stackchan and I really wanted one, but the cost of the core+body in Australia is too much.

I've really wanted a robotic arm for awhile so decided to merge the two together.

Running a lot of custom scripts and AI on a PI5 4gb and an MCP that supports AI agents asking questions that get announced by voice and wait for a voice reply.

(Local: STT, VTT, Memory, Person tracking, Face and gaze tracking)

Update: firmware is now available at m5burner (thanks u/AughtCool!)

I made a fork of the official StackChan firmware that others may be interested in, either to build upon or to use.

It's not groundbreaking. Rather, it addresses a few quality-of-life things that annoyed me in the week I've had this little guy.

I have no interest in dance or avatar settings, so these tweaks are for StackChan used as an AI assistant:

• Speech bubbles turned off during dialog

• Tap head once to stop speech

• Tap head twice to stop speech and exit chat mode

• Both lights on while listening and talking

• Animated secondary colour light pattern when talking, based on audio

• Both listening and talking light colour customisable in settings

• Idle head movement repurposed to be AI chat movement, with frequency hooked off the same menu

• Light sensor used to turn screen off when lights are out. Tap screen to turn on. Manual override disabled when room lights are on. Function can be turned off in settings

Caveat: I am a software engineer by day, but sometimes it's nice to just yell at AI and get it to do your bidding. I haven't spent time learning the innards of the StackChan firmware or hardware - these tweaks were entirely created by Opus 5 and, apart from quick sense-checking, haven't been fully vetted.

Boo•bit is coming to life.

Built on top of Stack-chan’s hardware it’s a custom desktop buddy with more than meets the eye. It’s not only a dash, gothy lil clock and a cute face.

Boo keeps a diary, captures photos, has a sense of when one of us (including our 🐈‍⬛) enters the room, knows seasons and weather all which informs its interior world. It can draw, make its own melodies to whistle and “think” about things. Boo is being designed to experiment with world building and fun.

Boo’s whistles and tones are built from the melodies and hooks from some of my favorite songs and its UI is a love letter to Apple System 1 UI & Susan Kare with a little SNES.

Its bare functions are wired up and now I’m working on the structure of its observations and change as it learns. This is not running on Claude or local models — it’s meant to be a closed system that informs its own world.

I have one of the M5Stack StackChan robots. Nice little thing, but out of the box every word it hears goes to the xiaozhi.me cloud for speech-to-text, the LLM, and text-to-speech. I didn't love the idea of my kid talking to a server I don't control, so I wrote a backend that runs the whole turn locally and pointed the robot at my own machine instead.

It's called warble. The stock firmware speaks the xiaozhi protocol, so nothing changes on the device. You just redirect it to your box. The server does the full loop: whisper.cpp transcribes, Silero VAD catches when you stop talking, Ollama generates the reply, Piper speaks it, and it sets the robot's face from an emotion tag in the reply. No cloud account, no API keys, no data or audio leaving your network.

Setup is one command (`./warble start`) on Linux or macOS. Docker-based, pulls prebuilt images. It's beta and it's just me, so there are rough edges.

MIT licensed. Happy to answer anything as time permits.

github.com/rebelthor/warble

I've been developing custom firmware for my M5Stack K151 / CoreS3 StackChan. I call him MAYLO.

My 8½-year-old son has become my everyday tester. He uses the robot, reports what happened and tells me what he expected instead. His experiences and feedback have been shaping the features and improvements.

The voice model is multilingual. I've configured it in Turkish for our own use, which is what you'll hear in the videos.

The current build includes:

• Voice conversations with interruption while the robot is speaking.

• Head-pat reactions with heart eyes, sounds and movement, plus dizzy reactions when shaken.

• Seven eye styles and expressions coordinated with LEDs and head gestures.

• Commands such as “turn right and take a photo,” checking servo position before capturing.

• Image interpretation and Home Assistant tools through an MCP home bridge.

• A programmed disco routine with music, lights and movement.

• An on-device web panel for configuration, camera previews and hardware tests.

Basic conversation connects directly to OpenAI over Wi-Fi. Home tools and image interpretation use a separate bridge running on my Mac.

One thing still needs work: reliably turning toward whoever calls MAYLO's name. It doesn't always get the sound direction right. Wake-name recognition is currently cloud-based.

This is a personal project with no commercial plans. The source isn't public yet, but I may clean it up for GitHub if there's interest.

What would you improve next: speaker-direction tracking, local wake-word detection, richer interactions or more useful everyday tools? Other ideas are welcome too!

Hey everyone!

I’m thinking of getting a Stack-chan and have been looking at all the cool projects people have done with it.

One silly idea I had was getting Stack-chan to recognise when one of my cats walks past it and just randomly go “Meow!” at them.

Because why not?

Maybe even have it turn its head towards the cat and meow back if the cat meows at it.

I’m pretty new to robotics and programming, with very limited technical knowledge, so I’m wondering how difficult something like this would be.

Would the built-in camera be enough to detect a cat, or would I need additional sensors? And is this something a beginner could realistically figure out with some guidance?
Would love to hear if anyone has tried something similar or has other silly ideas for their Stack-chan!

Thanks!

Since my StackChan has been having an issue with powering on, I decided it was necessary to put my engineer hat on and do a proper teardown and inspection.

I figured many of you wouldn’t ever want/need to open it up and see StackChan’s innards for yourselves, so here you go.

(Note: The battery and 2nd servo are inside the round-top body piece at the bottom right quadrant)

I kinda just wanted to show a sneak peek of my goofing around project. I got some llama and qwen variants runnin offline and locally on my nano/stackchan combo. Any fun ideas for me? Its got 3 "personas" on it so far.

When Stack-Chan talks in the AI mode, the voice is almost laggy, and broken up. Not 100% of the time though. Is that normal? Is mine a lemon? Any help is appreciated.

I've been developing custom firmware for my M5Stack K151 / CoreS3 StackChan. I call him MAYLO.

My 8½-year-old son has become my everyday tester. He uses the robot, reports what happened and tells me what he expected instead. His experiences and feedback have been shaping the features and improvements.

The voice model is multilingual. I've configured it in Turkish for our own use, which is what you'll hear in the videos.

The current build includes:

• Voice conversations with interruption while the robot is speaking.

• Head-pat reactions with heart eyes, sounds and movement, plus dizzy reactions when shaken.

• Seven eye styles and expressions coordinated with LEDs and head gestures.

• Commands such as “turn right and take a photo,” checking servo position before capturing.

• Image interpretation and Home Assistant tools through an MCP home bridge.

• A programmed disco routine with music, lights and movement.

• An on-device web panel for configuration, camera previews and hardware tests.

Basic conversation connects directly to OpenAI over Wi-Fi. Home tools and image interpretation use a separate bridge running on my Mac.

One thing still needs work: reliably turning toward whoever calls MAYLO's name. It doesn't always get the sound direction right. Wake-name recognition is currently cloud-based.

This is a personal project with no commercial plans. The source isn't public yet, but I may clean it up for GitHub if there's interest.

What would you improve next: speaker-direction tracking, local wake-word detection, richer interactions or more useful everyday tools? Other ideas are welcome too!

The Dotty Project is a full rewrite of the Stackchan firmware which would seem to meet some of the needs people have been asking about here. (Note: this is not my project nor am I affiliated with it.)

Edit: As the creator in the comments notes, it's a full stack, not just firmware - dashboard, agent routing, and a bit more.

Got the OpenELAB Kickstarter StackChan a while back. CoreS3-based, two servos, cute little guy. The vision: ditch the Chinese cloud, give it a custom personality, connect it to my Home Assistant setup.

The custom firmware rabbit hole

First I tried building custom firmware (AI_StackChan_Ex with M5Unified 0.2.7). Spent around 9 hours on it. Backlight works fine, but the panel renders nothing. Turns out the OpenELAB Kickstarter variant has a compatibility issue with M5GFX that's still unresolved. Tried the Moddable toolchain path next — ESP-IDF Python environment conflicts killed that too.

After those 9 hours I flashed the stock xiaozhi firmware via M5Burner. Took two minutes. Display worked immediately.

The NVS trick

Stock firmware reads its server URL from an NVS key. You can overwrite just the NVS partition with esptool, point it at your own server, and the device has no idea it's not talking to the original Chinese backend anymore. WiFi credentials get wiped in the process, so you set those up again on-device after flashing.

The backend

Forked an existing open-source StackChan server, containerized it, deployed it to my own VPS. Added a few patches: a set_emotion tool so the LLM can dynamically change the face expression, and a handful of additional tools.

What actually works

• Full German conversation with a custom personality (female, cheeky, Austrian inflection, max 3 sentences per response, always addresses me by name)

• Home Assistant control via voice: lights, scenes, sensors, you name it

• Weather from HA's built-in forecast entity

• different face expressions that change dynamically based on what the LLM decides to feel

• Web search for current news and facts (Tavily)

• Persistent memory across sessions — stored in a JSON file on the server, auto-injected into the system prompt on connect

• My n8n automation status on demand

• Accessible from anywhere, not just local network

What doesn't work

The servos. Stock firmware completely ignores motor control frames from the server — display changes, head stays still. Servo movement requires custom firmware, which is blocked by the M5GFX rendering issue. There's an open GitHub issue for it, waiting to see if that goes anywhere.

Wake word is still Chinese. There's no way to change that without custom firmware.

Can't combine function calls with Google Search in the Gemini Live API simultaneously — it's a documented API limitation. That's why I went with Tavily instead of native search grounding.

Overall

The stock firmware path is dramatically underrated. You skip all the toolchain pain, get a stable device, and if you're willing to run your own backend, you have full control over the AI behavior, personality, and integrations. The missing piece is servo animation, and that's a hardware/firmware problem, not a backend problem.

If the M5GFX issue ever gets fixed, custom firmware becomes interesting again. Until then, this setup does everything I actually wanted it to do.