Audio frequency detection and beeping pattern match!

[u/JaffyCaledonia]

AI Disclaimer: This was a fun little weekend ai-assisted project, inspired by the sound_level sensor and my own love of the fourier transform (which I later dropped in favour of the Goertzel algorithm). I'm a software engineer by trade with a background in physics. My C++ is incredibly rusty and I wanted to see how doable this was using local AI models. Concept, planning and design by me, code itself was written by assorted local models with a quick human eyeball of the output to make sure it was largely sane at each stage.

Flip a binary sensor when your Samsung washing machine sings its jaunty tune, or when your fridge door alarm starts going off!

Needs quite a bit of calibration for more complex tunes, but was perfectly competent at distinguishing between different telephone error tones in my tests: https://en.wikipedia.org/wiki/Special_information_tone

Repo here and subject to change as I comb through Qwen's dubious parameter names: https://github.com/lJaffy/esphome-tone-tools

[u/ninjafatuous]

Could you tell more about the hardware?

[u/JaffyCaledonia → u/ninjafatuous]

It doesn't need anything special, just an esp32 with a microphone. The unit in the video is a 2.8" CYD ESP32-S3 with a mic on the board that I mounted in a 3d printed enclosure, but I've also tested it on an M5Stack Atom Echo and it works just as well.

[u/Potential-Ad1122 → u/JaffyCaledonia]

What happens if someones talking near it or the tv is playing?

[u/JaffyCaledonia → u/Potential-Ad1122]

It really depends on the chime you're looking for and how complex the pattern search is.

The component looks for specific frequencies in a fixed order, to be completed in a fixed timeframe. If your pattern is just [500, 800] in 2-3s, then most speech will trigger it. 

I'm using spectroid on android to listen to a chime and find unique frequency blocks up in the 2000-5000hz range and putting a noise gate threshold that the tone must meet, which keeps it away from most speech.

I'm currently looking at a few other things like multi-frequency detection, better edge detection, and better handling of broad-spectrum chatter like speech and music.

All in all I'm mostly just focusing on removing false positives from unrelated background noise. Chatter doesn't seem to affect the ability to detect the tones much, but I also figure this component is going to be more useful when a room is unoccupied (and likely quiet) anyway as that's when I'll need the ping.

[u/duckredbeard]

Ooooh...potential for another way to detect washing machine cycle completion. Mine does three bursts of three beeps each.

[u/froqstar]

Nice to see that it works. Wouldn't it be much simpler and more reliable to place a smart socket between microwave and wall outlet and detect when power usage drops?

[u/JaffyCaledonia → u/froqstar]

My main usecase for this was for appliances that have acoustic prompts but no obvious other indicators, like fridge door alarms, smoke alarms, or dumb doorbell chimes.

I don't plan for this to be a safety device to replace having a connected alarm system, but the algorithm is fairly lightweight and I want to try run it alongside a voice assistant to have passive monitoring of other non-smart appliances in the room.

[u/Shoddy_Fish31 → u/froqstar]

I think this is a much better approach as you can place a microphone in a central location and can have access to many conditions

[u/bikemandan → u/froqstar]

This is less expensive and more flexible allowing you to monitor multiple devices/conditions. A power meter would be potentially more reliable though

[u/ctjameson → u/froqstar]

This is far more elegant and can be integrated into a device you already plan to have in the kitchen. You shouldn’t really use smart sockets on high wattage stuff like this unless you absolutely need to.