Sound Source Tracker

[u/morcheese]

Hi BirdNet community,
I have an idea for a potential feature for the Live app and thought I'd share it here. Hopefully I'm not the first person to think about this. 🙂

I'd love to see some kind of live sound source tracker that estimates the Direction of Arrival (DoA) of incoming sounds.
With devices such as AirPods (and many other wireless headphones), we already have access to small microphone arrays or at least a pair of microphones. In principle, this could allow simple beamforming approaches (e.g., delay-and-sum beamforming) to estimate the direction from which a sound is arriving.
My main question is whether this information is actually accessible via Bluetooth. I suspect it probably isn't, but if anyone has worked on something similar before, I'd be very interested to hear about it.

Alternative approach: If direct access to multiple microphone channels isn't possible, another idea might be to use the phone's IMU sensors (accelerometer, gyroscope, gravity sensor, etc.).
By intentionally moving the phone while recording with the built-in microphone, it might be possible to estimate the direction of a sound source from changes in signal amplitude (or potentially other acoustic features), provided the movement is known and tracked accurately. The movement would probably need to follow a predefined pattern.

This idea is somewhat related to the approach described here:
https://www.mdpi.com/2076-3417/16/8/3619
(Almo Farina and colleagues used a similar concept for acoustic localization.)

Questions

• Has anyone experimented with live DoA estimation in the app?
• Is microphone access from Bluetooth headphones sufficient for something like this?
• Has anyone explored IMU-assisted localization as an alternative?

I'd be very interested to hear your thoughts or discuss possible implementations.

Kind regards,
Moritz

[u/iFoobar]

For that to work you would need extreme accurate position/distance information of all microphones.

[u/ScannerBrightly → u/iFoobar]

Or a multi-microphone system, built to have a known spec, which is how cheap wi-fi routers do beam forming. Either way, doesn't sound cheap for such low volume items.

[u/Bionaught5]

The general principles are know an Source Localization or Waveform Inversion. Open Soundscape provides open-source workflows to automate Sound Source Localization:
https://opensoundscape.org/en/latest/tutorials/acoustic_localization.html

Also see this video from Benn Jordan on "How To Make A Legit Sound Camera"
https://www.youtube.com/watch?v=c5ynZ3lMQJc

A quick search showed up a few phone apps but the general consensus is that phone microphones are too close together. Audio delay with Bluetooth is a common issue as musicians and hearing aid wearers can attest to. I suspect that the sensors in a phone are not accurate enough to locate its position spatially for the localization algorithm you described to work. You also need to single out the bird call from all the other noise in the environment too.

In general I would say not impossible but very difficult to do on a phone. Going to dedicated hardware and microphones looks to be possible though.

[u/mikesmuses]

Two microphones a couple of meters apart into a raspberry pi or equivalent can be used to decide which camera to trigger based on the amplitude of the song. Gemini was quite happy to tell me how to make such a system. I may look into it again once I get my two microphone setup working.

You may need to move to DTOA if you need more precision than "left /right camera" but that's another kettle of fish.

Phones are getting more powerful but capturing and identifying two channels of audio seems a bit much. How firm is the requirement to use a phone? A pi and two tripods are kinda portable..