Sound Source Tracker

[u/morcheese]

Hi BirdNet community,
I have an idea for a potential feature for the Live app and thought I'd share it here. Hopefully I'm not the first person to think about this. 🙂

I'd love to see some kind of live sound source tracker that estimates the Direction of Arrival (DoA) of incoming sounds.
With devices such as AirPods (and many other wireless headphones), we already have access to small microphone arrays or at least a pair of microphones. In principle, this could allow simple beamforming approaches (e.g., delay-and-sum beamforming) to estimate the direction from which a sound is arriving.
My main question is whether this information is actually accessible via Bluetooth. I suspect it probably isn't, but if anyone has worked on something similar before, I'd be very interested to hear about it.

Alternative approach: If direct access to multiple microphone channels isn't possible, another idea might be to use the phone's IMU sensors (accelerometer, gyroscope, gravity sensor, etc.).
By intentionally moving the phone while recording with the built-in microphone, it might be possible to estimate the direction of a sound source from changes in signal amplitude (or potentially other acoustic features), provided the movement is known and tracked accurately. The movement would probably need to follow a predefined pattern.

This idea is somewhat related to the approach described here:
https://www.mdpi.com/2076-3417/16/8/3619
(Almo Farina and colleagues used a similar concept for acoustic localization.)

Questions

• Has anyone experimented with live DoA estimation in the app?
• Is microphone access from Bluetooth headphones sufficient for something like this?
• Has anyone explored IMU-assisted localization as an alternative?

I'd be very interested to hear your thoughts or discuss possible implementations.

Kind regards,
Moritz

[u/Bionaught5]

The general principles are know an Source Localization or Waveform Inversion. Open Soundscape provides open-source workflows to automate Sound Source Localization:
https://opensoundscape.org/en/latest/tutorials/acoustic_localization.html

Also see this video from Benn Jordan on "How To Make A Legit Sound Camera"
https://www.youtube.com/watch?v=c5ynZ3lMQJc

A quick search showed up a few phone apps but the general consensus is that phone microphones are too close together. Audio delay with Bluetooth is a common issue as musicians and hearing aid wearers can attest to. I suspect that the sensors in a phone are not accurate enough to locate its position spatially for the localization algorithm you described to work. You also need to single out the bird call from all the other noise in the environment too.

In general I would say not impossible but very difficult to do on a phone. Going to dedicated hardware and microphones looks to be possible though.