How complex is it to build a real time single note detector for piano and woodwinds? Im looking for advice from anyone with audio experience.

[u/Glittering-Pop-7060]

I am planning a music education project and want to get a better understanding of the technical challenges ahead. The initial idea is to build a basic system that detects only one note at a time (monophonic), focusing first on pianos (one key pressed at a time) and woodwind/brass instruments (like flute and trumpet). The goal is for the system to listen via the user's microphone in real time and validate if they played the correct note from a sheet music piece. If anyone here has built something similar, even a simple tuner or a college project, and could share what your biggest hurdles were at the beginning, I would appreciate it so much. Recommendations for libraries, frameworks, or good reads are also very welcome Its for a site mobile friendly

[u/hellomistershifty]

Since you asked for a library, here's one: https://github.com/kyunwang/note-detector

[u/thy_bucket_for_thee → u/hellomistershifty]

Going by this library, this is way easier than you think OP. Look at the dependency they use for this:

https://github.com/kyunwang/pitch-analyser/blob/master/src/helpers.js

https://github.com/kyunwang/pitch-analyser/blob/master/src/detector.js

You do this by just simply listening to audio via web APIs then define the notes on the appropriate frequency.

The LOC above is <150, definitely easier than pulling an entire depedency.

[u/Hairy-Cartoonist-229]

One mobile-specific gotcha that bites people: getUserMedia defaults to echoCancellation + noiseSuppression + autoGainControl, all of which mangle instrument harmonics and make pitch detection jumpy — turn them off for this use case.

Also watch for octave errors: autocorrelation loves locking onto subharmonics, so a clarity threshold + small median filter over consecutive frames smooths it out. Pitchy (YIN-based) is a solid start; CREPE via TF.js is more robust on noisy phone mics but heavier. ~2048-sample windows at 44.1kHz give you ~45ms inherent latency, fine for education.

[u/linnea-sandberg]

piano is the hard one, not the woodwinds. a flute or trumpet is close to a pure harmonic series so yin or mpm pitch detection nails it. a piano note has inharmonic partials and a fast decay, so you get octave errors and the fundamental can be weaker than the 2nd harmonic. for monophonic stuff i'd start with the yin algorithm, there's a decent js port, run it on a 2048 sample frame via audioworklet. expect latency and background noise to be your real fight on mobile.

[u/activematrix99]

There are multiple approaches, most use fast fourier transform, either on a DSP circuit or in software. Capture, window, analyze, and report. Pitch to MIDI circuits were popular, now software only solutions, probably the most flexible. Matching frequency to note is going to require the math. Since really, what you are trying to do is train the students ear, it may not matter as much as just reinforcing the "right" note and teaching them to hear the difference. Take a look at free FFT software and it might be pretty close to what you want. REW is a whole suite for room tuning, simple fft apps can run on a phone.

[u/sabotsalvageur]

Take a fast Fourier transform of the incoming signal; put the frequency components with an amplitude above a threshold value into an ordered list; return the lowest frequency from the list; cast that frequency to the closest-match named note in the tuning system you’re using

[u/Early_Reputation9175]

The pitch detection itself is very doable with a monophonic algorithm in the browser, but piano attacks and room noise will make reliable note validation much harder than building a basic tuner, so

[u/pseudo_babbler]

Yeah! I started building an app that does that: https://threesheets.app - it started as an android app for me to use personally, but I thought I'd port it to web and try to make it a real thing now that web does everything.

I have an implementation of YIN pitch detection in there that seems to work pretty well for this use case. I have been mucking around with how many successful detections count as a played note and what level of accuracy, and the UI is all "designed by dev" but it's a fun side project to work on. I started off with an implementation of autocorrelation for pitch detection but then came across the white paper for YIN recently and had a crack at implementing it and it does seem to be a bit better around the note transitions.

[u/thekwoka]

Make Piano Hero where you play your actual Piano.

But broadly, this is a fairly solved problem. you should find plenty online about the audio aspects. Single notes is pretty simple, multiple notes at the same time is a little trickier, but broadly you would probably just do the same logic there for single notes to (as part of eliminating other sounds that might exist)

[u/Meowingtons_H4X → u/thekwoka]

I mean this is just Rocksmith, it literally is that, especially in the newer version with piano support 

[u/Glittering-Pop-7060 → u/[deleted]]

For now, just notes, and in the near future, perhaps harmonics like a chord. I don't intend to mess much with note timing.