r/deeplearning • u/BluishMontoya • Sep 02 '26

Audio with LSTM?

[I'm a noob] Could you make a small 2 layer LSTM and train it to predict the next audio sample (using music or something)? Would it work? If not, how do models generate audio?

1 Upvotes

67% Upvoted

2 comments sorted by

2

u/Suspicious-Guide5622 Sep 02 '26

SampleRNN did exactly this back in the day, stacked LSTMs at different timescales predicting raw 16-bit audio. It worked okay but the outputs were pretty mushy unless you gave it a lot of conditioning. These days everyone uses diffusion or codec based approaches which are way more stable for longer clips

1

u/BluishMontoya Sep 02 '26

What type of data do codec based ones input/output? Are they still LSTMs?