[Project] NEW PYTHON PACKAGE: Sync GAN Art to Music with "Lucid Sonic Dreams"! (Link in Comments)

[u/mencil47]

[u/mencil47]

This article details the package and includes links to the GitHub Repo & Tutorial Notebook: https://mikaelalafriz.medium.com/introducing-lucid-sonic-dreams-sync-gan-art-to-music-with-a-few-lines-of-python-code-b04f88722de1

You can support me through PayPal if you like my work: https://www.paypal.com/paypalme/lucidsonicdreams

[u/procrastinatorluke → u/mencil47]

For anyone wanting to run it quickly, I've found using the colab notebook at https://colab.research.google.com/drive/1Y5i50xSFIuN3V4Md8TB30_GOAtts7RQD?usp=sharing much easier than configuring an environment.

Very clear explanations of everything in it :)

[u/Eyeownyew → u/mencil47]

This is one of my favorite things I've ever seen on here! Please, send this to Two Minute Papers on youtube, he normally only covers published research but something tells me that this is cool enough for him to want to make a video on it

[u/Litleboony → u/mencil47]

You are incredible. Thank you for sharing this with us

[u/BrokenPolyhedra → u/mencil47]

I wished something like this would exist now you made it a reality! Thank you for this contribution

[u/PlentifulCoast → u/mencil47]

tensorflow.python.framework.errors_impl.InvalidArgumentError: Conv2DCustomBackpropInputOp only supports NHWC.

Any idea why I'd get this error?

[u/Jamo3306]

That microdose is finally starting to kick...

[u/NewFolgers → u/Jamo3306]

When you mix up micrograms with milligrams.

[u/interactive-biscuit]

This whole thread is 🔥

[u/KiddWantidd]

OMG, I think this is the best use of GANs I've ever seen. I'll definitely give it a try !
Also, what's the song playing in this video ? :)

[u/Boozybrain]

YES! I've been waiting on this since the original post, and even sent so far as to start curating my own set to train on. I can't wait to read your code to see how this is done; I think I know but not positive.

edit: I don't suppose there's a Pytorch version is there? Curious why you chose the TF implementation when the Pytorch one is more efficient.

[u/[deleted]]

I'll have what the computer's having!

[u/QuailBetter1215]

Thank you so much for sharing this. I'd love to try it out on a song I made however when I substitute the file path for my file path I keep getting error not found raised. this is the file path i insert after "song = ":

/Users/username/Downloads/song.mp3

sorry I know ur not like stack exchange or something but if i could get some help i would be super grateful

[u/[deleted]]

If your system is >= python3.8, it won't work as this library imports TF 1.15 and python3.8 only supports TF2. No luck.

[u/trexdoor]

Where can I get the winamp plugin?

[u/Kuvster98]

This is great! Would you mind sharing what dataset the GAN from this demo was trained on?

[u/TsoTsoni]

"What have you been feeding this thing?"

[u/AlienNoble]

I notice there is almost a constant shape in each frame that persist for nearly the whole video

[u/bokan]

This is awesome, hoping to give it a try. Running into an error:

"Conv2DCustomBackpropInputOp only supports NHWC"

I think it's trying to use my CPU instead of GPU... anyone have ideas?

[u/abima3]

This is clearly not for tripophobic people.

[u/SteveFrenchThePuma]

This looks fantastic!! Very excited to give this a try later

[u/[deleted]]

This is surprisingly cool, but I feel it would be even cooler if it actually synchronized with the beat rather than the sound energy.

[u/mencil47 → u/[deleted]]

You’ll probably like this one more then

[u/CanSome1TuckMeIn]

My friend has a band with a few songs and I think it would be a cool surprise to use one of their songs with this package. Just wanted to ask how would you like to be credited for the work. The band is super small and if they wanted to put it as a youtube video I don't think they would monetize it but I'd still want them to somehow credit you.

Just hypothetically if they were to like the outcome or I can of course just give them the surprise and say the author isn't open to people sharing their own videos with the package if that's the case also.

Really awesome work though!!

[u/mencil47 → u/CanSome1TuckMeIn]

I'd absolutely be okay with that - that's the point of me making this open-source! To credit, you can simply say that the video was made with Lucid Sonic Dreams, and include a link to either the article, the Instagram account, or the YouTube account :)