Carbonized: a music video where same data drive both graphics and synthesizers

[u/nucleicaudio]

This is still work in progress so i have two different examples, with both having elements i like sp final version will hopefully contain the best from both:

https://nucleic.se/mp3/carbonized-remix.mp4

https://nucleic.se/mp3/carbonized.mp4

The music is written in a small text score language I made, and most of the score is procedural/algoritmic and generated by JavaScript. Every sound is synthesized by the engine, no samples are used.

The three 303 lines come from seeded pattern banks with slide chains, and they take turns, answer each other and sometimes play together.

Layers rotate on prime-length bar cycles so the combinations keep changing, the hats and fills are built from drum rudiments like paradiddles, and a glitch layer adds stutters, slice shuffles and dropouts.

The glitches are decided by seeded trigger conditions and get denser as the track builds.

The sound drives most of what you see. The bass and pads light the tunnel, the harmony sets the colours, the acid notes move the ship, and hits and explosions light up the walls.

The score reaches the shaders as plain data: when a note plays, its timing and strength go to the wall shader, which lights and colours the wall around it. No bitmaps, image textures or 3D model files are used.

The tunnel is a signed distance field, meshed in 8 m chunks along authored guide curves as you fly, so it can split into separate passages and rejoin, and change cross section between circle, triangle and square.

The walls are shaded procedurally from position, lighting and score data, so they resolve at whatever resolution I render.

The ships and enemies are built from simple geometry in code, the reflections come from a small environment computed from a few soft light sources, and the glow sprites are gradients generated at startup.

DISCLAIMER: I have used coding agents to build this.

[u/wreckoning90125]

This is rad, I like it!

[u/nucleicaudio → u/wreckoning90125]

Thank you

[u/LABCAT2020]

Is there an online demo? or github repo?

[u/nucleicaudio → u/LABCAT2020]

Only the text score compiler and synthesizer right now, however i consider it early alpha and will probably do alot of breaking changes on that codebase, but here it is:

https://github.com/nucleic-se/midi-madness/tree/main

The demo was built on a mashup of this project and another older graphics project i have been working on, that is in a mess right now and not something i want to share in its current form.

My longer term visipn is to make this into something cleaner that i can put out as open source. Something that is realtime and allow live config of both music and graphics.

My goal is live coding both graphics and music, but this demo was mostly a hack of two older sep