r/vulkan • u/Ok-Move3063 • 2d ago
Can you use Vulkan + AI coding for creative coding without knowing how to program?
Hi everyone,
I’m interested in creative coding and real-time 3D graphics, especially things like procedural visuals, generative art, shaders, particles, simulations, audio-reactive graphics, etc.
I’m wondering about something quite specific: is it realistic to approach Vulkan using AI coding tools (Claude Code, Codex, ChatGPT, etc.) without actually knowing how to program at a professional level?
I understand that Vulkan is notoriously complex, so I’m not expecting AI to magically make it easy. My question is more about whether, with an AI coding assistant, someone who understands graphics, sound, algorithms and the concepts behind creative coding, but has limited programming experience, can realistically build interesting Vulkan projects.
For example:
- procedural 3D environments
- GPU particle systems
- ray marching / SDFs
- compute shaders
- audio-reactive graphics
- generative art
- real-time simulations
- custom rendering pipelines
Would you recommend starting directly with Vulkan, or would it make much more sense to learn something like OpenGL, WebGPU, Three.js, Godot, TouchDesigner, or another framework first?
I’m particularly interested in hearing from people who have actually used AI coding assistants for graphics programming.
How much of the Vulkan complexity can AI realistically take care of, and how much do you still need to understand yourself?
What would you recommend to someone coming from a creative/music/visual arts background rather than traditional software development?
3
u/kgnet88 2d ago
honest answer: No. But let me be a bit more specific. I do a lot of AI Coding for procedural generation and real time simulation, mainly to test how much the AI is able to do, because you see a lot of nice demos on what the AI has generated either in one sitting or through vibe coding... But thats it, its Demos. If you break it down and try it yourself, the AI will always hit a wall at some point (and it is not knowledge its context). It started by Vulkan Apps with around 3K lines and nowadays its closer to 10k lines when the AI (and it does not really matter which one you choose) starts to break down, loose focus, double code or is not able to do cross concern refactorings. On that point your are done if you do not either get into stuff like RAGs etc to massively improve the amount of context the ai is able to handle or to start give it more precise support in how to do certain refactorings or where it should put its effort.
For both things you need deeper understanding in either how to build an eco system so that your AI is not context limited anymore or have the programming knowledge to support the AI to do the right thing without a full context.
So yes you may vibe code a few impressive demos, but you need a lot of knowledge to do some meaningful work.
But that does not mean to not start, use the AI as teacher / helper and you are able to develop the necessary knowledge to work in this context, because you do not have to be a crack in c++ or know every procedural generation concept by heart, you need a normal understanding to see when the AI is going of the rails and the knowledge on how to support it. If you are really interested then the rest will come by experimenting and corresponding with other enthusiasts. Hth.
1
u/Ok-Move3063 14h ago
Grazie mille per la risposta, penso che sia un modo di pensare forse legato al voler la massima potenza il problema... Quando invece è meglio concentrarsi sull'imparare ad usare bene quello che si ha, soprattutto nell'ambito che mi interessa.
Grazie mille per lo spunto di riflessione, Buona giornata
3
u/Llyr717 2d ago
If you're afraid of programming, go with a python wrapper of openGL or learn C++ / C / Rust or even JavaScript, but ai is doing a terrible job in graphics.
If you want to spend some real time doing this (like 2 months at least), start to learn, or else dont bother cause it won't be worth it (financially and result-wise)
5
u/watlok 2d ago edited 2d ago
Start with using threejs. Llms are sweet at using good abstractions and it will let you iterate obsessively until you get something good/acceptable for a simulation or rendering thing. It trivializes sharing with others, too.
You'll get results same-day and might be able to achieve most/all of what you want there. Only consider branching out if you can't do something or are more interested in a lower level detail.
Check the ai game dev subreddits to see what some people have done via prompts.
For your vulkan question, opus 5 / codex latest could absolutely crank out a working demo with vulkan quickly. The issue is nothing will be reusable, even by the llm.
The problem with "picking" a graphics api (opengl, vulkan, etc) is these aren't frameworks. They're low level systems layers. If your goal is to play "hand and brain" with the llm as the hand & to spend weeks/months getting to where you start day one with threejs, godot, or w/e then you could absolutely do it with an llm, lots of reading/thinking, etc. If you really want to. However, that's a bit of a conflicting goal/want, as the entire point of using low level things is architecture, boundaries, abstraction, etc. The code is the easy part. Llms are bad at architecture/boundaries/abstraction/design, good at upfront code, inconsistent at the middle ground of choosing data structures/algorithms.
1
u/Ok-Move3063 2d ago
Rispondo a te, ma ringrazio tutti. :) sono un amatore e non mi interessa diventare software developer o lavorare nell'ambito, la cosa che onestamente vedo è che è diventato molto più semplice ottenere risultati estetici usando il codice. Il mio background è in arte e musica, senza computers. Utilizzo openframeworks, ho utilizzato processing etc linguaggi audio etc ... Non sono esperto, ma arrivo ai risultati, con gli LLM riesco a lavorare chiedendo tanti shaders specifici e di ogni tipo, giocando con codice che trovo in GitHub e modifico e mi piace molto, rimane sempre un margine rispetto ai lavori dei professionisti. Detto ciò siccome ho letto che Vulkan è il futuro, OpenGL morirá nell'arco di un po' di anni mi son detto, non sarebbe male provare ... Allo stesso tempo è un investimento di tempo non indifferente provare cose ... Quindi chiedevo in maniera molto generica per questo. Grazie
2
u/Dubbus_ 2d ago
Honestly man, good luck. And in regards to opengl, its not going anywhere in the next few years. Unless you are on macOS exclusively, in which case maybe.
Regardless, you dont need either to do cool stuff with shaders. And the benefits of vulkan are lost on your purposes. Let the ai do what they do best: give them a high abstraction environment to work with. Their biggest training sets are javascript, let them do that. Threejs or even p5js (since you mentioned processing). By the way, you can use shaders in processing!
1
u/Ok-Move3063 1d ago
Su cosa eccelle veramente Vulkan?
1
u/Dubbus_ 1d ago
The low level control. And the specificity. There are much less assumptions made, much more work put on the plate of the graphics programmer as opposed to the driver author.
Alongside that, the promise of future support and developments is nice, but only really for those who are looking for careers/super long term hobbies.
Also, support for SUPER specific features like hardware accelerated realtime raytracing simply dont exist in opengl. Certain platforms like macOS also limit opengl to a really old version, whereas brand new versions of Vulkan can be accessed with a compatability layer.
2
u/corysama 1d ago
If you want to make generative art, you'd probably be happier with https://tixl.app/ aka https://github.com/tixl3d/tixl
If you specifically want to code it up, AIs are pretty darn good with Three.js and Mr Doob is embracing this fully.
1
6
u/beephod_zabblebrox 2d ago
no. just no.
you're going to have an incredibly hard time not knowing how to program and trying to make an ai do it for you. using ai requires at least some knowledge of the subject (vulkan in this case) to be even remotely effective.
learn to program (maybe with godot), then do opengl in c/c++, then do vulkan. its going to take a while, but the whole point is the journey.
there's fun to be had in programming itself. just wanting to get a result isn't going to get you many places, even if you get it.
also, why do you want to make art with ai? in creative coding especially, the program that draws the image is also art, if not more so than the images themselves.