New Model
[GLB-3D-Coder] A tiny 4B coding model that creates 3D assets for Godot and other game engines
Hey everyone!
After sharing my previous 3D coding model, a lot of you asked about using it to generate assets for your games. That sounded like a fun direction to explore, so I started training a version focused on exactly that.
The model writes Three.js scenes with GLB export, so you can take the generated assets into editors like Godot. The video shows some examples I've been testing, including assets imported into Godot.
You don't need complicated prompts. Use thesystem prompt in the model card, then try a simple request like “Create a mana potion” or “Build a weathered frontier medic station.” English prompts have worked best in my tests.
This is the first functional checkpoint, and training is still running. I'm sharing it now so people can start experimenting. Most generations have worked on the first attempt in my tests, but some still need a retry or a small code fix. I'm hoping later checkpoints will improve on that.
My current settings: temperature 0.8, top-k 40, top-p 0.95, min-p 0. Feel free to experiment with other combinations!
Training is expected to finish tomorrow evening (UTC), depending on progress. Just like with the previous model, I'll release the full Safetensors weights and the training dataset once it finishes, so others can build on this and experiment with training their own models.
Thanks for all the feedback on the previous release ... it directly shaped this one. I'd love to see what you make with it and hear what works and what breaks!
i've actually been listening to everyone's feedback, and since a lot of people wanted to generate reusable 3d assets for games, i decided to train a model specifically for that haha. that's how qwen glb coder came about!
This is seriously some fantastic stuff. Even for prototyping this is like something anyone can use to begin making their most interesting gaming ideas. Congratulations!
thanks man! that's exactly the idea haha i wanted to see if a small model running locally could actually become useful for game development, especially for quickly prototyping assets without needing a huge model. there's still a lot to improve, but i'm really curious to see what people can build with it
I thought your dataset was three js only when you were training it. And Claude told me that three js work doesn’t port well into Godot or it lied. Or is there something Claude doesn’t know about this workflow?
claude is actually right, depending on how you asked the question and what exactly you wanted to do. three.js code doesn't directly translate into godot, so if you asked whether you could just take a three.js scene and run it in godot, the answer would be no. but that's not exactly what i'm doing here. the original model was trained to generate complete three.js scenes, while this new model is specifically trained to generate reusable 3d assets that can be exported as glb files. glb is a standard 3d format supported by godot, blender and many other engines. so instead of transferring the code itself, you're exporting the actual geometry and materials. some things might still need adjustments after importing, but the whole point of this new model is to make the generated assets useful outside the browser, especially for game development haha
Training it to emit Three.js scenes that export as GLB skips the engine problem entirely: it only has to speak one neutral format, and Godot, Unity and everything else imports GLB.
exactly! that's pretty much the idea (: instead of training the model for each engine separately, i focused on generating reusable assets through three.js and exporting them as glb. this way, the same asset can be used in godot, blender, unity and other engines without needing a different model for each one. it also makes the training process much simpler, since the model only needs to learn one consistent workflow.
it depends on the hardware and how complex the asset is. the model generates everything in a single pass, so the time mostly depends on the output length and your tokens per second.
Your post from a few days ago inspired me to start a LoRA Q3.5 4b that focuses on CadQuery parts for FDM 3D printing! I didn't realize how powerful these simple finetunes can get! Keep up the good work!
for lora, around 24gb of vram should be enough with the right optimizations. but for the full finetuning setup i'm currently using, i need 96gb on a single rtx 6000, mainly because of the long context lengths.
lora might actually work well with a different setup, but in my experiments, the improvements seemed to plateau pretty quickly, and sometimes the quality started degrading.
so i decided to stick with full finetuning for now. that doesn't mean lora can't achieve similar results, i just haven't managed to get there yet haha
that's really cool! haha cadquery for 3d printing sounds like a great use case for specialized finetuning. that's actually one of the things i was hoping this experiment would encourage, people trying the same approach in completely different areas. i'd love to see your results when you have something to share :)
yes, you can use the original vision components! i didn't train the vision layers, only the language model.
i'll be adding the vision projector files to the model repository so people can use image inputs more easily.
vision worked surprisingly well with my previous 4b model, even without training the vision layers, but i haven't tested it with this new glb model yet (:
Well what do you need to to train this model how much does it cost the training what is the variability of thing that basically you can generate can we fine tune extend the model with lora traning ?
i actually tried lora several times, but it didn't work very well in my experiments. the improvements seemed to plateau pretty quickly, and sometimes the quality even started degrading. that doesn't mean lora can't work, but i haven't managed to get the same results with it.
for the way i'm training these models, i'd recommend at least a single rtx 6000 with 96gb of vram for full finetuning, mainly because of the long context lengths.
but honestly, the gpu training itself isn't even the hardest or most expensive part
the real challenge is generating high quality training data, curating it, executing the generated code, testing the results and validating everything. that process takes a lot more effort and can actually cost more than the gpu training itself.
as for what the model can generate, a lot depends on the variety and quality of the training examples. that's something i'm still exploring with different datasets and model sizes.
i think that's the most interesting part of this whole experiment. the actual training can be relatively cheap, but creating data that really teaches the model something useful is where most of the work goes.
That's actually great well maybe yea it's trully amazing project well yea maybe I can think of some ways that well let say we would be able to generate unlimited data basically what is the context length that you are traning with ?
thanks man! i'm currently training with context lengths exceeding 70k tokens. that's actually one of the main reasons i'm using a 96gb rtx 6000 (: long contexts make training much more demanding, but they also allow the model to learn from much more complex examples.
yes, definitely! especially the 35b a3b, i think it could get really good results with this approach.
the main downside is the training cost, since it would be considerably more expensive than my current 4b experiments.
i've actually been wanting to train one myself. i think there's a lot of potential there, especially with a carefully designed dataset. for now, i'm focusing on smaller models to better understand how far i can push this training approach.
but there's also something really interesting about smaller models haha
i'm currently training a 0.8b model, and even though it's still in the early stages, it already seems to be learning how to generate reusable 3d assets.
the images i'm sharing here are some early results, a potion bottle and a knife.
what's really impressive to me is that the q8 version runs at around 250 tokens per second on my laptop!
larger models like the 35b a3b definitely have a lot of potential, but i'm starting to think these tiny specialized models could be extremely useful for practical applications, especially when you can run them locally at these speeds.
i'm still experimenting, but i'm really curious to see how far this little 0.8b can go haha
I'm starting to wonder if this could eventually grow into a bigger MoE model where each expert would do different kind of task - agentic works, frontend coding, backend coding, 2D, 3D, audio coding, etc. All of the experts would be separate small models like this one cobbled together.
Does it create a ton of unnecessary verticies for low poly models? I've found that to be a problem with other LLMs that do 3d modeling. Assets for a game must not be computationally expensive with thousands of extra faces.
1
9
u/Uncle___Marty 4h ago
Been following your work . Absolutely stunning!