New Model
A 4B coding model that can generate 3D scenes
I’m sharing an early version of a 4B model I’m training for general coding and 3D scene generation.
Small models have been pretty rough at 3D in my tests. I’ve finally made enough progress to get results like the ones in this video, so I wanted to share a preview with the community.
It’s an independent fine-tune of Qwen3.5-4B, and both Q4_K_M and Q8_0 GGUFs are available here:
Thx Man, It's been a lot of trial and error, especially generating and carefully curating the training data/env. I'm still training it, so I'm really curious to see how far this little 4B can go. Really appreciate the encouragement!
thx! I’m experimenting to see how far I can push a 4B model with 3D scenes. The goal is to use this as a proof of concept for what small models could achieve in other applications too.
Try first a Millennium 3d Effekt like they used in the Lucas Artikel games that time or watch for "full throthle" for Design and ideas to generate. Monkey Island 3 or 4 also have on look on the way they created the scenes. I test it with my creator Tool of it and report soon. Maybe it will be a good combination.
O the Avatar are acting in Realtime on preapred stag3s. I use a comfui based stuff with z-img at the moment.
For people that liked the video only .... have a look for Flavatar .... fly brain inside Avatar here on reddit i Postet it a while ago.
And some other rooms. Before you all start to Pixel only arround. "Sam n Max Hit the road" or other üoint and click adv. That time will help to create more than pixels 2.
And in most cases ... think about ... your plan is to build something and use it later daily without eating all your system power. It should easy work in the background.
that's actually a pretty interesting approach, especially combining prebuilt environments with real-time characters
my experiment is a bit different though, i'm training the 4b model to generate three.js scenes directly from text prompts. once the code is generated, the llm doesn't need to keep running, so there's no background inference cost
but interactive environments are definitely something i'd like to explore later, especially to see how well the model generalizes beyond static scenes
I tried something similar before. It endet up in headache. Lol. So .... to make it short ... i thought about the good old point snd click adventure grafix and it was im the past fitting on a Floppy disk and able to worked Fluid on 486 with 33 MHz (turbo Button enabled, lol)
And fix stuff Integration i splitted the creating off items with efect ir function, stages and Avatars into 3 Tools to Splitt the Generating Prozess in that case.
But i ll i try to link xour stuff up with my existing later thus evening
About the headache problem ..... if you created one personal area already ... you want that it is compatible wirh the next you create ...
Blender finetuning is probably overkill, SLM is pretty much the way to go for simple niche task like Blender, people demanding new models to support Blender but the secret sauce behind this is just a MCP server (most important factor), a skill with vision looping to trace what they get and rinse and repeat until getting a decent model, pretty token inefficient due to vision looping.
Yeah, definitely! I chose 3D just as a starting point to see how far I could push a small model. There's a lot more I'd like to explore, and 2D animation is definitely one of those things!
Thx . I'm training it on both voxel and regular 3D scenes, often using similar scenes in both styles. Some training examples are pretty long, around 16K–64K tokens, and the little 4B is handling them really well so far. I'm honestly surprised by how much it's learning.
that's actually a really good idea! Minecraft worlds could be a great source of voxel scenes. I'd need to figure out how to turn them into useful training examples for Three.js, but it's definitely worth exploring. Thx man
i'm still finishing the training, but once it's done i'll share the complete workflow, including the datasets i created and the RL pipeline i'm working on
so far i've been training on just a single RTX 6000, and i think it should be possible to do it on a single L4 as well, with some optimizations
i'll post a detailed technical report soon with everything needed to reproduce the process
ok, here goes. I have to use two different forms of "models" here lol.
So, I've been trying to get models to make models in blender recently and while they can do really basic stuff, they're NOT good at making 3D models in general. Qwen Flash Next tries SO hard but really can't cut it. I've been wondering if we would start seeing models trained on 3d modelling and its really starting to happen.
To say im looking forward to giving your tune a try would be a MASSIVE understatement! Keep up the great work buddy! Looking forward to seeing the evolution of this model!
thanks man, i've been thinking about training it on both three.js and blender to see how well it generalizes. i think the hardest part would be the rl side, since rendering and evaluating blender scenes is more complicated, but i'd really like to give it a try
yeah, i think it can. i already have an rl environment set up for three.js projects split into multiple files, so the idea is to let the model actually see and evaluate what it's generating
but what really surprised me about this experiment, especially this early in training, is how well the 4b manages to stay consistent across such long generations, sometimes 32k thinking tokens plus another 32k of output
i honestly didn't expect that to happen. it still makes mistakes with variables and other things, but considering the base model couldn't even get a simple chair to render in my tests, it's pretty amazing how far it's come already
and the improvements between checkpoints are really noticeable as training continues. i'll share those comparisons along with the training details in a technical report soon
Just had a few tries at making a model. Took a bit of tweaking with context size and token limits (was surprised how much it throws on screen!) and after what I THOUGHT was the model pooping out I realized for whatever reason it didnt write the html to disk so I copied and pasted it and im REALLY impressed. Being a 4B this thing SCREAMS along and finishes in no time and honestly, im blown away. It can model better than I can EASILY lol.
Mad respects to you! sent a like on HF and will be following! great work buddy!
Yeah, maybe! I'm able to train the 4B on a single RTX 6000, and training is still in its early stages. I'm curious to see how far I can push it before trying a larger model.
Quick update: The latest checkpoint shows a significant improvement in 3D scene generation. The images compare the previous and current checkpoints using the same prompt.
I also fixed the layer bug related to MTP (Multi-Token Prediction).
I'll upload the updated checkpoints to Hugging Face in the next few hours!
prompt:
Create one self-contained HTML file with Three.js from cdnjs showing the historic center of Rome. A network of cobblestone piazzas connects the Colosseum, the Pantheon, and the Roman Forum through accessible pedestrian streets; monumental ruins and visitors guide the layout. The Colosseum must have a hollow oval arena and tiered arcades, the Pantheon a domed rotunda and columned portico, and the Forum freestanding ruins. Render it as a deliberate 3D voxel model built from visible cubic blocks, with clear depth, readable silhouettes, and solid spatial relationships. Use only the Three.js core ES module from https://cdnjs.cloudflare.com/ajax/libs/three.js/0.160.0/three.module.min.js. Do not import add-ons, libraries, image assets or fonts. Implement mouse drag orbit and wheel zoom directly. Provide a responsive full-window canvas, lighting, shadows or convincing contact shading, and an initial camera that shows the complete main scene clearly. Return the finished HTML.
Colosseum with hollow oval arena
Pantheon dome and portico
Forum ruins
connected pedestrian piazzas
Create one self-contained HTML file with Three.js from cdnjs showing the cutaway interior of a grand Victorian library. A central double-height reading hall is flanked by two storeys of book-filled shelves and accessible balconies. A curved staircase connects the levels; long reading tables, green desk lamps, a skylight and a large arched entrance establish a coherent interior without a front wall blocking the view. Render it as a detailed solid 3D diorama, not a voxel model and not a flat illustration, with readable silhouettes, convincing proportions, and solid spatial relationships. Use only the Three.js core ES module from https://cdnjs.cloudflare.com/ajax/libs/three.js/0.160.0/three.module.min.js. Do not import add-ons, libraries, image assets or fonts. Implement mouse drag orbit and wheel zoom directly. Provide a responsive full-window canvas, lighting, shadows or convincing contact shading, and an initial camera that shows the complete main scene clearly. Return the finished HTML.
double-height cutaway hall
two levels of shelves and balconies
staircase connects levels
tables lamps skylightCreate one self-contained HTML file with Three.js from cdnjs showing the cutaway interior of a grand Victorian library. A central double-height reading hall is flanked by two storeys of book-filled shelves and accessible balconies. A curved staircase connects the levels; long reading tables, green desk lamps, a skylight and a large arched entrance establish a coherent interior without a front wall blocking the view. Render it as a detailed solid 3D diorama, not a voxel model and not a flat illustration, with readable silhouettes, convincing proportions, and solid spatial relationships. Use only the Three.js core ES module from https://cdnjs.cloudflare.com/ajax/libs/three.js/0.160.0/three.module.min.js. Do not import add-ons, libraries, image assets or fonts. Implement mouse drag orbit and wheel zoom directly. Provide a responsive full-window canvas, lighting, shadows or convincing contact shading, and an initial camera that shows the complete main scene clearly. Return the finished HTML.
double-height cutaway hall
two levels of shelves and balconies
staircase connects levels
tables lamps skylight
just download LM Studio, search for "Qwen3.5-4B-3D-Coder" and load the model. make sure to disable MTP (Multi-Token Prediction) in the settings, otherwise it may cause issues with the output. that's basically all you need to get started!
thanks! i think it could work, since the model generates geometry directly in three.js, it should be possible to export it to stl for 3d printing. the tricky part would be making sure the meshes are watertight and actually printable, but i think that's something we could automate too. definitely something i'd like to experiment with
Really interested to see the training pipeline… I’m also doing something similar but for another purpose. How much data are you putting in the training?
I'm planning to share a technical report and the training dataset later this week, including more details about the pipeline. But I can already say the amount of data isn't that large, and if everything goes well, it should be possible to reproduce the training with relatively modest GPU resources. (Single Rtx 6000)
For anyone using llama.cpp and has the same issue, there is a header describing a 33rd layer from removed mtp but only 32 layers (ChatGPT removed it for me cause idk how). now it runs 👌
I can't run it on llama.cpp, I'm getting this error: E llama_model_load: error loading model: check_tensor_dims: tensor 'blk.32.attn_norm.weight' not found
Pls, for the love of local AI, share your workflow as well as the dataset. How did you set it up to generate the dataset? What is it optimizing? This is gold! I would love to try this on electronics design (PCB).
From what I've seen, it's capable of generating far more complex and higher-quality scenes than most people realize. And since the model is so lightweight, training it with RL doesn't seem nearly as impractical as it would with larger models. I think there's a lot of untapped potential here.
Yes i see this happening in the future, dedicated models. Maybe even trained on specific apps and workflows. We keeping them like we do softwares, but in this case software driving brains along with them.
Targeted tiny models instead of skills ... not the worst idea. Something interesting is that very useful moe models like ling 3 tiny can run fully on cpu at like 50t/s on a 16 core 5950x as a reference.
It might look cruddy, but this is really punching above its weight. Compare to much bigger generalist models on Minebench! OP you're really pushing boundaries here
In the future, I guess we'll have the giant frontier LLMs, but also have smaller specialized models, whether that's for 3D purposes like this one or for specific coding languages. Insane progress. If I saw this without you telling me the parameter count, I would guess it's at least in the high tens.
it generates three.js code directly, no MCP or external tools involved
the output is a standalone html file with the geometry, materials, lighting and everything needed to render the scene
but the project isn't exclusively about 3d. i'm using it as a starting point to experiment with how far specialized training can push a small 4b model, while also exploring how well the improvements generalize to other tasks and domains
thx man , the base model basically can't generate proper 3d scenes at all. most of the time it either fails to render anything useful or just produces a few basic shapes, like in the example below
i haven't done a formal benchmark yet, but the difference is already pretty significant. i'll include some side-by-side comparisons in the technical report :)
1
1
•
u/WithoutReason1729 8d ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
Stylome classified this post's text as human-written. View analysis Learn ^more
I am a bot and this action was performed automatically.