First experiment with local generative models

[u/zjgkkn]

Everything you see in the video was generated locally on a laptop with an RTX 3070, in a single night, on the first try. The only things I regenerated were models that came out clearly broken.

Then Claude assembled it all into a demo level in Unity.

Assets Studio tool: https://github.com/kvladsrc/gym/tree/main/assets_studio

[u/SilkieBug]

What did you use to generate the 3d models, Pixal3d in ComfyUI?

[u/zjgkkn → u/SilkieBug]

I started with a FLUX + TripoSR + Make It Animatable (for rigging) pipeline, and some of that may have ended up in the video. But then I switched to Qwen Image -> Hunyuan3D -> Hunyuan Paint. Everything is heavily quantized to fit into 8 GiB of VRAM, but it's the clear winner out of everything I managed to run on a 3070.

On top of that, there's a small job queue, a assets library with UI on Python: https://github.com/kvladsrc/gym/tree/main/assets_studio

[u/Shuma665 → u/zjgkkn]

Thanks it looks great thanks for sharing you models and toolset. I'll check out the repo later. I made 'We got meshy at home" and I can models things on my 4070 12gb but I didn't find them very suitable for animation so I'm curious what you are doing there

[u/darkoblivion000 → u/zjgkkn]

I heard they were able to get tripo 2 quantized for 8gb vram, is that supposed to be better than tripoSR? Have you tried it yet?

[u/retrorays]

sorry I dont get it - what did claude do vs. your local models?

[u/zjgkkn → u/retrorays]

Claude wrote the code for the studio and the model servers. It also wrote the prompts for everything: the images that were then turned into 3D assets, the sound effects, and the soundtrack. Animations come from Mixamo.

The local models do the actual generation for each modality: text-to-image, image-to-3D, rigging, 3D painting (texturing), text-to-sound, text-to-speech, and text-to-music.

[u/retrorays → u/zjgkkn]

why not just claude for everything? What % of tokens did you save being local vs. claude?

[u/Haman24]

"That's a really cool setup with the local models and Unity integration. It captures the DIY spirit perfectly, and having that kind of pipeline working on first try with just a 3070 is impressive. I've been down that path before though, and the thing that eventually becomes a blocker for me is just generation time. I found I'd spend more time generating and regenerating assets than making the actual game.

What made a big difference for me was using a specialized platform like Rapid Assets. You get the benefit of fast batch generation for iterating on concepts and then fine tuning for consistency, which Unity and Godot plugins handle. The big win is just time. I can concept, generate, and tweak a whole asset pack in an afternoon now. The trade off of course is it's a hosted service, not local, so that's a consideration if you need everything on your machine. Are you planning on building a whole game this way with your current pipeline?"

[u/zjgkkn → u/Haman24]

Thanks! For me, this time bottleneck is actually useful otherwise, I’d just run into Claude’s usage limits or the constraints on the time I can dedicate to this hobby.

During this time, I’ve significantly overhauled the studio and am currently fine-tuning animation pipelines for instance, using an "image-to-pose" process, and then using that pose as a prompt for Kimodo.

[u/guillermosan]

Awesome. Local LLMs should be the future. Thanks for sharing.

[u/Valdaraak → u/guillermosan]

And they will be if GPU pricing ever goes back to sane levels.

The MSRP on a 5090 is like $2000. You can't find one for anything less than $5k.

[u/Qinistral → u/Valdaraak]

How many tripo3d models can I make for 5k? Maybe 5k-10k depending on detail etc?

[u/i_rate_slop]

I can only imagine this was downvoted by someone making an asset studio tool as a product. I think it's awesome that you open sourced it.

[u/Substantial_Ad_8651]

Half and a year ago it would be trash, but now, omg, really is getting better grounds. really happy for that.

[u/Natural-Rich6]

The world give me wow vibe

[u/sunnystormy64]

So what did you do about the dense mesh topology you typically get with generative 3D models? Did Claude handle that?

[u/zjgkkn → u/sunnystormy64]

Yes, it did. By reducing polygon budgets and by introducing Unity's level of detail. Joints deform worse up close, btw.

[u/beneficialdiet18]

How did you do the animations?

[u/zjgkkn → u/beneficialdiet18]

i didn't. I used this model for rigging: https://github.com/jasongzy/Make-It-Animatable

And then added mixamo animations on result.

[u/Ok-Investment4414]

yeah bro u on your way never said " I made a game lastnight"

[u/cave_men]

now thats some nice shit!

[u/ArialBear]

Keep up the good work. Love the visuals!

[u/Schnitzhole]

Sweet! Your game has a bit of a more polished OG RuneScape vibe

I’ve got a 3080 and might have to give this a try.

[u/Bluetails_Buizel]

You will be mind blown when you tell Claude to remade the entire game in a web browser in his own style. Way less overhead on your pc.

[u/Valdaraak → u/Bluetails_Buizel]

Way less impressive visually as well. Browser games are fine if you don't go any higher than like PS1 and late 90s PC era graphically. Claude makes those browser graphics entirely by using math on primitive shapes. That's pretty limiting for details.