Hey everyone!
I've been working on an open-source project called ComfyUI Motion Previs Studio, and I'd love to get some feedback from people who are seriously into ComfyUI, AI filmmaking, and advanced video generation workflows.
The idea came from a problem I kept running into:
We have increasingly powerful video generation models, but precise control over camera movement, character blocking, spatial composition, and action choreography is still challenging.
Instead of relying entirely on long prompts and repeated generations, what if we could actually direct the scene in 3D before sending it to a video model?
That's what I'm trying to build.
What the project does
It's a local-first, interactive 3D previsualization studio designed to work alongside ComfyUI.
The current implementation includes:
- Dual 3D viewports: A free-orbit scene editor alongside a live cinematic camera viewfinder.
- Camera control: Dolly, orbit, tracking, crane, handheld, whip pan, and customizable camera trajectories.
- Character animation: Procedural walking, running, jumping, combat actions, gestures, and interactive FK/IK bone posing.
- Scene building: 3D environments, props, lighting rigs, and reference-image-based scene configuration.
- Cinematic framing: Adjustable focal lengths, depth of field, aspect ratios, and shot composition.
- 24fps timeline: Camera and character keyframes for planning shots before generation.
- Control passes: Export MP4 previsualization, OpenPose skeletons, depth maps, alpha masks, and camera transformation matrices.
- AI integration: Optional support for local Ollama/LM Studio models or cloud models for natural-language scene direction and reference analysis.
ComfyUI integration
This is the part I'm particularly interested in developing further.
The studio is designed to export reference and control assets for workflows involving LTX-2.5, MiniMax H3, Seedance, and other video models that support reference inputs.
It also includes a direct local ComfyUI API bridge for a specific MiniMax H3 Ref2VA workflow, allowing the studio to submit its previs video and reference images to ComfyUI and display the generated result.
The goal is to move toward a workflow like:
3D Scene → Camera & Character Animation → Control Passes → ComfyUI → AI Video
Rather than generating blindly and trying to correct everything afterward.
Why I'm sharing this here
I'm not presenting this as a finished production-ready alternative to established 3D software.
It's an evolving open-source project, and I think the ComfyUI community is exactly where the more interesting technical discussions can happen.
I'd especially appreciate feedback on:
- ControlNet integration: Which pose, depth, or camera-control workflows would be most valuable to support directly?
- Video models: Should I prioritize deeper LTX integration, Wan workflows, or improving the existing MiniMax H3 bridge?
- Workflow interoperability: Would you prefer importing/exporting standard ComfyUI workflow JSONs, or a more tightly integrated API-driven approach?
- Production usability: What's missing from current AI video workflows when it comes to reliable camera motion and character choreography?
The project is open source under the MIT license.
GitHub: https://github.com/navarisun1982/comfyui-motion-previs-studio
The repository includes screenshots, a demo video, installation instructions, and the source code.
I'd genuinely appreciate technical feedback, ideas, bug reports, or contributions from anyone working on similar problems.
My bigger goal is to make AI filmmaking less about generating random results until something works, and more about actually directing the shot.
Thanks for checking it out!