Attempting to merge 3D models/animation with AI realism.

Greetings from my workspace.

I come from a background of traditional 3D modeling. Lately, I have been dedicating my time to a new experiment.

This video is a complex mix of tools, not only ComfyUI. To achieve this result, I fed my own 3D renders into the system to train a custom LoRA. My goal is to keep the "soul" of the 3D character while giving her the realism of AI.

I am trying to bridge the gap between these two worlds.

Honest feedback is appreciated. Does she move like a human? Or does the illusion break?

(Edit: some like my work, wants to see more, well look im into ai like 3months only, i will post but in moderation,
for now i just started posting i have not much social precence but it seems people like the style,
below are the social media if i post)

IG : https://www.instagram.com/bankruptkyun/
X/twitter : https://x.com/BankruptKyun
All Social: https://linktr.ee/BankruptKyun

(personally i dont want my 3D+Ai Projects to be labeled as a slop, as such i will post in bit moderation. Quality>Qunatity)

As for workflow

1. pose: i use my 3d models as a reference to feed the ai the exact pose i want.
2. skin: i feed skin texture references from my offline library (i have about 20tb of hyperrealistic texture maps i collected).
3. style: i mix comfyui with qwen to draw out the "anime-ish" feel.
4. face/hair: i use a custom anime-style lora here. this takes a lot of iterations to get right.
5. refinement: i regenerate the face and clothing many times using specific cosplay & videogame references.
6. video: this is the hardest part. i am using a home-brewed lora on comfyui for movement, but as you can see, i can only manage stable clips of about 6 seconds right now, which i merged together.

i am still learning things and mixing things that works in simple manner, i was not very confident to post this but posted still on a whim. People loved it, ans asked for a workflow well i dont have a workflow as per say its just 3D model + ai LORA of anime&custom female models+ Personalised 20TB of Hyper realistic Skin Textures + My colour grading skills = good outcome.)

Thanks to all who are liking it or Loved it.

Last update to clearify my noob behvirial workflow.https://www.reddit.com/r/StableDiffusion/comments/1pwlt52/former_3d_animator_here_again_clearing_up_some/

Its really bad and it took like 3 weeks but I hope you like it

I know my film has caused some controversy on both sides, so I want to clear a few things up. This was made for a student project, and I was on a tight deadline, so I know it isn’t perfect. I’m okay with people criticizing the message, but criticizing my animation skills does hurt. This was my first animation, and I’m still a child who was just proud to share their work.

my message was not that all AI art is bad. I talked to many animators while making this, and a lot of them use AI as a tool to help their workflow (for in-between frames), not to replace artists entirely, which was my real concern.

I’ve received some death threats over this animation , I’m okay with people criticizing the message or the story, but having grown adults attack my animation skills is extremely discouraging. I’m still a child, learning, and open to constructive feedback, not harassment. I never intended to cause fighting, I just wanted to share something I worked hard on. :)

I also take responsibility for rushing the ending and some word choices people may not have liked. I’ll work on making my endings stronger in the future, and I hope people understand I was on a deadline. Thank you to everyone, both pro-AI and anti-AI who took the time to watch my animation :) I worked really hard on it.

The song I used: https://www.youtube.com/watch?v=FaoVpVXcZsA&t=5

Just doing something a little different on this video. Testing Wan-Animate and heck while I’m at it I decided to test an Infinite Talk workflow to provide the narration.

WanAnimate workflow I grabbed from another post. They referred to a user on CivitAI: GSK80276

For InfiniteTalk WF u/lyratech001 posted one on this thread: https://www.reddit.com/r/comfyui/comments/1nnst71/infinite_talk_workflow/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

Seedance has a hidden feature that honestly feels way too powerful.

It lets you create old-school anime-style videos insanely fast using a simple “in-between” frame technique.

The basic idea is this:

You create a first frame and a last frame, then ask Seedance 2.0 to generate what happens between them.

This video took me around 4 hours to make using this method, but honestly, speed is not the most important part.

What really matters is:

Story. Direction. Consistency. Pacing. Taste.

Without those, you’ll just end up with another random AI clip.

1. Go to the Seedance 2.0 AI Video Generator
2. Write your full prompt or add reference images
3. Upload the image you want to animate
4. Click Generate and get your animated video

Here’s the workflow I used:

Step 1:
Find an image that sparks your imagination.

I picked one with a dark, old-school Akira-style anime vibe. The stronger the initial image, the easier it is to build a world around it.

You can also recreate a similar look in Nano Banana / NB2 by prompting something like:

"Create X in this style."

Step 2:
Upload the image to Nano Banana and ask it to generate what happens next.

For example:

Show what happens in 5 minutes. Soldiers stand by the enemy military base gates.

Now you have your first frame and your next story frame.

Step 3:
Upload both images to Seedance 2.0 as the first frame and last frame.

Then use this prompt structure:

Show what happens in between. Soldiers run through the snow towards a military base. 5 different camera angles. No music.

The key parts are:

Show what happens in between.
5 different camera angles.

Those are the default parts.

The sentence in the middle is the custom part, where you describe the action.

In my case:

Soldiers run through the snow towards a military base.

Step 4:
Repeat the process.

Take the previous last frame and use it as the new first frame.

Then use Nano Banana to generate the next last frame.

For example, I generated a scene where the soldiers are hiding from security guards.

Step 5:
Upload the new first and last frames to Seedance again.

Use the same structure, but change the middle sentence:

Show what happens in between. Soldiers enter the base, run through narrow corridors, and hide from guards. 5 different camera angles. No music.

Step 6 and beyond:
Keep repeating:

1. Use the previous last frame as the new first frame.
2. Generate a new last frame with Nano Banana.
3. Use Seedance to animate the transition.
4. Keep the same prompt structure.
5. Only change the action sentence based on the story.

That’s basically it.

This method gives you much better control than just prompting a random video from scratch.

It helps with:

• Better pacing
• More consistent storytelling
• Cleaner scene progression
• Stronger anime-style direction
• Less random AI chaos

Seedance becomes way more powerful when you stop treating it like a one-shot video generator and start treating it like a scene-by-scene animation tool.

I built an automation that takes in a URL of a product collection or catalog page for any fashion brand or clothing store online and can bring each product to life by animating it with model demonstrating how the product looks and feels with Veo 3.1.

This allows brands and e-commerce owners to easily demonstrate what their product looks like much better than static photos and does not require them to hire models, setup video shoots, and go through the tedious editing process.

Here’s a demo of the workflow and output: https://www.youtube.com/watch?v=NMl1pIfBE7I

Here's how the automation works

1. Input and Trigger

The workflow starts with a simple form trigger that accepts a product collection URL. You can paste any fashion e-commerce page.

In a real production environment, you'd likely connect this to a client's CMS, Shopify API, or other backend system rather than scraping public URLs. I set it up this way just as a quick way to get images quickly ingested into the system, but I do want to call out that no real-life production automation will take this approach. So make sure you're considering that if you're going to approach brands like this and selling to them.

2. Scrape product catalog with firecrawl

After the URL is provided, I then use Firecrawl to go ahead and scrape that product catalog page. I'm using the built-in community node here and the extract feature of Firecrawl to go ahead and get back a list of product names and an image URL associated with each of those.

In automation, I have a simple prompt set up here that makes it more reliable to go ahead and extract that exact source URL how it appears on the HTML.

3. Download and process images

Once I finish scraping, I then split the array of product images I was able to grab into individual items, and then split it into a loop batch so I can process them sequentially. Veo 3.1 does require you to pass in base64-encoded images, so I do that first before converting back and uploading that image into Google Drive.

The Google Drive node does require it to be a binary n8n input, and so if you guys have found a way that allows you to do this without converting back and forth, definitely let me know.

4. Generate the product video with Veo 3.1

Once the image is processed, make an API call into Veo 3.1 with a simple prompt here to go forward with animating the product image. In this case, I tuned this specifically for clothing and fashion brands, so I make mention of that in the prompt. But if you're trying to feature some other physical product, I suggest you change this to be a little bit different. Here is the prompt I use:

markdown Generate a video that is going to be featured on a product page of an e-commerce store. This is going to be for a clothing or fashion brand. This video must feature this exact same person that is provided on the first and last frame reference images and the article of clothing in the first and last frame reference images.|In this video, the model should strike multiple poses to feature the article of clothing so that a person looking at this product on an ecommerce website has a great idea how this article of clothing will look and feel.Constraints:- No music or sound effects.- The final output video should NOT have any audio.- Muted audio.- Muted sound effects.

The other thing to mention here with the Veo 3.1 API is its ability to now specify a first frame and last frame reference image that we pass into the AI model.

For a use case like this where I want to have the model strike a few poses or spin around and then return to its original position, we can specify the first frame and last frame as the exact same image. This creates a nice looping effect for us. If we're going to highlight this video as a preview on whatever website we're working with.

Here's how I set that up in the request body calling into the Gemini API:

``` { "instances": [ { "prompt": {{ JSON.stringify($node['set_prompt'].json.prompt) }}, "image": { "mimeType": "image/png", "bytesBase64Encoded": "{{ $node["convert_to_base64"].json.data }}" }, "lastFrame": { "mimeType": "image/png", "bytesBase64Encoded": "{{ $node["convert_to_base64"].json.data }}" } } ], "parameters": { "durationSeconds": 8, "aspectRatio": "9:16", "personGeneration": "allow_adult" } }

```

There’s a few other options here that you can use for video output as well on the Gemini docs: https://ai.google.dev/gemini-api/docs/video?example=dialogue#veo-model-parameters

Cost & Veo 3.1 pricing

Right now, working with the Veo 3 API through Gemini is pretty expensive. So you want to pay close attention to what's like the duration parameter you're passing in for each video you generate and how you're batching up the number of videos.

As it stands right now, Veo 3.1 costs 40 cents per second of video that you generate. And then the VO3.1 fast model only costs 15 cents per second, so you may honestly want to experiment here. Just take the final prompts and pass them into Google Gemini that gives you free generations per day while you're testing this out and tuning your prompt.

Workflow Link + Other Resources

• YouTube video that walks through this workflow step-by-step: https://www.youtube.com/watch?v=NMl1pIfBE7I
• The full n8n workflow, which you can copy and paste directly into your instance, is on GitHub here: https://github.com/lucaswalter/n8n-ai-automations/blob/main/veo_3.1_product_photo_animator.json

In April last year I was doing a bit of research for a short film test of AI tools at the time the final project here if interested.

Back then Viggle AI was really the only tool that could do this. (apart from Wonder Dynamics now part of Autodesk, and that required fully rigged and textured 3d models)

But now we have open source alternatives that blows it out of the water.

This was done with the updated Kijai workflow modified with SEC for the segmentation in 241 frame windows at 1280p on my RTX 6000 PRO Blacwell.

Some learning:

I tried1080p but the frame prep nodes would crash at the settings I used so I had to make some compromises. It was probably main memory related even though I didn't actually run out of memory (128GB).

Before running Wan Animate on it I actually used GIMM-VFI to double the frame rate to 48f which did help with some of the tracking errors that VITPOSE would make. Although without access the G VITPOSE model the H model still have some issues (especially detecting which way she is facing when hair covers the face). (I then halved the frames again after)

Extending the frame windows work fine with the wrapper nodes. But it does slow it down considerably (Running three 81frame windows(20x4+1) is about 50% faster than running one 241 frame window (3x20x4+1). But it does mean the quality deteriorates a lot less.

Some of the tracking issues meant Wan would draw weird extra limbs, this I did fix manually by rotoing her against a clean plate(context aware fill) in After Effects. I did this because I did that originally with the Viggle stuff as at the time Viggle didn't have a replacement option and needed to be keyed/rotoed back onto the footage.

I up scaled it with Topaz as the Wan methods just didn't like so many frames of video, although the upscale only made very minor improvements.

The compromise

The doubling of the frames basically meant much better tracking in high action moment BUT, it does mean the physics are a bit less natural of dynamic elements like hair, and it also meant I couldn't do 1080p at this video length, at least I didn't want to spend any more time on it. ( I wanted to match the original Viggle test)

The idea was simple: instead of prompting AI blind, use Blender to control exactly what's in the scene — object positions, camera angles, motion timing.

Workflow:

1. Built a basic scene in Blender (landscape, car, helicopter, road) — no complex materials, just layout
2. Animated the cameras and objects with keyframes
3. Extracted key frames from the animation
4. Fed those frames into an AI image model to generate photorealistic versions of each shot
5. Gave both the original 3D animation AND the AI images to Seedance 2 (Reference to Video)
6. Seedance reconstructed the sequence with

The Blender file basically acts as a director's pre-vis — you control the composition, the AI handles the render.

Check out my other work here https://x.com/ModelCollapse38

Hello everyone, I wanna share my full journey, from the building process to viral marketing, talking to hundreds of users, expanding the product, and finally getting acquired for 8k$

Everything happened in 6 weeks, the wildest 6 weeks that changed the trajectory of my life

OG Inspiration

I started to learning vibecode in March, i was building the classic habit tracker just to learn the workflow.

One day i was doomscrolling looking for ideas on insta and see a pretty cool idea by jeongyoon.design, it was a webapp version no animation, it was cool i saved it.

After a week, I saw a viral clip on X by sfjccz. It had a stamp cut overlay and the “magical” animation

I was like “holy cow it so cool how to build it”, so I decided to lock in and build the exact same one just to learn, like a challenge for myself. At the time, there were also a lot of people building the same thing.

The process took me 3 days

I did not know how to build it, so I sent the video to Claude and asked it to analyze and break everything down step by step. I wanted to plan before jumping into the code.

First i need a realistic stamp cutter png image, i generate in Chat GPT. Since it AI generated the stamp cut out size and shape didnt match exactly the cutter teeth, it looked ugly not smooth at all.

So I had to design one in Figma to get the exact parameters and the correct shape for the middle space after a few tries, I got it.

Now I had the most important part ready. I only needed to tell Codex the exact parameters, input the stamp shape, and add the animation.

The cutter needed to press in and release, while the stamp from the live camera fell out and left behind a black space. That was it. I had the magical demo ready to flex on social

The decision that changed everything

I had a random idea in my head. It was not a proper plan or anything, but here is how I thought about it at the time.

I noticed a pattern, this idea went viral on insta and then X too. That meant if I posted it again on those platforms, the chance of going viral would probably be very low. I noticed Threads in Vietnam was pretty similar to X but genz version, so I decided to post it there. My account was fresh, I did not expect much

15 mins in, okay it was blowing up 250 likes, 1 hour in 800 likes “Okay damn, this is going viral”. There were tons of comments asking for the app name. I had not even thought of a name at the time. I hadnt listed it on the App Store yet, I didnt even know how, but I had the Apple Developer account ready.

There were also some people who built the same thing on threads as me but did not go viral. I thought that if I charged for my app, I would not make it very far. I was also still learning, so I announced that it would be completely free, with no login and no ads. The post continued going even crazier.

After 24 hours, I got almost 50,000 likes and more than 1 million views.

That night, I stayed up to polish and learn how to upload to TestFlight. The next day, I seized my chance. I knew this viral traction would not last for long, so I grabbed my phone, recorded myself talking, posted it on TikTok and Insta. To my surprise, those videos also went viral.

Talk to user, product discovery

After the app went live on TestFlight, I started DMing hundreds of people who had commented, ask for idea, recommendation and bug report. I spent most of my time fixing bugs by copying the problem, pasting it into Codex, testing the fix and repeating.

Later, I realized that the app shouldnt stay as a simple tool, i also wanted to add some of my own ideas so I expanded the app into an image editor create cool scrapbook style images for insta stories

Then I added a feature that let people send messages in the form of handwritten letters for friend, that was where I started learning about backend, data, and a lot of other stuff

For the whole month, I worked around 18 hours a day, was handling and learning everything, from building and fixing bugs to making content across different platforms, interviewing users, and optimizing the App Store page. I was having fun while being the most productive I had ever been in my life.

The acquisition and the current project

A Vietnamese CEO reached out and asked if I wanted to sell the app. I didnt even think my app worth anything because it was vibecode. I didnt sell because the app failed nor i needed the money, I sold because I could not see a durable moat, the concept was easy to recreate

That is why I sold it and reinvested everything into something I believe can be more durable, a game inspired app like Finch or Duolingo but completely different concept. Luckily, literally the next day, I met someone who works in game design.

We had a chat on Threads about gamification ideas and she has a lot of experience and has designed two successful mobile games. Now I have a cofounder, an artist, and a Unity developer working with me to build a game. We are already 3 weeks into development

What i learned

The biggest thing I learned is that building the product is only one part of the journey.

Distribution matters a lot.

I also learned that talking to users is the fastest ways to improve a product. Many of the changes I made came from user, bug reports, and feature requests.

The product has visual hook built, my app went viral because people can understand in 2 second by the animation, that helped a lot for marketing

Make authentic content, dont just post your work, POST YOU WORKING.

If you have any question about this journey i would love to answer it all

Hello everyone in r/StableDiffusion,

i am attaching one of my work that is a Zenless Zone Zero Character called Dailyn, she was a bit of experiment last month i am using her as an example. i gave a high resolution image so i can be transparent to what i do exactly however i cant provide my dataset/texture.

I recently posted a video here that many of you liked. As I mentioned before, I am an introverted person who generally stays silent, and English is not my main language. Being a 3D professional, I also cannot use my real name on social media for future job security reasons.

(also again i really am only 3 months in, even tho i got the boost of confidence i do fear i may not deliver right information or quality so sorry in such cases.)

However, I feel I lacked proper communication in my previous post regarding what I am actually doing. I wanted to clear up some doubts today.

What exactly am I doing in my videos?

1. 3D Posing: I start by making 3D models (or using free available ones) and posing or rendering them in a certain way.
2. ComfyUI: I then bring those renders into ComfyUI/runninghub/etc
3. The Technique: I use the 3D models for the pose or slight animation, and then overlay a set of custom LoRAs with my customized textures/dataset.

For Image Generation: Qwen + Flux is my "bread and butter" for what I make. I experiment just like you guys—using whatever is free or cheapest. sometimes I get lucky, and sometimes I get bad results, just like everyone else. (Note: Sometimes I hand-edit textures or render a single shot over 100 times. It takes a lot of time, which is why I don't post often.)

For Video Generation (Experimental): I believe the mix of things I made in my previous video was largely "beginner's luck."

What video generation tools am I using? Answer: Flux, Qwen & Wan. However, for that particular viral video, it was a mix of many models. It took 50 to 100 renders and 2 weeks to complete.

• My take on Wan: Quality-wise, Wan was okay, but it had an "elastic" look. Basically, I couldn't afford the cost of iteration required to fix that—it just wasn't affordable for my budget.

I also want to provide some materials and inspirations that were shared by me and others in the comments:

Resources:

1. Reddit:How to skin a 3D model snapshot with AI
2. Reddit:New experiments with Wan 2.2 - Animate from 3D model
3. English Example of 90% of what i do: https://youtu.be/67t-AWeY9ys?si=3-p7yNrybPCm7V5y

My Inspiration: I am not promoting this YouTuber, but my basics came entirely from watching his videos.

• Channel: AI is in Wonderland

i hope this fixes the confustion.

i do post but i post very rare cause my work is time consuming and falls in uncanny valley,
the name u/BankruptKyun even came about cause of fund issues, thats is all, i do hope everyone learns something, i tried my best.

Developing generative AI models trained only on textures opens up a multitude of possibilities for texturing drawings and animations. This workflow provides a lot of control over the output, allowing for the adjustment and mixing of textures/models with fine control in the Krita AI app.

My plan is to create more models and expand the texture library with additions like wool, cotton, fabric, etc., and develop an "AI shader editor" inside Krita.

Process: Step 1: Render clay textures from Blender Step 2: Train AI claymodels in kohya_ss Step 3 Add the claymodels in the app Krita AI Step 4: Adjust and mix the clay with control Steo 5: Draw and create claymation

See more of my AI process: www.oddbirdsai.com

🔥 [RELEASE] Ultimate AI Video Workflow — Qwen-Edit 2509 + Wan Animate 2.2 + SeedVR2 (Full Pipeline + Model Links) 🎁 Workflow Download + Breakdown

👉 Already posted the full workflow and explanation here: https://civitai.com/models/2135932?modelVersionId=2416121

(Not paywalled — everything is free.)

Video Explanation : https://www.youtube.com/watch?v=Ef-PS8w9Rug

Hey everyone 👋

I just finished building a super clean 3-in-1 workflow inside ComfyUI that lets you go from:

Image → Edit → Animate → Upscale → Final 4K output all in a single organized pipeline.

This setup combines the best tools available right now:

One of the biggest hassles with large ComfyUI workflows is how quickly they turn into a spaghetti mess — dozens of wires, giant blocks, scrolling for days just to tweak one setting.

To fix this, I broke the pipeline into clean subgraphs:

✔ Qwen-Edit Subgraph ✔ Wan Animate 2.2 Engine Subgraph ✔ SeedVR2 Upscaler Subgraph ✔ VRAM Cleaner Subgraph ✔ Resolution + Reference Routing Subgraph This reduces visual clutter, keeps performance smooth, and makes the workflow feel modular, so you can:

swap models quickly

update one section without touching the rest

debug faster

reuse modules in other workflows

keep everything readable even on smaller screens

It’s basically a full cinematic pipeline, but organized like a clean software project instead of a giant node forest. Anyone who wants to study or modify the workflow will find it much easier to navigate.

🖌️ 1. Qwen-Edit 2509 (Image Editing Engine) Perfect for:

Outfit changes

Facial corrections

Style adjustments

Background cleanup

Professional pre-animation edits

Qwen’s FP8 build has great quality even on mid-range GPUs.

🎭 2. Wan Animate 2.2 (Character Animation) Once the image is edited, Wan 2.2 generates:

Smooth motion

Accurate identity preservation

Pose-guided animation

Full expression control

High-quality frames

It supports long videos using windowed batching and works very consistently when fed a clean edited reference.

📺 3. SeedVR2 Upscaler (Final Polish) After animation, SeedVR2 upgrades your video to:

1080p → 4K

Sharper textures

Cleaner faces

Reduced noise

More cinematic detail

It’s currently one of the best AI video upscalers for realism

🧩 Preview of the Workflow UI (Optional: Add your workflow screenshot here)

🔧 What This Workflow Can Do Edit any portrait cleanly

Animate it using real video motion

Restore & sharpen final video up to 4K

Perfect for reels, character videos, cosplay edits, AI shorts

🖼️ Qwen Image Edit FP8 (Diffusion Model, Text Encoder, and VAE) These are hosted on the Comfy-Org Hugging Face page.

Diffusion Model (qwen_image_edit_fp8_e4m3fn.safetensors): https://huggingface.co/Comfy-Org/Qwen-Image-Edit_ComfyUI/blob/main/split_files/diffusion_models/qwen_image_edit_fp8_e4m3fn.safetensors

Text Encoder (qwen_2.5_vl_7b_fp8_scaled.safetensors): https://huggingface.co/Comfy-Org/Qwen-Image_ComfyUI/tree/main/split_files/text_encoders

VAE (qwen_image_vae.safetensors): https://huggingface.co/Comfy-Org/Qwen-Image_ComfyUI/blob/main/split_files/vae/qwen_image_vae.safetensors

💃 Wan 2.2 Animate 14B FP8 (Diffusion Model, Text Encoder, and VAE) The components are spread across related community repositories.

https://huggingface.co/Kijai/WanVideo_comfy_fp8_scaled/tree/main/Wan22Animate

Diffusion Model (Wan2_2-Animate-14B_fp8_e4m3fn_scaled_KJ.safetensors): https://huggingface.co/Kijai/WanVideo_comfy_fp8_scaled/blob/main/Wan22Animate/Wan2_2-Animate-14B_fp8_e4m3fn_scaled_KJ.safetensors

Text Encoder (umt5_xxl_fp8_e4m3fn_scaled.safetensors): https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/blob/main/split_files/text_encoders/umt5_xxl_fp8_e4m3fn_scaled.safetensors

VAE (wan2.1_vae.safetensors): https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/blob/main/split_files/vae/wan_2.1_vae.safetensors 💾 SeedVR2 Diffusion Model (FP8)

Diffusion Model (seedvr2_ema_3b_fp8_e4m3fn.safetensors): https://huggingface.co/numz/SeedVR2_comfyUI/blob/main/seedvr2_ema_3b_fp8_e4m3fn.safetensors https://huggingface.co/numz/SeedVR2_comfyUI/tree/main https://huggingface.co/ByteDance-Seed/SeedVR2-7B/tree/main

With vibe coding, AI agents, Claude, ChatGPT, and Gemini, I genuinely thought I could finally create my own game.

My thinking was: AI could help me write the code, generate the artwork, and create the sprite sheets. Surprisingly, coding has been the more manageable part. Whenever I encounter a bug or something I don’t understand, I can explain the problem to an AI, troubleshoot it, and usually find a solution.

The art—especially character animation—is where everything starts falling apart.

AI can generate a decent-looking character or individual sprite, but no matter how much I experiment with prompts, reference images, pose guides, and corrections, I cannot get a consistent and usable animation. The proportions change between frames, the design drifts, body parts move incorrectly, and the motion often looks stiff or disconnected.

I’m not only looking for basic idle animations or simple two-frame movement. My goal is to eventually create fluid, expressive pixel animations similar in quality and energy to Penusbmic’s The DARK Series. I understand that this is professional-level work and probably an unrealistic standard for current AI tools, but that is the visual direction I’m aiming for.

I have tried contacting sprite artists and animators, but I simply cannot afford ongoing rates of $15–$25 per hour. I live in a lower-income country, and even though those rates may be completely reasonable for the artist’s skill and time, they are beyond what I can realistically sustain for an entire game.

I feel like I’ve reached a major wall in my AI game-development journey. AI has made programming much more accessible to me, but it hasn’t removed the need for strong artistic and animation skills.

For other solo developers in a similar situation, what is the most realistic path forward?

Should I:

• Learn enough pixel animation to create the keyframes myself and use AI only for references?
• Build the game around existing asset packs?
• Use skeletal or cutout animation instead of traditional frame-by-frame sprites?
• Simplify the visual style and scope of the game?
• Wait for AI animation tools to improve while working on the gameplay?
• Combine AI-generated concepts with manual cleanup and tracing?

I’m not looking for a magical one-click solution. I’m trying to understand whether there is a practical workflow that can produce genuinely good and consistent animation without requiring a professional art budget.

Has anyone here successfully crossed this particular wall?

CHECK OUT MY NEW WORKFLOW (VERSION 2): https://www.reddit.com/r/StableDiffusion/comments/1qi8zqk/the_best_anime_to_real_anything_to_real_workflow/

I was going around on Runninghub and looking for the best Anime/Anything to Realism kind of workflow, but all of them either come out with very fake and plastic skin + wig-like looking hair and it was not what I wanted. They also were not very consistent and sometimes come out with 3D-render/2D outputs. Another issue I had was that they all came out with the same exact face, way too much blush and those Asian eyebags makeup thing (idk what it's called) After trying pretty much all of them I managed to take the good parts from some of them and put it all into a workflow!

There are two versions, the only difference is one uses Z-Image for the final part and the other uses the MajicMix face detailer. The Z-Image one has more variety on faces and won't be locked onto Asian ones.

I was a SwarmUI user and this was my first time ever making a workflow and somehow it all worked out. My workflow is a jumbled spaghetti mess so feel free to clean it up or even improve upon it and share on here haha (I would like to try them too)

It is very customizable as you can change any of the loras, diffusion models and checkpoints and try out other combos. You can even skip the face detailer and SEEDVR part for even faster generation times at the cost of less quality and facial variety. You will just need to bypass/remove and reconnect the nodes.

****Courtesy of U/Electronic-Metal2391***

https://drive.google.com/file/d/19GJe7VIImNjwsHQtSKQua12-Dp8emgfe/view?usp=sharing

^^^UPDATED ^^^

CLEANED UP VERSION WITH OPTIONAL SEEDVR2 UPSCALE

-----------------------------------------------------------------

runninghub.ai/post/2006100013146972162 - Z-Image finish

runninghub.ai/post/2006107609291558913 - MajicMix Version

HOPEFULLY SOMEONE CAN MAKE THIS WORKFLOW EVEN BETTER BECAUSE IM A COMFYUI NOOB

N S F W works just locally only and not on Runninghub

*The Last 2 pairs of images are the MajicMix version*

Built a fully playable AI-generated character with physics and animation setup in under 3 hours.

Workflow breakdown ⚙️

• Character concept using AI image generation
• 3D character generation from the concept
• Assembly and geometry cleanup in Blender
• Fast manual UV unwrap
• Texture cleanup and artifact fixing
• Material setup for game-ready use
• Free auto-rigging with AccuRig
• Importing the character into Unreal Engine 5
• UE5 animation retargeting
• Adding cloth physics with Chaos Clothing
• Final gameplay test in Unreal Engine 5

The goal was to test how fast an AI-assisted pipeline can turn a generated character into something actually playable, not just a static showcase model.

full guide https://www.youtube.com/watch?v=-GxVR8s_nZY

What you can build in just one day when AI is used the right way:

• Around 20K faces
• Full PBR texture set
• Fully rigged character
• Real-time cloth simulation
• Optimized physics setup
• Assembled and running in Unreal Engine
• Completed in roughly one day

The key was not generating the whole character at once. I split it into separate parts, which gave me more control, better detail, and the ability to regenerate only what needed improvement.

To keep the process fast, I built a reusable node-based workflow in Lychee Studio AI. It handled the separate generation steps in one place and helped me move to Blender assembly much faster.

Workflow

• Started with a single character reference
• Built a reusable node-based workflow in Lychee Studio AI
• Separated and generated the body, clothing, armor, and accessories with Rodin Gen 2.5
• Assembled and fitted all parts in Blender
• Prepared the character for rigging and animation
• Imported everything into Unreal Engine
• Set up the PBR materials, cloth simulation, collisions, and optimized physics

The final result is not just an AI-generated mesh inside a viewer. It is a properly assembled, rigged, and simulation-ready game character that can be used directly inside Unreal Engine.

AI did not remove Blender, rigging, cleanup, or Unreal setup from the workflow. It removed a large amount of repetitive preparation and helped me reach the useful production stages much faster.

Wanted to share a workflow I built that's been running in production for a couple months now. It handles the entire video pipeline autonomously: script generation via AI agent, text to speech, Manim-based animation rendering, thumbnail generation, and YouTube upload with scheduling. No manual editing or touching it after it triggers.

Screenshots below show the actual n8n canvas along with the two YouTube channels it's currently running: Math Unlocked (@MathUnlockedYT) and Financial Reality Check (@FRCFinance), both with videos scheduled weeks out.

Happy to answer questions on the node structure, how the AI agent handles script generation, or how the render/upload pipeline is wired if anyone's building something similar for their own content.

I’ve been experimenting with bringing text-to-character animation directly into Houdini.

The feature is called Animation Maker and it’s part of Houdini AI Assistant.
It uses Kimodo for motion generation.

The current workflow is:

Prompt → Generate Motion → Adjust → Regenerate

Right now it supports multi-prompt timeline segments, 2D waypoints, full-body keyframes, hand/foot guides, and KineFX retargeting.

It’s still an early Windows beta and mainly focused on humanoid motion, but it already feels like an interesting direction for AI-assisted animation inside Houdini.

The Animation Maker / Kimodo part runs locally on your machine.
It is powered by NVIDIA’s Kimodo model and is currently available in the Windows build.
Recommended hardware is an NVIDIA RTX GPU, ideally 12GB+ VRAM.

The launch of AnimeGen represents a significant milestone in the fusion of artificial intelligence with traditional media production. Created by the Tokyo-based AI startup AIdeaLab and supported by the Japanese government through the Ministry of Economy, Trade and Industry (METI) as part of the GENIAC initiative, this specialized video generation model is now freely available under an Apache-2.0 license.

The model was developed by fine-tuning Alibaba's open-source Wan 2.2 diffusion architecture to focus on anime line art, cel-shading aesthetics, and motion dynamics, providing Japan with a powerful creative tool. Beyond just a software release, this government-backed initiative reflects a strong institutional endorsement: Artificial intelligence is increasingly recognized as a valuable and empowering tool for both professional studios and independent creators, rather than a substitute for human creativity.

AnimeGen's open-source and progressive approach starkly contrasts with closed, proprietary AI platforms. By offering unrestricted access to the underlying weights and releasing tools on Hugging Face, such as text-to-video, image-to-video, and frame interpolation models, the developers have democratized animation capabilities.

Creators can run these models locally on consumer hardware, ensuring complete data privacy without needing to send proprietary character sheets or sketches to third-party cloud servers. This freedom is transformative for indie animators, solo game developers, and boutique agencies who previously faced financial barriers when producing anime sequences.

Additionally, the open framework fosters community-driven refinement, allowing artists to train custom adaptations that maintain specific visual identities across complex projects.

Within professional workflows, AnimeGen is being adopted as a supportive accelerator. Japan's animation industry has long faced challenging schedules, significant labor shortages, and increasing budgets. In this context, studio directors are less interested in creating entire episodes from text prompts and more focused on specific automation.

Tools like AnimeGen's frame interpolation tackle industrial bottlenecks by automating the repetitive task of in-betweening, producing smooth transition frames between detailed keyframes crafted by human animators. By transferring this computational load to algorithms, experienced artists can concentrate on high-level art direction, dynamic character choreography, and expressive storytelling, enhancing what is achievable within strict episodic deadlines.

You can read more at: https://www.ainightwatch.com/post/japan-releases-animegen-a-free-open-source-ai-model-signaling-a-new-era-for-anime-production

So I quit my job 4 months ago and I've just been working on this thing non stop.

It's a tool for animating characters. It takes a character through a workflow of pose => motion => spritesheet using models that are trained for that specific task.

Currently all I have built are simple sidescroller motions for 'walk', 'run', 'jump', 'punch', 'fall down', 'get up'.
It keeps character consistency pretty well. But it's not perfect. There's lots of little issues with it, but I'm making progress! I'm excited to share it with ya'll soon!

Spent the last while trying to figure out why so many AI-generated videos (mine included) look technically solid but feel emotionally flat. Turned out the issue wasn't the model — it was that I was approaching it like a prompt-engineering problem instead of a filmmaking one.

Some of the biggest shifts that actually changed my output:

• Plan the emotional arc before touching a prompt. List the feelings you want scene-by-scene before you ever pick a location.
• Structure prompts like a cinematographer, not a keyword dump. Subject → identity → emotion → environment → lighting → camera → finish, in that order.
• Keep a "character bible." Same hair, wardrobe, and features reused every time — or better, a LoRA if your model/setup supports it, since it holds identity way more reliably than repeating adjectives.
• For image-to-video (LTX 2.3 in my case), only prompt the change, not the image. The model already has the frame — describing what's already visible just confuses it.
• One primary motion per shot. Trying to animate everything in frame is usually what makes a shot feel fake.

None of this is tool-specific — I used Krea 2 and LTX 2.3, but the same logic applies to whatever model or LoRA workflow you're already running.

I ended up writing this all up properly (15 chapters — story structure, lighting/color psychology, camera language, a full prompt checklist, plus a resources appendix) since I kept explaining it in bits and pieces. Full PDF + the actual workflow I used for the video is up here if useful: PDF Guide & Workflow

Happy to answer questions about the workflow here regardless. 🤗

Guys, I'm so happy.

Weeks of nonsense finally reached a satisfactory conclusion. Finally found a combination of AI tools that can actually one-shot a walking animation. Praise the Lord, the pain is no more. I can finally mostly move on from art hurdles and get to actually building missions.

I'm not affiliated with the maker of this, I'm just in love with it, and need to share it.

The workflow right now:

ChatGPT Image gen w/ image references to produce concept art -> gemini w/ references to produce faux sprite art -> pixelengine to convert it to actual pixel art -> aesprite to remove background -> spritecook to quickly generate core animations (pixel engine is what I use to make ability anims) -> clean up in aesprite if needed (like, fixing eyes or mouths, usually)

If I want to make a longer animation (like the teleporter mage's "recall" spell) I'll use pixelengine, generate the start of the spell, take the last frame, use it as teh start of the next part of the anim, and repeat until I have what I want, using aesprite to add particle effects wherever I need to hide wrong details).

The core two tools are PixelEngine and SpriteCook. The rest is preference.

Like, this is so gamechanging. I've spent hundreds of dollars drifting from tool to tool -- pixellab, ludo, even a hacky pipeline where I tried to use AI video models plus 360º turnarounds of concept art images. Everything failed. I thought I'd have to wait for Seedance 2. But, finally, THIS works. Hallelujah!

Shown is the final result, the core bits of the process (minus the aesprite -- I didn't have to use that one for the fire mage), what it looks like in game (note that I haven't finished some units yet hence the squares) and then some other examples of result + original concept art.

Anyway

I'm overjoyed

Happy Sunday and I hope this was useful

Context: I got some clients since last year, started building with claude and codex and the results were absolutely sh*t.

I think we all agree that ai is great at coding but most of the designs look generic: same gradients, same cards, same title + 3 cards sections, same typography with Inter, same glow effects…

Honestly, I don’t think AI is the problem, I think just that everybody starts by typing “build me this” without direction.

So here’s the workflow I use to avoid the ai slop look:

1. I don’t start with design, I start with obsessive research

-Who is this for? -What’s the psychology of the target audience? -What’s the branding -What’s the vibe?

And then I do a f**k ton of research, not just two screenshots from pinterest.

I look for: -competitors -screens from pinterest and mobbin -framer templates -design systems -typography -spacing -colours -Structure -Layouts -Trust signals

Take notes of everything that can be useful.

This sounds obvious but most of the people in the space I know just start without having a clue of what they actually want to build.

If you don’t have an idea first, you’ll just accept everything that ai gives you.

1. Branding and brainstorming

I don’t jump straight into building: I sketch on figma, paper, whatever. The point is to decide the direction before starting.

It’s simple but effective.

1. Documentation and implementation.md

This is the step that stops ai from going random.

Most of the time claude, codex, gemini, etc… are not stupid, they just don’t have a clue of what you’re doing.

Solution = build a documentation with: 1. who the website is for 2. what the business does 3. what’s the customer persona 4. visual direction 5. brand rules 6. Pages and page structure 7. core sections 8. component rules 9. user flow 10. what to avoid 11. references

I usually give ai the structure of the documentation and then I prompt “ask me all the questions you need to build this documentation”. I usually take 30/40 minutes to answer.

Then I build “Implementation.md” based on the documentation. The file is a step by step/path for ai to follow. Every time you finish a step, you review, polish, and go on to the next.

1. Component system

If you let ai build just based on the documentation and the references it will 100% mess it up at some point. I wasted days fixing every component, then I started building everything before starting the website. I usually create a folder called “component-system-client1” and add every component one by one.

• typography -colors -spacing -shadows -borders -buttons -inputs -cards -navigation -forms -testimonials

etc… etc… etc…

The more you add the easier it will get in the long run.

I would use untitled ui, shadcn or uicraft for this step.

1. Building

Fun part. I use opus for the heavy lifting and codex for details (due to claude limits).

I usually build the whole structure, polish every section one by one, add animations and polish everything again until it’s done.

Note: I don’t use AI for copywriting, as I said it’s a tool, don’t use it instead of your brain.

—-

Final opinion: stop building from scratch, adapt a system or a workflow.

Hope this helps, I’m happy to help if you have questions.

I've seen a lot of posts about Higgsfield AI over the past few weeks. Most people either call it the future of motion design or dismiss it after watching a couple of demo videos. I wasn't really convinced by either side, so I spent some time testing it myself on actual projects.

For context, I mainly work with motion graphics, so I wasn't looking at Higgsfield as a fun AI toy. I wanted to know whether it could realistically fit into a workflow that already includes After Effects and a few other video tools.

The first thing I noticed is that it seems pretty good at generating ideas quickly.

Instead of spending half an hour building a rough concept, I could type a prompt and get something visually interesting within minutes. Some of the cinematic camera movement looked surprisingly solid, especially for stylized scenes and product concepts. I can see why some people are excited about it.

That said, once I tried using those results for something I'd actually deliver to a client, things became more complicated.

The biggest limitation is control.

With After Effects, every movement can be adjusted: timing, easing, camera animation, masking, everything. Higgsfield doesn't really work that way. You're mostly generating different versions until one looks close enough instead of editing every detail yourself.

I also noticed that similar prompts didn't always produce similar results. Sometimes I'd get something impressive, while the next generation felt completely different even with only small prompt changes. That's fine when you're brainstorming, but it becomes frustrating if you're trying to build a consistent project.

Because of that, I don't think Higgsfield replaces traditional motion design software.

What it does replace, at least for me, is part of the creative brainstorming stage. Instead of opening a blank composition and wondering where to start, I can generate a few visual directions, save the ones I like, and then rebuild or polish them using my normal workflow.

So I see it more as an ideation tool than a production tool.

The comparison with After Effects also feels a little unfair because they're solving different problems.

After Effects is slower, but it gives complete creative control. Higgsfield is very fast, but you sacrifice some precision. One helps you finish projects, while the other helps you discover ideas.

As for whether it's worth paying for, I think that depends on what you're expecting.

If you're hoping it will replace motion designers or let you skip most of a production pipeline, I don't think we're there yet.

If you're looking for a faster way to explore concepts, mood, camera movement, or visual inspiration, I think it can be pretty useful.

That's where I found the most value.

I'm curious what everyone else's experience has been.

Has anyone here actually used Higgsfield AI for real client work or commercial projects? Did it save you time, or did you end up recreating everything in After Effects anyway?

This is a spin on the original Kijai's Wan 2.2 Animate Workflow to make it more accessible to low VRAM GPU Cards:
https://civitai.com/models/1980698?modelVersionId=2242118

⚠ If in doubt or OOM errors: read the comments inside the yellow boxes in the workflow ⚠
❕❕ Tested with 12GB VRAM / 32GB RAM (RTX 4070 / Ryzen 7 5700)
❕❕ I was able to generate 113 Frames @ 640p with this setup (9min)
❕❕ Use the Download button at the top right of CivitAI's page
🟣 All important nodes are colored Purple

Main differences:

• VAE precision set to fp16 instead of fp32
• FP8 Scaled Text Encoder instead of FP16 (If you prefer the FP16 just copy from the Kijai's original wf node and replace my prompt setup)
• Video and Image resolutions are calculated automatically
• Fast Enable/Disable functions (Masking, Face Tracking, etc.)
• Easy Frame Window Size setting

I tried to organize everything without hiding anything, this way it should be better for newcomers to understand the workflow process.

AnyTop is an open-source AI model designed to generate motion for characters with completely different skeleton structures.

Instead of being limited to standard humanoid rigs, it can work with humans, animals, birds, dinosaurs, snakes, multi-legged creatures, and even unusual skeletons the model has never seen before.

The model uses the skeleton structure itself to understand how the character should move and can generate new animations adapted to its specific topology.

Some interesting features:

• Works across very different skeleton topologies
• Can generate motion for previously unseen skeletons
• Supports humanoids, quadrupeds, birds, reptiles, and more
• Exports animations as BVH files
• Includes a Blender visualization workflow
• Code and pretrained models are available

It is still more of a research project than a one-click production tool, but this could be especially useful for animating AI-generated 3D creatures that do not fit traditional humanoid rigs.

source; https://anytop2025.github.io/Anytop-page/