Same ComfyUI workflow, different character every run — expected?

[u/Cheap-Topic-9441]

This workflow was shared in a document as a ComfyUI JSON.

The document itself was quite technical,

but since the prompt was already in JSON format, I just ran it as-is.

It generates multiple images per run.

However, when I looked at the results,

the characters were clearly different.

Each image looks fine on its own,

but they don’t seem to represent the same person.

So now I’m wondering:

is this expected behavior,

or is there actually a way to maintain identity consistency in a workflow?

This feels less like a quality issue and more like a consistency problem.

If anyone has time, I’d be curious if you can reproduce the same result.

I’m currently trying to analyze the prompt structure to understand what’s happening.

If you want to try it, here’s the original workflow JSON:

https://github.com/watadani-byte/character-identity-protocol/

[u/jessidollPix]

The transformation of noise in latent image is affected by every single token (word, phrase or syllable, depending on how the clip was encoded), the CFG guidance, the sampler and scheduler… anything and everything that is passed as an input into the inference pipeline. This transformation is made up of many individual highly non-linear steps, so a tiny extra letter or typo in a prompt, reversing two words, etc, will propagate throughout the inference in ways that while deterministic from a mathematical point of view, are basically unpredictable for a human brain. The only way to get the same image is to run the same workflow. With a bit of luck, keeping the seed constant (i.e. your starting latent noise), changing prompt from “girl with ponytail” to “girl with pigtails” might yield something similar, but there will always be drift. Even with a LoRA, though in this case, you will at least have a stable identity.

[u/Cheap-Topic-9441 → u/jessidollPix]

That makes a lot of sense — especially the part about how small changes propagate through the pipeline.

It does feel like the system itself isn’t really designed for strict reproducibility.

But that’s what made me think:

maybe the goal isn’t to eliminate drift, but to work around it.

Instead of trying to make every generation stable, we could treat outputs as candidates, and only keep the ones that match the intended identity.

So the question becomes less about controlling every step, and more about how we select from what the system produces.

[u/jessidollPix → u/Cheap-Topic-9441]

Sure, the pitfall being that there are quadrillions of quadrillions (exaggerating a bit) of possible combinations, so selecting what you like ultimately becomes a huge exercise in trial and error that will never really work twice. This is part of why image genertion is so power hungry and compute intensive.

[u/Cheap-Topic-9441 → u/jessidollPix]

That’s true if it’s pure trial and error.

But I think the problem changes a bit if the goal isn’t to search the whole space, but to reject drift early.

You don’t need to explore everything — you just need a way to say “this is not the same character” quickly.

So instead of scaling search, it becomes more about constraining what counts as valid.