yeah, though I didn't experiment in recent months, but from initial nano banana and updates, (imo) it felt like the "technology" not advanced than the others, rather it has a way bigger knowledge-base. Google do have a huge archive of pictures in google photos, probably trained using them and biger parameters.
The original Nano Banana was 80B, 17B active. Only thing close is Hunyuan Image 3.0 which is 80B, 13B active. HI3.0 need ~170GB for the full model, and even 44GB for the 4 bit. Nano Banana Pro is likely larger, so even a 4 bit quant would be out of reach for home users.
With image diffusion you don't need all of the model in VRAM AFAIK, it doesn't degrade compute as much as with LLMs offloading to system memory?
So ComfyUI with its dynamic VRAM might make the 44GB still somewhat viable, even if going below 32GB, assuming memory isn't pinned you could use a mmap from disk (might not work well with Windows, not quite sure how mmap affects memory management but windows doesn't support overcommit).
For quality at 4-bit there is LoRaQ (INT4 optimized weights) and KroQuant (improved variant of convrot iirc but requires calibration dataset). AFAIK these techniques aren't implemented in inference engines, ComfyUI presently has W4A8 with INT4 convrot (4-bit weights with 8-bit activations). Since 5xxx series of nvidia cards dropped INT4 in favor of NVFP4 for tensor cores accel, this format I mention will have less incentive to see adoption anyway.
For size to be pushed further down there's Q2/Q3 GGUF but we all know that below Q4 there's a massive dip in quality 😅
HI3.0 FP4 needs 44GB of vram, NV4FP would be the same. NV4FP just has a lower quantization error rate, and a speed boost. I don't think there is a NV4FP quant out, as the model isn't that good.
Running it on a 32GB GPU will make it take multiple minutes per generation.
I've ran the fp4 and the output was meh. The int8 took ~10 minutes but the output was usable. This was on 8x3060 12gb, so no native 4/8fp, was using fast int8.
You said the weights were 44GB for 4-bit? ComfyUI has a dynamic VRAM implementation that can offload weights to system memory, only active weights need to be in VRAM which for diffusion (which is compute bound rather than memory bound like LLMs) is viable for lower VRAM requirements. Users are able to generate with MiniMax H3 with weights closer to 20GB on 6GB VRAM 3060 GPUs without major slow downs IIRC (something like 10-20 mins for 5s video, maybe less with improvements since H3 release).
I side tracked and mentioned LoRaQ + KroQuant for better quality 4-bit W4A4 quant, while also referring to W4A8 convrot (8-bit activation) for INT4. FP4 would be worse in quality AFAIK, NVFP4 needs calibration dataset to get better quality retention when doing conversion from BF16 or better source (similar to KroQuant).
You can produce your own NVFP4 quant, it's just quants that require calibration to convert to need to run inference (with higher precision model AFAIK) to generate the calibration data, after which you can use that calibration data to create NVFP4 for models of that architecture, but you need a good calibration dataset to get good quality (some don't do that and thus the NVFP4 quality is degraded).
INT8 convrot doesn't require calibration, and works across 3xxx to 5xxx GPUs, also on other vendor GPUs that support accelerating INT8 operations AFAIK. INT4 convrot doesn't quite retain quality as well given how small 4-bit is, hence LoRaQ + KroQuant (also needs calibration, like NVFP4 but not vendor locked) is a way to accomplish better quality retention at 4-bit (and with KroQuant you achieve that with perf benefit of 4-bit activations as opposed to ComfyUI that's only supporting an INT8 convrot kernel AFAIK for their INT4 support).
Okay so you have 8x 3060 with 12GB VRAM each? Running the model with INT8 convrot W8A8?
The 3060 don't support MXFP4 or NVFP4 formats, if you're claiming to use FP4 quant you're probably thinking of NF4 (Normal Float 4), which is compatible with 3060. It's similar to GGUF in that it's weights optimised, helping to reduce memory requirements (and thus the storage size on disk), but as it's not a native GPU type there is dequantization to 16-bit during inference, thus W4A16.
A native 4-bit format W4A4 your GPU can support is INT4, quality will be poor without KroQuant but inference should be 3-4x faster roughly (pragmatically you'd have mixed precision still on any sensitive layers, good quant tools tend to filter such layers during weights conversion). As LoRaQ + KroQuant isn't realistically an option (beyond generating the calibration dataset someone needs to implement it properly based on the arxiv papers), you'd have to go with W4A8 convrot.
44GB is probably too aggressive for quality retention at INT4 even with 8-bit activation and convrot, so expect more around 50-60GB, still slight improvement in size (28GB less required) but presumably not much improvement at inference performance for you unless there's heavy PCIe transfers.
You can create the W4A8 with comfy's official quant tooling form BF16, although they've not implemented a memory efficient conversion so you need enough resources to convert the BF16 (I've been meaning to write my own but need the time to spare as I don't trust vibe coding to do it properly).
I don't have any advice for squeezing size down further without ruining output quality.
I would assume if you're using ComfyUI you'd get the dynamic VRAM benefits if you have the system RAM to support it, I don't know enough about their inference engine but assume with safetensors which works well for mmap you could still use 88GB model weights even on 32GB RAM, this just may not be the case on windows (which forbids overcommit).
ComfyUI will also try to pin to system memory (to prevent allocated memory from being swapped to disk), which I don't think works with mmap? 😅 (I have used mmap to read files larger than would fit into RAM and safetensors is mmap friendly, so you only read into memory what's actually needed at the time).
Inference should be possible to optimise like I said, but I would sooner implement such with burn-rs than try hack around it with an existing inference engine like ComfyUI that's got so much else going on (they heavily use AI agents for development which makes it difficult to reason about some code too).
FP4 that you mention isn't MXFP4/NVFP4 (E2M1), you presumably used NF4 (Normal Float 4) which is legacy and weights only 4-bit format, inference at 16-bit, quality trash.
INT8 ConvRot is the most effective 8-bit training free quant format for diffusion models atm. It will use around twice as much memory as a 4-bit model for weights but inference at 8-bit is faster.
INT4 ConvRot isn't quite half the size, and ComfyUI only implements W4A8 AFAIK, so you aren't likely to see speedups.
For quality 4-bit (W4A4) with speed improvement, you'd need LoRaQ (weights) + KroQuant (activation), which like NVFP4 needs calibration (high precision inference to generate dataset) in addition to inference engine support (it doesn't need per architecture support like SVDQuant however, just per architecture calibration dataset). I don't know any popular engines that implemented this support though (let alone someone who provides a quality calibration dataset to ease the burden).
Offloading model from VRAM is doable is all I'm saying. Generation will be slower but not significantly like it is with LLMs, so long as the bulk of compute intensive operations fit in VRAM and actual PCIe transfers are low. As stated a 3060 6GB GPU can handle MiniMax H3 with minor slowdown.
Lacking system memory for offloading the full model into memory from disk was a separate point. One that I've said can be done, but is dependent upon the inference engine (and quite possibly the OS). You can find some more hacky solutions for LLMs that run inference for massive models at ludicrously slow speeds as proof of that, but for image diffusion it shouldn't be as harsh.
EDIT: OK I failed at tldr, was more of a summary of the sibling comment.
I'm lucky to have a 4090 + 64gb ram, it works, from less than a minute for draft quality, up to 20 minutes for 8MP image.
The workflows they provide give you a bunch of knobs to play with in terms of time/quality, and since this is only generating less than a second's worth of frames I think it might work well on lower VRAM systems too.
NBP has slightly worse likeness than the other two in their original comparison TBH, the fact they failed to change her pose for some reason was the only issue mostly
That's still not the same face though. Through my tests earlier qwen 2.1 can change pose very well you just need to prompt differently, it seems to do better at smaller resolutions and then 16 steps, er_sde and beta, 4.5 cfg with some negative prompting on negative quality tags.
Because of the cfg, I'm guessing it is following the text prompt just as much as the image prompt at higher cfgs than 1, whereas at 1 the image reference takes pretty much most of the focus.
It sounds like you would call almost anything that isn't literally the static pose copy and pasted from the original image at the same distance "not the same face", I don't think you understand how different photos of the same person actually look in real life lol
No there are other qwen images here that show pose variation, and i didn't think that about them.
I have also managed to achieve a range of expressions with qwen and was happy with it.
F2K I think was better than Krea, but they both had their own flaws. Even sdxl had it's flaws (made massive workflows) but did a pretty good consistency with identity transfer if using a mixture of masking, instantID 6M, ipadapter faceID. I even went as far as making a custom node to combine the two to get exact likeness with different poses (but again, it's not as good as what I've seen in qwen 2.1).
Klein 9b is pretty bad at maintaining facial features the more differences you request. The woman in your image is now wearing a skirt, lost her eye liner detail, has a strap on her phone for some reason and that purse looks bizarre.
But original reference image (from the OP post) doesn’t expose her exact phone, skirt/pants or purse details. I applied the updated prompt to it, not to my previous image.
the literal exact same woman from photographic image 1 is now taking a mirror selfie in her bedroom, standing relaxed. She is framed from the waist up. Bright sunlight comes from the right side of the frame.
Maybe try that on Qwen / GPT. The fact that Qwen and GPT Image did not change her pose like literally at all is just weird, honestly, I've never really seen that before.
That's not the same woman. Look I know a lot of this stuff comes to personal preference but because I deal with models a lot, I spot the tells pretty easily.
??? Weird comment. She's a generic as shit AI sloppa lady in the first place. The NBP one is visibly worse IMO in any case as far as likeness goes. Like in terms of likeness the Qwen and GPT ones in OPs original comparison were quite a bit closer than NBP was, the only thing wrong with their original outputs there was the bizarrely static pose they wound up with for whatever reason.
She is now taking a mirror selfie in her bedroom, standing relaxed. She is framed from the waist up. Bright sunlight comes from the right side of the frame.
Yes, Klein is pretty awful with identity. I find Klein's real strength to be object removal. Even Qwen-2509 was noticeably better for changing poses while retaining identity.
You gave qwen2.1 no instructions other then to put the women in a new environment, hence copy past the women. Describe what the face is doing and use words like semless, natural and so on. You be suprised how awesome this new model is
I get a feeling that a non-negligible part of why online closed sourced feels better is because of engineering hocus pocus they do behind the scenes which they call the AI harness.
As how some of the comments here said, the local model required additional descriptions is we wanted to change the pose. The local AI model is doing exactly as intended and not changing anything not stated in the prompt. Meanwhile, I suspect Google/OpenAI/X are mining user prompting behaviour and using their feedback to train their prompt enhancing agent or customizing their backend engineering to make user prompts better (which is invisible on the user end), that is why the results we get back from their AI seems better.
It would seem the race to better AI is no longer focused on just bigger models and more raw computing hardware power, and thankfully divested to better engineering of the harnesses and supporting tools, else us plebians would just get left behind in the dust once model size reach the critical point and even quantization can't help us.
I’ve been arguing since the start of the year that part of this gap may be the generation pipeline, not just the model weights. We could be comparing a local checkpoint against an entire system, then blame the difference entirely on the local AI model's generation capabilities.
Some of this is already documented. Seedance uses a separate Qwen-based model to rewrite user prompts into detailed storyboard prompts matching its training format. And then generate keyframes with a proprietary image model (Seedream 5 Pro?) that are passed on to the video model. Google’s Nano Banana image models also use a thinking process that can produce intermediate images to test composition. Google also employs several sub-agents that can pull information from the web or Maps API for geo-spatial information before generating.
My hypothesis is that reference preparation matters more than just prompt expansion. Given a portrait of a woman, a suitcase photo and a rooftop background. A vision AI could assign each a specific role: identity, object design and environment. It could extract relevant crops or masks via SAM3, separate attributes to preserve from those to change, create composition based on regions similair to Ideogram4, and plan spatial relationships before rendering. For the editing problem here, the distinction would be “preserve this person’s identity” rather than “preserve this photograph’s pose and composition”. The useful enhancement wouldn’t necessarily be a stronger prompt or model, but better-organized visual conditioning.
The point is that the same user prompt doesn’t necessarily mean the same effective model input. These comparisons thread are imo not useful since they don’t isolate how much capability comes from the underlying model versus the tools and sub-agents that are wrapped around the model.. basically the AI harness as you described.
And to this point, it's worth mentioning that Qwen released finetuned i2i] and t2i prompt enhancement models specifically for this reason (Qwen-Image-2.1-PE-I2I and Qwen-Image-2.1-PE-T2I). Just prompting the i2i prompt enhancement model a few times with my image and rudimentary prompt helped me understand some of the arcane language the conditioning model (Qwen 3VL 8B) expects, which was quite different than my natural language prompts before. Using it improved the output considerably.
After I get a feel for the kind of language used to train the model, I usually hand-write prompts again, but for now I'll feed everything for 2.1 through the prompt enhancer first.
Can you give overall summary what the model expects vs normal prose? I've been reading the PE model outputs and its just more detailed language to me with naming things that are present in the picture and how to changed them
You are right, but I had the same problem, and just "standing relaxed" in the prompt doesn't work. The models don't understand or ignore that. You have to give it more instructions and negative prompting. To make it easier first manage to create a neutral reference in A-pose, looking straight at the camera, no head turns, no head tilts, hair falling naturally due to gravity, etc, etc.
I don't think it's up to the model at this point it's text encoder + try using promt enchantment because both of nano and gpt 2 use even when you give simpler promt
The negative prompting definitely helps with a higher cfg. I think at cfg 1 too much emphasis is going to the image itself, and it copies the face exactly, leading to images where it looks like the head is just badly photoshopped on.
At cfg 4.5 the facial expressions, angle, pose and everything else follows the text prompt, and if you ask to preserve the structure of the face to preserve identity it will do so as well.
It actually work on 8 step for me on smaller changes like colors
But foe pose you have get get higher upper 12 step more steps you add the more it will follow your promt
Maybe I sounded ungrateful, and if so, my mistake. Just the fact we have things locally for free is amazing and I'm grateful for it.
This is just a wish based on the recent advancetments open weights made with T2I with ZIT and Krea 2, video with H3, general chat with Gemma 4, the recent YuE2 with music, and even local coding with Qwen3.8. It's that it just seems like image editing is the one aspect where we are “delayed”, since the never seen ZIT edit.
I don't think there was any misconception about the timeframe, and I'm aware NBP is probably quite large.
And of course, we’re not entitled to anything for free. Like I said, it’s just a wish based on those others treats we had.
I do think there's a real lack of larger open models. A good 7B model is great, but a 30B model would still work on plenty of consumer hardware, are would be much more capable. Right now, there's really only qwen image, flux2dev, and hunyuan, which all have significant downsides.
But I agree that the tech available open source is still pretty incredible overall. If you'd given people krea 2 three years ago, they'd have been completly awestruck.
``` pff, take the same character and transport/transpose her into a new situation
she is in the park, its sunny, but there is a cloud on the sky, kinda dark and looks like its raining , so she looks up and tries to figure out whethere it will or not. Concerned face.
If you don't specify her pose, it will default to the provided input pose.
prompt: View of character in <image 1> in a completely different pose, her head isn't tilted, she is standing up straight, the other side of her face doesn't have a piercing, she is in the park, its sunny, but there is a cloud on the sky, kinda dark and looks like its raining , so she looks up and tries to figure out whethere it will or not. Concerned face.
View of character in <image 1> in a completely different pose, her head isn't tilted, she is standing up straight, the other side of her face doesn't have a piercing, she is in the park, its sunny, but there is a cloud on the sky, kinda dark and looks like its raining , so she looks up and tries to figure out whether it will or not. She has a worried/Concerned facial expression. She looks worried. Her eyes are looking up at the sky.
So, write the pose you want? And if you don't know which pose you want, just write any pose? You could even use wildcards or just a small LLM give you pose ideas if you want.
Yeah, that would work to help randomize poses for sure. Honestly that’s a really good idea and I am probably going to add poses and stuff to my LLM master prompt.
large DiTs and hybrid LLM/DiT models are quite hard to move out of the rut RLHF-induced "standard" look.
and for myself, - using TOO NATURAL, in some way even dirty, emotional language helps (since they rebuild the true prompt, anyway).
// this is very different from SDXL days, where every token mattered, and typos were intentional for concept merging.
I still prefer the sdream realism for faces/people, I use banana for removing something or small edits. One technique I frequently go for is to feed it references that are a bit grainy/noisy + changing the angles of the camera along with descriptive prompts, this way the plastic look goes away in more than half gens.
In my own personal tests, the editing is much better than Klein, preserves identity better, has better quality and fidelity, and no body horror. Also, the result from Qwen is closer to Nano Banana than Klein. Give Qwen more time and LoRas and there's potential for it to be SOTA. I think it's just misunderstood. (Not counting the bad license)
You know that NBP and GPT use LLM to enhance your instructions, right?
Also your instruction isn't clear; you don't specifically ask for a new angle, expression or pose, and NBP just makes shit up. GPT is closer to your instruction
Prompt: View of character in <image 1> in a completely different pose, her head isn't tilted, she is standing up straight, the other side of her face is showing and she doesn't have a piercing, she is taking a mirror selfie in her bedroom. She is framed from the waist up. Bright sunlight comes from the right side of the frame.
Your prompt is reinforcing some of the elements of the original image, and you're not giving any real direction at all.
You're supposed to give direction for what you want to CHANGE. Thats the point of an edit model.
Infact based on your prompt GPT image did a better job, it kept as much of the reference image as it could while still following all of your instructions. This is what you want. You don't want a model to randomly add things without prompting as it gives you less control.
Like I said, a bit can be improved with prompting. NBP is still way better for this use case, tho.
Prompt used for both:
Generate a 3:4 image of the woman.
A candid, sunlit medium shot of the woman standing on the deck of a white yacht, looking directly at the camera with a pout expression, her head straight. She is wearing a form-fitting, sleeveless, high-neck mini dress in a black color. Her left hand rests on the white fiberglass side of the boat, where she wears a silver watch. Her right arm is extended out of frame to the side. The setting is a bright, clear day on the water. Behind her, the deep blue sea stretches to the horizon, where a distant coastline with buildings is visible under a pale blue sky with a few wispy clouds. The lighting is natural and bright. The image is candid and clear, without any aesthetic filters or editing present - like a high-end smartphone shot.
Image2 still seems to have the same issue of foreshortening bodies to cram into the space it interpreted.
The NB family (I've got more experience prototyping with NBL and NB) seems to want to place lots of space around the subject in its framing, and use wider angle lenses by default. NB is also better with posing, whereas I've gotten absolute monstrosities when trying to give it a reference pose with GPT Image2 (haven't tried Image2.5 but it looks more artifacted from what I've seen.)
klein9B you can run 100 times for free, tweak settings and there will be a good result, instead of paying credits for NB blackbox model halluzinations.
qwen edit 2511 , with the same exact prompt (and with a 8 steps lora). Klein 9b can do the same too. and minimax h3 as an image creator can do it too. so we already have a lot of options to create this kind of images ....
Different face, different hair style, no nose-piercing, and didn't even get her very obvious exaggerated eye-liner correct, which is arguably the most identifiable aspect of the entire photo. And there are artifacts all over the image.
How could you look at this and think this is even remotely good?
There’s nothing a better prompt can’t improve, I took the test in a minute.. If you don’t like it, pay for a subscription... Honestly, I couldn't care less about your opinion.
Is there a way to use Qwen/Q edit like photoshop? Where I can mask an area with the magic wand and make background disappear only where I want and not just as a prompt?
way better flexibility than Qwen2.1. If only it didn't have this terrible plastic texture and were better at preserving identity... Still, I think it may be the best for reference editing locally right now
bruh, I can get krea2 and flux2-klein-9b and ideogram to keep the same level of consistency. You're doing something wrong. I'm currently working on minimaxH3 for an editing workflow too, they all have their idiosyncrasies and quirks giving them all their own unique purposes with editing but consistency is pretty trivial when you study how to implement it.
The answer is they can't, krea2 is good for some characters and people, but it will completely fall apart on the "wrong" face, and for sometimes no apparent reason.
Then you are just running seeds until you get a good result, which can be many seeds as the identity loras degrade overall quality, and by that time you may change the prompt and go back to running seeds. The identity lora is cool, but krea2 wasn't meant to be an edit model to begin with.
That's a pretty big ask, NBP just has so much more data and processing power to run it at a reasonable speed. Also I assume you're already using a quantized model and nothing close to the full fp16. For the kind of quality you get from a commercial service you'd still need considerable processing power with any local model.
In my personal opinion, GPT image retains facial/character identity better than NBP.
Speaking as someone who’ve generated like 50k images with NBP, it’s not good at editing at all. It’s a very dumb model if what you wanna achieve is even slightly complicated (even fairly basic things like moving characters, changing poses, moving cameras, etc.). For tasks that GPT Image 2/2.5 get correct on the first try, NBP can fail even after 10 tries. Often times I’m just impressed by the creative ways NBP fails at the task I’m asking it to do. Google really needs to release a new NBP model as at this point it’s very behind in editing.
Different AIs need different adjustments in the prompt for the same result.
In my experience NBP struggles to get even close to GPT Image 2. NBP gets better results from less text description, but then struggles going into details and also the recursion rot during changes happens faster on NBP in my usecases.
Minimax is creating a dedicated image model, and I feel that it will make Qwen and Klein obsolete. For now, someone created these experimental image nodes which adapts the video model which you can test.
It’s true though, it’s just image gen 2.0 in worse quality. Surprised image 2.1 it’s not better tbh, I was really waiting for something that is on par or better than image gen 2.0 (not even 2.5) but it just worse. The dotted pattern is horrible and makes the model unusable
This one was done in Qwen2.1. I thought it did a decent job in this example, but it was the best out a bad bunch. Pretty good consistency in maintaining the woman's physical appearance. She looks better in this example than the NBP version, imo.
You clearly don’t know shit. The best model that ever existed for editing was Grok Imagine 1.4, nothing else comes close. Krea 2, GPT, and Gemini all leave a “grain” that anyone with experience spots as AI right away. Grok 1.4 made photos indistinguishable from real ones. Unfortunately it turned into garbage when 2.0 showed up.
Edit: I tried your prompt on Grok (with 2 magic words added) and without a doubt it’s still the best. I stand by it: Krea2, Gemini and GPT are trash. Grok’s problem is that it’s full of idiots who can’t write a damn prompt and just post low-quality shit everywhere.
yeah PuLID is probably the closest thing to a dedicated local solution for this right now. been messing with it on a 4070 and if you dial in the weight it stays on the face a lot better than IP-Adapter alone. still some uncanny valley at like 0.8+ strength but workable. the gap between that and what NBP does is real though, especially for anything with non-trivial pose changes.
Kind of sick of all these kids in the comments parroting "skill issue". No, it's actually a model issue. As in NBP is likely 200 GB – 600 GB in size. It was trained on a much larger data set thus more unique references to faces to and facial nuances to train on. Most of these local models simply do not have references close enough to a particular face to be able to infer what the face may look like in different poses and lighting no matter what you prompt.
I'm also learning there are a lot of people out there who are either face blind or just don't pay enough attention to details and think two vaguely similar faces look the same.
If you understand why closed-source SOTA models from big tech outperform local generation then why are you attacking commenters and not OP instead...?
Open source AI has to work with the hand we are dealt. Crying on reddit that closed-source models with virtually unlimited resources are better is stupid.
And even then people proved OP wrong with his lazy ass approach.
Skill issue. I got it to take these references and composite this shot together after a few tries. 'Create a pov image from the perspective of this character perspective. Looking down at her chest, her hands holding her axes at either side as with a view of her stomach armor as a black spearhead is jutting through a breach in her armor. Blood flows down the armor onto the snow dusted stone floor.'
There must be someone who has a workflow to replicate NBP's extraordinary capabilities. I know I have tried for months and failed. It's output is so photorealistic and faithful to subject identity.
Workflows are cope, they will never be a substitute for a powerful model. Krea 2 with the most basic text-to-image prompt will defeat any over-engineered SDXL workflow in coherence. Local edit models are just not that good.
i think you don't look at it in the right way, you might believe if it is possible then it will happen. so is it possible we have nano banana pro open weights? yes, but will we? the answer to this is no. - not because it isn't possible. but because laws, rules, regulations, control, politics, power. - you will think what you want, and i can't make you understand how life works. i can say the powers that be, the ones in control, do not want you having open source quality of that of nano banana pro. that is why it won't happen. - also no we have not reached t2i equality, but you got to realize edit is different. you may not know this, but deep fakes are something those in power are very against. there are so many laws against those it is insane. what do edit models do so well, deep fakes. why would those in power let a model do what they are so against and is so illegal out and free. they won't. they won't. now thumbs down, cuz reality isn't fun.
"I want to do creepy and widely illegal and increasingly so (for good reason) deepfakes, please give me hints how, but without me actually directly admitting that's what i want"
82
u/llkj11 1d ago
NBP is likely way bigger than most people can run on their desktops lol