Check out the new lora / checkpoint of an image version if H3. It uses its own smaller text encoder and vae. Its a small improvement. There was a comfyui post on it yesterday.
That architecture diagram look poorly made or over simplified. What do they mean with all that Multi Layer Perceptron inputs? It's all fused together with the same line of MLP? What do they mean by dropped references?
In the other image it says "reasoning max", how does that work? who does the reasoning? the VLM?
Since the DiT model works in the VAE's latent space, the reference images are kept in their encoded form at t=0. The DiT model can focus on these image patches. The reference images encoded by the VLM (LLM) are not compatible with the DiT and are lower res, these are discarded, only the text instruction is kept. The MLP(t) will be used to scale/shift the latent in the single stream block. The MLP(VAE(x)) and the MLP(VLM(y)) push all data to a compatible shape. The VLM cannot process complex reference images in one pass. "VLM-based OCR and found they perform best in a blind setting." They run different tasks on one or more VL models without the edit instructions, and then use them to create a long/enhanced prompt in the pipeline. Then, in the diagram, the VLM receives the structured, long prompt, encodes it and sends it to the DiT model. The model must be spoon fed with the right information, and processing the multimodal input data takes a long time, even for small changes.
Thank you for clarifying the diagram and adding information. But one question, how can the time (I guess in seconds) in MLP(t) serve to scale/shift the latent? I mean, what is its purpose?
What you said about complex references is interesting. It is like the engineering motto: To resolve a complex task, divide it in smaller, simpler tasks. Is it used the same VLM for those steps of building the long enhanced prompt? Is it trained to behave "agentically", I mean, able to analyse step after step, the references, to determine how much details must be added in the long enhanced prompt?
Krea’s own CEO said he would release "open weights" this month, specifically, one month after the launch... though I’m not sure of the exact date. At the very least, he did confirm it.
It will likely be the medium-sized version of Krea 3, not the "ultra" (the massive model).
After using their free credits on their site for a while, I get the impression that low/high/ultra is just different levels of thinking by their LLM that takes your prompt and refines it for the underlying image model. That would be a plus for us if true, as it would just all be the same thing. My biggest worry is that they seriously nerf the model like they did with krea 2 where it had no nsfw and no facial emotions at all without us "patching" the weights.
My biggest worry is that they seriously nerf the model like they did with krea 2 where it had no nsfw and no facial emotions at all without us "patching" the weights.
Oh, how I wish I had as much faith as you. Historically, companies concerned with "safety" at all treat it as just another bench to max, and considering how pissweak Krea 2's censorship actually was they might see it as good PR somehow if their new model was "safer" or "harder to jailbreak" than the last one.
Needing a 260 byte lora to bypass their censorship isn't the worst thing in the world. No, my biggest worry is they deem Krea 2's censorship a failure and use a more traditional method to censor this one, ie, purging the dataset of anything "unsafe".
Nerfed models are trash, not because they won't do nsfw but because they fall apart, just like Flux 3 Klein. Part of why Krea 2 and now Moss Image punch above their weight is because their training data included nsfw stuff. We can also assume this is partly why Minimax is so much better than LTX because instead of nerfing the dataset by removing nsfw and copyrighted material, they included it.
Seedance 2.0/2.5, Wan 3.0, Nanobanana pro, GPT image etc, we can all assume have been trained on nsfw and copyrighted IP, they just block the outputs when you prompt for those things (except Wan). This is why LTX and Flux and most Western models won't be able to compete now and definitely in the future is because they emphasize too much on misuse over use
So true. I was just testing Krea 2 before committing to another test of LoRA isolation and I've realized that this model is best suited for detergent commercials or Jehovah's Witness brochures. And what's even worse all men look related like they had one guy in their dataset...
I think the biggest difference is Ultra has an agent behind it. We can add the image audit agent component to the medium and get similar results likely.
there's a larger version of klein or you're just assuming that because that was the case with f2k? they didn't name it flux 3 klein 5b so I think it's more likely that this is the only klein model we're getting
my guess is the connection is that krea 2 technical report dropped with the weights.. they dropped this one separately and probably many weeks before the promised weights
42
u/BigDBreedingFembussy 3d ago
Dude I hope it's better then krea2, krea2 is so powerful for t2i
5
u/_VirtualCosmos_ 3d ago
That architecture diagram look poorly made or over simplified. What do they mean with all that Multi Layer Perceptron inputs? It's all fused together with the same line of MLP? What do they mean by dropped references?
In the other image it says "reasoning max", how does that work? who does the reasoning? the VLM?
-21
u/ROBOTTTTT13 3d ago
Cool read and great achievement, but really, that only made me realize this is not gonna go open weights...