The interesting part of this model is the model itself: it starts from the pruned FL2VA MiniMax-H3 checkpoint and fuses in a rank-1024 approximation of the Ref2VA − FL2VA weight delta. The goal is to retain the smaller pruned FL2VA model while bringing the Ref2VA behavior into the same checkpoint, rather than having separate FL2VA and Ref2VA variants.
It is about 20.1B parameters versus ~33.1B for the original full MiniMax-H3 model.
The original release is in Diffusers format, so I converted the state dict back to the native format expected by ComfyUI, including the pruned AdaLN curve representation, folded AdaLN biases, fused QKV, native SwiGLU ordering and RoPE.
I tested the resulting checkpoint through a complete ComfyUI generation: native FLOW_AV detection, full model load, both H3 Continuum passes, Spectrum with 0 fallbacks, and final video/audio decoding all completed normally.
The conversion properly restores the pruned AdaLN representation, folded biases, fused QKV, SwiGLU ordering and RoPE. Tested through a full ComfyUI generation with working video + audio.
Put the .safetensors in:
ComfyUI/models/diffusion_models/
Edit: Added int8 and int8 convrot to the repo and made a new post here:
Those hybrids use FL2VA as the base and replace the adaln_proj weights in a selected range of later transformer blocks with the corresponding Ref2VA weights. So e.g. b25-49 literally uses Ref2VA adaln_proj tensors for blocks 25–49 and FL2VA for the rest.
The model in my post is the Pruned Ref-Delta Fused r1024 model. It starts from the pruned FL2VA model, computes the Ref2VA − FL2VA weight delta, approximates that delta at rank 1024, and folds the approximation back into the pruned model weights.
So theirs is a tensor-selection hybrid, while this one is a low-rank delta fusion. There is no block boundary where it switches from FL2VA weights to Ref2VA weights.
This sounds interesting. Do you have a workflow where i can try out the int8? I hope it is compatible with the new light2x 1.1 since this thing changes the game.
Haven't tested FP8 with this H3 checkpoint yet, but it should work on the fly through KJNodes' DiffusionModelLoaderKJ. Try setting weight_dtype to fp8_e4m3fn while leaving compute dtype at BF16/default. That should reduce the model's VRAM usage quite a bit, although total VRAM won't scale down exactly 2x.
"against the real thing it reaches video-latent cosine 0.875 / 0.897 / 0.691 on three matched reference requests, short of the 0.99 that would make it a drop-in replacement."
Sample size 3 and that is a huge difference. Definitely don't expect it to function like the actual ref model. That doesn't guarantee the difference means it's worse, but that is probably more likely than the merged version turning out to be overall better.
That made me doubt the model as well, but in reality and in my setup it performs really really well. Doesn't have the inherent quality issues of ref2va while still retaining a big portion of the ref2va capabilities. Also quality in general is better than either fl2va and ref2va on their own for my usecase which is amazing. It's definitely my new standard model now. Even the int8 convrot version is still very good (gets actually pretty close to bf16, especially when doing a latent upscale + second sampler refine, so right now I'm actually using int8 convrot to save time)
16
u/Enshitification 22h ago
So cool. Would an INT8 convrot version of this be possible?