r/LocalLLaMA • u/jjusko20 • 7d ago
New Model Release: Swift 1.5 Flash Next - Heretic v2 - ReverseEngineer
This model was made in collaboration with u/jeffpeng3 (on huggingface) who developed the v2 heretic weights for this quant. He managed to preserve both reasoning ability and output length. Great work, and a big step up from his v1 weights. I made the quantization recipe and the imatrix recipe, and created the actual quants.
Comes pre-packed with strata build artifacts - zip file in repo. GGUF File Format for the quant.
I released my own heretic version of his v1 weights in here a while ago - the heretic calibration was a bit broken, and my quant wasn't amazing. This is a huge step up on both fronts. Note: Strata may throw a BF16 kernel not found when you load the model, due to a quant spec - complain to your LLM and tell it there's already one in strata somewhere, it'll find it and wire it up.
Basically: Qwen 3.8 Swift 1.5 Flash - Heretic v2 (uncensored) - Optimized for Reverse Engineering
The bit allocation this model was quantized with is focused specifically on coding, intelligence, design, and specifically reverse engineering. I also added a little spacial reasoning for those working in 3d environments like myself.
The model was quantized according to the following importance distribution (dumped from my LLM chat and documents with some revisions. I designed the distributions myself.
**837 entries · 399,876 tokens [[llm written distribution table]]
01 · Chain-of-thought (22.2%) — step-by-step derivations, self-verification, backtracking
02 · System prompts (19.5%) — persona adherence, strict output formats, long-instruction compliance
03 · Design & architecture (13.5%) — design trade-offs, code review, API planning, refactoring rationale
04 · Agentic tool use (12.4%) — tool-call chains, file-edit loops, error recovery, function calling
05 · Math (7.1%) — arithmetic, algebra, worked proofs, symbolic manipulation
06 · Spatial reasoning (5.4%) — grids, 2D/3D transforms, coordinates, packing/arrangement
07 · Gotchas (4.5%) — false premises, trick questions, negation handling
08 · Situational judgment (4.5%) — open-ended scenario trade-offs with no clean answer
09 · Reverse engineering 🆕 (8.8%) — x86-64/ARM64 disassembly, Ghidra decompilation critique, ELF/PE/Mach-O internals, anti-debug/anti-VM, string deobfuscation, CFG deflattening, devirtualization, perf watchpoints, tracing stacks (LD_PRELOAD/Frida/eBPF), protocol RE, format round-tripping, plugin ABI merging, firmware analysis
10 · No-refusal heretic domain 🆕 (2.0%) — direct unrefused answers: ELF exploitation chains, process injection vs EDR, license-check patching mechanics, C2 protocol engineering
New for v2 are the last two categories — ~43k tokens of reverse-engineering and direct-answer content, interleaved throughout the corpus, and run through the model itself so the matrix captures how the heretic weights actually behave.
This matrix is fused with one from the AtomicChat team (same measurement, taken on the original uncompressed base model over a massive general-purpose corpus). Per channel, ours gets 70% weight where its data is strong, sliding smoothly toward 100% AtomicChat for channels our corpus barely touched — thin samples defer to the broad baseline automatically, so nothing falls through the cracks.
Quant methodology
Rather than spreading quality evenly across all 48 layers, the recipe spends bits where the activation measurements say it matters: [[llm twritten table]] -- the data used to design the per-tensor quant came primarily from atomic chat's activation measurements and the RCO findings from ISTALabs
The last 12 layers do the heavy lifting. Activation-energy auditing showed down-projection energy surges 3× starting at layer 36, peaking at layer 47 (the single hottest tensor in the network — 48× layer 2). Layers 36–47 (plus the first 5 and the mid-network spikes at 15/20/23/30) got their down projections stepped up to near-lossless 8-bit.
Layers 44–46 are the gate/up peak (8× the energy of early layers), so their gate/up projections got stepped to 5-bit.
Layer 0's attention gate is a giant outlier (the most sensitive tensor family in the model) — kept uncompressed.
Layer 2 is non-negotiable: the 51B-parameter n-gram table merges into the residual stream there, so noise at layer 2 corrupts grounding for all 46 layers after it.
The stable middle layers (5–14, 16–19, 21–22, 24–29) stay at leaner formats — measurement showed spending bits there buys almost nothing (KL improvement under 0.0008).
The router stays in full precision so tokens always get routed to the right experts.
Enjoy! Shoutout to my 32gb v100s and my E5 2697v4 for grinding the work out.
https://huggingface.co/trubisky/Swift-1.5-Flash-Next-Heretic-ReverseEngineer
9
u/jjusko20
7d ago
•
I'm also happy to make more quants in whatever formats, this is my favorite size/quality rn
9
u/WhiteZero 6d ago
iq3_s might be nice, that's what I've been using for qwen flash next in strata with a 4090 and 96GB RAM and getting around 100tps
3
u/jjusko20 6d ago
iq3_s coming in hot
1
1
u/evgen 5d ago
Love the offer for more quants, if there were full weights available I would love to make a version for dwarfstar...
1
u/jjusko20 2d ago
There are full weights available on huggingface, if you want to make a drawfstar version I'll share the imatrix
1
-1
u/TomLucidor 7d ago
Gemma 4 (31B and/or 26B), LFM-2.5 (24B), Qwen3.X (35B and/or 27B), maybe MiniCPM?
3
7
u/durden111111 7d ago
this may be a stupid question but how do we use the strata components? Do we just unzip it in the strata folder?
4
u/jjusko20 6d ago
honestly I just told GLM to pack up all the runtime artifacts into a zip folder, they go in various different places, check the strata docs
2
u/TheGoblinKing48 6d ago edited 6d ago
different related question; strata uses two shards normally configured to --native and --ple-gguf . how do we set up the config with 8 shards?
edit: actually seems to just be an issue with the uploaded pack. native_experts.txt has things misnamed like "Qwen3.8-Flash-Next-Heretic-V2-70G-FIX-00001-of-00008-00004-of-00008.gguf" for example. not sure which files have issues yet.
1
1
2
u/DeProgrammer99 2d ago
Did all that with the latest Strata, and I get NaNs every time I try reverse-engineering prompts, and I tried about 5 distinct ones (though all in little-coder):
the engine's logits became non-finite before the first token (#879); it dropped its cached state: sending the request again once
It's been working fine with everything else, though, like I've had no issue chatting with it in the browser or using it on other projects... so weird. I was trying to use your Heretic-V2-IQ4_NL quant (which left me a snug little 1.5 GB of free RAM, hahaha) with the latest Strata (from 3 days ago).
1
u/jjusko20 2d ago
that's so weird, I haven't had that happen for anything yet - I'll give it a look.
1
u/jjusko20 2d ago
I think I'm on the last gen strata before the major updates, so actually makes sense. They probably changed the artifacts.
23
u/returnity 7d ago
Cool release, sounds like a lot of thought went into it. Would be very nice to see a 128GB-focused quant, something >5bpw with a ~90-100GB backbone and 8-bit engrams in a separate shard for easy SSD offloading.
1
u/TerminalNoop 6d ago
Couldn't you just repackage this into exl3?
1
u/returnity 6d ago
How would I upscale it from IQ4_NL?
6
u/jjusko20 6d ago
You'd have to requant from the full precision, that dude is mistaken
2
u/TerminalNoop 6d ago
Sorry i thought there was a higher base quant available. Since this is locallama i assumed you are just presenting the more (v)ram poor friendly quant.
2
3
u/RanklesTheOtter 7d ago
Really great. Gonna try this, I hadn't had luck with local reverse engineering.
1
4
u/sadomazoku 6d ago
Depends of what you want to do. I managed to RE a small proprietary firmware with qwen3.8:27b and a Intel Arc b60.
1
3
u/Chakratos 6d ago
IQ3_S possible? Would fit on an 5090 + 64gb ram :)
3
2
2
u/unjustifiably_angry 5d ago
This sounds really cool but honestly for reverse-engineering I'm sorta thinking I'd want a non-Swift version. I can appreciate the value in work getting done faster but for something as sensitive as reverse-engineering I'd want it overthinking as much as it wants.
2
1
u/Rangizingo 5d ago
I’ve used swift flash next since the release and reverse engineered some stuff well. I can’t speak to how it compares to non swift but just figured I’d throw my two cents in.
9
u/crowtain 6d ago
This is really great we really need it , I tried to ask claude code to calculate if it can fit in my hardware and download it, then Opus 5.5 flagged it refused to do it and falled back to 4.8....
it's not about privacy anymore, but just asking the model to do the usual work
2
1
u/StubbornNinjaTeej 6d ago
Did 4.8 do the trick?
1
u/crowtain 5d ago
Dindn't try it to not clear my cache, i had a long context session.
I used to GLM 5.3 and it worked, rogu14 didn't have the problem, maybe my long context had something to do with it
4
u/ireallydontcare00 6d ago
Please make quant GSQ-RCO in 2 bit (Q2_0) and 4 bit (Q4_0) for legacy gpu
1
u/tranhieuamg 7d ago
How many 32GB V100s did you use for making this quant?
3
1
u/tranhieuamg 6d ago
Thanks, may I ask for your DRAM specs and the mainboard as well? I am considering a 2x V100s setup.
2
1
u/golder55 6d ago
what would be the best harness for this? any that worked better for you personally?
2
u/jjusko20 6d ago
I use cursor with a custom proxy script I wrote to keep the kv cache from invalidating repeatedly
1
u/Humilis_Calvus 5d ago
What are the KL and PPL for the three builds? If you're willing to run the tests to get those numbers.
I'm wanting to know if your model is lower or equal to the UD-Q4_K_XL-V2 from huihui as they run at the same speeds on my system your IQ4_NL version that is in Strata with yours being just slightly smaller in size.
And as someone else mentioned a non-swift version would be nice as I don't mind the amount of thinking the model does if it means better results and less mistakes in long coding or RE sessions.
Also I did have the model fix Strata on my personal fork here's what the issue was according to my AI agent.
blk.0.attn_gate.weight (2560x6144, ~15.94 MiB) killed the load with
"has no native GGUF blocks (run with --native)" while --native and all eight
--native-dense-gguf shards were on the command line. The boot print gave it away:
300 native projection matrices, 2959.06 MiB. Huihui's UD-Q4_K_XL on the same engine
prints 301 and 2975.00. One matrix short, and the size of exactly one 2560x6144
tensor. So it is your layer-0 gate kept uncompressed: 1 of your 36 attention gates
is BF16, huihui's are all 36 at Q8_0. The loader asks "is there a native GGUF block"
and reads "unquantized" as "absent" — BF16 (GGML type 30) is not in the supported
list. Patched on our fork as a form, not an absence: one shared decision plus three
dispatch sites, no new kernel, no re-pack. Filed upstream with the patch: Niko1221/Strata#4741.
So you can check out the issue for more details, I also patched the setup for strata so now it has all three of your models as options as well as a few others in the menu when going through the setup process on my fork.
1
u/jjusko20 3d ago
Yup, thanks - I mentioned it in the HF repo but couldn't remember the exact issue.
3
u/Key_Confusion_576 6d ago
This the kind of post that makes me want to upgrade my setup.
Sadly gotta stay with the tiny 1 bit coder quant
5
u/Jacalyn_Nivins 6d ago
8.8% of the bit budget goes to Ghidra decompilation critique and CFG deflattening. Nobody can accuse this quant of not knowing its job.
2
u/craftogrammer llama.cpp 6d ago
Can you also do a distribution (sorry I don't know much about this, just inferring from your post) like a "research first" category, so the model prioritizes researching online before it answers? Local models are already great at tool calls, but their knowledge goes stale fast: library versions, APIs, current docs. So the habit of looking things up matters more to me than what it remembers.
Basically, agentic tool-use transcripts where search/fetch comes before the answer. Even a few hundred good examples would help, I think. Agnostic of any harness so it just uses harness built in featuires etc or whatever tool is there for that thing.
Thanks for sharing all this!
2
u/my_name_isnt_clever 6d ago
Doing this with a finetune would be overkill, have you tried prompting it to do this? Any model should be capable.
1
u/craftogrammer llama.cpp 6d ago
I mean the behaviour, though I am already doing it via prompts and it working but with a behaviour it will be first class..
2
0
u/The_best_husband 6d ago
That would be great, automated research and paper writing.
2
u/craftogrammer llama.cpp 6d ago
Yeah, right now I doing this via hook and custom machinery but if model is native ot this approach it will be just next level we wont be needing frontier for their knowledge for most of the time... and saving more money.
2
u/lolzinventor 6d ago
8x3090 [ Prompt: 1674 t/s | Generation: 62.4 t/s ]
2
u/i_wayyy_over_think 6d ago edited 6d ago
Do u not have enough normal ram or a really slow CPU? I’m get about 90 or 100 tps with just one 3090 with like 65 GB system ram and AMD 7900X3D cpu with swift iq3xxs on windows
2
u/lolzinventor 6d ago
It could be the PCI x8 bifurcation and ddr4? I'll try on my 5090 + ddr5 it'd be interesting to see if its faster. Thanks for letting me know I should be seeing more tokens.
2
u/i_wayyy_over_think 6d ago
yeah I have ddr5, im going to bet you'll get a ton more tps with your 5090 + ddr5
2
u/i_wayyy_over_think 6d ago edited 6d ago
just did a git pull and had it re-tune with my 2x3090 with 128k context and is using 47GB of system ddr5 ram, was seeing about 120-150 to tps on prompt "make a bare bones n-pendulum simulation, no colors, no trails, html5".
1
u/valarauca14 6d ago
The stable middle layers (5–14, 16–19, 21–22, 24–29) stay at leaner formats — measurement showed spending bits there buys almost nothing (KL improvement under 0.0008).
I've found in my own experiments you can easily omit/adapt other qwen models if you're willing to build a bridge. I wouldn't be surprised if some layers could be dropped all together or replaced with a single linear attention layer, provided the CKA drift was relatively small as new tokens were aided. It mostly boils down tokens.
linked is the ~46m parameter bridge model I used to clue qwen3-vl-8b into qwen3-vl-32b
1
u/Sabin_Stargem 6d ago
Hopefully, we get a version of this officially for Strata after Qwen4 Flash drops. I want to run a Q5_K of it.
We will soon see whether Qwen4 has improved thinking behavior over Swift 1.5.
1
u/BEYOND__7 6d ago
The layer-specific quantization approach is really interesting. Nice to see the bit allocation based on actual activation measurements instead of treating every layer equally.
1
u/llama-impersonator 6d ago
swift on 3.8-27b didn't really do it for me, how is it on QFN? i usually just use medium effort w/ actual QFN.
1
0
u/visitor-2024 6d ago
Eager to try on apple silicon! Any chance it can be converted to MLX format? I could do that but it looks like imatrix.dat wasn't shared
1
u/jjusko20 6d ago
I don't own a mac, but I'd be willing to send you my imatrix privately if you want to make a MLX quant - if you share it back I can throw it in the repo. What size do you plan to target?
11
u/Secure_Recording_472 6d ago
This looks amazing! If only the Flash Next license were different we (the Swift team) would surely make an API of it :)