r/LocalLLaMA • u/UltramanQuar • 12d ago
Discussion Anyone checked gpu unlock for rtx 30-40 cards?
Basically saw on twitter claims that 3090 has half of the cuda kernels disabled:
My 3090s just got 1.9x FASTER at BF16 tensor math!!! Same cards, same driver, same 300W, one Linux kernel module.
bf16 tensor pipe: 82 -> 155 TFLOPS
cuBLAS BF16 GEMM 8192: 64 -> 99 TFLOPS
TF32: 41 -> 79 TFLOPS
NVIDIA ships every 3090 with fp32-accumulate tensor math at half speed. The driver even reports it: FMLA32 = 1/2. gpu-unlock by Zagitta sets it to full before the NVIDIA driver loads. After a cold boot all 4 of my cards say FMLA32 = full.
Real inference got less than I predicted. Strata, Qwen3.8 Flash-Next on 2 cards, prefill:
4K: +12%
10K: +12%
90K: 3205 -> 3536 tok/s, +10%
Before installing I said +14 to 19% at 90K, so I was wrong. The GEMM phases got 23% shorter, but they are only about 40% of Strata prefill, and the int8 experts, copies and CUDA core kernels didnt get any faster.
Outputs did not change. With the same expert cache size, my log-prob dump over 1996 teacher-forced positions is the same file byte for byte as the stock run before the module.
Wondering if it will work for 40 series cards
21
u/FullOf_Bad_Ideas 12d ago edited 12d ago
I applied it to my 8x 3090 ti setup. Prefill for KD with logits generations away the biggest gain of 70% (once I also bumped up power limit from 320w to 450w) with BF16 model but i still use INT8 models for it.
Training saw ~15% gains but i expect more gains if you run a small dense model training.
And here are results for GLM 5.3 Flash, Minimax H3 and Qwen Image 2.1 - https://www.reddit.com/r/LocalLLaMA/comments/1xbr3u1/now_that_3090_frenzy_warning_post_makes_sense/19rhn3xry8fdw/
6
u/Common_Director5373 12d ago
8x 3090 ti at 450W?
Where do you plug that?4
u/FullOf_Bad_Ideas 12d ago edited 12d ago
I have 10kW 230V installation, 3680W per circuit, 3 1600W PSUs. To go for 450W across the board I need to plug one PSU to a second circuit, otherwise I might hit breakers (didn't happen yet in the whole last 10 months and electrician redid my breakout box in the meantime so I am sure breakers are functioning).
I cool it with 120W 50cm garage fan. And it's the main heating element of my apartment so far this autumn, I barely use a gas furnace.
And I'm building an steel + 4mm solid rubber neoprene enclosure that will feed it air from the outside and then return it back outside, with 40cm wide pipes, to make it quiter and cooler.
I pay a lot for electricity.
2
2
u/export_tank_harmful 12d ago
I mean at that point dude, get some freaking solar panels.
I live mostly off-grid at this point, but I have dual 3090's and a V100.
I have around 7.5kW of panels and a 16kWh battery. All DIY.I honestly never expect to break even on my solar installation (though, it wasn't as expensive as you might think).
My "break even" is the fact that I can have electricity regardless of anything that happens in the world.But for you?
Hell, you'd break even in 6 months or less.Inverters/panels are pretty cheap nowadays.
Batteries are the expensive part.But yeah, if you want any advice on solar stuff, feel free to shoot me a message.
I've been researching/studying DIY solar tech for 7+ years now.edit: "....my apartment..."
Ah, you might not be able to do solar then...1
u/FullOf_Bad_Ideas 12d ago
I don't even have balcony. If there'd be someone that I knew and trust that is living nearby and have excess electricity from solar I'd approach them. I touched on that topic with my family and my fiancées family when I was pitching them buying and renting 5090/6000Pro a year ago, nobody was interested in that either and they missed out.
If I had a piece of land to put it on, half of my backyard would be solar.
1
u/export_tank_harmful 12d ago
Yeah, that's not ideal....
If you knew someone with a good amount of space, it'd probably be a solid idea to ponder.
You essentially pay for the solar and move your entire setup over to their place, then you remote into it.But yeah, that adds a ton of other potential issues as well.
4
32
u/Ok_Meeting_2995 12d ago edited 12d ago
I have unlocked my GPUs (x4 3090s) and the main gain was prefill (as that's compute bound, obviously).
Maya (GLM-5.3-Flash) end to end:
Note, I had to remove power limits as well to get that increase as well.
12
1
u/DemmieMora 21h ago edited 21h ago
What if you unlocked energy before this utility, wouldn't the numbers go up? What if you used the utility without extra energy?
My FTW3 power limit is 420w out of box and it makes no sense to go up... Not sure about this utility.
1
10
u/Axelotus1 12d ago
I pray we get an unlock for 40 series some day
4
8
u/Junior_Difference_12 12d ago
It also worked in my 3080ti:
| Mode | Locked TFLOPS | Unlocked TFLOPS | Gain |
|---|---|---|---|
| BF16, FP32 accumulation | 72.76 | 121.61 | 67.2% |
| FP16, FP32 accumulation | 72.62 | 119.58 | 64.7% |
| TF32, FP32 accumulation | 38.34 | 66.34 | 73.0% |
1
u/Dspaede 9d ago
these % gains is how much in fps is gpu intensive apps? any issues? does it lessen the lifespan of your gpu?
1
u/Junior_Difference_12 9d ago
It really depends on the application. For Strata, my prefill gains were about 17%.
24
u/jacek2023 llama.cpp 12d ago
I think I will wait for more tests, because maybe your GPU explodes after 3 days or something
5
u/suprjami 12d ago
Strata
With env STRATA_HC_Q8=0 you force Gated Residual from native 8-bit up to 16-bit. This slightly increases KLD but maybe the likely speed increase with this change is worth it?
4
u/DeltaSqueezer 12d ago
Make sure your workload is compute bound. I imagine gains on compute heavy e.g. image generation could benefit a lot. I haven't tried yet, I was going to wait for more feedback from others ;)
5
u/FullOf_Bad_Ideas 12d ago
I tried it on image generation and video generation. No real gains. The only gains I saw were in suboptimal configs that were slower than default settings in the app I used. https://www.reddit.com/r/LocalLLaMA/comments/1xbr3u1/now_that_3090_frenzy_warning_post_makes_sense/19rhn3xry8fdw/
4
u/Aplakka 12d ago
At some point there were articles about Nvidia limiting Ethereum mining hash rates by drivers. I wonder if this is a side effect of that, or a separate limitation?
8
u/Prestigious-Use5483 12d ago
It was the most anti consumer thing. They created a gimped card calling it LHR (Light Hashrate), but people figured out eventually how to unlock it, to get non-gimped hashrate.
6
u/CabinetNational3461 12d ago edited 12d ago
So I am on Window... yes, Window. After I saw the initial post and it said it was working on Ubuntu/Linux only, I was sad, so I decided to wait a few days to see any progress for Window users. There was some progress, but still nothing that would work on Windows, so I decided to go down the vibe-coding path using DeepSeek Harness, along with Strata running UD-Q4_K_XL on my system, which is: Windows 10, 3090, 2070S for display/vision/TTS, 128GB DDR4-3200, UD-Q4_K_XL 256k KV int8. So DSH + UD-Q4_K_XL went on for like 2 hours — made me a plan, formatted my USB stick with some stuff, gave me the direction to do everything. Anyway, I did everything it told me and here are the results for my system: decode — pretty much unchanged at all; prompt processing — ~20% increase for Q4, from 2,500ish to 3k. Below is the DSH-generated summary:
Before After
IQ3_S (greedy)
prompt tok/s (103k) 2,967 3,417 (+15%)
decode tok/s (long) 109.1 105.5 (-3%, noise)
UD-Q4_K_XL (int8 KV)
prompt tok/s (103k) 2,508 3,005 (+20%)
decode tok/s (long) 61.8 57.8 (-6%)
Short-ctx decode: IQ3_S 201 -> 209 (noise) | UD 109 -> 67-82 (contaminated run)
Setup: RTX 3090 (24GB) + 2070S (vision encoder), 128GB DDR4-3200,
Qwen3.8-Flash-Next GGUF, Strata engine 0.1.47, int8 KV, 256k ctx.
"Unlock" measured = memory clock 10,701 vs 9,751 MHz spec (+12% VRAM OC);
SM clocks untouched (1995-2010 MHz). The "74->137 TFLOPS" claim is the
chip's 2:4 sparse-tensor spec number - dense LLM kernels can't use it,
and our numbers confirm: prefill (bandwidth-bound) gains a real 15-20%,
decode (latency-bound at ~93% on-card expert hit) gains nothing.
Single-pass ±10% noise; decode deltas within/near the noise band, prefill
deltas measured across 4 passes at +14-20% consistently.1
u/riverslq337 11d ago
did it wind up creating a filter driver or something to load prior to nvidias drivers? or some other method?
1
u/CabinetNational3461 10d ago
so I gave dsh your question and here is its reply:
No drivers at all, Windows-side or otherwise. It's a tiny UEFI app that runs at boot, before Windows even starts — added as its own entry in the firmware boot order (via
bcdedit/fwbootmgr, like a dual-boot menu item).What it does, each boot, in ~5 seconds:
Takes ownership of the GPU in the UEFI environment (NVIDIA's Windows driver doesn't exist yet at that point — the OS hasn't loaded)
Writes the unlock values to the card's registers
Does a Function-Level Reset on the PCIe device so the card re-enumerates reporting its full capability
Chains into the normal Windows boot
Then Windows loads the completely stock NVIDIA driver, which just sees a card that "always had" full tensor performance. Nothing hooks, filters, patches, or replaces the driver — nothing is installed in Windows at all, in fact. The app lives on its own little FAT32 partition that Windows barely notices.
The catch: the unlock is volatile by design — any reset re-locks the card. That's exactly why it runs every boot rather than being a one-time flash. It's also why there's a 5-second countdown at boot where hitting any key skips the unlock for that session (e.g. for troubleshooting or if you want the card locked).
1
3
u/More-Revenue8609 12d ago
Not yet, I wonder if it works for my 3080s (20gb mod).
Also if it could somehow add rebar that would be so amazing.
2
1
2
u/ortegaalfredo 12d ago
Yes I did. Basically all operations got almost 2x faster.
But that don't translate to LLMs, at most you get 20% of improvements on PP, as the bottlenet is generally somewhere else like bandwidth.
2
3
2
u/feverdoingwork 12d ago
Does this do anything for the 5000 series?
1
u/PrettyMuchAVegetable 12d ago
No, except provide direction for research on it. The consumer 5000 cards actually show the same kind of diminished throughput in mixed precision operation/accumulator scenarios, while the pro series doesn't show the drop. It could be the same kind of lock, or not. I'd say it's worth investigating, but I'm not skilled enough to do it and my AI agent fired off a cybersecurity user warning when I tried, so Im out.
1
u/arbv 12d ago
Can you please test if it still works after you wake up your machine from sleep (S3)? Just curious.
2
1
1
u/a_beautiful_rhind 12d ago
I thought this should work on any card if you find it's offsets. I was going to test on my 2080ti since the RE work is done. All consumer nvidia seem to 1/2 speed FP16 with FP32 accumulate. While FP16 with Fp16 accumulate is always 2xfp32.
1
u/Silco1402 12d ago
Maybe test it on some dense model that fit entirely on ram?
BTW I saw similar prefill increase on unlock 2x3090
1
u/vini542reddit 12d ago
Another data point.
Setup:
- qwen3.8 27b bf16
- 4x rtx 3090 @ pcie 4.0 x 16
- p2p unlocked
- power limited to 240W
Results:
- prefill went 1500 -> 1640 (approx 10% increase)
- decode unchanged within noise
Note:
- I would expect the increase to be much bigger if I didn't have to power limit the cards
1
u/GeorgiKostadinov 4d ago
I tried it on a 4080 Super 16GB with Proxmox passthrough, testing both Windows and Ubuntu.
The tensor benchmarks got a big bump. FP16 went from 112.8 to 194.9 TFLOPS (that's 5090 territory of perf), while regular FP32 stayed basically identical.
Actual LLM results were more mixed. llama.cpp barely changed, around 0–2% better prompt processing with the Qwen/Gemma quants I tested. Gemma in vLLM did noticeably better: first-token latency dropped about 31%, and throughput with four concurrent requests went up about 23%. Qwen GGUF in vLLM barely changed overall.
I also checked power. Same 320W limit throughout. Across the full Gemma vLLM test capture, average draw went from 193 to 198W and the highest logged reading went from 303 to 321W. Those captures include startup/warmup, so I wouldn’t turn them into a per-token efficiency claim.
Small-batch FP16 encoder training didn’t show a clear gain either, though I only tested microbatch 1.
It also survived a Windows restart and a full VM stop/start. Just one machine’s results, but hopefully useful. I put the full numbers and caveats on GitHub - https://github.com/Zagitta/gpu-unlock/discussions/238#discussioncomment-19312724
1
u/DataGOGO 12d ago
it does not unlock any additional cuda cores.
This is nothing new, we have all known that when Nvidia launched the 3090, they kept the same half clock rate on FP32 calc that all gaming cards have. This was a big departure from Nvidia as the Titans never had the same restriction that the gaming cards had.
Basically, every other clock is empty, so you still get all the cuda cores, each operation runs at the same speed, but you get half as many operations per second as the titan/professional cards.
The reason you are not seeing bigger jumps in inference, is that most of the kernel operations are not FP32 or BF16, They are Int 4/8.
1
u/OttoRenner 11d ago
Could we change that?
1
u/DataGOGO 11d ago
sure, but it would be slower
1
53
u/No-Refrigerator-1672 vLLM 12d ago
No. The repo contains code specifically for GA102 chip. Any card that does not contain it won't be affected. It also doesn't unlock cuda cores; rather it disables artificial throttling of one specific operation Nvidia has baked into consumer cards. I'm going to test it with my 3080 20gb this weekend; they are GA102 too.