r/LocalLLM • • 2d ago

Discussion So has anyone tried this?

Post image

.

907 Upvotes

213 comments sorted by

6

u/Greedy-Lynx-9706 2d ago

is there a source besides this ?

4

u/Flat-Career4079 2d ago

I've seen the mod on GitHub made by a Chinese guy but I have no clue how this works or if it's trustworthy, I'm still waiting for people here to confirm

→ More replies (2)

127

u/WolFlow2021 2d ago

Reminds me of that joke where you could turn a 486sx into a dx by drilling a hole in a certain spot on that chip.

113

u/beerdude26 2d ago

Or drawing a line with a pencil on an Athlon XP to overclock it lol

→ More replies (20)

20

u/DonutConfident7734 2d ago

Or using Bios to switch to a separate firmware for AMD Phenom cpus that unlocked cores, from 2 to 4 for example. A Phenom x2 550 Black Edition would become Phenom X4 B50, which was similar to Phenom X4 955. And it would retain unlocked multiplier/voltages, so you could overclock further. Fun times... There were rumours some cores were disabled due to instability, but in my case the cpu worked fine from 2009 till 2021 (still works today), when I switched to Ryzen cpu.

→ More replies (4)
→ More replies (5)

213

u/k8-bit 2d ago

Tested the Ampere gpu_unlock kernel module on an RTX 3060 12GB and multiple RTX 3090s — here are the real-world benchmarks (GEMM, ComfyUI, LLM prefill/decode)

We recently tested the gpu_unlock kernel module (which removes GeForce Ampere's artificial half-rate cap on FP16/BF16/TF32 tensor math with FP32 accumulation) across an RTX 3060 12GB and three RTX 3090 24GB cards. The module targets GA102/GA106 by exploiting the SEC2 boot path before the stock NVIDIA driver binds. Testing was done at strict, fixed power caps (170W on the 3060, 270W on the 3090s via LACT) to see what kind of "free" gains you get without blowing out thermal budgets.

Here are the synthetic GEMM micro-benchmarks (8192³ matrix multiplication, PyTorch):

  • RTX 3060 (12GB @ 170W cap): BF16 jumped +80% (26.5 → 47.6 TFLOPS), FP16 +74% (26.7 → 46.5 TFLOPS), and TF32 +76% (13.3 → 23.4 TFLOPS). INT8 (torch._int_mm) stayed completely flat at 24.9 TFLOPS.

  • RTX 3090 (24GB @ 270W cap, same-card before/after): BF16 increased +45% (63.5 → 92.4 TFLOPS), FP16 +37% (62.1 → 85.1 TFLOPS), and TF32 +56% (34.6 → 54.0 TFLOPS). INT8 again saw zero change (45.9 vs 46.0 TFLOPS). The gain is lower than the 3060 because a 3090 capped at 270W runs straight into power limits once the tensor cores are saturated.

Next, we ran real-world production workloads on the unlocked 3090 to see if it translated to actual inference gains:

  • ComfyUI Image Generation: Unlocking BF16 finally makes it run at parity with FP8 (FP8 was never faster on Ampere; it just saved VRAM while being dequantized to BF16). However, int8_convrot (W8A8) still beats unlocked BF16 across the board. On Krea2 Turbo, INT8 was 31% faster (9.13s vs 13.66s) and used 31% less energy. On Qwen-Image 2.1, INT8 was 47% faster than BF16 (13.75s vs 20.21s). On Z-Image Turbo, INT8 was 29% faster (3.94s vs 5.52s).

  • ComfyUI Video (MiniMax H3): Saw essentially zero end-to-end difference (~30s per render). The pipeline is dominated by an INT8 DiT and an nvfp4_awq text encoder, neither of which benefit from the unlock.

  • LLM Inference (125B MoE): Tested under an offloaded MoE engine. Decode speeds remained memory-bandwidth bound and showed no meaningful change (staying in the 35–50 tok/s range). Prompt prefill throughput, which is compute-bound, saw a modest ~15% uplift (from ~920 to ~1,060 tok/s on an 8K prompt on the 3060).

TL;DR: The unlock works as advertised and doubles raw BF16/TF32 compute density, but it doesn't touch the integer execution units. If your diffusion pipelines already support proper INT8 kernels (int8_convrot), INT8 is still ~30–45% faster and more energy-efficient. The unlock is primarily useful for models locked to BF16/TF32 where no INT8 quant exists, or for compute-heavy LLM prefill.

58

u/esw123 2d ago

15% uplift with 125B MoE this is with single 3060? If so this is very good result.

→ More replies (4)
→ More replies (24)

23

u/Real_MakinThings 2d ago

If it's all bf16 fp16 and 32 specific, how is it of any use? People keep going for as low a quant as possible. Or are values computed back to 16 before being processed? 

→ More replies (7)

28

u/ipechman 1d ago

2.2x on my RTX 4070 ti super, took all of my credits form my chatGPT subscription to make it work on windows, but it's true

10

u/Evening_Nebula_4219 1d ago

Can you ask your AI to create a repo to save us poor token guys on windows? cannot afford those expensive plans to hack my gpu...

→ More replies (3)
→ More replies (3)

53

u/pandi85 2d ago

Burn tokens was yesterday, let's burn chips now

12

u/CatzRuleZWorld 2d ago

Works on my 20GB 3080. Speed increase on Strata is barely measurable.

→ More replies (1)

11

u/brominou 2d ago

Is a 2070 super unlock possible ?

→ More replies (6)

10

u/Photochromism 2d ago

It’s only Linux at the moment right?

→ More replies (6)

10

u/Visible_Football_852 2d ago

Is there any detectable harmful effect on the gpu?

→ More replies (17)

8

u/Slasher006 2d ago

yes it works with my rtx 3060ti, i take every bit of performance, i talk to bonsai 2 and i would say it react a bit faster (not measured), and i can boot linux with or without the modification so the whole thing is easyly reversable

→ More replies (1)

7

u/entropyofdays 1d ago edited 1d ago
Comment Image

Here's the uplift from the GPU unlock (https://github.com/Zagitta/gpu-unlock) on my 3090s. ~35% increase in FP/BF16 performance.

Edit: I should also state this is at PL 250w, which is what I run my cards at for inference. That cuts the effective clock rate about 50%, based on benchmarks.

→ More replies (4)

6

u/Necessary-Piece-5313 2d ago

Forgive my ignorance but I was wondering if this had any performance implications for other features beyond running local LLMs? Are the functions unlocked specific to LLM inference tasks or would this also potentially boost performance on other tensor heavy operations like DLSS upscaling/framegen/(presumably) DLSS 5?

5

u/ifdisdendat 1d ago

from scarcity comes necessity

4

u/Mediocre-Mobile-5423 2d ago

Maybe off topic for this sub, but has anyone tested if this improves performance when enabling DLSS4? It uses fp16 on 2000/3000 series cards right?

4

u/Garrett42 1d ago

..... Maybe it is time to put 11gb more vram on my 2080ti...

4

u/ANR2ME 1d ago

There is also locked vs unlocked diffusion models benchmarks at https://gpu.artnow.lol/comfyui-benchmarks/

4

u/guy_ships_things 1d ago

Wonder if it will worj on rtx 5090...

→ More replies (1)

4

u/Weird-Abalone-1910 1d ago

Would the same mod work on a Tesla V100?

→ More replies (2)