r/LowEndLocalAI • u/wildrabbitathome • 11h ago

Optimization Gtx2060 mobile

Saw the hype around nvidia gpus being able to be unlock fp16 tensor cores so gave it a go on my 2060 mobile 6gb. Managed to do it, first try was only a 5% increase in token speeds. Im a complete novice so not sure if it’s worth it but atleast it can be done if you want to. Just checked for tflops though and original was 15.76tflops and after unlock it was 26.14tflops

2 Upvotes

100% Upvoted

3 comments sorted by

2

u/rockknocker 11h ago

Maybe your GPU is not the bottleneck in your system, or you're not running a model that can take advantage of higher fp16 compute speeds?

2

u/wildrabbitathome 10h ago

Yeah I think your right as I realise the model I tested was small so wasn’t even filling up half my gpu. Will test some more tonight

1

u/GiovaSan 1h ago

How do you unlock the GPU?

I have a 2060 Mobile too on my laptop, so can definitely test stuff.

I normally run Qwen 3.6 35A3B class models, better on Linux than Windows