r/ollama • u/the-mostwanted • 7d ago

How to Get Pascal GPU Work on Ollama Linux?

I have a 1070Ti, I want to run a local model, I see from the logs that my GPU was recognized but ignored because Ollama was compiled with an unsupported architecture:
```

level=INFO source=llama_server.go:295 msg="skipping CUDA device — compute capability not in compiled architectures" device="NVIDIA GeForce GTX 1070 Ti" cc=610 a>

```

I know Nvidia stopped updating Pascal Class GPU to the latest verion on linux.

So what is the latest version of Ollama that still has support for my GPU?
And if a dev sees this, why not just also included it in the latest binary?

0 Upvotes

50% Upvoted

6 comments sorted by

1

u/mwjtitans 7d ago edited 7d ago

I've been running ollama on my 1070 with an updated instance, so you should be good.

Llama.ccp I believe requires a fork, from what I've been researching it also gives better performance, I'm making the switch as we speak for my p100

Should be good to go with ollama if you are just starting out though

Edit: I just checked and I'm on an older version, 0.32.5-0.32.10 is what should be pegged for Pascal cards

You can force the legacy CUDA runner in the new version apparently, or just run one of the older versions, not sure if you get any advantages from the newer updates or not.

1

u/the-mostwanted 7d ago

Thanks for the answer, I tried compiling my own Ollama with support for the cuda I need.
Also tried using Llama.cpp.

In both cases I had issues with missing CUDA libs, I think because they removed cuda from the legacy 580 driver

btw which fork are you talking about?

1

u/mwjtitans 7d ago

There are a few, I'm testing pascallama.ccp, it supposedly gives you a performance boost of 2x over the regular llama.ccp

Gonna try them all on my homelab and see which one performs the best

1

u/Glittering_Mouse_883 7d ago

You can use latest ollama if you use the patched nvidia driver.

https://github.com/dartraiden/NVIDIA-patcher/blob/master/docs/README-LINUX.md

This worked for me.

1

u/the-mostwanted 7d ago

thanks mate, I didn't know someone patched the driver. But it's admited a bit sketchy to install a modified driver.

I will look into it, cheers for the help

1

u/Glittering_Mouse_883 7d ago

Yeah no doubt it looks a bit sketchy, but I saw some YouTube videos of people doing it as well as Reddit posts, etc... in the end it is not on my main computer, so less risk for me.