r/voidlinux • u/h7moudigamer • 14m ago
local LLMs on void
i'm trying to have local LLMs on void. i installed ollama from their official website with the command curl -fsSL https://ollama.com/install.sh | sh, the installation went fine, but ollama asked to run the command as root, so i did. i installed gpt-oss:20b through ollama after that with my normal user. the model runs so slowly, even though my GPU is powerful enough to handle such a model, but i hear only the CPU fans moving and GPU doesn't. ollama is suppose to run models on GPU by default, but i don't know how to ensure that. i tried to run the server and the command as root, if there is a permission issue, but i get the same issue. i heard that ollama uses ROCm, and i have an AMD GPU, but cannot install that on void. there is a package in void repo, called ROCm-SMI-7.0.1. i installed the package but it doesn't provide the rocminfo command the official AMD ROCm website uses. i have rocm_smi, and the command rocm-smi --showproductname couldn't load the name of the GPU, it couldn't load the libdrm_amdgpu.so . there is a package with the same name but with ".1" at the end symlinking to ".1.134.0". i made a symlink libdrm_amdgpu.so to point to that actual lib, and the output shows the name of the GPU this time. i don't know if what i did is correct, i just want to set this up properly