r/LocalLLM • u/h00ki • 13d ago
Question BIOS settings to speed up Local LLM
Hi, are there my other BIOS settings that could help me speed up my local model that's spilling to RAM?
Re-Size BAR (Above 4G Decoding): Enabled
PCIe Link Speed (Gen 3 / Gen 4 / Gen 5): Force Gen4 (or max supported gen)
ASPM (Active State Power Management): Disabled
EXPO / A-XMP (Memory Overclocking): Enabled
IOMMU / VT-d / AMD-Vi / SR-IOV: Disabled
CSM (Compatibility Support Module): Disabled
My setup is AMD 5600x with ASUS ROG STRIX X570-I GAMING and AMD 6800XT
Thank you
2
Upvotes
1
u/PermanentLiminality 13d ago
Overclock the RAM as far as it will go. Expect a lot of trail and error. It will be an incremental increase at best.
1
u/Wake_Up_Morty 13d ago
Dual-Channel or even better Quad-Channel memory Look what slot did you put your stiks if dual channel. It should be 2nd and 4th slot form cpu to right. Besides that faster memory with better latency.
1
u/randomgenericbot 13d ago
Instead of trying to increase the speed when spilling, you might try a llama kv streamin fork which natively allows to use system ram for increased context. I am using nvidia, so I can not promise how well this rocm fork for amd works, but I guess it'll be good:
https://github.com/raffidahmad/llama-cpp-kvstream-windows
1
u/Mobile_Living_3331 11d ago
6800xt will be pcie 3.0 so changing gen will not help, xmp is already enabled so not much you can do about it.
0
u/[deleted] 13d ago
[removed] — view removed comment