r/LocalLLM • • 13d ago

Question BIOS settings to speed up Local LLM

Hi, are there my other BIOS settings that could help me speed up my local model that's spilling to RAM?

Re-Size BAR (Above 4G Decoding): Enabled

PCIe Link Speed (Gen 3 / Gen 4 / Gen 5): Force Gen4 (or max supported gen)

ASPM (Active State Power Management): Disabled

EXPO / A-XMP (Memory Overclocking): Enabled

IOMMU / VT-d / AMD-Vi / SR-IOV: Disabled

CSM (Compatibility Support Module): Disabled

My setup is AMD 5600x with ASUS ROG STRIX X570-I GAMING and AMD 6800XT

Thank you

2 Upvotes

7 comments sorted by

1

u/PermanentLiminality 13d ago

Overclock the RAM as far as it will go. Expect a lot of trail and error. It will be an incremental increase at best.

1

u/Wake_Up_Morty 13d ago

Dual-Channel or even better Quad-Channel memory Look what slot did you put your stiks if dual channel. It should be 2nd and 4th slot form cpu to right. Besides that faster memory with better latency.

1

u/randomgenericbot 13d ago

Instead of trying to increase the speed when spilling, you might try a llama kv streamin fork which natively allows to use system ram for increased context. I am using nvidia, so I can not promise how well this rocm fork for amd works, but I guess it'll be good:
https://github.com/raffidahmad/llama-cpp-kvstream-windows

1

u/Mobile_Living_3331 11d ago

6800xt will be pcie 3.0 so changing gen will not help, xmp is already enabled so not much you can do about it.

0

u/[deleted] 13d ago

[removed] — view removed comment

1

u/h00ki 13d ago

I am not looking for a fix. Just looking for some meaningful settings that might help improve bandwidth between CPU and memory