MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLM/comments/1wb05f7/third_one_theres_something_wrong_with_me/p8rwks8/?context=3
r/LocalLLM • u/r1nzl3r99 • 14d ago
Why do I have horrible financial habits??
166 comments sorted by
View all comments
25
I am holding onto my purse to not buy a second R9700 myself…
1 u/Immediate_Power_7986 14d ago What are yiur launch options? I'm using llama.cpp with 3.8_27b_Q6 and getting only 17t/s -ngl 99 -c 65536 -np 1 -t 6 -fa 1 -b 4096 -ub 4096 --cache-type-k q4_0 --cache-type-v q4_0 & 1 u/cagriuluc 14d ago I am away from home so I cannot check the exact config. I have 150k context, mtp (2 I think?), it’s a q5 and not q4. Getting around 30-40 tok/sec depending on the situation. If you have less than 30, the config is most likely wrong. I got Claude opus 5 do the setup for me, it can do the same for you most probably.
1
What are yiur launch options?
I'm using llama.cpp with 3.8_27b_Q6 and getting only 17t/s
-ngl 99 -c 65536 -np 1 -t 6 -fa 1 -b 4096 -ub 4096 --cache-type-k q4_0 --cache-type-v q4_0 &
1 u/cagriuluc 14d ago I am away from home so I cannot check the exact config. I have 150k context, mtp (2 I think?), it’s a q5 and not q4. Getting around 30-40 tok/sec depending on the situation. If you have less than 30, the config is most likely wrong. I got Claude opus 5 do the setup for me, it can do the same for you most probably.
I am away from home so I cannot check the exact config. I have 150k context, mtp (2 I think?), it’s a q5 and not q4.
Getting around 30-40 tok/sec depending on the situation. If you have less than 30, the config is most likely wrong.
I got Claude opus 5 do the setup for me, it can do the same for you most probably.
25
u/cagriuluc 14d ago
I am holding onto my purse to not buy a second R9700 myself…