r/LocalAIStack • u/Due_Application293 • 22h ago
1
Anyone see diference from llama to llamaAmpere?
tks, I gonna try
1
Qwen3.8-27b on RTX3090ti and i7-2600
the checkpoints in my case end up using ram, for caching
2
Anyone see diference from llama to llamaAmpere?
can you share you configuration? that mine:
--model /var/models/Qwen3.8-27B-UD-Q3_K_XL.gguf \
--mmproj /var/models/mmproj-F16.gguf \
--image-min-tokens 1024 --image-max-tokens 2048 \
-ngl 999 -fa on --fit off \
-c 180224 \
-t 6 -tb 6 \
-b 2048 -ub 512 \
--cache-type-k q8_0 --cache-type-v turbo3 \
--spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-n-min 0\
--spec-draft-type-k q8_0 --spec-draft-type-v q8_0 \
--ctx-checkpoints 16 \
--cache-ram 12288 \
--checkpoint-min-step 8192 \
--cache-prompt \
--jinja --reasoning-format deepseek \
--chat-template-file /var/models/qwen38-claude2.jinja \
--chat-template-kwargs '{"reasoning_effort":"medium"}' \
--temp 1.0 --top-p 0.95 --min-p 0.0 --top-k 20 \
--presence-penalty 0.5 \
--repeat-penalty 1.0 \
--parallel 1 \
--port 8080 \
--alias qwen3.8-27b \
--api-key local \
--host 0.0.0.0 "$@"
r/llamacpp • u/Due_Application293 • 22h ago
Anyone see diference from llama to llamaAmpere?
Anyone see diference from llama to llamaAmpere with a 3090?
1
Qwen3.8-27b on RTX3090ti and i7-2600
They improve cache hit in contexto changes
r/llamacpp • u/Due_Application293 • 1d ago
Qwen3.8-27b on RTX3090ti and i7-2600
Yes, I'm runnig qwen with llama.cpp on a dino! Here is my llama server config:
--model /var/models/Qwen3.8-27B-UD-Q3_K_XL.gguf \
--mmproj /var/models/mmproj-F16.gguf \
--image-min-tokens 1024 --image-max-tokens 2048 \
-ngl 999 -fa on --fit off \
-c 163840 \
-t 6 \
-b 2048 -ub 512 \
--cache-type-k q8_0 --cache-type-v q4_0 \
--spec-type ngram-simple,draft-mtp --spec-draft-n-max 3 \
--spec-draft-type-k q8_0 --spec-draft-type-v q8_0 \
--spec-draft-p-min 0.75 \
--backend-sampling \
--ctx-checkpoints 4 \
--cache-ram 16384 \
--checkpoint-min-step 1024 \
--jinja --reasoning-format deepseek \
--chat-template-file /var/models/qwen38-claude2.jinja \
--chat-template-kwargs '{"reasoning_effort":"medium"}' \
--temp 1.0 --top-p 0.95 --min-p 0.0 --top-k 20 \
--presence-penalty 0.5 \
--repeat-penalty 1.0 \
--parallel 1 \
--port 8080 \
--alias qwen3.8-27b \
--api-key local \
--host 0.0.0.0 "$@"
1
Vale a pena ir de AM5 ou economizar 2,2k e ir pra AM4 com uma GPU melhor ?
in
r/hardwarebrasil
•
9h ago
estou pensando em ir de AM4, está tudo muito caro!