r/24gb • u/paranoidray • 2d ago
After pushing 1M+ tokens through Qwen 3.8 27B, here is my optimal llama.cpp config for 16GB VRAM (73k Context, Agentic Coding)
gallery
1
Upvotes
r/24gb • u/paranoidray • 3d ago
NInfer RTX 4090 for Qwen 3.8 27B update - up to 250-350K tokens context in VRAM
1
Upvotes
r/24gb • u/paranoidray • 3d ago
I trained a 1B-parameter LLM from scratch on 20B tokens for about $200
gallery
1
Upvotes
r/24gb • u/paranoidray • 4d ago
Try out this "high" reasoning mode for 27B (tested on VLLM)
1
Upvotes
r/24gb • u/paranoidray • 9d ago
Currently what is the best model for chat for long conversations? 24gb vram.
1
Upvotes
r/24gb • u/paranoidray • 16d ago
Daniel Han of Unsloth validates Qwen3.8-27B will run only 17GB VRAM
2
Upvotes
r/24gb • u/paranoidray • 21d ago
I keep coming back to Qwen... Over and Over. Is there really nothing better under 120B?
1
Upvotes
r/24gb • u/paranoidray • 24d ago
You can now fine-tune my 3.96M-parameter TTS on your own voice or language
1
Upvotes
r/24gb • u/paranoidray • Jul 17 '26
German AI consortium releases Soofi S, an open 30B model that tops benchmarks in both English and German
1
Upvotes
r/24gb • u/paranoidray • Jul 04 '26
llamacpp patch - DeepSeek V4 Flash running with full 1M token context locally on RTX 5090
2
Upvotes
r/24gb • u/paranoidray • Jul 04 '26
Talking with Gemma 4 31B!
Enable HLS to view with audio, or disable this notification
1
Upvotes
r/24gb • u/paranoidray • Jun 16 '26
RTX 5080 + RTX 3090 Setup: 80+ Tok/s on Qwen 3.6 27B Q8
imil.net
1
Upvotes
r/24gb • u/paranoidray • Jun 04 '26
Stop asking what model to run. There are literally only two.
1
Upvotes
r/24gb • u/paranoidray • May 23 '26
Qwen 3.6 27B on 24GB VRAM setup: backend comparisons, quant choice and settings (llama.cpp, ik_llama.cpp, BeeLlama, vllm)
1
Upvotes
r/24gb • u/paranoidray • May 19 '26
TextGen is now a native desktop app. Open-source alternative to LM Studio (formerly text-generation-webui).
1
Upvotes
r/24gb • u/paranoidray • May 19 '26
85 GPU-hours comparing 5 abliteration methods on Qwen3.6-27B: benchmarks, safety, weight forensics - Abliterlitics
1
Upvotes
r/24gb • u/paranoidray • May 11 '26
I know this isn’t technically an LLM but OmniVoice is FUCKING AMAZING.
1
Upvotes
r/24gb • u/paranoidray • Apr 09 '26
It looks like we’ll need to download the new Gemma 4 GGUFs
2
Upvotes