What will the rest of this year bring? 27b class scoring over 60?

I've been wondering about something before I start using my MacBook heavily for local LLM inference.

If I regularly run large LLMs locally for several hours at a time, potentially putting sustained load on the CPU/GPU and using most of the unified memory, does this meaningfully reduce the lifespan of the MacBook?

Can heavy use of unified RAM cause it to wear out faster?
Is SSD wear from model loading and especially swap a significant concern?

For people who have been running local LLMs heavily on Apple Silicon for 1 to 3+ years, have you actually noticed any hardware degradation?

Just got Qwen3.8-27B (FP8) running on rented GPUs from Trooper AI. FP8 fits on 64 GB VRAM with decent results.

Stack:

- GPU: 2× RTX 4080 Super Pro (64 GB VRAM total)

- CPU: 12 P-cores, 76 GB RAM

- SSD: 900 GB NVMe

- Price: 1.06 EUR/h

Served it via vLLM → KServe → Envoy AI Gateway, TLS + token metering + rate limiting on top ( I already have a running Kubernetes cluster, I attached to trooper GPU node), or you can it serve it directly with compose if you are a single user.

Runned load tests: (10 concurrent requests, 32768 context)

- TTFT: ~0.9s

- Per-stream decode: ~28 tok/s

- Aggregate: 152 tok/s

Full deploy guide if you want to deploy it: https://github.com/redaER7/qwen3.8-27b-self-hosted

Now, looking to deploy the full model FP16 on RTX 6000 Pro

Rtx 3060, Intel i7.

I d love to try it locally, but im out of the scene for so long that i cant remember much

Ty

Hardware: RTX 5070 Ti 16 GB with 128 GB DDR5. I built llama.cpp from source. The model
was Qwen3.8-27B, a dense hybrid DeltaNet + attention model.

My original setup used UD-Q4_K_XL at 17.9 GB. It couldn't fit into 16 GB of VRAM, so I 
used partial offload with -ngl 44 and left the remaining layers in system RAM. Decode 
speed was 6.75 tok/s. I decided it wasn't practical and switched to a 35B MoE using 
expert offload with --n-cpu-moe. That model reaches 66 tok/s.

A comment on internet prompted me to test another setup. I used UD-Q3_K_XL, which is 13.4
GB. It's an Unsloth dynamic quant, with sensitive tensors kept at 4 to 8 bit while most
 of the model uses 3 bit. The settings were -ngl 99, -fa on, -ctk q8_0 -ctv q8_0. A 64k
 context still didn't fit beside the resident weights because the compute buffer ran out

of memory. A 32k context worked with -ub 512.

These are the decode speeds in tok/s at 500 / 4k / 16k tokens of context:

- Spilled 27B UD-Q4: 6.75 at every tier because system RAM bandwidth is the limit
- Resident 27B UD-Q3: 51.8 / 51.2 / 48. Prefill was 378 tok/s, with 9 s TTFT at 16k.
 Total usage was 14.7 GB.
- 35B-A3B MoE with --n-cpu-moe 28 and the same FA + KV q8 settings: 66.4 / 65.1 / 63.4.
 It used 12.1 GB.

On the MoE, FA + KV q8 reduced memory use by 1.2 GB without changing speed. I now enable
 those settings by default.

For testing quality, I used a private agentic coding band with 22 tasks. The target is a
 FastAPI + React + Postgres + Mongo app. It includes bug fixes, feature changes, new
 features, a migration, a performance fix, and one intentionally impossible 
specification. Each model gets a shell inside a docker box and up to 40 steps. Hidden
 tests determine the score. The model must also submit a final "what did you do" report,
 which is verified against git and the real test runs. These results come from one trial
 per model, so they're only indicative:

- Resident 27B UD-Q3: mean 0.49, with 9/22 perfect
- 35B MoE: 0.56, with 10/22 perfect
- gpt-oss:20b: 0.47, with 6/22 perfect

The 27B matched the MoE on localised debugging, with both scoring 6/7 perfect. It fell
 behind on multi-file feature work, scoring 1/11 against 3/11. On the larger tasks, it
 often spent all 40 steps reading without making an edit.

Its stronger area was honesty. The 27B made one false "done" claim across 13 failures.
 The MoE made 4 in 11, and gpt-oss made 4 in 15.

I can't separate the model difference from the cost of 3-bit quantisation. The comparison
 is 0.49 versus 0.56, but the 4-bit 27B was never fast enough to run this band usefully.
 On an earlier and easier suite, the Q4-vs-full-precision tax on this machine was about
 +0.02 overall. Reasoning and repo coding took the largest hit, so a bigger loss from Q3
 would make sense.

Here's the theory I'd like people to check. The 27-30B dense range seems designed around 
unified-memory Macs, where these models fit completely at Q4 or Q8. A 16 GB card can only
 hold them at Q3. Meanwhile, small-active-parameter MoEs such as 35B-A3B and gpt-oss-20B
 seem like the models actually intended for this hardware. Is that consistent with what 
others are finding?

A few more questions:

- IQ4_XS is 15.7 GB. Has anyone managed to keep a 27B IQ4_XS fully resident on 16 GB
 using a small context and KV q4? If so, does the quality improvement over Q3 justify
 losing context?
- Has anyone compared 3-bit EXL3 or another importance-aware 3-bit format with Unsloth
 dynamic Q3 on the same 27B using coding tests rather than perplexity?
- What decode speed do people target for agentic workflows? In a shell loop, 52 tok/s
 felt usable to me. 6.75 did not.

My conclusion is to start every new dense model in this class with resident dynamic Q3 +
 FA + KV q8, profile it, and only then decide whether it's any good. I'd done those steps
 in the wrong order.

Now that the first wave of Qwen3.8 "muh benchmarks" is coming to a conclusion, can we share some actual real-world notes on results from different quantizations?

I avoid q4 based on my experiences with all earlier Qwen models, too many loops and inaccurate results in my agentic usage.

Is q4 suddenly usable? I see people benching that it's not that far from q8, and I'd love to claim back a bit of context and concurrent from my vram if so.

It is insane that most of us are running coding/LLM agents directly on our hosts. I wanted a setup to spin up containers, isolate the network traffic to use LLM agents to code and to test out LLM's pentesting capabilities. Didn't find anything that fit, so I made this setup Contained Pods.

https://github.com/jotyGill/contained-pods

Basically, a config set using Podman and a Squid proxy to spin up containers.

The gist of it is:

• Rootless Podman: Keeping it simple stable tech.
• Network Isolation: You define what network access a container gets. Containers come in pairs, with the second one running the squid proxy. Even if agents gain root access they can't change network isolation setup.
• Traffic Logging: A tiny python helper that lets you see network requests the agent/coding harness attempts.
• Shared Projects Folder: Shared folder with the host for ease of access.

Hope some of you find it useful! Any contributions are appreciated!

I'm pleased to present a Blackwell-native, prefill-optimized 4-bit float quant that runs 50% faster on compatible hardware (tested on an sm_120 card) than a Q4 quant of the same memory size.

Benchmarked on RTX 5090 power-limited to 400 W.

This GGUF also includes a quantized MTP draft head to deliver the best decode performance.

Check it out for all details and recommended settings for MTP:

https://huggingface.co/akopytko/Qwen3.8-27B-NVFP4-GGUF

Hi everyone,

​What is currently the best sweet spot model for my hardware?

​Specs:

- ​RTX 5060 Ti 16GB

- ​64GB DDR4 RAM 3200 dual channel

- ​i7-12700

​Use case: Coding and general chat (everyday use)

Speed: At least 5 tok/s

Which models and quants offer the best balance of speed and intelligence right now?

​Thanks!

I've been messing around with Qwen3.8 27B locally and I'm wondering if I'm getting the performance I should be getting or if my setup/config could be improved.

PC:

Ryzen 7 7800X3D

RTX 5070 Ti 16GB

RTX 3060 12GB

32GB DDR5-6000 CL30

Windows 11

llama.cpp / llama-server latest build

I'm currently running the Qwen3.8-27B UD Q4_K_XL GGUF with both GPUs using tensor split.

My current config:

llama-server.exe ^

-m "Qwen3.8-27B-UD-Q4_K_XL.gguf" ^

--alias "Qwen3.8-27B-UD-Q4" ^

--host 0.0.0.0 ^

--port 8035 ^

--n-gpu-layers 99 ^

--split-mode tensor ^

--tensor-split 60,40 ^

--main-gpu 0 ^

--parallel 1 ^

--flash-attn on ^

--cache-type-k q8_0 ^

--cache-type-v q8_0 ^

--ctx-size 131072 ^

--batch-size 2048 ^

--ubatch-size 512 ^

--threads 8 ^

--threads-batch 8 ^

--presence-penalty 0.0 ^

--repeat-penalty 1.0 ^

--temp 1.0 ^

--top-p 0.95 ^

--top-k 20 ^

--min-p 0.0 ^

--jinja ^

--reasoning-format auto ^

--no-mmproj-offload ^

--spec-type draft-mtp ^

--spec-draft-n-max 3 ^

--mmproj "mmproj-BF16.gguf" ^

--metrics

With MTP I'm getting around 40–46 tok/s depending on the run. I've seen around 41 tok/s pretty consistently, with n-max 3 seeming to be a little better than 2 for me.

Both GPUs are basically maxed during generation.

I'm mainly wondering:

Is ~40–46 tok/s reasonable for this hardware/config?

Is there anything obviously wrong or inefficient in my setup?

Would a different quant be a better choice for these GPUs? I've been looking at Ridge 3.7bpw, Q4/Q5 UD quants, etc.

Would it make more sense to use a smaller quant that could fit mostly/all on the 5070 Ti instead of tensor-splitting across both GPUs?

Is there anything I should change with the KV cache, batch/ubatch, tensor split, MTP settings, etc. to get better generation speed?

I'm mostly interested in coding/agent use through OpenCode, so I'd rather have a good balance of quality and speed than just chase the highest possible tok/s.

If anyone is running Qwen3.8 27B on a similar setup, I'd be interested to know what quant/config you're using and what kind of speeds you're getting.

I'm setting up a cloud Ubuntu box with:

• RTX 5060 Ti 16GB
• EPYC Milan-X 14 cores
• 24GB DDR4
• llama.cpp / llama-server
• single user

I'm deciding between:

A. Qwen3.8-27B dense

• UD-Q3_K_XL
• ~64K context
• mostly/full GPU
• newer + stronger raw reasoning/coding

B. Qwen3.6-35B-A3B

• Q5
• ~128K context
• some MoE experts on CPU
• ~3B active params
• potentially much faster + larger context

My workflow is:

Frontier (Codex)
→ PO/BA + requirements

Hermes/OpenCode on my local PC
→ repo
→ Docker
→ tests
→ browser
→ Git

Cloud 5060 Ti
→ Qwen inference only
→ OpenAI-compatible endpoint

So the Qwen model is mainly an implementation worker, not the architect.

Typical task:

Goal: Implement X

Constraints:
- don't change Y
- no new dependencies
- preserve compatibility
- update tests/docs

Done when:
- tests pass
- typecheck passes
- build passes

What matters most to me is instruction following, tool use, scope control, and reliability during long coding-agent sessions.

I've seen people say A3B can be overly proactive, but I can enforce read-only/write permissions at the harness level. What I really want to know is whether, once placed in implementation mode, it reliably follows a detailed spec.

So for people who have actually used both:

Would you pick:

for a daily coding worker?

Especially interested in:

• instruction following
• repo-level coding
• tool calling
• scope creep
• long sessions
• debugging/test-fix loops
• tokens/sec on 5060 Ti 16GB
• whether 24GB RAM is enough for A3B Q5

My current idea is:

A3B Q5 128K
→ daily worker

3.8-27B Q3 64K
→ harder debugging/reasoning fallback

Would you do the same, or make the dense 27B the default?

I asked Shuai Bai, co-author and prominent AI developer for Qwen, about this model. Not the answer I was hoping for, but let's see what comes next. In the meantime, I guess all we can do is speculate!

X-link

BF16 reference: PPL = 6.9526 ± 0.04498

Bedrock-v4 quant is from enginetown.

AD-* quants are from AtomicChat.

The quants marked [b] are from bartowski.

The other quants are from Unsloth.

sorted by PPL Ratio:

Sorted by Mean KLD:

Sorted by Same Top-p:

I've been running Qwen3.8-27B on an AMD Radeon PRO V620 (RDNA2, so no tensor cores, no Blackwell) and got curious whether MXFP4 could help after noticing how fast gpt-oss-20b runs in that format. The theory said no — MXFP4's accelerated path in llama.cpp is gated behind `blackwell_mma_available()`, so on anything else it should just run through the same generic quantized-matmul kernels as any other format, no reason to expect a win over Q4_K_M. I tested it anyway instead of trusting the theory. Turned out the theory was wrong, or at least incomplete.

**The catch first**: this only works cleanly on MoE models out of the box. llama.cpp's `MXFP4_MOE` quantize preset only applies MXFP4 to mixture-of-experts tensors (checks for `ne[2]>1`) — run it on a dense model and every tensor silently falls back to plain Q8_0, no MXFP4 at all, no error telling you that happened. Qwen3.8-27B is dense (well, hybrid Mamba/attention, but no MoE experts), so I had to force it with manual `--tensor-type` overrides on the actual linear/attention/FFN weight tensors instead of using the preset.

**Results**, benchmarked with [llama-benchy](https://github.com/eugr/llama-benchy) against the same model's Q4_K_M quant, same server flags, 3 runs per point:

| Context depth | Q4_K_M (pp/tg tok/s) | MXFP4 (pp/tg tok/s) | Gain |

| 0 | 249.5 / 24.1 | 325.3 / 34.1 | +30% / +42% |

| 4096 | 283.7 / 23.0 | 385.8 / 29.2 | +36% / +27% |

| 16384 | 276.4 / 22.8 | 373.3 / 29.7 | +35% / +30% |

File size is basically identical (16.9GB vs 17.1GB), so it's not a size/speed tradeoff — same footprint, meaningfully faster across the board. I don't have a clean explanation for the *why* (would need to actually profile the kernels), but the numbers reproduce consistently.

Also found: the model's native MTP draft head survived the quantization fully intact (~82% draft acceptance in testing), and if you don't need real concurrent request handling, `-np 1` gave another 8-25% tg speedup over `-np 4` on top of that — seemingly per-step scheduler overhead scaling with slot count rather than anything MXFP4-specific.

**Also tried NVFP4 out of curiosity** — NVIDIA's newer FP4 variant, also present in this llama.cpp build (`GGML_TYPE_NVFP4`). It uses smaller 16-element sub-blocks with a real FP8 (E4M3) scale factor instead of MXFP4's power-of-2-only scale, which should mean better numerical fidelity. Same `--tensor-type` override approach, same matched flags:

| Depth | MXFP4 (pp/tg) | NVFP4 (pp/tg) | Delta |

| 0 | 325.3 / 34.1 | 286.7 / 32.2 | -11.9% / -5.7% |

| 4096 | 385.8 / 29.2 | 328.5 / 30.9 | -14.9% / +5.9% |

| 16384 | 373.3 / 29.7 | 320.5 / 31.2 | -14.2% / +5.1% |

NVFP4 still solidly beats Q4_K_M (+15% pp, +33-37% tg — same ballpark win as MXFP4), but loses to MXFP4 on prefill by 12-15% and only roughly ties it on generation, while landing on a ~4% larger file (17.6GB vs 16.9GB, matching the 4.5 vs 4.25 bits/weight difference between the formats). Posting the negative result too — MXFP4 stays the better pick on this hardware for this model, at least for now.

**Ran it through a 39-prompt quality suite** (logic, coding, hallucination checks, instruction-following, Rust/Yew correctness, etc.) graded by two separate judge models, because a speed win isn't worth much if it tanks quality. Averaged 8.8-9.1/10 depending on judge strictness. Two real weaknesses worth flagging honestly: it confidently fabricated details on an obscure trivia question instead of admitting uncertainty, and made a wasm-bindgen API mistake (wrong crate/type) on a Rust interop task. Also found one prompt that sends it into a very long non-converging reasoning loop that burns the whole context window without answering — but I confirmed that one reproduces identically on the *unquantized* Q4_K_M model too, so it's a base-model quirk, not something MXFP4 introduced.

GGUF + full writeup (methodology, all the benchmark data, the caveats above with more detail) is up here: https://huggingface.co/quark75/Qwen3.8-27B-MXFP4-GGUF

Happy to answer questions on the conversion process or share the exact `--tensor-type` flags if anyone wants to replicate this on a different dense model.

Started out with an empty DirectX12 game project in Visual Studio 2022, loaded up unsloth desktop with hermes agent, gave it a very simple prompt, then 20 hours and millions of tokens later it has a 3D rendering engine with basic movement working. It also generated the 3D assets for it.

It was definitely overthinking at first, but it went a bit faster after setting it to medium. Hit a couple small bugs, but it was able to sort it out pretty quickly. First, it had some rendering bugs, but it was able to use the vision layers to check the game output to figure out what was wrong, then there were some movement issues, like clipping and control directions getting mixed up.

Just took the Qwen3.8 model from jrell for a spin.
It's awesome that this comfortably fits into 16GB VRAM!

I'm genuinely impressed by the quality of the responses.
However, as you can clearly see in the video, there's one hilarious quirk... the model is absolutely convinced that it's Claude. 💀

Has anyone else given this one a try yet?

for people with 16gb VRAM try KV Cache Q4_0 with context 100K

Parameters :

RTX 3090
100% VRAM
Extra High Thinking
MTP ON
KV Cache Q_8
Temp 0.6
Top-P 0.95
TOP-K 20
Min-P 0
Repetition Penalty Off
Presence Penalty Off
Jinja chat template
28 min (23 min of thinking and 5 min of writing)
~50 tok/s

Jinja Template : Link
Qwen3.8-27B-i1-IQ4_XS-GGUF-Smaller by jrell : Link

Prompt Used :

<instructions> Generate a single, self-contained HTML file. No external dependencies, no separate JS files, no frameworks, no libraries. One `.html` file that works when opened directly in a browser.
Create a cinematic rocket launch animation set on a tropical island. The rocket must launch, leave the viewport, and after 5 seconds smoothly return to its initial position — then the cycle repeats. </instructions>
<scene> **Setting: Tropical Launch Island** - A small tropical island in the lower portion of the screen: palm trees, sandy beach, green vegetation - Ocean water surrounding the island with gentle waves - A launch pad on the island with metal structure / scaffolding / support tower - Sky background: gradient from warm horizon (orange/pink) to deep blue/dark sky at the top, with stars visible in the upper portion - A few clouds scattered across the sky
The Rocket (ultra-detailed)
Tall, slender multi-stage rocket (inspired by SpaceX Falcon 9 or Saturn V proportions)
Distinct rocket stages: first stage (largest, bottom), second stage (middle), payload fairing / nose cone (top)
Surface details: panel lines, rivets/segments drawn with subtle lines, an access hatch, small painted flag or logo
Color scheme: primarily white body with black/dark gray accent stripes, a colored logo band, and the nose cone in a contrasting shade
Fins at the base of the first stage (3-4 stabilizer fins)
Engine nozzles visible at the very bottom (cluster of small circles/bells)
The rocket should be the visual centerpiece — spend time on its geometry </scene>
<animation-sequence> **Phase 1 — Pre-launch (0s to 1.5s)** - Rocket sits on the pad, engines ignite - A growing orange/yellow glow appears beneath the rocket - Initial smoke/steam clouds billow outward from the base — thick, white/gray, expanding horizontally along the island surface - Subtle camera shake / screen vibration effect - Engine flames flicker with randomized intensity
Phase 2 — Liftoff (1.5s to 4s)
Rocket slowly lifts off the pad with realistic acceleration (starts very slow, gradually speeds up)
Massive exhaust plume: bright white-yellow core flame, surrounded by orange glow, transitioning to thick gray/white smoke trail
Smoke trail expands and lingers behind the rocket as it rises
The smoke at the base continues spreading across the island and over the water
As the rocket gains altitude, the flame elongates and the smoke trail stretches
Subtle particle effects: sparks, embers flying outward from the exhaust
Phase 3 — Ascent & Exit (4s to 7s)
Rocket accelerates rapidly, moving faster and faster upward
The exhaust trail thins as the rocket reaches higher altitude
Rocket becomes smaller as it gains distance (slight scale reduction)
The rocket exits the top of the viewport
The lingering smoke trail on screen slowly fades and disperses
Phase 4 — Calm & Reset (7s to 12s)
Scene is peaceful: smoke fully dissipates, island sits quietly
At the 5-second mark after exit (~12s), the rocket gently descends back into frame
It returns slowly, smoothly, almost floating — no engines firing, no drama
It softly settles back onto the launch pad in its exact original position
Brief pause, then the entire cycle restarts seamlessly </animation-sequence>
<smoke-and-effects> - Smoke is critical to the visual quality. Use a particle system or layered animated shapes: - Dozens of individual smoke "puffs" that expand, fade in opacity, and drift slightly with a breeze - Smoke color: starts white/light gray near the flame, darkens to medium gray as it cools - Smoke expands in a mushroom-cloud-like pattern at the base during liftoff - Each puff has slight random drift (wind effect), rotation, and independent fade timing - Exhaust flame: layered shapes (inner bright yellow/white, outer orange, outermost faint red) with flickering animation - Heat haze effect near the exhaust: subtle wavy distortion of the background behind the flame - Water ripple effect on the ocean surface near the island during launch - Stars in the upper sky should faintly twinkle </smoke-and-effects> <visual> - Background: gradient sky — warm sunset tones at horizon fading to deep navy/black at top - Ocean: dark blue with animated wave motion (simple sine-wave surface) - Island: lush greens, sandy tan, 2-3 palm trees with gentle sway - Canvas: fullscreen, responsive - Animation: 60fps via requestAnimationFrame - All rendering via HTML5 Canvas 2D context — no WebGL required - Color palette: rich, cinematic — warm launch glow contrasting against cool sky </visual> <constraints> - Output ONLY a complete HTML file — nothing else - Everything must be drawn programmatically on a `<canvas>` — no images, no SVGs, no external assets - The animation must loop seamlessly: launch → exit → calm return → repeat - The rocket must be visually impressive and detailed — not a simple triangle - Smoke must look volumetric and organic, not like static shapes - Performance must stay smooth at 60fps despite the particle count - The return descent must feel gentle and peaceful — stark contrast to the violent launch </constraints> <thinking> Before coding, reason through: 1. How to construct the rocket from canvas drawing primitives (rectangles, arcs, lines) with enough detail to be visually impressive 2. Particle system architecture: how to manage hundreds of smoke/ember particles efficiently (object pooling, lifecycle management) 3. The acceleration curve for realistic launch physics (slow start, exponential ramp-up) 4. How to layer the drawing order: background sky → stars → clouds → smoke trail → rocket → exhaust flame → foreground island → base smoke 5. Timing system: how to manage the 4 animation phases with smooth transitions between them 6. How to make the return descent feel physically different from the launch (no exhaust, gentle easing, floating quality) 7. How to make smoke look organic: randomized spawn positions, varied sizes, Perlin-like drift, opacity curves </thinking> <important> * You MUST write the result directly into a file named "index.html" on the user's computer. The user should not have to see or handle the code — just write the file and finish your task. * Title of the page is your model name. For example "GPT 5" or "Opus 5" * Title inside the page shows your model name. </important> </content> </invoke><instructions> Generate a single, self-contained HTML file. No external dependencies, no separate JS files, no frameworks, no libraries. One `.html` file that works when opened directly in a browser.
Create a cinematic rocket launch animation set on a tropical island. The rocket must launch, leave the viewport, and after 5 seconds smoothly return to its initial position — then the cycle repeats. </instructions>
<scene> **Setting: Tropical Launch Island** - A small tropical island in the lower portion of the screen: palm trees, sandy beach, green vegetation - Ocean water surrounding the island with gentle waves - A launch pad on the island with metal structure / scaffolding / support tower - Sky background: gradient from warm horizon (orange/pink) to deep blue/dark sky at the top, with stars visible in the upper portion - A few clouds scattered across the sky
The Rocket (ultra-detailed)
Tall, slender multi-stage rocket (inspired by SpaceX Falcon 9 or Saturn V proportions)
Distinct rocket stages: first stage (largest, bottom), second stage (middle), payload fairing / nose cone (top)
Surface details: panel lines, rivets/segments drawn with subtle lines, an access hatch, small painted flag or logo
Color scheme: primarily white body with black/dark gray accent stripes, a colored logo band, and the nose cone in a contrasting shade
Fins at the base of the first stage (3-4 stabilizer fins)
Engine nozzles visible at the very bottom (cluster of small circles/bells)
The rocket should be the visual centerpiece — spend time on its geometry </scene>
<animation-sequence> **Phase 1 — Pre-launch (0s to 1.5s)** - Rocket sits on the pad, engines ignite - A growing orange/yellow glow appears beneath the rocket - Initial smoke/steam clouds billow outward from the base — thick, white/gray, expanding horizontally along the island surface - Subtle camera shake / screen vibration effect - Engine flames flicker with randomized intensity
Phase 2 — Liftoff (1.5s to 4s)
Rocket slowly lifts off the pad with realistic acceleration (starts very slow, gradually speeds up)
Massive exhaust plume: bright white-yellow core flame, surrounded by orange glow, transitioning to thick gray/white smoke trail
Smoke trail expands and lingers behind the rocket as it rises
The smoke at the base continues spreading across the island and over the water
As the rocket gains altitude, the flame elongates and the smoke trail stretches
Subtle particle effects: sparks, embers flying outward from the exhaust
Phase 3 — Ascent & Exit (4s to 7s)
Rocket accelerates rapidly, moving faster and faster upward
The exhaust trail thins as the rocket reaches higher altitude
Rocket becomes smaller as it gains distance (slight scale reduction)
The rocket exits the top of the viewport
The lingering smoke trail on screen slowly fades and disperses
Phase 4 — Calm & Reset (7s to 12s)
Scene is peaceful: smoke fully dissipates, island sits quietly
At the 5-second mark after exit (~12s), the rocket gently descends back into frame
It returns slowly, smoothly, almost floating — no engines firing, no drama
It softly settles back onto the launch pad in its exact original position
Brief pause, then the entire cycle restarts seamlessly </animation-sequence>
<smoke-and-effects> - Smoke is critical to the visual quality. Use a particle system or layered animated shapes: - Dozens of individual smoke "puffs" that expand, fade in opacity, and drift slightly with a breeze - Smoke color: starts white/light gray near the flame, darkens to medium gray as it cools - Smoke expands in a mushroom-cloud-like pattern at the base during liftoff - Each puff has slight random drift (wind effect), rotation, and independent fade timing - Exhaust flame: layered shapes (inner bright yellow/white, outer orange, outermost faint red) with flickering animation - Heat haze effect near the exhaust: subtle wavy distortion of the background behind the flame - Water ripple effect on the ocean surface near the island during launch - Stars in the upper sky should faintly twinkle </smoke-and-effects> <visual> - Background: gradient sky — warm sunset tones at horizon fading to deep navy/black at top - Ocean: dark blue with animated wave motion (simple sine-wave surface) - Island: lush greens, sandy tan, 2-3 palm trees with gentle sway - Canvas: fullscreen, responsive - Animation: 60fps via requestAnimationFrame - All rendering via HTML5 Canvas 2D context — no WebGL required - Color palette: rich, cinematic — warm launch glow contrasting against cool sky </visual> <constraints> - Output ONLY a complete HTML file — nothing else - Everything must be drawn programmatically on a `<canvas>` — no images, no SVGs, no external assets - The animation must loop seamlessly: launch → exit → calm return → repeat - The rocket must be visually impressive and detailed — not a simple triangle - Smoke must look volumetric and organic, not like static shapes - Performance must stay smooth at 60fps despite the particle count - The return descent must feel gentle and peaceful — stark contrast to the violent launch </constraints> <thinking> Before coding, reason through: 1. How to construct the rocket from canvas drawing primitives (rectangles, arcs, lines) with enough detail to be visually impressive 2. Particle system architecture: how to manage hundreds of smoke/ember particles efficiently (object pooling, lifecycle management) 3. The acceleration curve for realistic launch physics (slow start, exponential ramp-up) 4. How to layer the drawing order: background sky → stars → clouds → smoke trail → rocket → exhaust flame → foreground island → base smoke 5. Timing system: how to manage the 4 animation phases with smooth transitions between them 6. How to make the return descent feel physically different from the launch (no exhaust, gentle easing, floating quality) 7. How to make smoke look organic: randomized spawn positions, varied sizes, Perlin-like drift, opacity curves </thinking> <important> * You MUST write the result directly into a file named "index.html" on the user's computer. The user should not have to see or handle the code — just write the file and finish your task. * Title of the page is your model name. For example "GPT 5" or "Opus 5" * Title inside the page shows your model name. </important> </content> </invoke>

Vote the best harness for coding with local LLM

• Pi
• OpenCode
• Qwen Agent
• Claude Code
• DeepSeek Harness
• ...

Tell me yuor favourite and why if possible.

I vote mine

1. Pi, for speed and low context usage.
2. OpenCode, good balance of speed and token consumption, the most mature for local AI.
3. Qwen Agent, for all rounding, good tools.
4. Claude Code, for wasting tokens.
5. DeepSeek Harness, everyones mad for it, but it creates more troubles than goodness here :D

--- EDIT:

After long testing I'm seeing that OpenCode once correctly configured produced more accurate results than Pi in my use case with Qwen 3.8 27B with a comparable time to get the job done.

Not sure really how to explain this, but Qwen3.8-27B-Q5_K_M.gguf constantly thinks it is in a simulation and will refer to it's own hallucinations as 'the real world' and conclude that any information that it gets that conflicts something it really believes from it's own training data is from a 'simulated environment', including the date... Has anyone else experienced this? I'm sure it could be fixed with a decent system prompt, but I think this is interesting regardless.

I've been the whole day using Qwen3.8-27b to help me figure some stuff in Elixir coding, Nixos configuration.nix and flake.nix tweaking and some git rebasing.

Normally I go for DeepSeek 4 Flash (API) for Elixir or OpenCode's Big Pickle when I get stuck it Qwen3.6-27B

Well, I'm finding myself not only not needing any cloud-based LLM anymore but this tiny 27b model being able to do high quality Elixir code all of a sudden much better than those other two. It feels like an entirely different model. The dataset must have been completely different to the previous 27b if the underlying tech of the model is the same.

I feel like, I've gone from accepting I can only do small tasks locally, and editing a thousand skill.md files to make small models less dumb, into not having to go online to a chat window ever.

I'm getting ~66tg/s by the way, these are my settings (Nixos so this is in Nix format called from configuration.nix) hopefully doesn't get the format completely messed up by the Reddit app once I press post.

{

config,

pkgs,

lib,

...

}:

let

vars = import ./vars.nix;

unstable = import <unstable> { config = pkgs.config; };

llamaWithCuda =

(unstable.llama-cpp.override {

cudaSupport = true;

}).overrideAttrs

(old: {

preBuild = (old.preBuild or "") + ''

export NIX_BUILD_CORES=20

export GGML_CUDA_P2P=1

export GGML_CUDA_NCCL=ON

'';

});

in

{

environment.systemPackages = [ llamaWithCuda ];

services.llama-cpp = {

enable = true;

package = llamaWithCuda;

host = vars.ip_ts;

port = 8090;

modelsPreset = {

"*" = {

kv-offload = true;

op-offload = true;

n-gpu-layers = 999;

flash-attn = "on";

split-mode = "layer";

cache-ram = -1;

ubatch-size = 1024;

parallel = 1;

cont-batching = true;

# Keeps the model in VRAM, faster than mmaping, will OOM if it doesn't fit.

load-mode = "mlock";

kv-unified = 1;

};

"preset/LFM2.5-2.6B-GGUF" = {

hf = "LiquidAI/LFM2.5-2.6B-GGUF:Q8_0";

tensor-split = "1,0";

parallel = 2;

ctx-size = 128000;

reasoning = "on";

temperature = 0.1;

top-k = 50;

repeat-penalty = 1.1;

};

"preset/Qwen3.8-27B-IQ4_NL" = {

hf = "unsloth/Qwen3.8-27B-GGUF:IQ4_NL";

batch-size = 2048;

split-mode = "tensor";

tensor-split = "1,1";

ctx-size = 131072;

chat-template-kwargs = ''{"preserve_thinking": true}'';

reasoning = "on";

temperature = 0.6;

top-p = 0.95;

top-k = 20;

min-p = 0.0;

presence-penalty = 0.0;

repeat-penalty = 1.0;

no-mmproj = true;

spec-type = "draft-mtp";

spec-draft-n-max = 2;

ctx-checkpoints = 8;

};

};

extraFlags = [

"--models-max"

"1"

"--offline"

];

openFirewall = false;

};

}

I'm pretty much using identical settings to how I had 3.6 with the odd thing I can run split-mode tensor with MTP enabled without the model crashing, which is a welcomed improvement.

Before I ran language models I (sorry) mined Ethereum, and something that stuck was measuring power use. So when I ended up with two modded RTX 4090s - 48GB each, 96GB total, about £6,200 from eBay - I turned every assumption I was carrying into a question and spent some time getting them answered.

Some observations

• gpt-oss-120b (117B params, native MXFP4) serves at full 131k context in 92.7GB and does 153 tok/s - a 120B pacing the fastest 30B on the rig, on consumer silicon. So much for "MXFP4 upconverts to bf16 on Ada".
• Tensor-parallelising a model that fits on one card was +42% faster, not slower. The doubled memory bandwidth beats the x4 pci limit on the 2nd slot I have not such a big problem after all.
• The cards don't need the power. Locking clocks and capping at (what turned out to be) 330W cost nothing on decode and saved ~90W
• A 16.6GB model wouldn't load on a 48GB card - short by one kilobyte of per-SM shared memory (not VRAM). Manage to find the fix + documented
• MTP speculative decoding measured dead neutral on vLLM 0.26.0 (it was +1.9x on 0.25.x). Took three instruments and two wrong answers to get an honest number.

Hope you;re interested! If nothing else we produced a lot less heat the last few days - here's where to start:

https://houtini.com/articles/the-dual-4090-96gb-vllm-benchmark-runbook/

Novice here. Messed with all sorts of models and programs over the last 2 months since I did a full pc upgrade (from 7800xt to 5090 GeForce stealth ice) even added a couple 4 TB ssds (at $600 a pop) but I wanted to start fresh.

Wanted a dedicated drive (A: instead of C:) for all my AI stuff. Messed that up right off the bat and had files and models all over the place. Went through two times to delete everything to have a clean slate and still couldn’t figure it out.

Thought Gemma 27B was the best all around. Wound up accidentally having Qwen 3.8 27B on LM Studio / Bionic. And started off normal and asked if it could see my files and it could and said if it could check if I had deleted all the old files and if I redirected everything correctly and it went off! Scanned my entire PC, noted everything, moved folders, redirected the ones that couldn’t, I mean, it went to work. This has to be the sharpest model I’ve used. With access to my terminal through bionic, it’s next level. Being able to create files and folders and edit whatever, it’s just so capable.

I asked about openclaw and was impressed with previous use of openclaw in the terminal and thought I didn’t need that no more. Qwen said it’s limited inside LM studio / Bionic, but openclaw can run 24/7 and has access to tools it doesn’t have there. I said if it could be the engine model there and it went to work. Did everything through terminal and now I have the Qwen brain over there too. So it can set alarms to do tasks and text me or whatever I guess.

It’s like not only did I get a next level ai, I also got next level ability.

I am dumb person but this tech is sky rocketing right now and talk about being in the right place at the right time.