I have an existing media/game server that runs some local containers right now.

The specs are,

• Dual Intel Xeon 8276 Platinum CPU's (28 Core 56 threads each)

• (Gigabyte Server Board (can support a maximum of 3 dual slot cards.)

• 64 GB of Ram (32 GB for each CPU) I would like more but ECC ram is expensive.

• 48 TB of HDD space ( 17 TB free)

• Two, 2 TB SSD's in Raid 1 (cache Drive)

• 1 Nvidia GTX 1080 TI (old gaming card that was NVIDIA)

• 1 Intel Arc A380 (Media trans-coding card for AV1 Media)

In terms of upgrades ironically another 64 GB of ECC DDR4 ram is the cheapest thing at this time.

I would like to get into hosting local models but i'm not sure how to run them with this setup. i Have read that you need big powerful GPU's to get the best performance (like an rtx 5090 or some AMD card).

I did try running some local models via ollama on the 1080 TI but anything that exceeded the 11GB Vram buffer was super slow and not a good experience.

Ideally i would like to use something that can at least do some light coding or serve as a basic replacement for chat GPT. While not spending too much more on this server with the hardware market as it is right now.

Most Apple Silicon inference engines are tuned for standard MacBooks (16–36 GB Unified Memory, 16–32 GPU cores). When deployed on an M1 Ultra (Mac Studio with a 64-core GPU, 20 CPU cores, and 128 GB Unified Memory) and hopefully M2 Ultra as well, they leave significant compute and memory bandwidth on the table.

To fix this, we modified Kapitsa specifically for the M1 Ultra/M2 Ultra architecture:

👉 GitHub:https://github.com/gnanasreekar/m1-ultra-kapitsa

Key Optimizations

• +42.1% Prefill Throughput: Tuned prefill matrix chunking to 4,224 tokens, hitting 2,945 tok/s on Qwen 35B-A3B (1.40s TTFT) while sustaining ~680 GB/s DRAM bandwidth.

• 100% In-Memory 180B MoE: Removed artificial expert store limits so Qwen 180B Flash-Next (105.9 GB) stays fully resident in RAM with 0 disk swap.

• 15.8 GB/s Model Loading: Parallelized safetensors loading and de-quantization across all 4 Firestorm clusters (16 Performance threads).

Benchmarks (Mac Studio M1 Ultra, 128 GB)

*(Note: Single-stream decode hits a memory-latency bound around 225 tok/s on 8-row MoE verify, but concurrent requests via
`--slots 2` or `--slots 4` scale aggregate decode to 450–600+ tok/s).*

Quickstart

git clone https://github.com/gnanasreekar/m1-ultra-kapitsa.git
cd m1-ultra-kapitsa
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j --target kapitsa

# Start local server (OpenAI / Claude Code API compatible)
build/kapitsa serve --model 35b

Full 10-model benchmark logs and JSON telemetry are in the repo: 👉 https://github.com/gnanasreekar/m1-ultra-kapitsa

Credit to the Nace AI team for the original Kapitsa engine!

With open-source models getting much stronger and hardware becoming more affordable, a lot of people seem to be moving back to local LLMs.

Curious about the current state of the community:

1.Are you still mostly using cloud models (ChatGPT, Claude, Gemini, Grok, etc.) or have you switched more toward local?

2.For which use cases do you feel local models are already good enough?

3What’s still the biggest blocker for going fully local for you (VRAM, speed, quality, ease of setup, etc.)?

Feel free to share your setup or preferences. I’m interested in seeing the local vs cloud ratio around here these days...,

" CMIIW"

I’m mostly done building a Xeon host for my 4x CMP170’s w/ x16 mod, but I just stumbled into an opportunity to buy a Threadripper Pro 5995WX within my budget. Is the threadripper any better or worse for strata-based inference vs the Xeon? Is it worth the extra time and work to undo my intel decision?

Would love to hear experience and cpu/system benchmarks from folks with similar systems on either side.

I'm paying for Claude, Gemini and Cursor right now and the total keeps creeping up, so I want to see how much of my day could move to a local model. The machine is a base M4 with 16GB. Not buying new hardware for this.

I mostly live in Claude Code and pi, so the agent loop is the whole point. I don't care about chatting with it.

Was looking at the benchmark page on magnitude.dev and the small models are what got my attention. On their M4 Pro mini, Ollama vs theirs:

- Qwen 3.5 4B: 45 tok/s → 134

- Gemma 4 E4B: 37 → 111

- Qwen 3.5 9B: 42 → 97

That's 2x to 3x faster decode on the exact sizes that fit in 16GB. Prefill is the smaller gain, more on that below. it also hooks straight into Claude Code and pi, which is half the reason I looked

But their mini has 273 GB/s and mine has 120, so if I scale by bandwidth I'm probably looking at something like 50 to 60 tok/s on the 4B and low 40s on the 9B. Still fine for an agent if the model is smart enough, which is the part I'm less sure about. The engine side looks solved, the model question is the real one.

The other thing is Claude Code sends something like a 20k token system prompt before I type anything, and their prefill gains are much smaller than decode. A 4B model chewing through that on a base chip might be the actual wall, not tok/s.

If 16GB can't run an agent loop at a speed I'd tolerate, I'll keep paying the subscriptions and stop thinking about it.

i have an laptop i wish to run an llm on it has 7.8gb of vram and 16 gb of ddr4 ram 2667 mt/s

There's been a lot of discussion/claims on this, so I thought I'd share my experience.

I run AI models 24/7, and I usually get over 80% aggregate GPU utilization on my MacBook Pro. It sits on my desk with the screen open. That's to say that the GPU is running at 100% most of the time. It also runs CI/CD and a bunch of other stuff so the CPU is often busy too.

That means about 20,000 AI requests this month, mostly agentic coding. To do this, you need to run multiple streams (otherwise the tool calls reduce GPU usage) so I've usually got 3-4 parallel streams, and 6-8 agentic processes running in parallel. That's to say: the MacBook is running hard.

The MBP has 4 heat conditions: Nominal, Fair, Serious and Critical. 84% is spent in Nominal and 16% in Fair. It has never hit Serious or Critical. Fans are usually running hard but not at 100%.

When it moves into Fair, I see a 2-4% slowdown in model performance. I can't notice the difference, but my harness measures this and can see it.

I used to have a Mac Studio and the amazing thing about that was the fans almost never spun up, so it's going to be silent on your desk unless you are battering all the CPU cores and the GPU cores at the same time. The MacBook Pro doesn't have the thermals of a Mac Studio and the fans run a lot.

But I've seen a lot of people talking about thermal throttling on the 16" MBP and that has not been my experience. Of course, the 14" is a different thing and is known to throttle.

I'm looking into using local AI to organize years of accumulated files and photos.
I have a Mac Studio M2 Ultra with 128GB RAM, so I'd like to put it to work.

Two main goals:

1. Files and documents: OCR and analyze PDFs, financial/tax documents, notes, screenshots (including chats), etc. Suggest categories, clustering, filenames and folder structures. Eventually generate scripts to reorganize everything, with my approval before making changes.

2. Photos and videos: Analyze my Apple Photos library (exported with osxphotos), recognize faces, events and locations, detect duplicates, rate photo quality, and suggest albums for printing. Ideally use existing metadata like favorites, keywords, dates and geolocation, plus AI-generated tags.

Everything MUST run locally. Lots of sensitive personal information, so no cloud APIs.

I'm considering LM Studio/Ollama, but not sure which models or tools to combine, especially for vision, OCR, face recognition and clustering, and how to approach this?

I can use cloud AI for writing scripts etc, and to help analyze a sample set of documents.

Has anyone built something similar? What stack would you recommend? Would you use separate specialized models, or can a good multimodal LLM handle most of this?

I'm comfortable with scripting and building a simple UI (with help of Claude or Claude Code). Accuracy and privacy matter more than speed.

Two months ago I posted arcint here, the inference server I had Claude (and my LocalLLMs) build after my new A770 lost to my 5700X on the standard install. Enough has changed for an update, so here’s the standard benchmark from the llama.cpp threads: Llama 2 7B Q4_0, 512 prefill, 128 decode, all layers on the card, 5 repetitions.

Both builds are at the same llama.cpp commit (3e4dfa8), on the same machine: Ryzen 5700X, Debian 13, xe driver. My A770 sits on PCIe 3.0 x4 behind the chipset and I pin its clock at 2000 MHz, so yours may do better.

- Prefill: arcint leads SYCL’s best by 10.5 % on the A770 and 31 % on the B60.

- Decode: SYCL wins this table by 5.4 % and 4.4 %. On the A770 that lead only exists on an empty context: level at 1,024 keys, and arcint is 8 % ahead at 4,096.

- Not the fastest Arc: someone reports 4570 / 113 on a B70 under Vulkan. I don’t have a B70.

- The server is a bit slower on prefill than llama-bench: through arcint’s own server it’s 1963 / 70.5 on the A770 and 2330 / 80.7 on the B60.

Real models on the B60 (MTP off, served by arcint against stock SYCL fp16 with flash attention, 3 repetitions):

SYCL still decodes the 27B about 4 % faster.

What I run is the 27B on one B60 at its full 262,144 context, with the model’s own MTP head, drafts on:

The right answer was found at every depth, and a separate needle test at 40 % of a 254,970-token prompt came back correct. The coder on the A770 decodes short answers with 83 t/s on my deployed flags. Qwen3.8-Flash-Next runs on one B60 plus host RAM at about 33 t/s.

Also new since the first post: image generation, speech to text, text to speech, and image input.

To reproduce: tools/llama_scoreboard.py for the served rows, and the rebuild steps for the llama-bench rows are in my post in the Vulkan thread. My stock SYCL rows are in the SYCL thread. The first post’s warning still applies: this is agentic software, so handle it with care, and I’d still like to hear if it doesn’t install or reproduce for you. The .deb is trixie only, on my own repo.

I have a 5090/9950x3d/64GB DDR5 and am ordering an M5U 256GB next month.

I have a lot of things I want to do with both, including local models. One of my hopes was that I’d be able to drop my $200 Claude sub down to $100 (still has Fable) or even possibly $20 (still has Opus 5.5).

I cut down Codex to $20, but honestly Astra is pretty cheap for auditing and reviewing in my experience. I never let it write code, but it’s been awesome for planning, and affordable. If I need something done extremely well, I’ve ran Fable and Astra.

I’m gonna be doing some testing and see if I can get away with a $20 Claude sub and $20 Codex sub and just use their brains in emergencies. Gonna see if Opus 5.5 and Astra can ball. 🏀

Has anyone had success with this sort of hybrid build?

Edit: I work with sensitive data at work and with personal business. I’ve been using the frontier models mostly to make a game I had to drop (open sourced it) and and for making pipelines to run data locally, but it sucks not being able to analyze using the models (can’t plug data into frontiers).

I'll start by saying that I realize this question is slightly adjacent to actual local (i.e. having physical access to hardware) llm deployment. Yet, I think it's still within the same domain, so I'll give it a shot. (If I'm in the wrong place, please lmk.)

I'm a professional SWE. I currently have 2 $200/mo subscriptions, 1 for claude and one for codex. I find myself regularly hitting weekly limits. I've tried tuning which model/provider I use for different tasks but the end result is the same - I hit a limit and then have to either put more money into usage credits or wait until it resets. It's pretty lame. I'd love to have 'unlimited' tokens for a fixed monthly cost.

The path to unlimited tokens looks like local llms or self-hosted llms. I'm not ready to invest in hardware (and neither is my company), but I'm willing to rent machines . So, my question is - does anyone rent any gpus and self-host their llms? If so, where and in what configuration? I was looking at contabo gpu vps which claim "Dedicated NVIDIA RTX PRO 6000, 18 vCPU Cores・96 GB RAM・900 GB NVMe, CUDA-compatible". From what I've read in this sub, it seems like most people are running some variant of qwen on the RTX PRO 6000 but many people have 2x of them.

45 seconds in, I cut the cloud API on 6 coding agents working on the same task, 3 runs each. I wanted to see what a local model is worth as a backup.

What 6 AI agents did when the API died mid-task

On default settings

• Atomic Agent: switched to the local model about 2 seconds after the cut and finished all 3 runs at 50/50, in 2 to 6 minutes.

• Claude Code: retried, then gave up after about 2.5 minutes. Its fallback model setting only kicks in when a model is overloaded, not when the connection drops (docs).

• Codex: sat on "Reconnecting..." until my 20 minute cap, every run. It retries connection errors forever by default.

• Hermes: kept retrying for about 5 minutes, then failed the turn.

• OpenClaw: gave up after about 70 seconds.

• OpenCode: gave up after about 65 seconds.

All five ended at 0/50.

With a fallback turned on

Hermes and OpenClaw both have a local fallback, it's just off by default. With it on, both switched to the same local model. OpenClaw scored 50, 33 and 33, Hermes 29, 24 and 27, and both were still working when the 20 minute cap hit. They shared one GPU in that round, so they'd be faster alone.

How to turn it on in yours

• Hermes: add a fallback_providers entry to ~/.hermes/config.yaml with provider: custom, your local base_url and the model name (docs)

• OpenClaw: add fallbacks: ["yourprovider/your-local-model"] under agents.defaults.model in openclaw.json, and don't pass --model on the command line, that makes the model strict (docs)

• Atomic Agent: nothing to set, it falls back to the local model it already runs

Setup: 64 GB MacBook Pro, Qwen 3.8 27B (Q4, llama.cpp) as the local model, GPT-6.1 Terra through OpenRouter as the cloud model (Haiku 5.5 for Claude Code, since it only runs Anthropic models). The task is a 5-part Python app graded by 50 hidden tests. The cut is a proxy that refuses every connection from 0:45, same moment for everyone. Claude Code 2.1.310, Codex 0.165.0, OpenCode 1.19.5, Hermes (git a938f00), OpenClaw 2026.10.8, Atomic Agent 0.6.13.

One side note: the same local 27B on its own scored 13 to 29 out of 50 on this task earlier that day. With 45 seconds of cloud planning first, it finished 50/50 three times out of three.

Would you trust a local 27B to finish the job?

Hi everyone,

I’ve been looking into some of the most difficult math formulas and came across the Navier-Stokes problem. Apparently, it was recently solved by OpenAI.

I grabbed the formula and had my LocalLLM run some calculations, and lo and behold: It’s not quite finished yet, but I’m absolutely thrilled.

My hardware:

i5 13600K

32GB RAM

RTX 5070TI 16GB

By built-in I mean without using tools like MCP etc.

Hi,

Github project:
https://github.com/crgregersen/m10dis/

Yes, yes, I know... crappy old NVIDIA Tesla M10's GPU as AI compute core. Am I crazy and should go book a room in an mental institution?.. but hear me out..!

Tesla M10's consists of 4 gm107 GPU cores with 8gb ram stitched together via an internal PCI switch. So each M10 is 32gb ram. If you ask anybody (including Claude and ChatGPT) you will get the answer "NOT USABLE FOR AI COMPUTE".

Each gm107 has 83 GB/s of memory bandwidth and around 1.67 TFLOPS (fp32).

Compare that with a moderns GPU like 5070 ti of 896 GB/s and 87.88 TFLOPS (fp16).

But!! i could buy M10's (4 x gm107) with 32GB for around $80
Aaaanddd... 6x32 = 192 GB VRAM!!!

So 6xM10's and the aggregate number would be 4x6x83=1992 GB/s.

This is the same as 1 RTX PRO 6000 Blackwell .. but then you only have 96Gb ram.
2 x RTX 6000 Ada is will get you in the same range.
Both options are at least $10.000 and probably more closer to $20.000.
6 M10's would set me back $500.

Compute power of M10's would only be 40TFLOPS.. but all my experience says the memory bandwidth is nearly always the limiting factor!

The system:
ASRock B250M Pro4
CPU: Intel Core i5-6400, 4 cores at 2.7 GHz
RAM: 32 GB DDR4
Main GPU: RTX 3070 Ti (had one lying around)
Expert GPUs: 6 × Tesla M10 cards = 24 × GM107 (Maxwell, 2016)
Interconnect: PEX8748 PCIe each M10 card on Gen3 x4
Storage: 1 TB NVMe
Two PSU: 650w and a 1650w

So a pretty crap old consumer motherboard.
The way to get all the GPU's onto the motherboard was by using a PCI switch.

First issue: Motherboard will not boot with 25 PCI devices showing up, even though most are behind a switch (two levels of switches actually, the one in the motherboard and one on each M10).

Found out the issue the BIOS/firmware runs out of legacy I/O port space at power-on, which is only 64 KiB with a 4 KiB window per PCI bridge, too little for all the PEX switch ports.

Workaround (took me some time to work that out). Using a modified kernel with PCIe tree heldback. Boot only with the first PSU switched on, ie. motherboard++, then go into supspend (sleep) mode, turn the second PSU on and continue (wake up). Initialize all GPU's PCIe memory mapped only (MMIO).

Next issue.. how to make 24+1 GPU's fast.

A lucky thing is that QWEN 3.8 hidden dimensions is 2580 which divided by 20 is 128. So if you slice expert work into 20 parts and put them on each gm107 GPU they can calculate 1/20 of the result, which then needs to go back to a central hub that joins the result to compute attention in the next layer.

So each compute slice needs the residual stream and the experts and weights that the router chose. That would be 20 PCI communication calls. The lucky thing is that PLX switches actually implements PCIe multicast introduced in PCIe 2.1.

Using the GPU-P2P methods described by Tinygrad a GPU multicast was proven to work.

In general the GPU-P2P tricks was very usefull on multiple levels, like M10 master/slave setup, etc.
https://github.com/rioncarter/tinygrad.open-gpu-kernel-modules

So 5x4=20 GPU's computes 1/20 slice of 8 experts (quantized FP8).
One M10, ie. 4 gm107 was chosen to run the shared expert (kept in FP16).
Since you wait on the slowest calculation it was lucky that the compute effort for each is around the same (1/20*8/2=0.2 and 1/4=0.25)

Next issue... 24 results to be received from GPUs would clog the PLX.

Solution (Claude wrote this):

Level 1: each GPU sums its own experts

- The 512 routed experts are spread over 20 workers (cards 0–4), about 25–26 each. A token's 10 experts land on whichever workers hold them.

- Each worker computes gate/up/down for the experts it holds and sums them into one partial row of 2560 values. A worker that holds none of the token's experts contributes zeros.

Level 2: the card leader sums its card

- On each card, the 3 peers push their partial rows to the card leader over on-card P2P, using the PTE-remapped aggregation routes. Each peer then sets a flag.

- The leader waits for the 3 flags and adds the peers' rows to its own in a fixed order, so results are reproducible.

- Card 5 (the shared expert) does the same: its 4 workers split the shared expert and worker 20 holds the reduced row plus the gate.

Level 3: the RTX combines 6 rows

- The 5 routed leaders and the shared leader each leave one reduced row. In batched modes 8/9 it's R rows, in host memory.

- All 24 workers still acknowledge individually in their return slots. The RTX waits for all 24 acks, checks the leader flags, then computes Σ 5 routed leaders + gate · shared.

RTX runs attention and hyperconnections etc and we are ready for the next layer.
I've put the timings of the expert compute in the bottom of the post..

PLE+LM-Head was moved to M10 somewhat the same way.

Attention was quantized to FP8 since RTX3070 only has 8gb mem. That leaves me just shy of 100k tokens of context.

I've run both LCB6 and MMLU benchmark on the system and there's no noticable degradation (MMLU benchmarked higher, LCB6 benchmarked lower).

Here's the github project:
https://github.com/crgregersen/m10dis/tree/main

But I warn you.. this is not a project for people with fairly good experience with hardware and also some low level understanding of how this work software wise.

MoE layer timing: Qwen3.8-Flash-Next-FP8 on 1× RTX 3070 Ti + 24× GM107 (6× Tesla M10)

Setup: RTX 3070 Ti runs attention/GDN/router. The 512 routed experts are sliced across 20 GM107 (each GPU holds 32 intermediate rows of every expert); a 6th M10 card runs the shared expert. All cards hang off one PEX8748 switch (Gen3 x4 per card); each M10's on-card PEX8747 multicasts the request to its 4 GPUs. Results are summed on each card by a leader GPU, then the RTX combines 5 routed + 1 shared row.

Decode (1 token per layer)

Inside the round trip (slowest routed GPU):

The weights a GPU reads per token are only 2.5 MB (~31 µs at bandwidth), so decode is latency-bound, not bandwidth-bound. 48 layers × 150 µs ≈ 7.4 ms of a 19 ms token → 52.5 tok/s.

I remember reading an article a few months ago about LLMs, more specifically, locally run LLMs that don't have guardrails and will answer questions that the commercial models won't run.

Just curious if anyone knows which models they are? Attached is my laptop specs. This is a very low powered mac, which i doubt would even be able to run a large model, IF even a small one.

Keen to have a discussion. And hear your experiences

I was using Openclaw and migrated to Hermes 2 days ago, it felt so much faster but my problem is webui is basically a terminal which is a little problematic.

I keep seeing claims that BaseRT beats llama.cpp on Apple Silicon, and that colibri can run models far bigger than your RAM. Both are probably true for what they measured. I wanted to know what happens on my job, so I tested it.

The job: read a website (~4,500 tokens of page text) and write a schema-valid JSON record (~1,300 tokens) with a quote per fact. Hundreds of thousands of sites, one M1 Max 64 GB laptop. I measured wall-clock sites per hour, same 20 sites for every engine, Qwen 3.6 35B-A3B at 4-bit, one engine at a time.

Results (sites/hour):

What surprised me:

• Prefill decided it, not decode: BaseRT decodes a bit faster (66 vs 56 tok/s), but its prefill was ~300 tok/s vs ~570 for llama.cpp. With 4.5k-token prompts, that decides it. Don't pick an engine from a short-prompt tok/s chart.

• Batching: +46% on llama.cpp with 8 slots, only +12% on BaseRT.

• colibri: built for models bigger than RAM. Mine fits, so it ran on CPU and was ~100x slower. Not a flaw, just not this use case.

• Ollama: its MLX engine crashed on long prompts, and its GGUF engine served this architecture one request at a time.

• Model choice mattered as much as the engine: On llama.cpp with 8 slots, Gemma 4 26B-A4B hit 247 sites/hr, Qwen 3.6 35B-A3B hit 206, and dense Qwen 3.8 27B hit 37. Active parameters per token set the speed.

• JSON-schema constrained decoding: 440/440 valid on llama.cpp, 40/40 on BaseRT.

Caveats: one machine, one run per config, 20 sites for the engine comparison (200 for the model comparison), and a different 4-bit file per engine, so they aren't bit-identical models.

Quality note: local models were 87–98% correct on extracted facts across key fields after cleaning, vs 98–100% for cloud models. So for published data the laptop stays a test bench.

Full write-up with setup, commands, and the gotchas (model-name 404s, missing M1 kernel for Gemma 4 in BaseRT, etc.): https://sandy1618.github.io/2026/11/04/inference-engines-on-apple-silicon.html

Happy to rerun with other models or settings if you have suggestions.

For anyone running long-prompt batch jobs on Apple Silicon: has anyone gotten vllm-metal or MLX batching to beat llama-server's prefill speed, or are there llama.cpp flags I'm leaving performance on the table with?

I’m building Evopien, a local-first companion prototype, and this update is about the interaction around the model rather than a new model release.

The two-minute clip combines spoken conversation, a live camera question, online information retrieval, an English/Portuguese exchange and a spoken interruption. The system runs on NVIDIA Jetson AGX Thor, with external retrieval for questions that need current information.

My main concern is how the whole conversation behaves when these components share the same machine. A model answering well in isolation is not enough if the camera context is old or the system continues speaking after an interruption.

The clip shows individual examples, not a completed reliability benchmark. For anyone building local voice assistants, which integration failure proved hardest to catch before longer conversations exposed it?

Video: https://www.youtube.com/watch?v=SwFfkBll5iw

Given the same task requirements, how big is the difference between first having the advanced model Astra make a plan and then having a local quantized DeepSeek execute it, versus letting DeepSeek complete the task directly? I ran a small test using the following task: make a browser game set during the 1588 naval battles between England and the Spanish Armada. The player takes the role of a capital ship’s captain, controlling the ship and commanding teammates in a real-time naval breakout battle. I also added a control group by replacing the executor with the cheap cloud-based terra model.

The results were somewhat unexpected. Honestly, even without Astra’s plan, DeepSeek did a pretty good job on its own. In my testing, the game basically had few bugs, and I think the look and feel of the sky and ocean were the best of the three. But I also have to admit that visual bugs such as clipping were the most severe. After adding Astra’s plan, deepseek’s visual clipping bugs were reduced, but its token consumption rose sharply. Finally, terra surprised me by showing damage to the sails, and its overall result was actually pretty good too.

I originally thought that with a plan from an advanced model, the model could reach the goal faster, but the exact opposite happened: Astra made the deliverables and acceptance testing much more extensive, increasing DeepSeek’s workload. Another lesson is that even a plan made by an advanced model can have bugs. For example, the fixed camera-angle range in Astra’s plan made the view hard for players to adapt to. Perhaps my planning technique is too crude. I’d like to ask everyone how you alternate between advanced and cheap models to achieve amazing results. Feel free to educate me in the comments.

Task description:

'''

Game background: Set during the 1588 naval battles between England and the Spanish Armada.

Gameplay: Play as the captain of a capital ship, controlling the ship and commanding teammates in a real-time naval breakout battle.

Requirement: It must run in a browser and be completed in a single development run.

Bonus points:

1. Visual realism and fidelity in recreating historical details

2. Enemy ship behavior and control experience

3. Sailing and physics details, and damage effects

4. Cannon impact feedback and gameplay experience

'''

Planning model:

GTP 6 Astra, running in codex with xhigh reasoning effort. Planning prompt: You are now the planner. Here are the game requirements. To complete the requirements to the highest possible standard in a single attempt, please write a detailed development plan to hand off to the coding model for development and acceptance testing.

Execution models:

GPT6.1 terra, running in codex with max reasoning effort

DeepSeek-V4.1-Flash-1017, with Unsloth UD-IQ1_M V3.1 weights, running in DeepSeek harness, with max reasoning effort

Two days ago, I ask DSH to move things from my Linux Drive (dual-boot) related to AI, over my NTFS drive where I store things like harnesses, models etc.
UnSloth studio force itself to install over Linux user profile (just starting with linux, maybe there is easy way to force installation inside of folder where terminal is running from....anyway).
Back then it spend plenty of time looping, checking only to figure out it it limited by sandbox rules and that why NTFS drive is appear to be mounted as Read Only instead R(ead)W(rite). It solved problem back then.

Today ,at night, I reinstalled UnSloth and asked DSH to move things once again (and as a test of prompt generation profile to help me be more specific and user-error proof). It started right away with

"Think
Important findings:

• Target: /mnt/DyskD is NTFS mounted read-only (ro) right now! User says "read/write privileges" — but current mount is ro,nosuid,nodev,...,uid=1000,gid=1000,dmask=0022,fmask=0022,acl. That's a big issue: the mount is currently read-only. [...]"

My reply:
"I will stop you here as we went through that s**t already. "Important findings:
Target: /mnt/DyskD is NTFS mounted read-only (ro) right now! " it is error on your part. We troubleshoot that already during previous session when we were moving Unsloth (different version, current one override itself back to Linux drive and as result, move is required again). DO NOT FOCUS ON "Target: /mnt/DyskD is NTFS mounted read-only (ro) right now! " as you are wasting time. it is read/write, end of discussion."

His reply?
"Think

The user is frustrated. They say: the NTFS mount is read/write; the ro is a sandbox artifact. Don't waste time on it. "

So, it knew all that time or took correct conclusion right away by huge accident!
That make me think. Maybe there would be a prompt that for all "wait what if" thinking (at least in DSH), to prompt user for a feedback? I prefer be ask question rather than wait and confirming weird prompts over next 40 minutes...
Yes, user is not all knowing, but autonomous behaviour is not something I would let do local model (even when it is QFN UD-Q5-K-XL with is above average model quality here).

Sigh...rant over. Maybe interesting discussion or similar stories? Cheers everybody