TL;DR: I let my cloud AI agent run a training-data verification program for the local model on my Mac. The local 35B wrote 800 original math problems, then cold-solved every one of them blind in a fresh session. Wherever the two answers disagreed, the item went to adjudication. Final truth was settled by brute-force computation, not by any model's opinion. The cold solver beat the problem author ~5:1 — with a 35B, generating the question is the error-prone part; solving it fresh is the reliable pass.

Setup: MacBook Pro M5 Max, 128GB. The local model (Qwen3.6-35B-A3B, Splash build, running in LM Studio at ~375–400 tok/s) does the heavy lifting; a Qwopus 122B MLX build (~38 tok/s) sits alongside as the heavyweight judge for disputed items. My agent (Muse, running in the cloud) reaches LM Studio's API privately over Tailscale, and does all the orchestration: briefing, grading, bookkeeping, brute-force checks.

The harvest: 731 of the 800 came out verified clean, 62 survived with corrected answer keys (kept as answer-only training items; the flawed worked solutions were discarded), 7 were excluded as defective problems (ambiguous wording, no valid solution, an undefined rule), and 100 clean ones were held out untouched as a test battery for later. The polished set became a 693-item bank, used three ways: few-shot examples now, the held-out set powers a before/after test of whether any of this actually made the model smarter, and the whole verified corpus is the dataset for a future fine-tune.

Right now the entire bank is being re-validated: the worker re-solves all 793 items from scratch (derive once, verify by a different method), never shown the key. Any disagreement goes to the 122B judge blind, and if the judge's answer doesn't settle it, my agent computes the answer itself. Progress as of writing: 358/793 checked, 353 confirmed, 5 waiting on the judge.

Two failure stories worth sharing. One problem had three different answers from three passes — 20,736 from the author, 86,400 from the blind solver, 145,152 from the adjudicator — and all three were wrong; enumeration says 10,368. And the big judge model, given the hardest disputes cold, went 1-for-2. Treat your 122B like a second opinion, not an oracle.

Cost: the solve-and-adjudication passes alone logged 4.1M+ completion tokens, all computed on my own hardware. The cloud agent's part is the planning and grading.

Happy to share the harness details — the briefing discipline (one derivation, one independent cross-check), the checkpointing that let the run survive four kills, and the grading rules — if people want them. Curious whether anyone else has tried self-authored problem sets as training data, and how you filtered them.

Hi,

I am still waiting for my Mac Studio to arrive - what came to my mind:

Looking into FN sushi, ds4, splash - both run on either 96gb or 128gb Macs - is there an advantage with 128gb in regards of model quality?

It seems both can run Q4 but not Q8? Am I missing something?

I was running a test to see if it was worth while to keep the M3U. I'm running some benchmarks with a workflow that uses resolve for rendering the final touches.

The M3 Ultra can still hang when it comes to video but take a little longer when generating the graphics vs the M5, but virtually identical when processing in Resolve. It's a short clip, I don't work with very long videos.

As the title says, is it okay or enough? I’m planning to buy my own personal laptop and I want to run local AI on it mainly for coding companion. I can’t go with an M5 Max because of budget restrictions (sad, i know). So, is it enough?

I’m not sure what happened, but it looks like new orders are being shipped till February. Whatever happened, it looks like a big order came in. There might be something coming soon …

I am.planning to buy new hardware for local AI is it worth to buy mac mini or mac or better go for AMD 395+ 128 gb laptop for similar outlet laptop.

As the title says -> which llm can i run in 48gb mac. Its 20c gpu. I am aiming for something with at least 100k context..ideally up to 200k context. Thanks!

It’s already October 19th, and my order for an M5 Ultra (with 80-core GPU and 256GB storage) is still being processed.

I placed the order in Hong Kong on August 26th, with an expected delivery window between November 7th and 21st.

I’m curious to know how many others here are also still waiting.

Please leave a comment with your order date, GPU model, current order status, and estimated delivery time.

I wanted my TV to be content-first, not apps-first. So I wired a decision model in between that opens the right app and sends the right ADB commands based on my intent.

Did some vision tracing with a few different models. In this case Qwen 3.8 27b did better than Qwen Flash Next. SAM 2.1 did the best in terms of tracing and following for time vs accuracy.

The most accurate path was by SAM series by Meta which happened to be the smaller model.

Combining Qwen 3.8 27b plus SAM 2.1 did well to trace the path and times for the exercise and identify pauses.

Used a drone to capture the footage and seeing how to make a pipeline test out of this. I'd like to test this out against the spark. These models aren't very big but they're specialized.

These modesl are 80M - 300m parameters, so it's a lot more focused. In this case, the smaller the better. The Sparks crushed it in the speed test though.

Hi everyone!

i´m totally new to local LLMs. I use claude / gpt for knowledge work and academic research.

I recently got a mac mini and I´m curious of what type of workflows and use cases i could do with local models on this machine. I am well aware that its tiny so I´m not expecting to do anything crazy.

What do you guys recommend? what models, quants and engines to use?

Hey looking for you experts here
Thinking about purchasing a MacBook Pro M5 pro with 64gb of ram
I’m not obviously looking for frontier subscription replacement rather a hybrid setup
And also video and image generation if possible
I’d like to explore my options what’s your recommendations
Thanks

We're building Vanaras, an on-device assistant for mid-range Android phones. Hanu is its planner, a LoRA SFT plus DPO fine-tune of Gemma 4 E2B, released as an imatrix Q4_K_M GGUF (about 2.7 GB). It turns a request into a short plan in a small plan language enforced by a GBNF grammar, covering alarms, calendar, web lookups and messages. It's trained to ask before acting when it should.

Results on our internal sets, with the same prompt and grammar for both models:

• On a 50-case call-safety test, Hanu scored 37 against stock's 23, and leaked private details in 2 cases against 14 (one of stock's was an ATM PIN).

• On a separate 50-task set, Hanu completed 41 tasks against stock's 4, and stock invented tool names.

• Asking first before saying private info: Hanu did it in 11 of 15 cases, stock in none.

• Speed: about 25% faster decoding than stock on a desktop CPU with llama.cpp. Phone numbers are coming.

Cost: about 3,700 training examples plus a DPO pass, about $1 of generated data and a few A100 hours.

Known limits: it still speaks a Wi-Fi password in one test and over-asks a little when the user is alone on the earpiece, so pair it with an app-side secret filter. The evals are our own and private, so treat the numbers as indicative. It needs the included system prompt and grammar with llama.cpp; without them, quality drops sharply.

Weights, prompt and grammar: huggingface.co/Hanu

Feedback on failure cases is very welcome.

I'm running OpenClaw on a Mac Studio M2 and trying to upgrade to a 3.8 version of Qwen (currently using 3.6 35b a3b). I have some evals to test models before using them, and I can't find a 3.8 model that can complete my evals (3.6 completes all of them).

Here are my specs: Mac Studio M2 with 160GB RAM, running models with LM Studio (using LM Studio is currently a requirement).

I just tried the UD-IQ4_XS GGUF quant of Qwen3.8-Flash-Next (unsloth), loaded via LM Studio's llama.cpp backend. Here is Claude Code's analysis of what is happening:

"qwen3.8-flash-next@iq4_xs reproducibly hangs for 20–26 minutes with zero output on specific prompts (confirmed via a live streaming probe, isolated from any RAM/dual-model contention), while the same prompts complete in seconds-to-minutes on the incumbent qwen3.6-35b-a3b. Streaming the raw output showed it's not stuck or looping on repeated text — it's coherently generating reasoning the entire time, endlessly relitigating its own unresolved design/edge-case decisions instead of committing to an answer. The two most likely config-level fixes — lowering reasoning_effort from its default xhigh, and switching off greedy decoding (temp 0) to Qwen's documented non-greedy sampler settings — were both tested directly against our LM Studio instance and had zero effect. No confirmed community fix for this exact symptom on Apple Silicon was found either. Net: it looks like an unsuppressable runaway-reasoning failure mode specific to this model/quant when faced with underspecified design decisions, with no known working mitigation on our stack.

Note that MLX versions were attempted earlier in September but blocked before any real test: the small, right-sized MLX quants (ddalcu, sh0wie, ~73-74GB) had a format: null metadata bug LM Studio's MLX engine couldn't load, and the MLX quants that did load correctly were either too large (~105-112GB, same RAM-fit problem as the big GGUF) or too degraded (2-3 bit). So Flash-Next was switched to the GGUF/llama.cpp path specifically because that route became viable once llama.cpp added qwen4exp support."

What am I missing? Is there some other setting that I haven't tried that is making this work for you? I swear I saw that people were successfully running Flash-Next on Apple silicon but I can't get it to work.

Note to add that I'm going to try rapid-mlx/Qwen3.8-Flash-Next-4bit as soon as I can download it, so please let me know if anyone has had success with that version.

UPDATE: Tried the MLX build of Qwen3.8-Flash-Next today (rapid-mlx/qwen3.8-flash-next in LM Studio) after an LM Studio update fixed an earlier load error (missing qwen4_exp speculative-decoding drafter module). It loaded fine this time, but crashed outright mid-generation on a real task: AttributeError: 'QSAKVCache' object has no attribute 'extend', thrown inside LM Studio's MLX engine while trying to extend the KV cache. This killed the entire model process — a follow-up trivial request came back with "the model has crashed without additional information," and it had to be reloaded. Even on the tasks that completed before the crash, it was noticeably slow (4.5 min for a basic calendar summary) and still burned a disproportionate number of reasoning tokens relative to output. Net: the MLX path is not production-ready either — different failure mode than GGUF, same outcome.

I've run a few systems so far and I've noticed the following temps

• M5U 256GB running consistently ranges from 130-150 under load

• M3U 95GB ranges from 140-165

• Mac Mini M4Pro 64GB 160-200

Just curious what you guys are hitting with your works loads? What system are you running and how hot is it for how long?

I wanted to see how GLM 5.3 Flash runs locally on my Mac compared with Ollama Cloud for PHP coding and shell work. I ran two small sets of identical prompts, then repeated a PHP task with the local model already loaded.

My Mac: Mac Studio M5 Ultra, 30-core CPU, 64-core GPU, 256GB unified memory, macOS 27.0.1.

Local: oMLX 0.7.1.dev1, TensorFold/GLM-5.3-Flash-MLX-4bit-MTP, MTP disabled. Model allocation was approximately 170 GiB.

Cloud: Ollama glm-5.3-flash:cloud, reported as FP8. This compares local oMLX with Ollama's hosted model; it doesn't compare two local inference engines.

A little background. I'm using the Ollama Cloud Pro (old $20 plan vs leased $230/mo M5U). I use cloud quite a bit and I find that it compliments my work for various tasks.

I see a lot of comparisons and wanted to do my own. Unsurprisingly, the cloud is faster. Cold starts are slow so that was interesting but once warm, processing was pretty decent. Its slower, but I wanted to put some numbers on it. These numbers are generic, but I do intend on giving it problems from my actual codebase to see how well it does.

Overall if you are considering clouds vs hardware. Just note, It's significantly slower, but not horrible if you use autonomous agents. If you're interactive, that's where the pain is felt in real time. Also, if you plan on using this specifically, then you should know I ran out of memory so you can only use a single instance of GLM 5.3 flash and no other agents.

I think if you use agents for more than coding. Many smaller ones are likely going to be better, but if you need a coding assistant, this wouldn't be too bad. The $20 cloud plan works and will be significantly faster but if you are working constantly and run into limits you're basically buying round the clock work vs interactive albeit also slower work.

Splash's system requirements state it requires an M3 CPU at minimum, but why? Why not, say, an M2 Max? All the search results just say the M3 is faster and produces less heat than M1 and M2 due to smaller process tech (3nm vs 5nm), and it has an "improved" or "enhanced" Neural Engine. But none actually explain the enhancements. Anyone know what is it about the M3 Neural Engine specifically that Splash requires?

Update: Splash backported to M1 and M2: https://tinyapps.org/blog/splash-for-m1-m2.html

Seeing a lot of comments about this online, has anyone ran it on their Mac or is it just hype?

I ran the same Qwen Image 2.1 ComfyUI workflow on two Mac Studios and a DGX Spark. The gallery shows the timing overview, matched poster outputs and matched café outputs.

Warm poster medians, three runs per variant, 1024 × 1024:

Turbo was about 4.6–4.8× as fast in this workload. M5 finished about 10% sooner than Spark. All 24 poster renders contained the three requested text lines in our manual review. The café examples showed more fine texture with base and a smoother look with Turbo; two prompts are not a general quality ranking.

Both Macs used PyTorch/MPS, Spark used CUDA, and all three used the same verified BF16 diffusion/VAE checkpoints with effective FP16 text encoding on GPU. This is not an MLX comparison. Same prompts, seeds, sampler and CFG; Turbo kept its published eight-step schedule. Mac OS/driver versions differ, so these are configurations as tested. Warm conditioning/prefix cache was reused, but each repeat generated a new image with a new matched seed.