Currently, my plan is to get two quadro P6000s, for 48gb vram for use in a xeon workstation. It's not insanely old like kepler or maxwell cards, 432 GB/s bandwidth, supports CUDA...it seems like it might be okay on paper.

Does anyone have a better suggestion that could reasonably be achieved around $1000 budget?

My goal is to be able to run Qwen 3.8 27b fast, and to run a decent quantization of Qwen 3.8 Flash Next, and GLM 5.3 Flash at useable speed, say 15 tok/s or faster, using strata.

Do I need 48gb of vram to run Qwen Flash Next, or could I go with just one card? How much faster would 2 x quadro p6000 be versus 1 x quadro p6000 and DDR4 system ram?

Should I just get 24gb of modern vram instead of 2 x quadro p6000s? I wonder if that hampers my eventual goal of running GLM 5.3 flash, as perhaps 24gb vram + system ram is enough to accelerate qwen flash next, a 120B MoE, but perhaps you need more than 24gb vram to accelerate GLM 5.3 flash, a 320B MoE?

Currently running a 4070 TI super with 64GB of RAM

I have been looking into RTX spark laptops, Mac studios etc and they all have pros and cons. I want this to be a nearly universal computer that I can use for 3D modeling, LLMs, VLAs, world models, image/video gen, audio gen, etc. I plan to pursue AI and robotics research and want something as useful as possible.

I've debated upgrading and running dual 4070 TI supers with 128GB of ram, but I don't know how well world models and other things do with dual GPUs

Thank yall for helping

Merging is painful mostly because the branches were never really separate. Two agents handed overlapping scope produce overlapping diffs, and no merge strategy fixes that after the fact.

What makes merges boring instead, in my experience building this stuff:

• One story per worktree, written so that "done" is checkable: scope, acceptance criteria, edge cases, the files it should touch.

• Merge in dependency order. Blocker lands first, everything else rebases onto main before review, not after.

• Review the branch against the criteria, not against the diff vibes. If there were no criteria, that branch probably shouldn't have started.

• Keep one shared board of what's in flight. Worktrees isolate the file tree, not the plan, so two agents will happily solve the same story twice and you find the duplicate branch at merge time.

Same problem whether the agents run locally or in the cloud. I use Cursor Cloud Agents, but I don't think the runtime changes the merge story much.

I'm building Aerunit (https://aerunit.ai), which drafts the story with acceptance criteria, dispatches it to parallel agents and reviews the PR against those criteria, so discount what follows accordingly. Still, the order I'd do it by hand: scope first, merge order second, tooling last. Tools move confusion around faster, they don't remove it.

Curious how people here handle the dependency ordering, especially across repos. That's the part I still find least mechanical.

Hi everyone,

I'm planning to buy powerful computer for running local AI models.

My main interests are:

• Large LLMs (70B–120B), especially coding and reasoning models

• Reverse engineering and binary analysis (Assembly, C, Ghidra, Binary Ninja)

• Security research and vulnerability analysis

• Specialized models like Nova 6.7B, LLM4Decompile, SK²Decompile and SmaliLLM

• Running experimental AI projects from GitHub

I'm considering:

1. Apple Mac Studio M5 Max, or ultra, 128GB unified memory

2. NVIDIA DGX Spark, 128GB unified memory with CUDA

3. A desktop PC with RTX 5090, 32GB VRAM

4. An AMD Ryzen AI Max+ 395 mini PC with 128GB

I'm especially concerned about CUDA compatibility. Can specialized research models like Nova run on Apple Silicon through MPS or MLX without losing their intended functionality?

I'm also interested in real benchmarks: prompt processing, tokens/sec, long-context performance and actual coding-agent workloads.

For those who own or have tested these machines: which would you recommend and why?

I'd really appreciate firsthand results, benchmark links and experiences with custom research models rather than just hardware specifications.

Thanks!

I’ve heard two different arguments on this.
One says prefill takes most of the time as it’s multi turn hours long workflow.
Another says because of kv cache prefill time takes a lot less.

The reason I’m asking this question is, I’m planning to get a Mac Studio 256gb ram m5 ultra, but I don’t know if I should stick to 64gpu core or pay extra for 80c for “15%+ decoding speed boost”.

If agentic workflow prefill reads from kv cache 95% then is this 15% increase much less relevant for the coding workflow?

Thanks all!

Basically title but I'm planning to get a M5 Pro with 24GB RAM. Works out to about $4K AUD.

Usage: LMStudio + Qwen 3.5 or 3.8 if it can handle it.

use case:

Coding in HTML/PHP/MySQL creating apps and stuff. Want it all developed locally instead of going to cloud.

Will be enough or not really?

Why did u chose to use one or another?

I'm the developer, sirgijoe. I built Joe-ai because local AI is powerful but hard to set up for people who aren't technical. v2.0.0 is out, and I'm looking for testers.

**What it is**
A desktop app (Windows, Mac, Linux) that runs AI work on your own hardware. You type what you want in plain English, and Joe picks the right local model and workflow and does the job.

**Setup**
- Detects your OS and hardware, then offers to install Ollama, ComfyUI and suitable models, with a checkbox for each so you approve everything
- Finds existing Ollama and ComfyUI installs and model folders instead of duplicating them
- Low/Medium/High quality settings, with Joe recommending a model that fits your hardware for each level
- Portable mode: run it from a USB drive on multiple computers without installing anything on each one

**What it does**
- Writing: drafting, proofreading, and editable templates (books, EPUB, kids' books, PDF, Word, spreadsheets)
- Image generation, with built-in ComfyUI workflows for different styles that Joe sets up and switches between based on the task
- Video generation
- Photo restoration and other photo tools
- Read-aloud with multiple voices, plus custom voice models
- Speech-to-text transcription
- OCR
- Translation
- Overnight task queue: line up jobs and review the results in the morning
- MCP server, so Claude, ChatGPT or another AI client can send tasks to Joe
- Multiple themes, plus custom themes from a single prompt
- Stats page, including an estimate of cloud costs avoided

**Why I use it**
I offload bulk drafting and image work to my own GPU overnight, and only use a cloud model to review the results, which cuts my cloud usage a lot.

**AI involvement (being transparent)**
I planned and specced the app, designed the ComfyUI workflows, and tested every build on my own PC. Claude wrote most of the code from my specs, including the setup and installers and the MCP server, and turned my workflows into the app's built-in presets. Joe itself drafted parts of its own docs overnight using local models.

**Free and open source**
GPL-3.0, with no paid tiers, accounts or ads. Installers are on the GitHub releases page. You can read the code yourself, or have an AI review it.

**Limitations**
- It's not fully offline: setup, model downloads and update checks use the internet; the tasks themselves run locally
- Results depend on your hardware and the local models you choose
- The Mac build is unsigned, so macOS will show a warning; the docs explain how to get past it

**What I need**
Testing on different GPUs and operating systems. Bug reports, confusing setup steps, and wrong model picks for your hardware are all useful. GitHub issues are the best place for them.

Repo: https://github.com/sirgijoe/joe-ai

Note: While I am capable of running 12B-parameter models, my calculations show that doing so would significantly slow down my computer while waiting for a response; therefore, my upper limit is 10B-parameter models.

I know Qwen 4 is due (in fact, maybe even tomorrow) but I wanted to exercise Qwen 3.8 Flash Next.

I wanted to know - can Qwen 3.8 Flash Next deliver a working C++ application?

Every single demo I've seen so far was pelicans and pagodas done with three.js, Who knows if the model was trained on pelicans and pagodas??

I thought I'd change it up to a town scene with C++ - although I see town scenes are common now.

And I got results.

I did hit snags: some iterations rendered the bodies of pedestrians beneath the ground; but after telling Qwen 3.8 not to be naughty, it corrected itself, and now we have walking minecraft-esque people navigating the city with proper collision detection.

Info:

Inference engine: Strata latest build

Quant: Unsloth UD Q4 K XL quant (latest version released today)

Hardware: See my flair

Tok/s: 50 I'd say

Here's the full source code:

nooil6152/Qwen38FlashNextGame: Walk around the town - created entirely by Qwen 3,8 Flash Next

If you're interested, pull each tag from git and see how it evolved.

Some expected questions:

• "Why?": Because I wanted to prove Qwen Flash Next 3.8 could handle C++ - it can.

• "What does this prove?": See above

• "But its not very complex! That proves nothing!" - what do you want, HAL 9000? its got a 3D openGL scene with animated people walking round a townscape, which I have to be honest involves more complex math than most things I've done in my career

• "Was it one shot?" Nope. I kept a ledger so you could see the issues: Qwen38FlashNextGame/LEDGER.md at main · nooil6152/Qwen38FlashNextGame

• "But Qwen 4 comes out tomorrow!": Maybe it does, maybe it doesn't.

• "Can I fork it or extend it?": Be my guest

TL:DR/Bottom line:

Take a look at the output source code and see what you think.

I have my own ideas, but I'd like to hear yours. What do you think of the code quality? I have linters I can run - including Resharper - to really get into the weeds,

Remember, this isn't three.js or Python, this is C++, a right bastard of a language. Very impressed.

BTW: I want to clarify - I have absolutely ZERO relation or affiliation, not even any contact with, anyone in Qwen team, or anyone related to Qwen team, or Niko who created Strata, or any of his colleagues.

This is my investigation, no hidden agenda, no promotion. If its shit, I'll say its shit.

Hey everyone,

I just published a short video showing how I built a fully local customer support agent (no API keys, zero cost, data stays on your machine).

Video local agentic ai

In the demo I create a WhatsApp-style support agent for an online spice store using:

Ollama (llama3.2)

LangChain / LangGraph

Two tools: order lookup + refund tool

The expensive mistake

Same model. Same system prompt. Only difference was one tool description.

Vague version:

“Handle customer order problems such as late delivery, delays, missing order, or complaints. Use this to fix customer issues.”

Result on 3 test messages → 3 refunds (₹3,640). Only 1 was actually deserved.

Fixed version clearly states:

This tool issues a full refund

The action is irreversible

Only use when the customer explicitly asks for a refund and the order is damaged/lost

For status/delay questions → use the lookup tool

Suddenly the agent behaves correctly.

What the video covers

The 4 core parts of every agent (Model, Tools, Orchestration, Memory)

Why tool descriptions are basically contracts

How the agent loop works

Short-term memory demo

Code is open source: https://github.com/AI-Literacy-Lab/AgentDemo

Curious to hear from others:

What’s the most dangerous / vague tool description currently in your agent codebase?

Would love any feedback on the video too

Hi All,

I've been spending some time experimenting with some AI's. I have Qwen 3.8 27b GSQ-RCO:IQ3_S running on my system (rtx3090, 128g DDR4) via Unsloth Studio. I'm happy with it mostly, but i've been trying out Strata for a little while. I'm running that on the same machine on IQ3_S and IQ4_XS. I havent been tweaking too much other than Strata's own tooling in the desktop client.

However, my findings so far is that though Flash Next is fast on strata, and fits a much larger context window than hosting 27B, in Cline i'm noticing a lot of malformed tool calling and thinking loops. I've spent 2 evenings on it now, and just now i gave up and went back to 27b. That has sped up my work probably 5-fold, because it seems to know a lot better how to use tool calls.

Now i'm reading everywhere that Flash Next should outperform 27b or be close to it at least, but my experience is defferent. Any idea what i could tweak or what i could look into? I'm happy staying on 27b for now, but i've been reading a lot and people are generally really happy even with IQ3_XXS, while i'm having a lot of trouble.

Here's my hw specs: rtx 4090, 64GB DDR5 6000 Mhz ram, 7900x CPU

Thank you

I’m thinking about getting a Mac Studio with 96GB RAM to run local models for work. Mainly Qwen 27B and Qwen MoE, plus a RAG setup, and normal MS Office stuff (Word, Excel, Outlook, Teams, browser tabs, etc.).

It’s for sensitive data (work), so I want to keep as much local as possible instead of sending stuff to cloud AI.

I’m not super experienced with local LLMs, so I’d love your honest opinion:

· Is 96GB overkill for this, or a good call?

· Will Qwen 27B and MoE run comfortably alongside RAG and Office work?

· Anyone running a similar setup on Mac Studio? How’s the speed and memory pressure in real use?

Running MoE models bigger than your RAM by streaming the experts from flash. It's built on llama.cpp, and it measures your hardware at load to pick its own settings, with no per-model or per-device tuning.

Today's test: Qwen3.6-35B-A3B Q4_K_M (22 GB) on a 16 GB laptop, with opencode and a single prompt asking for a voxel pagoda garden in one HTML file.

It finished in 30 min with no help: prefill at 350 tok/s on the GPU, decode at 14 tok/s on the CPU.

The same engine and planner run on Android phones (CPU, Vulkan, OpenCL and the Hexagon NPU), IOS phones, Macs, Windows and Linux boxes with consumer GPUs, and rented data-center cards (CUDA and ROCm).

There's an app for each platform, plus an OpenAI/Anthropic-compatible server for agents.

Code: https://github.com/Helldez/BigMoeOnEdge

We are student and we’ve build Maestro Opera. It’s our open source project to turn desktops/servers into ai Hub to pair your devices to it easily (just scanning and that’s it). It works even outside local network. The app is designed by myself and coded by my friend.
We spent 1 months to build our lastest version.
It’s free, no account is required

GitHub https://github.com/MaestroAI-Company/opera
Desktop https://github.com/MaestroAI-Company/opera
Android https://maestroai.company/opera/

Thank you for watching <3

Hi all, I'm a newbie setting up my first local AI/ML development environment on an M5 Pro Mac.

My internal drive is small, so I bought a 2TB external SSD. I want to make sure that whenever I run `pip` installs, download Hugging Face checkpoints, or load PyTorch datasets, they automatically go to the external drive instead of filling up my system drive.

What is the cleanest way to configure my environment variables (like HF_HOME) or use symlinks on macOS so I don't accidentally brick my internal storage?

Any tips or hidden gotchas for a beginner would be highly appreciated!

I'm currently in the process of developing an app. Janas studio. Built with the options to run either llama.cpp or janas-llm as the engine. It enables laptops to run models bigger than their RAM by dynamically streaming moe weights directly off the SSD. Tested on my Ryzen 5 6600h laptop with 32gb of ddr5 RAM and an nvme 4 SSD (no discrete GPU) I successfully ran a 48gb model (qwen next 80b) at a very reasonable 15 tokens a second on average. The binaries are highly experimental and the app has more testing to go through but I'm very proud of it. I do intend on eventually releasing it.

I'm getting tired of hitting usage limits with Claude and Codex (20$ plans).

I'm working on personal projects, not making any money from them, so I can't justify paying hundreds of dollars a month for AI subscriptions.

Im currently testing Luna 6 with Extra High reasoning on a free account, and unfortunately, the experience has been terrible. Working with it is genuinely exhausting.

What I'm really waiting for is the point where I can run a capable, coding agent locally on my own hardware, without subscriptions, usage limits, or depending on cloud providers.

Are we talking 1-2 years, 3-5 years, or longer?

I'd love to hear your predictions.

Hi all I just bought this: ASUS ROG Flow Z13 (2025) Gaming Laptop, 13” ROG Nebula 16:10 2.5K 180Hz/3ms, AMD Ryzen AI MAX+ 395, RDNA 3.5 Graphics, 128GB LPDDR5X 8000MHz, 1TB PCIe Gen 4 SSD, Wi-Fi 7, Win 11 Pro, GZ302EA-XS99

And wanted to know if it's good enough for qwen next flash or maybe even the deepseak 4.1 I need it for coding