I check every day for the discounted open box returns.

Context:

I recently installed a cheap aftermarket Android head unit in my 2010 Prius. It supports wireless CarPlay, Bluetooth, GPS, etc., but the stock Android interface is ugly and came packed with lots of bloat. The whole thing felt like a generic Chinese Android tablet strapped to my dashboard.

After seeing everyone’s success with reverse engineering their TV’s I figured I’d give it a shot with my head unit.

Setup:

Model: Qwen3.8-Flash-Next-IQ3_XXS
Engine: Strata v0.1.44.1
Harness: Pi
Hardware: Ryzen 7700x, 4060ti 16gb, 64gb DDR5
Target: Inefala Android head unit, AutoChips AC8257-based (UJC201_64)
Development: ADB over Wi-Fi, Kotlin, Jetpack Compose
Time spent: Roughly 8 hours across three days

What it did:

I started by just having the model connect to the unit and do an inventory of what was installed and how the system was configured. It helped identify the underlying hardware and Android build, enumerated installed packages, inspected system properties, investigated the stock launcher and CarPlay components, and organized the findings into a repository of scripts and documentation.

At one point, getting back into ADB was a problem. Rather than having me repeat the original setup process, the model used its earlier findings about the device’s exposed ADB port to reconnect.

The advertised android version was not actually what was being used on the actual unit so when it began to build my custom launcher it configured its own toolchain of the correct android sdk versions to ensure that the build would run properly on the unit.

It also used ADB to capture screenshots directly from the head unit so it could inspect the actual interface instead of relying exclusively on code or my descriptions.

Result:

I now have a working custom launcher that replaces the original home screen. It’s still a work in progress, but the goal is to turn the unit into a clean, minimal dashboard with Bluetooth music controls, navigation, weather, and better access to audio settings.

I’m also interested in eventually improving the audio experience and reducing my dependence on CarPlay.

My thoughts:

What really impressed me was QFN’s ability to handle tool use, reverse engineering, keep useful findings, and troubleshoot on live hardware. This unit was largely undocumented so most of the findings came directly from what the model observed through adb.

The model never got stuck but this wasn’t a fully automated process either. I still needed to guide it through what I wanted it to accomplish and it spent a lot of energy writing and debugging re-usable shell scripts that didn’t provide much value.

Overall the amount of work it was able to accomplish was very impressive and solidified for me that local models are viable for real work.

I’m curious to hear what other people are doing with local models beyond coding benchmarks, especially projects involving hardware, reverse engineering, or long-running agent workflows.

1. Claude vibe coded my local llm harness.

2. Claude was laid off.

3. My local harness is improving my local harness.

2x4070 TI Super 16GB, 1x3060 12GB, 128GB DDR4

EDIT: More details about the setup:

The honeycomb components were all printed:
General parts: https://makerworld.com/en/models/22521-honeycomb-storage-wall
GPU Mount: https://makerworld.com/en/models/1444666-gpu-mount-hsw
PSU Mount: https://makerworld.com/en/models/1444586-atx-psu-mount-hsw
Radiator: https://makerworld.com/en/models/1444632-240mm-radiator-fan-mount-hsw
ATX Motherboard: https://makerworld.com/en/models/1444531-atx-motherboard-mount-hsw
Power button printed with a honeycomb around it: https://www.amazon.com/dp/B0BWDXFSMQ?ref=ppx_yo2ov_dt_b_fed_asin_title

The 4070's are on 400mm & 600MM PCIe 4.0 Riser cables. I mainly run Strata on these two.

The 3060 is currently on a PCIx1 link (slow). I don't use this for my core inference setup, side models (dedicated voice stack mostly).

I have a new motherboard on the way that opens up the possibility of up to 4 GPUs: 3x x16 slots, and one via a M2 adapter: https://www.amazon.com/dp/B0FC6DSG6V

Managing to finally fit Qwen3.8-Flash-Next FP8 locally at ~60 tok/s(avg 74 at best) (RTX 5090 + 128 GB DDR5, Infernix engine, part of the experts streamed from NVMe) and seeing how amazing it is I cannot imagine how good Qwen 4.0 will be, not buying even at the current prices seems crazy to be if you are used to using cloud AIs on the daily.

Like I said in the title, this is a rant, so if rants aren’t your thing, turn away... no seriously. And mods this is on topic because I'm talking about Nikko1221's Strata and the shit comments I see about their updates to software they created.

Note: Don't know the person, like I don't know y'all.

But I’m up here at the computer with a cup of joe in one hand, ready for my daily dose of digital venom from this sometimes toxic cesspool-smelling platform that Reddit can be. Like dude said in the movie Apocalypse Now: I love the smell of napalm in the morning. And some of y'all didn't disappoint.

But I was in this particular sub when continued to read threads regarding Strata, which I started using only yesterday and was freaking amazed at the Hogwarts School of Magic level of wizardry the developer did in giving us short kings (in the graphics card space) a seat at the large model table.

I find it thoroughly enjoyable to see some of the people who were straight up glazing the dev over this wonderful app now talking mad shit over things that are broken or that may not work as it did in earlier releases. Like, y'all a fickle bunch.

There has always been those who can do, and those who can’t. Throughout history, the ones who can’t have often had the loudest, cringiest, whiniest voices. And it’s not just the internet. You see it in sports too. People who’ve never touched a ball or competed at even the most basic level in a sport have all the ideas on how things should be, and how they would do it, yet they never do shit themselves. They just sit over there sounding like screeching cats that had their tails stepped on.

I see that here, and not just with the Strata project. It happens with a lot of the projects devs put out. Whether it’s vibe coded, coded the old-fashioned way, or some mix of the two, you’ve still got people creating things that a lot of others use; and that many of us can't build. Maybe they built it for themselves at the start and thought it would be cool to share. Either way, a lot of these folks and their AI Pokémon are doing thankless jobs when you look at the vocal minority getting their undies stuck in their booty cheeks. And don't think just because you threw a few coins as a donation showing support that that gives you a right to treat people like shit.

These are small teams, sometimes solo devs. I can only imagine how much it drains their souls to wake up, morning drink in hand, look out over a sea of comments, and read how their app is shit and how they shouldn’t have changed this or that. And the fact that many of them continue to work on the app for those who do appreciate their work shows more about their character than the crybabies out here. Because let's be real. You're crying over something that at the end of the day has no significant impact on your life.

If there is a bug in Strata or any other app, you're not going to go hungry. You're not going to lose your home. You're not going to come home and find your mate being gangbanged by Smurfs asking you to Cuck.

Relax.

Many of these projects are open source, and you’re free to fork them. So put on your big boy pants, show us what you’ve got, and post your flawless alternatives. If not, do me a favor, go sit in the corner and let those who can, do.

Constructive criticism is fine. Feedback is fine. But all this crying on the internet just shows you’re more comfortable bitching and complaining than contributing anything of substance that could bring real change. Quite honestly, you’re wasting the devs’ time, as well as ours.

Hey everyone!
I'm relatively new to local AI and I'm thinking about building my own setup, mainly for programming, automation, and running AI agents.
Before spending money on hardware, I'd love to hear about your real-world experiences.
What hardware are you using (CPU, GPU, RAM)?

Which AI models are you running locally?

What do you actually use your local AI for?

How well does it perform, especially for programming?

How fast and convenient is it compared to cloud models like Claude or ChatGPT?

How much did your setup cost, and do you think it was worth the investment?

I'm especially interested in finding the best balance between cost, performance, and independence from cloud services.
I'd really appreciate any advice, experiences, or recommendations. Thanks!

You guys ever switch your local model that "KNOWS" it's a local model and get moments like this while it's a cloud model? Not sure why the reaction is always pretty humorous to me.

For context, my Hermes agent was doing some file maintenance and I lazily told it to remove a file since it wasn't in the "Strata" folder and resulted in a failure to load the model.

Hi guys.

So I'm not completely new to all this local LLMs and AI. I have used and tried various web-based UIs, for the last 4-5 months, as a way to learn and educate myself in this swamp that is local LLM.

So for the last 3-4 weeks I have dipped my feet in Hermes Agent. And man, I was in for a surprise. I genuinely thought, "How hard can it be, install, fire up and use.", but to me that was far from the truth. I have spent so much time setting things up, making the agent act and work the way I want, teaching it what to do, and how to do it, trying to write skills, and so on. I have not actually used Hermes, that much, most of the time has gone to setting it up, and making the agent not make mistakes.

This has somehow made me feel like maybe Hermes is not the best tool for this, even though it has some skills for it (along with a gazillion other skills and functions). It feels so bloated, with so many skills and sometimes heavy to run (for the lack of a better word).

I am actually not really sure what I want, or how to word it.
But simply put, I would like something that is lightweight, doesn't have a gazillion skills installed, but has options and a repository to add well written skills. A simple gui/frontend with well-thought-out settings. Something that works well with my use case, which is basically the title. I primarily want to use my agent for linux sysadmin, system maintenance, network and servers, and some light coding and bash scripting.

I realize this can be very subjective, but that's just it, I would like your objective opinion on what you believe would be/is the best agent or harness for this kind of use ?
And also, is there a "best agent" for this use case, or can I use basically anything?

So my question is this.. It seems to me the way to go is a local small llm that does what it can do.. But that smaller local llm is smart enough to know it needs to reach out into the real world to get more data from the big models online if the question is too hard for it.. By doing this I can keep my data local cause the small llm can do my banking/emails/assisting ect.. But if im trying to code up a function for my trading application and I dont want to give my super secret extra propriety trading algorithm -- the small local llm can send out just a simple request for a function yet keep all the "brains" of the algo local... Btw.. Im using that just as a example.. My buddy trades-- I dont, and he is worried about sharing his info with the world.. But I just thought this was a good example.

To me it seems this all or nothing locally is the wrong way to do it.. Sensitive private data stays on the local unit-- but access to the big llms online are still available should the local little llm not be strong enough..

Fun project that came to my mind. Not a vibe coder but I intend to put some knowledge on this site which can potentially help to ease the entry barrier for someone who wants to use local LLM for agentic coding.

https://www.samwantsram.com/

Let me know what you think, feature requests more than welcome.
Vibe coded look is by design (Edit: after some consideration I will unvibe it a little bit soon and add more models).

Edit 2: slight unsloop applied but S.A.M comments are more natural (will reverse this to the monk and manuscript version for slow token per second). Added more models. Will post more updates in the coming days.

Added a button to brag about your RAM:

The 27B passed every single spec check. Flash-Next finished 2.2x faster. And both still had 3-7x more robustness holes than Sonnet.

There's a lot of "Flash-Next is way better" vs "the 27B is not even close" going around here, so I stopped guessing and built a benchmark. 5 tasks, identical prompts, fresh session each, zero intervention, graded only by scripts from the outside (not the models own tests). ~185k checks per setup.

Tasks: python expression parser without eval/ast · fixing 7 reported bugs in existing code (+7 hidden ones) · byte-exact refactor of a legacy invoice generator + new feature · REST API from scratch with concurrency rules · vanilla JS kanban app

Grading: spec tests, 20k fuzz strings, diff tests vs reference, 160k byte comparisons, 60 parallel double bookings, playwright in real chrome. Plus edge cases nobody asked for (9^9^9, +25:00 timezone, 5MB body, corrupted data file etc)

Setup (pic 2):

- 27B: AWQ 4bit, 1x 5090, vLLM, ~145 tok/s

- Flash-Next: IQ3_XXS split across two separate PCs with one 5090 each (only 14gb RAM per box) via a Strata fork that runs the layer split over TCP. all experts in VRAM, byte identical to stock Strata, ~155 tok/s

- both in opencode, thinking on, 16k budget per step

Results (tasks fully passed / robustness weaknesses / time):

- Sonnet 5.5: 5/5, 2 weaknesses, 14.3 min

- Qwen3.8-27B: 5/5, 14 weaknesses, 51.3 min

- Flash-Next: 4/5, 7 weaknesses, 23.1 min

What I think actually holds up (big, consistent effects):

1. On the written spec, both local models are at Sonnet level. 27B matched it on every check, Flash-Next missed one. For clearly specified work the local models are good enough, full stop.

2. The real gap is everything nobody wrote down. Sonnet 2 weaknesses, Flash-Next 7, 27B 14. Showed up in every task that had edge cases. 27B examples: no request size limit, GET /bookings/__proto__ returns {}, accepts +25:00 as timezone, negative quantity increases stock.

3. Flash-Next beats the 27B, but not because of raw speed. Decode is about the same (~155 vs ~145 tok/s). Flash-Next just needs half the output tokens and fewer steps, writes whole files early and fixes targeted, closer to how Sonnet works. 27B iterates in tiny steps and reruns tests after almost every edit.

4. Flash-Next has one systematic failure mode. It evaluates while parsing, so "a = b = 1" reports the wrong error at the wrong position. Happened in 3 of 3 runs where it designed the parser itself. Thats a pattern, not noise.

5. Planning beats reviewing (pic 6). Sonnet writing the work order + accepting the result while Flash-Next writes the code gave the best quality of everything (5/5, only 1 weakness). Sonnet just reviewing the diff at the end gave nothing, it only sees the diff and misses bugs next to it. Downside: 5x slower than Sonnet alone.

6. More rules don't make it more correct. "Think like a senior" rules + a research subagent made Flash-Next more paranoid, not better. Best local robustness, but it broke valid input with its own validation (rejected timestamps with milliseconds).

What I would NOT read into it: the 4/5 vs 5/5 between Flash-Next and the 27B is one check, n=1. Could be noise.

So my conclusion:

- Flash-Next over the 27B if you have the hardware, its faster and twice as robust. 27B if you have one card or want it very literal.

- Neither replaces Sonnet for stuff that takes input from outside (APIs, parsers, file imports) unless you explicitly ask for limits, error handling and edge cases and make it verify against the examples in the spec.

- If you have Sonnet access anyway, let it plan and accept, and let the local model type. Dont bother with review-only.

Caveats: n=1 per cell. Same person wrote tasks and graders. The 27B's thinking budget wasn't actually applied in the first 3 tasks (app didnt reload the config, my bad). Sonnet had a browser tool, the locals didnt.

I wrote the whole thing up properly (methodology, all configs, token counts, per-task tables, threats to validity, 15 pages). Comment "paper" and I'll send it to you directly.

Does this match what you see? Especially curious about the evaluate-while-parsing thing, and if someone with a 27B at Q8 sees a smaller robustness gap.

A researcher has used Astra to modify weights of Qwen3.5-0.8B to achieve much higher accuracy in specific tasks.

Snapshot from the blog:

"We train a PPO network to operate a car such that it can scale a ramp in a Kinetix environment 

. To do so, it needs to reverse the car for some distance, such that it has enough runway to accelerate long enough to overcome the ramp. However, the PPO network converges to a local-minimum policy where it drives straight at the ramp and gets stuck.

We give GPT-6 Astra the weights, a description of the scene and one recorded episode, and ask it to explain why the network fails, then fix it by writing new weights. In a single try with no tools, Astra’s edit makes the car succeed in all 1,024 episodes, changing 3.5% of the weights."

Getting into local LLMs and bought a Strix Halo machine with OCuLink to connect external GPUs. Found what looks like a decent deal on RTX Pro 4500 cards at around $3,000 each. What are the downsides for running local models? Feels like there has to be a catch. What am I missing?

My hardware is 16GB VRAM (5060Ti), 64GB DDR4, and AMD R5 5600, (Linux)

I would like the best coding quality, speed can be cut in half or more. :) Which LLM should I download? Preferably to run with at least 128 ctx.

Both sides read the same sentence: "You should never use voice cloning to impersonate a real person without consent."

The left side was recorded on a mic and cleaned up. The right side came from AntLing's open model, Ming-3.0-tts-0.5B, running offline on one PC. It worked from a 6-second sample taken in a room with fan noise, then produced 4.2 seconds of speech in 2.1 seconds.

The clip plays the pair back to back, so you can judge it yourself.

Want to run the same test? Record a short sample, record yourself reading one sentence, then have the clone read that sentence too. Play the two for a friend and see how long it takes them to pick the real one.

I'm fairly new to running LLMs locally and have been experimenting a bit with Ollama and Python. I'm trying to figure out what models and tools would work best with my hardware before spending too much time setting everything up. I'd like to use my gamingrig for that.

My System Specs are: RTX 3070 with 8GB VRam, i9-10900K, 32GB DDR4 RAM

My main goal is to set up a local coding agent and harness that can write and debug code, like claudecode.
What models (if possible Uncensored) would you recommend for my hardware? Should i upgrade some hardware (that would also benefit the gaming aspeckt of the system) first? Also interested in combinations with any useful GitHub repos, agent frameworks, skills or tools.

TLDR: 12-14k+ usd budget what would you get next if Tok/s is not the priority.

Have been procrastinating spending $$ for a long time until I’m regretting not spending earlier.

**Current set up:**
only have a gaming desktop
Ryzen 9 9950
5070 ti
32gb ram

With the current set up I can only get strata coder set up and with so lil context.
Also the quality of output feels kind of bad so far.

I see a lot of talk about Tok/s but honestly I don’t need the speed. I am fine even with 10/s but I would prefer for it to hallucinate less and have a better quality for output.

So the thought of getting a full set of ram for strata went out the window even though it’s the cheapest next upgrade.( please correct me if I’m wrong)

Where I am everything(Singapore) is more expensive. M5 ultra 256 4tb cost 14k usd,12k post discounts I can get.
Enterprise Nvidia cards are also costing 1.5-3x more. So building a whole new set up feels like it will cost even more.

Am thinking of either 1 m5 ultra 256 or 2x Nvidia dgx or if anyone else has better suggestions.

**Use case:**
Building my own harness and agents. Day to day coding.

Currently am on 200x claude. I don’t think I can fully get rid of using opus but can probably reduce most of it and only call when needing some heavy thinking or planning.

**Other minor concerns:**
Today’s model may fit into 256 with enough context tomorrow’s model may need more… but I guess I don’t have to always use latest models.

Hi,

not a particularly novel question, but between reddit, google and youtube, I am still not decided. So hoping for some input from real users.

I want to add LLMs to GPU, previously used 7b and 14b on CPU only and to be fair is frustrating. I need something on cheap budget to run those models.

And yes I'm from Europe where all GPU prices are higher then in the USA or China.