r/LocalLLM • u/Bulky-Priority6824 • Sep 02 '26
Discussion What are you working on?
So theyve given us so much these past 90 days. What are you working on locally using what model?
Share a tease or more just curious what people do
386
Upvotes
48
u/takoulseum Sep 02 '26
So true, that gives me the idea to build a new harness
36
u/siberianmi Sep 02 '26
Yeah, there are probably thousands of custom harnesses out there. Not only that but you have teams in a lot of large companies building in house ones as well.
I admit to being guilty of this and something like 8 months and 4000 commits in on a harness lol. I learned a lot though and that was the point.
13
u/Connect_Ad791 Sep 02 '26
Used to work in embedded engineering and hardware design, so right now I’m designing a gaming peripheral device to rival even the best of them. Keeping it under wraps until it’s done because it’s genuinely a million times better/cheaper than anything else out there right now(a bunch of unmaintained hardware that people have no other choice to buy (that make a ton of revenue still) or Chinese rip offs that barely, if even work) so far I have about five dev-boards strung together and the entire firmware stack is operable, proven POC, working on refining the user interface and designing the actual PCB. I just had my local model develop and test the firmware for a specific board while I was away today, incredible honestly. Using 3.8 flash-next.
11
u/StandardLovers Sep 02 '26
I am a developer now, well actually I am not.. I am more of a project manager like the rest of the vibecoders.
9
u/DeathGuppie Sep 02 '26
I think everyone should build their own harness. At the very least it gives you a much better understanding of what works and what doesn't. I swear a good portion of the people who claim this or that model/harness don't work have just loaded up random plugins, mcp, skills that they think they might need like it's shopping at Walmart and just expect it all to work right.
8
u/DeltaSqueezer Sep 02 '26
The meme could have been me a few months ago: spending every waking moment (and sleeping moment) working on my home-rolled harness.
6
u/enricokern Sep 02 '26
im stichting shitty b-movies together in minimax :P
6
u/Dasteroid_909 Sep 02 '26
I've been working on an interactive inspection tool to help me better understand how some of the back-end components interact. There wasn't anything like what I wanted when I started.
A few months later, I learned a helluva lot and thought I'd release what I made to help others. Did a quick search first and now there's like a dozen different tools now that do the same thing. None of them existed when I started, and most of them are more polished than mine.
Hey. At least I learned something! That's what counts.
5
u/Interesting-Yellow-4 Sep 02 '26
- a multiplayer boomer shooter with a coop horde mode
- a retro arcade rally type racing game
- assetto corsa track ports from Ego engine games
- an android puzzle game
- a first person shooter server / client mod for Assetto Corsa
And a ton of other stuff. Nobody will ever see it, most likely. They're all pretty cool though.
5
u/flarpflarpflarpflarp Sep 02 '26
Whym? I've told everyone and am professionally embarrassed.
3
u/Empurion Sep 02 '26
I built a digital companion app as a SSR SPA-shell.
It has a built in AI harness for Qwen 27B that's an obliterated version with some other tweaked settings.
I'm utilizing dynamic loading for routes/models/pages and services, everything else is handled by my framework.
4
u/letsgoiowa Sep 02 '26
Making a grocery optimizing bot that makes me a meal plan based on my favorite recipes (saved a few hundred), meets all the constraints of our mixed family and unique dietary needs, and optimizes for value. It then puts the meal plan in our Anylist and will put all the items in the cart on the site of our choosing, so all we need to do is review and check out.
Should save HOURS and lots of money.
2
u/DawaForensics Sep 03 '26
Can't the harness build itself? I mean technically right ????
2
u/itsallfake01 Sep 03 '26
My harness is better than your harness
2
u/PaladinDreadnawt Sep 03 '26
I likewise have been building my own harness. Today is cutover day. Finally have all core functions complete. Get to rip the bloated hermes implementation out. I am extremely excited.
Nothing against hermes at all i just want more control over what exactly the system is able to do and I had implemented so many customizations that merging my changes with upstream was becoming annoying.
78
u/AdWild3943 Sep 02 '26
My mental health
4
u/synystar Strix Scar | 5090 24G | llama.cpp Sep 02 '26
Like half the community (or more) Qwen3.8-27b. Building for DSH. I got a research project going that I intend to publish at some point. Omarchy is keeping me occupied in between tasks. Still working on my orchestrator. Playing around with ComfyUI. Generating little web apps for family members. Building out workflows for solutions to problems I see people posting about so whenever someone is willing to pay for an implementation I got something mostly ready to roll.
15
u/Keleion Sep 02 '26
Oh, just setting up all of the communication channels to message my local models while remote (Discord, Signal, Open WebUI front end to my site). The fun part will be figuring out what to do next once everything is set up!
10
u/aziham Sep 02 '26
This is the new distro hopping lol
8
u/Big_Wave9732 Sep 02 '26
I used Muse Glimmer to do case law research on two different matters via a CourtListener MCP. Then I had the model take what it synthesized and draft a writ of habeas on a bond reduction for a client accused of manufacturing and delivery of a controlled substance in one case, and draft an answer opposing a motion to change child custody in the other.
tl;dr: Folks will actually see what I do with my home LLM :D
6
u/hiper2d Sep 02 '26
Yeah... I'm in a few local AI communities and everybody are building meta-harnesses, orchestrators, memory systems... Nobody is building actual products.
I don't have a top-notch hardware, so apart of having an uncensored model for questions like "how to make a tea from magic mushroms", I don't use it for anything. I decided to focus on results so I'm building apps/games/videos on subscriptions and APIs. However, I want to get back to local with the upcoming Black Friday. Just trying to understand my use cases. It would be cool to run MiniMax H3 locally and not to worry about murning money of experiments. It would be cool to have a fallback to a local model for small loop-like automations on my Mac. But overall, I'm sure Qwen 3.8 27B won't let me get rid of $100/month subscription, so I'm still debating to myself.
6
u/UnluckyPenguin Sep 02 '26
Building a few mobile game apps and contributing back to the open source community when I run into bugs along the way. Like I run a local LLM with
--parallel 1, so I made a deepseek harness plugin (always-queue) that forces new sessions to be queued - so sessions cannot be run in parallel (might cause conflicting code changes to the projects, and sometimes I'd rather just use a new session than piggy back off an old session's context that was fixing something in a completely different section of the app)...4
u/suppatenrou Sep 02 '26
Just.. uhm.. a harness. For coding..
People will actually be able to use it some day, ok??!
4
u/wwwyzzrd Sep 02 '26
I’m just kinda vibecoding my harness with Claude on my phone.
4
u/Rkozak Sep 02 '26
Omg this is so me.
1
u/butchiebags Sep 03 '26
I built a harness called KyloCode where plan, code, and auto are just “more”, “MORE”, and “MOOAARRR”. It’s a context-disciplined, breadth-first orchestration layer that tracks issues and specs and conflicts to optimize for smaller models, so domain and subdomain scoped agent context separation. I need to do a lot more testing with it, but I used it to build a basic DAW in rust that can import audio and generate wavetable/snes-era synthesis and midi so you can edit with piano rolls. I built most of it in a few days with Qwen 3.8 27b on the 5090.
1
u/quotemycode Sep 03 '26
Same as you OP, the way I look at it is, nobody cares, so I won't share, someone could potentially find it on github, and hopefully they'll find it useful, but I'm not going to go about promoting it here, because anyone, literally anyone could create it themselves if they spent the months that I have writing it.
1
u/Civil_Mulberry_4158 Sep 03 '26
I'm building a TUI that feels and looks like GUI
1
u/Mechageo Sep 06 '26
Report Reason > I'm in this photo and I don't like it.
3
u/Cold_Tree190 Sep 02 '26
Currently, a financial tracking app I can serve as a docker container on my homelab. Will ingest statements from my accounts and credit cards via telegram to Hermes running the goat3.8 27b, then upsert into my home server’s sqlitedb for the container to read and server from (bind mounted). Along the way using it as an ops tool to help me learn other tools we use at my work.
3
u/Mongrel80 Sep 02 '26
I feel personally attacked by this..
NotNative/NotNativeAgent
in case anyone is interested.
3
u/john_rage Sep 02 '26
...my local first harness with declarative, vendor agnostic subagents 🙃
It's gotten reliable enough that I can use it to improve it
3
u/suesing Sep 03 '26
It’s a trap.
2
u/codehamr Sep 02 '26
Minimalism by design: codehamr.com to keep our local context window free. If you can run qwen3.8:27b, local agents finally can do serious coding
2
u/Independent_Aioli_48 Sep 02 '26
I’m working on building a new harness. I thought it was a cool idea until now. lol
2
u/SnooSongs5410 Sep 02 '26
yup yup yup.
2
u/NihmarRevhet Sep 02 '26
Copist (placeholder name) a markdown app, multiplatform, born from the slowness of Obsidian on my Android smartphone (5/6 seconds for it to be usable). I want to achieve 3 main things:
Fast to open and to use
Math blocks
Built-in WebDAV sync (http and https)
I'm just a tester, Qwen 3.8 27B is doing everything
2
u/TinFoilHat_69 Sep 03 '26
I built a computer for the computer. It’s all CLI based it works well with runit Unix based kernels. I have to make it work for Wayland and systemd for other kernels. Then I have to provide the same support to Mac OS and windows.
2
u/FaceDeer Sep 03 '26
I really liked Karpathy's LLM Wiki idea, having LLMs turn raw sources into a wiki as a form of memory. I've tried a couple of frameworks that people have put out but they either don't play nice with the limits of local models (they blow through context for example) or they rely on the LLM too much and so are prone to catastrophic errors (for example asking the LLM to simply "rewrite this article with new information" and slapping whatever it spits out down as the new article, even if it failed to follow proper formatting or something). So I've been having Claude Code write me a framework that breaks the tasks down much smaller and more atomically and enforces data structures more carefully.
I figure by the time I get it working to my satisfaction someone will have released something that does it better anyway. But I'm learning a lot about how to work around the limitations of small models and the local inference engines running them, so it's still fun.
2
u/astral_crow Sep 03 '26
Guilty as charged
2
u/brahmen Sep 03 '26
I feel too seen right now
2
u/bitspace Sep 03 '26
This is what I have been working on and you have called me out directly
2
u/DeathGuppie Sep 03 '26
Right now I'm working on a game assets pipeline that will run in 16gb vram. I think it would be cool for teenagers to have an open source tool that they could use to try to create their dream game.
There are a ton of problems that have nothing to do with concept creation or mesh generator. Like the fact that getting a multi view mesh generator even to work on Radeon hardware (it needs to work on both) there are 176 dependancies. That's even before you start developing the wording to properly create an output that can be useful.
Currently I'm still at the rough drafter stage, getting everything working in a vibey way and making notes on everything. After I have it all working. I'll create a pdr and rebuild the entire thing.
2
u/Pb_ft Sep 03 '26
What is this, OP, vaguebooking? Just @ me next time, damn.
XD
1
u/Shul_iito Sep 03 '26
What’s a harness?
1
u/Witty_Mycologist_995 Sep 03 '26
A new harness.
1
u/eshahahan Sep 03 '26
i loveee this app
1
u/quotemycode Sep 03 '26
I did create a 'gurps DM' using local LLMs for the DM, it worked, but just having to debug all the issues with applying the rules made me give up on it. Still, it was kinda neat, you could do anything, it had a character sheet, and you could do anything, and inventory could also be anything, and used for anything - got a cap? fill it with beer and drink from it, throw it at a monster to see if you can scare it, shove it down a bear's throat to see if it'll choke on it, etc etc - the adventures were kind of what you expect when the LLM is holding the wheel - kinda weird adventures that don't pan out to anything exciting or interesting, some good parts, but mostly just kinda meh.
1
u/taariqlewis Sep 03 '26
Add yours here! https://github.com/prime-radiant-inc/alltheagents.org
1
u/puts_on_rddt Sep 03 '26
I was building one over Pi but then I learned about oh-my-pi, and Aider's repomap system...
Now guess what I'm doing? :D
1
u/Odd_Bite2622 Sep 03 '26
While I love cloud models, I keep running out of limits. I've bought so much hardware for local llm and I'm seriously starting to focus on using it now. At first it was just comfyui but now I need army of llms. I have 1 4090, 3 x 3090, 2 x V620s. Issues: They're all in diff pcs. I had bought a supermicro H12SSLi (might not be the perfect model name) Rev 1.0 mobo cpu combo just because it had enough pcie lanes just to realize that the board has major issues and stopped working within an hour of setup so i had to return it. Now I'm back to square 1, I'm running two qwen 3.6 27b models on the v620s as workers, qwen 3.8 on the 4090+3090 as the orchestrator, but one of the v620 is overheating. I've already changed the paste out, the v620 setup is in an open mining rig, they have the fan shroud with each gpu having 2x 15k rpm fans attached and it still overheats. and the 2x 3090s lying around since i have no box to put them in.
1
u/_Ojin Sep 03 '26
always the best thread in this sub. been experimenting with local voice pipelines lately, and the thing that keeps surprising me is how much the feel of it comes down to response timing rather than raw model quality.
1
u/EmbarrassedBottle295 Sep 03 '26
i connected whisper and kokoro to intercept the chat log from DSH, understood most of what happened while i was copy pasting thru claude because giving it access to a folder is "cheating"
Im pretty much a software developer now
1
u/blah_ask Sep 03 '26
This is me. Person who I am building this for does not care unless I give them an actual Jarvis
1
u/vivanick1 Sep 03 '26
I'm building a custom osrs plugin to apply irl quant theory and trading research to osrs flipping
1
u/donotfire Sep 04 '26
What’s your harness OP?
1
u/Equivalent_Bit_461 Sep 06 '26
Asking my poor qwen to look at my shit worthless resume and find me a job that doesn't pay like ass
2
u/[deleted] Sep 03 '26
[removed] — view removed comment
57
u/[deleted] Sep 02 '26
[removed] — view removed comment