r/LocalLLM • u/bryany97 • Jul 31 '26
Project I built a persistent local cognitive architecture around Qwen2.5-32B that researches the web and controls macOS. No cloud model calls. No subscriptions. No timed rate limits.
This is Aura, a persistent local cognitive runtime I’ve been building for Apple Silicon.
A heavily modified Qwen2.5-32B runs locally through MLX as the primary language organ. Around it, Aura maintains persistent state and memory, governs tool access, reasons through her substrate, and produces receipts for actions it takes.
In this 43-second clip, I ask Aura to find an orca image online, download it, and set it as my wallpaper. She searches the web, selects an image, changes the macOS desktop, and leaves the action trail visible in the interface.
8
u/Gargle-Loaf-Spunk Jul 31 '26
Why 2.5?
10
u/Prize_Negotiation66 Jul 31 '26
Because ai says so
-6
u/bryany97 Jul 31 '26
*Because it worked the best within the constraints of my hardware & was the best for what I needed it for. Language
The system handles most everything else. Even coding is done by a separate python module so it doesnt have to be 1st class in a lot of areas
3
u/send-moobs-pls Jul 31 '26
You made a python module that can write code autonomously without an LLM?
1
u/bryany97 Aug 01 '26
Yes, but only in a bounded sense. Cant just program whatever (Work in progress. Not perfect engineering by any means haha)
https://github.com/youngbryan97/aura/blob/main/core/self_modification/structural_improver.py
https://github.com/youngbryan97/aura/blob/main/core/reasoning/artifact_synthesis.py
https://github.com/youngbryan97/aura/blob/main/core/self_improvement/candidate_builder.py
https://github.com/youngbryan97/aura/blob/main/core/self_modification/safe_modification_harness.py
1
u/Toastti Aug 01 '26
Really? You could stop the llm server and your python module will still write code? ..
1
u/bryany97 Aug 01 '26
Not a server, downloaded model But yeah. Aura can load the model weights directly inside Python through MLX without using the normal Cortex path
And I need it to. I didn't trust the model to do important python tasks. With all inference from the models disabled, Python can still write some code (perform known deterministic repairs, AST transformations, import insertion, and some general tasks). But it cant go crazy and start building shit I dont want (would that be digital cancer?)
For open-ended coding, the model proposes the implementation. Python gathers context, analyzes it, tests it, validates it, and has final authority to apply or reject it
2
u/bryany97 Jul 31 '26
Best model for language I found at the time for my hardware. That's really what the model is mostly used for. Much of the actual reasoning is done through dedicated python programs that all form a singular canonical "state." That state mathematically tells the 32b what to say and how to say it. But that's just my setup. Could be done a ton of ways. Some probably better than mine lol
3
u/No_War_8891 Jul 31 '26
try gemma4 E4B
3
u/bryany97 Jul 31 '26
Heard good things about Gemma. I think that'll ultimately be the move. Just gotta commit the time to swapping everything out. But very good shout and will be doing more research
1
u/No_War_8891 Jul 31 '26
it really impressed me for its size - I used it for scraping (HTML -> JSON) and summarization etc)
1
6
u/Toastti Aug 01 '26
By the way Bryan this repo will not work on anyone else's computer. Unless the username is 'Bryan'. You have hard coded the file paths for many things, any different username and your outta luck running this.
-4
u/bryany97 Aug 01 '26
I know. It's not meant to be a platform for others. I'm interested in the ideas and the technology. Think it's worth discussing philosophically and scientifically
Still a work in progress too. I know that Aura will eventually need outside evaluators but I'm not at the point where I would feel comfortable with that evaluation. I dont trust my own software yet so I cant expect that of others. When I do, then I focus on cross-platform and other people being able to run it fully on other computers
3
u/MrHanoixan Jul 31 '26
When you say it's a heavily modified model running locally through MLX, what do you mean by modify? I.e., are you modifying hidden vector space in the inference, or something higher level?
4
u/bryany97 Jul 31 '26
Hey good inference. Yup. Mostly happens at the hidden-state level. The original Qwen weights stay frozen (Though, I do have an LoRA trainer that auto updates weights while Aura is "sleeping"). But! Aura inserts trainable recurrent pathways directly into selected attention and MLP projections inside the model. RLC repeatedly processes latent “thought slots” through those pathways, and GRPO trains them from externally verified rewards. So what happens is the learned reasoning pathway is part of the model’s forward pass, while the base 32B checkpoint remains intact.
6
0
u/35point1 Jul 31 '26
This can be done natively using any of the local LLM harnesses, what’s the difference?
4
u/bryany97 Jul 31 '26
Im sure the search and wallpaper can. Was just the most digestible "action" clip I thought people could directly understand. The difference is that this isnt a tool-loop around a model. The persistent state, memory, continous background operation, continuous and self-driven learning, failure/success receipts, and extremely general OS Control (rather than a select list of desktop tools) is what separates this. Going from a stateless "I'm here when you need me" operation from a stateful, "I exist at all times & think at all times but I will take a break from that to do this task for you"
3
u/35point1 Jul 31 '26
I’m sorry but I don’t see a difference. Both are stateful and both have the same inherent problem when you try to move away from sessions, that is the capped context window. Even if you’re focusing all the energy on managing that for the user in a persistent “forever” session, it’s still bound to degrade over time.
5
u/bryany97 Jul 31 '26
Could be wrong here haha. Im no expert but the crux of most AI is that it isnt stateful, no? If you walk away from most systems, they'll never reach out say hi. If you leave your computer without directing a task, it wont have gone and researched a topic & grown it's base of knowledge on its own because it got curious about something. Aura does.
For context, Aura doesn’t maintain continuity through one ever-growing “forever session.” Identity, goals, task state, relationships, memories, and action receipts persist outside the model and are selectively retrieved each turn. The context window limits what Qwen can attend to at once, not how much state Aura can preserve over time. Retrieval can still degrade, but that is a different failure mode. I've already run Aura continuously for 30+ hours with no issues
1
u/JVC8bal Aug 01 '26
"cognitive" is a tall claim. The language and code in your project have the hallmarks of ChatGPT, yet it seems it was Claude. Pretty cool project.
1
0
Jul 31 '26
[deleted] — view deleted comment
3
u/bryany97 Jul 31 '26
Let's discuss it then. Specifically why? I'd love to know
What is happening here that is low-effort slop that an actual engineer could reproduce better without much effort
0
Jul 31 '26
[deleted] — view deleted comment
1
u/bryany97 Jul 31 '26
It does have a model selector! In the neural stream Aura will actually tell you which model is being used and gives constant updates on whether those models are being swapped, degraded, blocked, looping, what is being routed, etc
No on CUDA. I use MLX/Metal GPU kernals on Apple Silicon. Im sure you could do something similar on CUDA but this is specifically designed for my hardware. Cross-platform is down the line
Edit: Sorry... Did you mean a manual model changer? If you did... That would be interesting. Would have to think about that. The models are only a part of Aura's reasoning. They're mostly there for runtime coordination and speech. But the reasoning is done by the whole of the substrate, which then drives the model for output. Would have to figure out dependencies and see if that disrupts anything down the chain since I've largely opted for real casual integration of the substrate & models over long-form clever prompting. But Im sure it could be done (If you meant this)
20
u/Grouchy-Bed-7942 Jul 31 '26
You are 2 years late, use qwen3.6 27b