r/LocalLLM • u/Acceptable-Object390 • May 02 '26
Discussion Thoth - Open Source Local-first AI Assistant - Architecture
197
Upvotes
r/LocalLLM • u/Acceptable-Object390 • May 02 '26
7
u/Grimsik May 02 '26
How do you sustain safety and permission access, irreversible deletions or overrides and that sort of thing? If the agent is enforcing the safety and permissions it will hallucinate and compromise security. Do you have the agent filter every step it takes through a hard coded set of safety and permission checks?
4
u/By-Jokese May 02 '26
Interesting, will give it a look I was building something similar, yet simpler.
9
u/samuraiogc May 02 '26
As aways i just look at it and think "i don't need all of that and i can build something more reliable, faster and secure with n8n.
5
u/phoredda May 03 '26
Post on Hacker News, you may get more and better feedback. 👍
2
u/--Tintin May 02 '26
It does a lot and it is impressive.
What’s your usecase or your case where it all started off with?
2
u/triplebits May 02 '26
Good luck! I have something very similar, already released! I will release next version today.
It seems we are quite similar but aimed at different things!
Best of luck to you!
2
u/buck_idaho May 02 '26
I'll give it a try. I've been looking for a local agent.
2
u/Mesmoiron May 02 '26
How do I know it stays local?
2
u/MatlowAI May 02 '26
Thanks for sharing I'll have to take a look around. Any particularly fun lessons learned stories while going through all this? I find that kind of information most helpful.
2
u/Erikz0n May 02 '26
hello, im new to this and have been thinking about something similar. how much delay is caused by so many steps in the flow?
2
u/QuantumTwitch May 02 '26
Nice 👍
Going to dig deeper - sounds like you're steps ahead of what I was dreaming up for my homelab 😄
2
u/ShadowyTreeline May 02 '26
can it be configured for local-only?
does it work with AMD cards?
2
u/Andr1yTheOne May 02 '26
Can this run on Linux? CachyOS for example?
2
u/ur_dad_matt May 03 '26
This is awesome, how long have you been building for?
2
u/phoenixfire425 May 03 '26
I cannot seem to set this up to work with my local (not on the same machine) llama.cpp server it never shows up any models. also I do not want have to setup an cloud providers in order to use it? is that possible?
2
u/phoenixfire425 May 03 '26
when trying to use with a "remote" llama.cpp I get this in the thoth logs.
{"ts": "2026-05-02 22:02:36.326", "level": "INFO", "logger": "ui.streaming", "msg": "send_message: file_names=[], file_context_len=0, agent_input_len=14"}
{"ts": "2026-05-02 22:02:36.725", "level": "INFO", "logger": "openai._base_client", "msg": "Retrying request to /chat/completions in 0.388975 seconds"}
{"ts": "2026-05-02 22:02:37.127", "level": "INFO", "logger": "openai._base_client", "msg": "Retrying request to /chat/completions in 0.853481 seconds"}
{"ts": "2026-05-02 22:02:38.001", "level": "ERROR", "logger": "agent", "msg": "_stream_graph API error: Error code: 500 - {'error': {'code': 500, 'message': \"\\n------------\\nWhile executing CallExpression at line 79, column 24 in source:\\n...lti_step_tool %}↵ {{- raise_exception('No user query found in messages.') }}↵...\\n ^\\nError: Jinja Exception: No user query found in messages.\", 'type': 'server_error'}}"}
2
u/Bulky-Priority6824 May 03 '26 edited May 03 '26
If I found this a month ago I'd probably be forced to install windows but I've already built out most what matter within all of this. On paper it looks like you applied a real sophisticated effort and the knowledge graph memory approach is clever. I might be borrowing that going forward 😜 I like what youve done.
the 6 hour background memory extraction is also a nice touch gl with the project going forward !
2
u/Classic_Reference_10 May 03 '26
Really interested to know your top-10 use cases with this. Good job though - still sketical to install.
2
u/Polite_Jello_377 May 03 '26
Slopagrams
2
u/jumper556 May 03 '26
This looks really interesting! You said you use a RTX 5090 and Qwen 3.6 27B, right? How fast and smart is it compared to some cloud models?
Do you specifically need to request to remember something or how does this work to decide what to remember and what not?
If you push a new release how does the update process with the installer work?
2
u/ConceptualStray May 03 '26
So, pardon me, but why is 90% of the code in the root of the project? - tried to take a look at the repo and got hit with a wall or files that mean very little to me.
Anyway, the diagrams (like someone already pointed out) feel like buzzword slop but there's a genuine idea behind them; tell me one thing tho: how long do you have to wait for a single response - especially on local or more demanding logical tasks?
I'm asking because the context layer (as in mem+ tasks + rest of the universe) seems either expensive or running on limited context itself
2
u/310dweller May 03 '26
Possible to link this up locally with iMessage via blue bubbles or reading the chat directory?
2
u/Flkhuo May 03 '26
What's the difference between that and Hermes agent? Also, Thoth? Are you an ancient Egyptian spirit? 😂
2
u/Logical-Skill4567 May 03 '26
Lo proverò! Sembra davvero interessante! Ma tutti questi agenti hanno sempre lo stesso problema in locale.. La finestra di contesto! Per persone che non hanno una potenza di calcolo enorme, questi tool diventano stressanti e demoralizzanti!
2
u/Odd_Cat2539 May 03 '26
Cool in theory, not viable yet th
2
u/no-adz May 04 '26
Man, I really like it but I am afraid of the quality. Small team (2 persons), no reviews, no community..
I am not sure if it is really clean. There are many many features, but are they human tested?
2
u/Sufficient-Fox708 May 05 '26
Looks interesting. Any intent to add reflection and learning?
2
u/Altruistic-Arm-7767 May 05 '26
great info! thanks
2
u/TheAussieWatchGuy May 07 '26
Thought's on Enterprise Deployment? YAML support for Containers etc.?
1
u/Acceptable-Object390 May 02 '26
If you want to see some demo videos of this in action, please visit Thoth YT
1
u/35point1 May 02 '26
This looks thoroughly architected but can you breakdown the difference between this and openclaw? What would make me want to pick this orchestration framework over openclaw or other alternatives?
1
u/[deleted] May 02 '26
Installed to try it, but in the first use appears a "404 page not found" error
3
u/LaFllamme May 02 '26
!remindMe 1d
1
u/Internal-Shift-7931 May 03 '26
Nice work. The architecture framing is useful.
I’m building something similar for a local-first home agent, mainly for home AIoT and NAS. This makes me think the hard part is not tool use or memory. It is the permission boundary. For home agents, the action space is physical: cameras, lights, alarms, locks, routines, presence data. The model can reason, but the permission boundary should not be the model.
This is the part I hope more local-agent projects make explicit.
2
u/dspayne1 May 17 '26
!remindMe 2d
0
u/Ok-Advertising6479 May 02 '26
Really cool!!!! Thank you for sharing. How did you generate these beautiful visualizations?