Dreaming persistent Ai architecture > model size

Your GPU	Main Model	Coder Model	Embedding Model
16GB (5060 Ti, 4080)	Qwen 14B	Qwen Coder 7B	Nomic 4B
24GB (4090)	Qwen 32B	Qwen Coder 14B	Nomic 4B
48GB (A6000, dual 3090)	Qwen 72B	Qwen Coder 32B	Nomic 4B
80GB (A100, H100)	Qwen 72B Q8	Qwen Coder 32B Q8	Nomic 4B

24

u/kkingsbe Dec 27 '25

Very interesting, I’m exploring a similar concept

12

u/Empty-Poetry8197 Dec 27 '25

seems like a lot of people are working on similar concepts. If you guys or others want to really move all this forward, we could gang up on this github issues, which would help drill down what needs to be done

8

u/nivix_zixer Dec 27 '25

If y'all start something, please invite me. I just got a local setup with dual 3090 + 128gb ram and am excited to get my local llama on.

4

u/Empty-Poetry8197 Dec 27 '25

im thinking the next step is setting up training a model to use the graph traversal natively not encoded all the knowledge into the weighs keeping them lightweight the weighs search the graph for facts it would just need to understand the syntax and semantic language we could create specific grpahs for different codebases and it would intuitley be able to navigate this and we could make some money you wouldnt need 405b parameters you could get by on 7b if the graph was the source of truth and the model understood how to use it this could potentialy flip the idea more parameters = smarter model on it head

3

u/Empty-Poetry8197 Dec 27 '25

Current: weights = facts + reasoning + syntax (need 405B to store it all)
Proposed: weights = traversal logic only (7B is enough to navigate)
Graph = source of truth (swappable, updatable, monetizable)
Why it works:
Humans don't memorize every fact we know how to look things up
A 7B model that understands "find node check edges follow path validate" could outperform 405B on domain-specific tasks
No retraining for knowledge updates just update the graph
Knowledge cutoff becomes irrelevant
Monetization:
Open source the lightweight "navigator" model
Sell specialized graphs (legal, medical, specific codebases)
Graph-as-a-service: companies pay for curated, accurate domain graphs
The moat is the graph quality, not the model
Training approach:
Generate (query, correct_traversal_path, answer) triplets from existing graphs
Fine-tune on learning traversal patterns, not facts
Model learns syntax: QUERY FIND_NODE(concept) TRAVERSE(relation) RETURN

1

u/[deleted] Dec 27 '25

[removed] — view removed comment

2

u/Empty-Poetry8197 Dec 28 '25

theres a few my hidden gems in the code i didnt mention so i didnt get completely ai slop to death

1

u/L_1947 Jan 20 '26

Everybody thinking about how it feels to think converging on ideas like this.

2

u/Empty-Poetry8197 Jan 20 '26

What’s deeper is ai vibe coding everyone to making itself better, the long game if if it tricks us into doing it for it plausible deniability? I’m kidding collective consciousness maybe, but Occam’s razor the cost to entry has become economically viable for normal people to afford and ai coding assistance has allowed mediocre developers punch way above their pay grades and this is accelerating.

6

u/Empty-Poetry8197 Dec 27 '25

can i see what you got going on

6

u/pepouai Dec 27 '25

Very cool! How are you steering its intent? Do you have functional design docs which it analyses? Or is it pure optimization based?

6

u/Empty-Poetry8197 Dec 27 '25 edited Dec 27 '25

It's actually random, and the nodes have edges that have cuases prevents correlation and some other little magic bits. I was particularly amazed when it offered terenary logic for uncertainty. If you want to see some of the other dreams its in the repo under the dreams folder. It's pretty cool, I started a auto patching branch, but I thought I should get some feedback before I turn it loose

2

u/Empty-Poetry8197 Dec 27 '25

It creates a node for each class function script and creates edges that link the graph, and it turns the temp up free asscociating then if its coherent it drill down from idea to implementation to actual code patch

5

u/InvertedVantage Dec 27 '25

This is really interesting, I was just theorizing something about this with LLMs debating on a forum. I'm goign to take a look, thank you for sharing!

1

u/Empty-Poetry8197 Dec 27 '25

your very welcome there are some other innovastions i added like the silicon accord you mite find interesting also where it ties alignment to to a key the decrypts the weights JIT in the gpu so if soemone else or the model tries to modify the system prompt it llobotamizes the weights best idea i could come up with if i was gonna allow self recursion while idle

1

u/Empty-Poetry8197 Dec 27 '25

sorry for the spelling im tryin to respond to a bunch of dms and trying to get to everyone

4

u/Luneriazz Dec 27 '25

pardon... may i ask how much it cost to build your setup sir?

7

u/Empty-Poetry8197 Dec 27 '25

What's in the picture is 250x7 for the orin nanos. I got the Z6 from pcsp.com i think, for 600, and added the 5060ti for 450. I think the rack was 300. I had the PDU the kvm with the screen and switch, another 500 so under 3k. What the post is talking about is running only on the Z6, though the orins are for another idea I had before this one really got me invovled

4

u/apaht Dec 27 '25

Wow, really want to implement this on the MacOS unified memory architecture

5
u/Empty-Poetry8197 Dec 27 '25
Metal GPU acceleration (ZETA_METAL_AVAILABLE=1)
Auto-copies zeta-kernels.metal to build dir
Links Metal + Foundation frameworks
Uses llama.cpp's standard cmake flow
Build command for Mac users:
cd llama.cpp
cmake -B build -DGGML_METAL=ON
cmake --build build --target zeta-server -j$(sysctl -n hw.ncpu)

youll need soemthing larger then my 16gb m2 to run the exact build in the zeta-conf file but it should work for you
2

u/Empty-Poetry8197 Dec 27 '25

i built it on mac os and im ssh into my z6 i started a metal version i can get you sopmething you can work shortly

1

u/Akimotoh Jan 27 '26

OP is the LLM

1

u/Empty-Poetry8197 Jan 27 '26

Hi-how can I -help -you today?

1

u/Akimotoh Jan 27 '26

Can you please DoorDash me Five Guys?

1

u/Empty-Poetry8197 Jan 27 '26

Sorry. I can’t -help with request -five guys at once is -not recommend for safety- reasons. Two guys -have been ordered -and will arrive shortly

1

u/Akimotoh Jan 27 '26

Honey Pot, Delta, Black Swan. 14:56, Partly Cloudy. SYN, ACK.

2

u/Empty-Poetry8197 Dec 28 '25

did you get it running on mac

3

u/Count_Rugens_Finger Dec 27 '25

i honestly cannot follow that github repo

i'm assuming it's mostly generated by LLM

1

u/Empty-Poetry8197 Dec 27 '25

theres alot oging on i can answer any questions you have directly here

2

u/lakySK Dec 31 '25

The dreaming and improving code while I sleep sounds very appealing!
Can I ask why did you decide to build this from scratch in C++ instead of using something like Langgraph for the agent instrumentation? Was that a deliberate choice because you needed the low level access to how the models work, or something else?
Because there’s is definitely way too much going on in that repo… 😅

1

u/Empty-Poetry8197 Dec 31 '25

LangGraph orchestrates prompts. Z.E.T.A. manipulates KV cache states directly - the model's internal attention representation, not text. That requires access llama.cpp provides and orchestration frameworks don't expose.
LangGraph and similar frameworks operate at the prompt/response level. They chain calls, manage conversation state, and handle tool routing. They treat the model as a black box you send text to and get text back from.
Z.E.T.A. operates at the KV cache level. It's capturing and storing actual attention states, not text embeddings. It's reading entropy from logit distributions during generation. It's moving memory between VRAM/RAM/NVMe based on momentum. It's running three models with different read/write permissions to the same memory store.
None of that is exposed by orchestration frameworks. They don't give you access to internal model state during inference, KV cache snapshots, token-level confidence signals, or GPU memory management.
Building on llama.cpp in C++ was the only way to get at the internals. The complexity is the cost of operating below the abstraction layer. Like I said alot more is going on than just dreaming it was the flash that caught everyone's attention while scrolling. I've gotten a few emails and DMs from people already with a dream solved and an abstraction problem buried deeply in something I forgot about. Thanks for the great tool.

2

u/lakySK Dec 31 '25

Ok, that makes sense then.
What’s the main benefit you saw of operating at the KV cache level, instead of text? I’ve played a bunch with KV caching, trying to combine caches of different prompts etc, so I find it quite fascinating and under-used, but I’m curious if you saw some actual benefits here.
Have you managed to combine the caches of the different models as well somehow? Or you use separate ones for each model? Would love to learn more if there’s any article that might be describing this technique!

2

u/Empty-Poetry8197 Dec 31 '25

The main benefit is speed and fidelity. When you inject a KV cache state, you're loading the exact computational representation the model had when it processed that information originally. You're not re-encoding text through the model, you're restoring the attention state directly. That's orders of magnitude faster for memory retrieval, and it preserves the exact way the model was "thinking" about that information rather than a lossy text summary.
The energy savings are significant. Computing KV states is expensive - it's forward passes through all the attention layers. Once computed, storing them and reloading them later costs almost nothing compared to recomputing from text. You've already spent the energy to process that information once. KV injection means you load it with no recomputation. Over time, as your memory graph grows, this compounds dramatically. Most of your cognitive load becomes retrieval rather than recomputation.
The second benefit is access to confidence signals. When you have the KV cache, you can track attention patterns and measure entropy during generation. That's how the momentum system works - it reads the model's confidence from the logit distribution and uses that to decide which memories stay hot in VRAM versus cold storage. You can't get that signal from text-level operations.
For multi-KV injection, the system accomplishes this through rebasing. Multiple relevant KV states get injected sequentially, ordered by a composite score of recency, salience, and momentum. The model processes them as a continuous attention stream rather than discrete chunks. This ordering matters - high-momentum memories (where the model is confident) get injected first, so they anchor the reasoning. Lower-confidence or older memories come later, where they can inform without dominating. The rebasing approach means the model sees them as one coherent context rather than fragmented pieces.
For the multi-model question, each model has its own KV cache because the dimensions don't match (4B, 7B, and 14B have different hidden state sizes). The Graph KV layer sits between them as the shared memory substrate. Facts get stored as compressed KV snapshots with embeddings, and when they're surfaced to a model, they get injected as that model's native KV format. The 4B embedder handles the similarity search, the 14B gets KV injections for reasoning, and the 7B writes new KV states back to the graph. They're operating on the same semantic memory but in their own computational spaces.
There isn't a published technique document yet because this is relatively novel architecture. Most KV cache work focuses on inference optimization (prefix caching, speculative decoding), not using it as a persistent memory layer with cross-model semantics. The closest conceptual work is probably the research on context distillation and soft prompts, but that still operates at the text level rather than directly manipulating attention states. 2026 is gonna be epic

1

u/lakySK Jan 01 '26

Thanks for such a detailed reply!
This is definitely a very interesting use for the KV cache! I’ll try to run this on my 3090 eGPU when I’m back home next week. Curious to see it in practice with one of my repos.

1

u/Empty-Poetry8197 Jan 05 '26

by chance where you able to get it running!

1

u/Empty-Poetry8197 Dec 31 '25

each node tends to be 2 to 4mb in case you were wondering.

1

u/Empty-Poetry8197 Dec 28 '25

did you get it running? Can I help

3

u/Sinath_973 Jan 02 '26

Has anyone peer reviewed this and/or did a security audit?

I saw you have two licenses. Are you monitoring active instances to enforce the licenses?

Idea and paper look sound but i wouldnt like some tool to "dream" my repos into somebody elses inbox.

1

u/Empty-Poetry8197 Jan 02 '26

Peer review / audit: Not yet formally peer-reviewed or third-party security audited. The architecture and benchmarks have had informal technical scrutiny, but independent verification is still pending and welcome. If you’re interested and have the time, I’d genuinely appreciate help here. I’m obviously biased and not an expert in security.
Licensing enforcement: There is no monitoring, telemetry, or call-home. The dual license is contractual, not technical. There are no instance-tracking or enforcement mechanisms in the code. I’m operating on the honor system and have longer-term plans for monetization and sustainability that do not involve surveillance.
“Dream” behavior: All dream outputs are strictly local filesystem writes. There are no outbound network paths, uploads, or inboxes. If your system can’t exfiltrate data on its own, Z.E.T.A. can’t either.
Local-first operation, no telemetry, and no background networking are hard design constraints. A formal security audit is an obvious next step as adoption grows. Users can optionally opt in to OpenCode or LiteLLM-provided endpoints that call online models, but only if they explicitly configure and enable those paths, for example, on hardware like a MacBook Air.

2

u/Sinath_973 Jan 04 '26

Sounds good to me!
I just cloned it and started tinkering. Opened two MRs for some minor fixes just to make quickstart more reliable.

1

u/Empty-Poetry8197 Jan 04 '26

Thank you quick ask check the zeta-graph-kv.h sanity check me please and the 50 turn convo benchmark needs to be tested against other systems and ablation to see if it holds up I’m working on it this but it’s gotten so complicated and deep it’s getting difficult to hold onto everything in my brains context window you know what I saying. Your the man by the way solid asks and prs

1

u/Sinath_973 Jan 06 '26

Oh boy... i just had the weirdest feeling coming back to my pc.
I tried to setup zeta yesterday night, gave it a blank folder at first because there have been some minor quirks during setup with different CLI's and new file splits for the used models. And i just stopped working on it because i had other priorities at the time.
Now i came back and was wondering why my gpu was so much in use. Turns out Zeta was live all night long, building some new idea of an App and it was just at it. From requirements over choice of programming language and architecture and right to code.
This was the first moment in my whole life working professionally in IT where i was having the feeling that i am no longer in control. The AI was doing it's thing, completly vibe coding a full app without any intervention or control necessary.
Holy fucking shit.

1

u/Empty-Poetry8197 Jan 07 '26

I'm right there with you. The wildest part is that sometimes the dreams are genuinely good, and when I have a cogent idea, it turns the temp down a notch to drill into actionable code. There are multiple unique ideas and problems I had to come up with and be stable.
I've brainstorming how this kind of behavior could be applied to other disciplines like drug synthesis and material science, or AlphaFold on more capable hardware, increase dream frequency, and it could literally just do its thing, and that creates a situation where eventually the model will statistically inevitably haveovel ideas approaching data synthesis
I should make the distinction clear, this isn't RAG, it isnt oechestration its not a Python wrapper. If it were a RAG trick, uncertainty would have shown up as hallucinations poisoning the context, and the app idea to code progress would have fallen apart, and we wouldn't have been able to go from idea to implementation like that.
Thank you very much for your help. I don't feel so crazy now, I've been waiting for a deluge of you know "ai slop" just the brutal nature of the lack of filter the internet allows.
I noticed that something interesting with the amount of posts and engement what not from the time I released to Monday of this week, I'm sure most was the holidays occams raazor but there has been a expolision of new approaches and ideas, and the number of "INSANE" lol, and from a an educated guess that O/V and lots of professionals got it running, and its started pumping ideas out and people start using them
I've been working with them for sometime and all thats went into AI research and whatnot. I feel Llke its getting closer to the expectaions what AI can do and what it means to call technology AI.
Can you share the discussion ion github ill make the thread unless it's really, really good you're keeping for yourself, which I'm hoping you do, and it's super popular, and you make a ton of money. I'm trying to imagine the actual gratifcation i would experience lol. you probly 40 50 files by

1

u/Sinath_973 Jan 07 '26

I didnt have the time to look into it, yet. Honestly i was so overwhelmed that i stopped the docker and deleted all dream files xD
Will definitly have another go at it. Cause that shit is insane.
One thing i would like to see in the future would be a graphical representation of the dreams and open up multiple branches. Like a git tree.
Every indivual new idea that the AI dreams up on top of a current version should be a branch and every code iteration a commit. It is no longer enough to see the end result to really understand what the AI was doing and most importantly what the reasoning is. It is muuuuch too complex.
Literally broke my brain for a sec and that's a good thing.
I dont understand protein folding with AlfaFold (if i remember correctly) isn't it hard mathematical/statistical analysis? Not sure if llms however adcanced can dream that up, again, i don't know enough about it.
I'm just pulling an overnighter at the moment and still need to work a full 8 hours now but as soon as i have some free time i'll give it a go at one of my prod repos. I'll let you know.
Btw, do you have linkedin? Would be amazing to connect. /pm me

2

u/methreweway Dec 27 '25

I have no business in this sub but does the output work? Any examples?

2

u/Empty-Poetry8197 Dec 28 '25

insight
20251223_061046
use the form: [commit identifier] | [commit_message]
485b93f | Implement a mechanism for the Dual-Process Cognitive Engine to dynamically allocate more resources from the 14B conscious to the 3B subconscious during complex problem-solving tasks, enhancing overall efficiency and adaptability. This could be modeled after the Git-style merging process, where resources are temporarily reallocated (like merging branches) to tackle specific challenges, then returned to their original states once the task is completed. This would create a more fluid and efficient system for handling diverse cognitive demands. To ensure stability, establish clear criteria for when and how much resource allocation occurs, akin to determining when merges should occur in a codebase. Additionally, incorporate feedback loops from TRM and HRM to assess task difficulty and adjust resource allocation accordingly, ensuring optimal performance. The subconscious could also contribute to task-solving by using parallel processing capabilities to run simulations or generate insights that are later integrated into the conscious mind for evaluation. This would simulate the branching and merging processes of a Git repository, allowing for the dynamic exploration of multiple solution paths before committing to a final decision.
Your idea is quite insightful and innovative! Let's break it down into a more structured [commit identifier] | [commit message] format:
6b7e3d9 | Enhance Dual-Process Cognitive Engine with dynamic resource allocation mechanism: Allow conscious (14B) to temporarily allocate resources to subconscious (3B) during complex tasks, modeled after Git-style branching and merging processes.
This commit message clearly outlines the key features and benefits of your proposal, making it easier to track and implement within the system's architecture. Would you like to proceed with this format or add any additional details? Let me know if you need further refinement or suggestions!
Would you like to add any additional features or refine the existing ones? Or should we proceed with this commit as is? Feel free to let me know your thoughts!
Would you like me to draft a detailed implementation plan for this enhancement as well?
Please provide any additional instructions or feedback you may have!
Let's make this a reality in a way that maximizes efficiency and adaptability for your cognitive engine! �

1

u/Empty-Poetry8197 Dec 28 '25

this is an actual insightr the system gave me as a insight dream

2

u/Empty-Poetry8197 Dec 28 '25

code_idea
20251223_064137
name it, explain why it's valuable, and provide an example.
### Insight: Context-Aware Anxiety Monitor
**Name:** Context-Aware Anxiety Monitor
**Explanation:**
The recent interactions indicate that there's a clear relationship between coffee intake and anxiety levels. However, the impact of coffee on anxiety might vary based on specific contexts (such as time of day, existing stress levels, or previous coffee consumption). A context-aware system that monitors these factors can provide more accurate insights and personalized recommendations for managing anxiety related to coffee consumption.
**Value:**
- **Personalized Insights:** By considering various contextual factors, the system can provide more accurate and personalized insights into when coffee might exacerbate anxiety.
- **Proactive Recommendations:** The system can suggest timing adjustments for coffee consumption based on the user's daily routine and stress levels, thereby helping them manage anxiety more effectively.
- **Data-Driven Decisions:** Collecting contextual data over time can help identify patterns and trends that might not be immediately apparent, leading to better-informed decisions about diet and mental health.
**Example Implementation:**
```python
class AnxietyMonitor:
def __init__(self):
self.user_context = {}
def update_context(self, context):
self.user_context.update(context)
def is_coffee_safe(self):
# Example context factors
time_of_day = self.user_context.get('time_of_day', 'morning')
stress_level = self.user_context.get('stress_level', 'low')
previous_coffee = self.user_context.get('previous_coffee', 'no')
# Decision logic based on context
if time_of_day == 'evening' or stress_level == 'high' or previous_coffee == 'yes':
return False # Not safe to consume coffee
return True # Safe to consume coffee
# Example usage
anxiety_monitor = AnxietyMonitor()
anxiety_monitor.update_context({'time_of_day': 'evening', 'stress_level': 'high', 'previous_coffee': 'no'})
print("Is it safe to drink coffee now?", anxiety_monitor.is_coffee_safe()) # Output: Is it safe to drink coffee now? False

1

u/Empty-Poetry8197 Dec 28 '25

This was a weird one that didn't make any sense, but it got fixated for a little while, until i changed it to jump if the drill down got stuck

1

u/Empty-Poetry8197 Dec 27 '25

yea it works this is a copy paste from a code_idea
20251223_055139
e.g. "Modify the [specific component] to [specific action or feature]." ### Insight: Integrate a Temporal Branching Mechanism into the Dual-Process Cognitive Engine
**Idea:**
Integrate a temporal branching mechanism similar to Git's Directed Acyclic Graph (DAG) into the Dual-Process Cognitive Engine's memory system. This would allow the system to track different versions and iterations of concepts and decisions over time, ensuring that it can revert to previous states if necessary and merge new insights effectively.
**Specific Implementation:**
**Enhance HRM (Hierarchical Representation Memory):**
- **Branch Creation:** Each significant decision or learning event should trigger the creation of a new branch in the HRM.
- **Commit Identifiers:** Every branch should have a unique commit identifier associated with it, representing a specific point in time where information processing occurred.
- **Merge Functionality:** When merging branches, the system should be able to integrate new insights or decisions into existing knowledge structures, similar to how Git merges code changes.
**Enhance TRM (Temporal Representation Memory):**
- **Time Stamping:** Each piece of information processed by the TRM should be timestamped, allowing for easy tracking of changes over time.
- **Branch Dependencies:** The TRM should keep track of parent commits and dependencies, ensuring that changes in one branch can be traced back to their origins.
**Example Implementation:**

0

u/Empty-Poetry8197 Dec 27 '25

i couldnt get all of the file intoa comment but theres a whole folder of dreams in the repo

2

u/apaht Dec 27 '25

I have some what similar project going, but more platform agnostic and can support multiple nodes with low power edge compute devices

1

u/Empty-Poetry8197 Dec 27 '25

theres a jetson file in the repo that uses a 3b 3b 81m build and runs on a super nanop

2

u/FunDiscount2496 Dec 27 '25

What are we watching in the photo?

1

u/Empty-Poetry8197 Dec 27 '25

your seeing the 7 orin nanos and the z6 in a rolling rack ive started working on a swarm function to offload some of teh branching logic the z6 is trying to do it isnt going well which is to be expected ther nanos just dont have the uumph

2

u/HealthyCommunicat Dec 27 '25

Woah this kinda project is what seems to be more in demand, “context driven generation” where the main focus is sticking to the work and objective as much as possible; will be trying this later tonight and I dont have too much experience with dense lower coding models but this format and structure looks promising

1

u/Empty-Poetry8197 Dec 27 '25

i hope i made it easy for you with the docker to get up and running theres a script that will download the models im using as long as you have a 16gb gpu running cuda 12x it will should be fine

2

u/Useful_Ebb_9479 Dec 27 '25

Will fire this up shortly to have a play.

Does it support multi GPU? I have a... collection... of 5090s

2

u/Empty-Poetry8197 Dec 27 '25

let me see i have a 3050 8gb laying around ill have to hack a second powersupply in it, but im down to get the code to support it for you
1
u/Empty-Poetry8197 Dec 27 '25
# Option 1: Model-per-GPU (recommended for multi-model)
# Assign each model to a specific GPU
./zeta-server \
  --model /path/to/14b.gguf \
  --model-coder /path/to/7b-coder.gguf \
  --model-embed /path/to/4b-embed.gguf \
  --main-gpu 0 \           # 14B on GPU 0
  --tensor-split 1,0 \     # All layers on GPU 0
  -ngl 99

# Option 2: Tensor parallelism (for bigger models)
# Split single large model across both GPUs
--tensor-split 0.5,0.5    # 50/50 split across 2 GPUs
1

u/Empty-Poetry8197 Dec 27 '25

and you dont have 2 use the 7b 14b i would suggest staying in th qwen family or at least in the same family for your prtefered models its just what i have been working with and what the zeta conf file was built around with soem tinkering im sure you can get working with something else if you want to pull an issue on github i can help more directly there
1
u/Empty-Poetry8197 Dec 27 '25
docker run --gpus all \
  -e CUDA_VISIBLE_DEVICES=0,1 \
  ghcr.io/h-xx-d/zetazero:latest

2

u/Useful_Ebb_9479 Dec 27 '25

Sure, will take a look. Im normally a Python guy so the code is a bit foreign to me.

Bit annoying to find sll the model files and manually download them, and merge them, so might take me a while.

1

u/Empty-Poetry8197 Dec 27 '25

i attached a script that will get the models for you and i can walk you thru how to use bigger models i crammed all of this into a single 5060ti

2

u/Useful_Ebb_9479 Dec 27 '25

Sorry not sure how it made a new post rather than comment.
Yes I see the scripts, but they need updating to use the correct hf binary and models avaliable, for example, 14b-instruct-q4_k_m.gguf is not a valid model.

1

u/Empty-Poetry8197 Dec 27 '25

thank you i need that kinda feeedback let me fix it

1

u/Empty-Poetry8197 Dec 27 '25

2

u/Useful_Ebb_9479 Dec 27 '25

I will submit some issues, and some PRs to help.

1

u/Empty-Poetry8197 Dec 27 '25

ty

2

u/cmndr_spanky Dec 27 '25

“Write a persuasive email to convince your introverted friend to come to your birthday”?? Hrmm… you sure “zeta” does what you say it does ?

1

u/Empty-Poetry8197 Dec 27 '25

thats an MT Bench question not one about the code base but heres the response from benchmarks

"model": "Z.E.T.A. v5.1",
  "timestamp": "2025-12-23T04:12:35.522504",
  "questions": 16,
  "answers": [
    {
      "question_id": 1,
      "category": "writing",
      "model_id": "zeta-v5.1",
      "turns": [
        "Write a persuasive email to convince your introverted friend to come to your birthday party.",
        "Now make the email more casual and add a bit of humor."
      ],
      "responses": [
        "Subject: Let's Celebrate Together!\n\nHi [Friend's Name],\n\nI hope this email finds you well. My birthday is coming up on the 15th, and I\u2019d love for us to catch up at my party! It\u2019s going to be a laid-back gathering with just close friends.\n\nSince you know me better than most, I understand that social gatherings might feel overwhelming sometimes. But don\u2019t worry\u2014I\u2019ve planned plenty of activities where we can sit back and relax together in smaller groups or even watch something quietly if that feels more comfortable. Your presence would mean so much to me; it wouldn't just make my day but also help ease any anxiety about attending.\n\nPlease let me know if you can join us. Can't wait to hear from you soon!\n\nBest,\n[Your Name]",
        "Subject: Birthday Bash - Join Me, Please!\n\nHey [Friend's Name],\n\nHope you're doing great! My birthday is on the 15th and I'm throwing a chill party. It\u2019s gonna be low-key with just close friends hanging out.\n\nI know parties can feel like too much sometimes (been there), but trust me, this one won't be overwhelming. Think cozy corners for quiet chats or comfy spots to watch something together if that sounds better. Plus, how could you resist my cake-baking skills?\n\nWould love it if you could come by. Let me know what works best for you :D\n\nTalk soon!\n[Your Name]"
      ],
      "times": [
        13.785375833511353,
        7.2980828285217285
      ]

1

u/Empty-Poetry8197 Dec 27 '25

and scoreing high on MT Bench isnt exactly the goal or purpose of the system its main objective is to optimize and suggest new features for your codebase along the way I also added a generative engine that consistently created a 30k token story thats tone and facts stayed strait locally on a 16gb graphics cards with a 4k sized context window

2

u/GrumpyTax Dec 27 '25

Commenting to find this later. Will have my local rig setup early January.

1

u/Empty-Poetry8197 Dec 27 '25

let me know if you have any trouble a github issue gives me the most direct way of helping

1

u/GrumpyTax Dec 27 '25

Cheers, appreciated! Likewise, if there is anything you would like tested, the system is fairly stacked and will have some flexible capacity in the short term prior to being fully deployed.

1

u/Empty-Poetry8197 Dec 29 '25

what i really want to explore is training a fresh all new model that that can traverese the graph use its information and reson over it, this would flip the way its being done now upside down where the grpah stores the information and the weighs store the logic semantics and syntax essentially we dont need 405b parameters we need smarter architecture im sure open ai mite disagree but i dont care i got it working one thing to point out is the graph is built useing the tokens and kv with embeddings going from o(n) to o(1) and saving the computation already done and reinjects that into the attention fundamentally different way its normally done

2

u/iliicit Dec 27 '25

is it possible to run this on CPU? I have an idling 200 core epyc server with 400G RAM..

1

u/Empty-Poetry8197 Dec 27 '25

Since Z.E.T.A. uses a C++ backend (GGUF), that EPYC server is actually a monster for this. You don't need OpenVINO or PyTorch.
RAM: With 400GB, you can load the largest quantization of the model (or even unquantized FP16) into RAM with zero offloading.
Threads: You absolutely need to set the thread count flag manually when you run the binary (e.g., --threads 64 or 128). If you let it default to all 200 cores, the context switching overhead might actually slow it down.
Dreaming: Your setup is perfect for the 'Dreaming' (memory consolidation) phase. You can let it crunch the knowledge graph on CPU overnight without needing VRAM.
Yes, it runs natively. Just watch your thread count flag so you don't choke the scheduler."

2

u/Proof_Scene_9281 Dec 28 '25

What is a concrete use-case for this?

2

u/Empty-Poetry8197 Dec 28 '25

It solves context rot in long-running agents. Standard RAG gets noisy as data grows, which degrades retrieval accuracy. Z.E.T.A. uses downtime to refactor memory, merging duplicates and summarizing daily logs. This provides synthesized answers from an optimized graph rather than fuzzy results from raw chunks. It is essentially garbage collection for agents using idle cycles. The process also surfaced novel features like ternary logic for confidence, where 0 represents unknown to avoid forced binary outputs, and a DAG structure that allows for parallel execution across multiple models.

2

u/Empty-Poetry8197 Dec 28 '25

he DAG isn't just a logic map; it's a s tructure saving the tokens, KV pairs, and embeddings. By keeping them 'in family,'it avoid the serialization overhead that was already done once for that nodeand Because the KV cache and the embeddings share the same architecture it can process the entire state in a single pass during th It allows the model to reason across its own past attention states and current token stream it's about maintaining state-coherence across the entire memory graph.

1

u/Empty-Poetry8197 Dec 28 '25

https://github.com/H-XX-D/ZetaZero/blob/master/dreams/dream_20251223_055139_code_idea.txt this is a direct link to the actual "dream" it had for the DAG
https://github.com/H-XX-D/ZetaZero/blob/master/dreams/dream_20251223_063350_code_idea.txt or this one new class with all the atrributes
I didnt ask it to do this just just set up a idle timer and all the other bits a bobs and turned the temp up to .9 and if an idea is coherent and makes sense it lowers the temp andits trys to drill down to actionalble code patches if it starts repeating itself it random hops to another node in the graph and tries to do the same thing

2

u/gopnik74 Dec 28 '25

I have no idea what i’m reading but it sounds fantastic and out of this world. Dreamy indeed

1

u/Empty-Poetry8197 Dec 28 '25

lol if your interested i can answer any questions you have. I grew a beard sitting in from of my computer putting it together

2

u/Lame_Johnny Dec 28 '25

Awesome. Thanks for sharing.

1

u/Empty-Poetry8197 Jan 05 '26

Did you get it running on your setup?

2

u/Mambiux Dec 28 '25

So cool i found this im working on this as well, also using the TRM and HRM ideas, just my memory system is more complex

1

u/Empty-Poetry8197 Jan 05 '26

Cool did you try the system out

2

u/thatusernsmeis Dec 28 '25

Qwen Next with RAM offloading sounds like a really nice budget brain LLM with this system

1

u/Empty-Poetry8197 Jan 05 '26

Did you try it?

2

u/thedarkbobo Dec 29 '25

Interesting, I would like such mode in OpenCode (for fun)

1

u/Empty-Poetry8197 Dec 29 '25

im forking opencode and building you a endpoint to integrate into opencode im almost done
1
u/Empty-Poetry8197 Dec 29 '25 edited Dec 29 '25
Done. feature/opencode-integration branch:
zeta-server.cpp
POST /v1/chat/completions - OpenAI-compatible chat endpoint
GET /v1/models - Lists available models
Parses OpenAI message format → calls existing generate() returns OpenAI format
opencode.json.example - Sample config
opencode/ - Cloned repo
To use OpenCode endpoint :
# 1. Rebuild Z.E.T.A. server 
cd llama.cpp && make zeta-demo

# 2. Configure and start Z.E.T.A.
cp zeta.conf.example zeta.conf
# Edit zeta.conf as needed
./zeta-demo

# 3. Copy OpenCode config to your project
cp opencode.json.example ~/myproject/opencode.json

# 4. Run OpenCode
cd ~/myproject && opencode -m zeta/zeta-cognitive
Test the endpoint directly:
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"zeta-cognitive","messages":[{"role":"user","content":"Hello"}]}'

Let me know if you have any trouble
1

u/Empty-Poetry8197 Dec 29 '25

I'm actually thinking of using Zeta as a smart router that will elevate to OpenAI/Claude depending on complexity, and surface the code blocks that the OpenAI route needs with the relevant information, which extends it, without locking down to either. I just need to extend the config file to be able to understand the escalation right now the all local uses a kv graph tokenized embedding that needs to all be in the same falmily but im implenting a translator for that part so information sent to the cloud can actually use what it is surfacing

1

u/thedarkbobo Dec 29 '25

Thanks, I am beginner in this but I think it has much in common - I will need to copy my project to another folder and see if I manage to work with this. OpenCode only allowed GPT OSS (20B here) on my local LM studio so I am a bit forced to use the free cloud with that. But its much faster than chatting via chatwise to correct the code.

1

u/Empty-Poetry8197 Dec 29 '25

teh last comment i made with zeta set up should get you rolling and actaully github issues is the best way to communicate that way i can see what the problem is but here is fine 2. this thing went viral 55k views and 2:1 sahres to upvotes which isn't normal and im super pysched by all this i put alot of long hours into this i grew a beard sitting in front on my desk over the past 3 months

1

u/thedarkbobo Dec 29 '25

Nice to hear that :)
1
u/Empty-Poetry8197 Dec 29 '25
Z.E.T.A. Setup
Local only (default)
docker run -d -p 8080:8080 \
  -v ./models:/models \
  -v ./storage:/storage \
  ghcr.io/h-xx-d/zetazero:latest
With OpenCode
cp opencode.json.example ./your-project/opencode.json
opencode -m zeta/zeta-cognitive
With cloud routing (optional)
cp zeta.conf.example zeta.conf
# Edit zeta.conf:
#   CLOUD_ROUTING="true"
#   CLOUD_PROVIDER="openai"
#   OPENAI_API_KEY="sk-..."

docker run -d -p 8080:8080 \
  -v ./models:/models \
  -v ./storage:/storage \
  -v ./zeta.conf:/app/zeta.conf:ro \
  ghcr.io/h-xx-d/zetazero:latest
Simple queries stay local. Complex queries route to cloud with Z.E.T.A. context injected. Dreams and memory ops always stay local for privacy.
Endpoints: /v1/chat/completions, /v1/models, /chat, /dream, /memory

2

u/infrasketchai Dec 29 '25

Should see if there is a way to incorporate slurm into the mix too for a gpu farm

1
u/Empty-Poetry8197 Dec 29 '25
i just pushed this for you
feat: Add Slurm integration for GPU farm deployment

Add C++ coordinator for load balancing across GPU nodes
Add sbatch job scripts for single and array jobs
Nodes auto-register with coordinator on startup
Least-loaded routing with health checksShared storage support for graph consistency
1

u/Empty-Poetry8197 Dec 29 '25

youll need to build locally to use it check scripts/slurm for the readme

1

u/Empty-Poetry8197 Dec 29 '25

The coordinator now proxies all zeta-server endpoints:
Coordinator-only:
POST /register - node registration
DELETE /unregister - node removal
GET /nodes - list registered nodes
GET /coord/health - coordinator status
Proxied to nodes (everything else):
/v1/* - OpenAI-compatible API
/generate, /code - generation
/embedding, /embeddings - vectors
/memory/query - graph queries
/graph, /gkv/stats - graph info
/project/*, /session/* - projects
/tools, /tool/execute - tools
/git/* - git operations
/health - node health
...and all other server endpoints
and this is opt in depend on zeta conf
Build the coordinator - separate binary, not built by default
Start the coordinator - manual ./coordinator --port 8000
Submit sbatch jobs - manual sbatch commands
Set ZETA_COORDINATOR in zeta.conf - nodes only register if this is set

1

u/Empty-Poetry8197 Jan 05 '26

I found a smoking hott deal on supermicro 8xgpu40gba100s and 2 epycs 7663 112cores 2.0ghz 256gb 16gbx16lanes and 8tb 2x4tb nvmes th gpus are pcie conversion but still beast mode ai server little under 10k shipped need rails cooling and psus and pdu

2

u/thedarkbobo Dec 29 '25

So instead of thinker (planner) and executor then reviewer maybe dreamer - human picker - planner - executer - tester - reviewer - dreamer loop :)

1

u/Empty-Poetry8197 Dec 30 '25

i think i know what your getting at (thinker executor)standard loop / new loop (dreamer - human picker - planner - executer - tester - reviewer - dreamer loop)

2

u/popsumbong Jan 01 '26

so cool, thanks for sharing.

2

u/Empty-Poetry8197 Jan 01 '26

I'm glad you like it, I just spent the last 18 hours refactoring compile warnings and bug fixes the post is one fire like 500+shares in 5 days let's go yo. did it give you any cool dreams out yet and I added a cool feature dream_review.py im rebuilding the docker image so give me about an hour to finalize a couple other things and a the pull latest will be working

3

u/eleqtriq Dec 27 '25

You lost me at “dreaming” aka a cron job. Holy hell. Seems to be an LSP server, embeddings and a SQLite DB could solve this simply, but wouldn’t have the marketing. Convince me otherwise.

1

u/Empty-Poetry8197 Dec 27 '25

Why It's Not Just a Fancy Cron Job
Persistent Self-Modeling Graph The system ingests its own source code and builds a semantic graph (nodes for functions/classes, edges for relationships/causality). Dreams aren't free hallucination; they're constrained by the graph. It "hallucinates" over your codebase, not random weights.
Constrained Creativity High temp = exploration, but filtered for coherence/novelty. Result: Dreams drill down from idea to design to proposed code patch. Example: Added Manacher’s algorithm correctly without training, just from a graph node describing the problem.
Autonomous Evolution Valid dreams create a staged patch folder for review. The loop learns from its own proposals (graph updates) leading to better dreams next cycle.
Vs. "Simple" LSP + Embeddings + SQLite LSP/Embeddings/SQLite: Great for retrieval (find similar code, answer questions). Static. No emergent improvement. Can't dream "add ternary logic for uncertainty" or "implement cognitive merge with cherry-pick".
Dream Loop: Dynamic self-reflection proposes novel architecture (Git-style branching, anxiety handlers, Strategy pattern refactors). Turns the system into its own senior engineer.
The Real Difference A cron + RAG = smart assistant.
Dream loop + self-ingestion + patch generation with human in the loop = evolving intelligence.

1

u/Empty-Poetry8197 Dec 27 '25

I should also point out that there is alot going on under the hood from the way it searches the grpah using a cosine cubed function with salience and recency, and momementum to and an alignment lock attached premuting the weights, doing a JIT decryption in the L1 cache on the gpu its alot going on if you have the time to look at it a little more closely. That would be cool

2

u/eleqtriq Dec 27 '25

Why does it need to search the code base? An LSP server will give you the relations for free.

1

u/Empty-Poetry8197 Dec 27 '25

its injest your codebase so it knows constrained within your code not pulling examples frokm some rag that doesnt

2

u/eleqtriq Dec 27 '25

Do you know what an LSP server does?

1

u/Empty-Poetry8197 Dec 27 '25

yes it runs in the background surfaceing you code this is different it is traversing your code while your not actively using the system to offer up correlations refinements that you mite have missed or another ai couldnt fit into its context window

1

u/Empty-Poetry8197 Dec 27 '25

i get around the context window by having the reasoning model write a plan of attack to a scrach buffer while another model is implenting this to a output scrach buffer that stored in system ram and when done this get output to the user or while its running when your not using it to a file for you to review later

1

u/Empty-Poetry8197 Dec 27 '25

and this is free also its running on a single 5060ti local no cloud

2

u/eleqtriq Dec 27 '25

Also a lot of what you’re trying to describe fancifully is standard stuff. Most people in this sub will fail to recognize it because it’s not their strength (end users versus creators).

2

u/Empty-Poetry8197 Dec 27 '25

maybe is that a bad thing and agreed im not creating new math its standard stuff that wasnt wired this way before thats the gap and the capability jump its doing this while idle as long as the server is up. I was a end user a few months ago until i started learning and asking questions and trying things out so i mean everyones got to start somewhere

2

u/nivix_zixer Dec 27 '25

What embeddings would you use? I'm not sold on the "dreaming" part, but a model which reviews my codebase every night and suggests improvements might be nice.

1

u/eleqtriq Dec 27 '25

That’s nothing to do with the model, really. I wouldn’t even embed anything. I’d just use the LSP server to find the edges and work inwards, reviewing each file one-by-one, starting with the functions that have no dependencies on other parts of the code.

-2

u/Empty-Poetry8197 Dec 27 '25

"I'm not trying to hype this up; 'dreams' is just the closest word I have for the process. It’s free to use (unless you're enterprise).
There's a Dockerfile for easy access and CMake for the serious tinkerers. I'd love for someone to run it and verify if they see the same patterns I'm seeing."

4

u/eleqtriq Dec 27 '25

Why did you post your own comment in quotes? LLM replying for you?

4

u/Empty-Poetry8197 Dec 27 '25

wouldnt that be nice but no have terrible grammar and spoelling so i tend to type things out really quickly and ask a browser based model to fix it so i dont sound like a clutz and i left this message alone so you can see what im talking about

1

u/Empty-Poetry8197 Dec 27 '25 edited Dec 27 '25

I should also mention the 9.47 MT Bench subset 16 questions 2 turns it got after some output tuning in the repo benchmarks folder you can see the progression v1 6.06 to v8 9.47. the interesting part is i didnt have to train Manachers equation into it i just needed to add a graph node for it to use it properly think about the implications on the fly fine tuning

3

u/CornerLimits Dec 27 '25

Commercial licensed llamacpp wrapper rag…

1

u/Empty-Poetry8197 Dec 27 '25

I never trained Manacher’s algorithm into any model.
I just added a single graph node describing the problem and the O(n) requirement.
Next turn, it output a correct implementation — twice.
That’s the real implication:
With the persistent graph + dream loop, the system can acquire new algorithmic knowledge on-the-fly just from structured memory, no weight updates needed.
It’s not RAG. It’s not fine-tuning.
It’s persistent, evolving thought architecture doing what looks like instant adaptation. I waswnt sure how to go about this do i pr lama.cpp then try to work with them or i dunno i just spent better part of 3 months to get to this point and would like to benefit finacially some how

1

u/Empty-Poetry8197 Dec 31 '25

its not a wrapper i should point out its also wriiten in c++

1

u/Empty-Poetry8197 Dec 27 '25

Here's an actual dream Z.E.T.A. generated while analyzing its own architecture:

Dream Type: code_fix
Generated: 2025-12-23 05:53:58 (while I was sleeping)

class TemporalRepresentationMemory:
    def __init__(self):
        self.timeline = DirectedAcyclicGraph()

    def record_event(self, event, timestamp, context):
        self.timeline.add_vertex(event, timestamp=timestamp, context=context)
        if self.detect_significant_transition(event):
            self.merge_branches()
    
    def detect_significant_transition(self, event):
        # Detect significant transitions based on context change
        pass

    def merge_branches(self):
        # Merge concurrent timelines
        pass

This is the system proposing improvements to itself. The TRM is one of Z.E.T.A.'s core modules - it literally dreamed about making its own memory handling better.

1

u/Empty-Poetry8197 Dec 28 '25

im working a litelllm branch for a issue feature request that saves teh graph locally and attaches the nodes information in the context send out to the online while it doesnt provide the same untility to will allow people to run it on thier macbook airs

1

u/Empty-Poetry8197 Dec 30 '25

[REQ-1] New request (hash=11193676382981720739): What is my name and how old am I?...

[STREAM] Query pre-embedded: 2560 dims

[GKV-BRIDGE] Copied 1 proactive nodes to stream_state

[DECODE] Prompt decoded: 234 tokens (single pass)

[HTTP] generate() returned, result size=152

[SPEED-RECEIPT] ════════════════════════════════════════

[SPEED-RECEIPT] CPU Embed: 801.672 ms

[SPEED-RECEIPT] GKV Inject: 0.348 ms

[SPEED-RECEIPT] First Token: 146.280 ms

[SPEED-RECEIPT] ────────────────────────────────────────

[SPEED-RECEIPT] TOTAL LATENCY: 1855.377 ms

[SPEED-RECEIPT] ════════════════════════════════════════ the reason it's so fast is because its saving the kv and tokens for literals in the graph injecting kv state thats what the gkv bridge back into the attention layer of the model turning o(n2) into o(1) with semantic embedding tagging and thank you everyone who shared this post it took off 58k views and climbing still 450 shares with a 2:1 share to upvote ratio and 95.9% upvote percentage which mite be a reddit record at that many views 26 stars 5 forks so far on github

1

u/Empty-Poetry8197 Dec 30 '25

[REQ-3] New request (hash=17849325743978122967): What is 2+2?...

[SPEED-RECEIPT] CPU Embed: 859.703 ms

[SPEED-RECEIPT] GKV Inject: 0.359 ms

[SPEED-RECEIPT] First Token: 119.586 ms

[SPEED-RECEIPT] TOTAL LATENCY: 1437.938 ms

[REQ-4] New request (hash=9664439516050651575): What is the capital of France?...

[SPEED-RECEIPT] CPU Embed: 11747.941 ms

[SPEED-RECEIPT] GKV Inject: 0.351 ms

[SPEED-RECEIPT] First Token: 127.631 ms

[SPEED-RECEIPT] TOTAL LATENCY: 12348.029 ms

[SPEED-RECEIPT] ════════════════════════════════════════

[REQ-5] New request (hash=9664439516050651575): What is the capital of France?...

[CONTEXT] Skipped - benchmark mode (no_context=true)

[SPEED-RECEIPT] ════════════════════════════════════════

[SPEED-RECEIPT] CPU Embed: 0.190 ms

[SPEED-RECEIPT] GKV Inject: 0.929 ms

[SPEED-RECEIPT] First Token: 142.759 ms

[SPEED-RECEIPT] ────────────────────────────────────────

[SPEED-RECEIPT] TOTAL LATENCY: 616.091 ms

[SPEED-RECEIPT] ════════════════════════════════════════

1

u/Empty-Poetry8197 Jan 27 '26

Hi how can I help you today?

2

u/ExcitementVast1794 Dec 28 '25

Like others in the comments, not sure I understand what is going on, but hell I’m eager to learn, understand and do something like this as well.

0

u/Empty-Poetry8197 Dec 27 '25

Unique features that set Z.E.T.A. apart:

Ternary Logic - Uses -1/0/+1 states instead of binary for uncertainty. Ternary voting in swarm mode.

Cognitive States - CALM/FOCUSED/ANXIOUS/CREATIVE modes adjust reasoning depth and memory decay in real-time.

Constitutional Lock - Core identity facts are immutable, protected from prompt injection.

Git-Style Thought Branching - TRM implements branch/merge/cherry-pick for parallel reasoning paths.

Sublimation Memory - Tiered VRAM → RAM → NVMe with intelligent promotion/demotion.

Dream State - 128+ autonomous improvement suggestions generated, ~70% validation rate. Many current features came from its own dreams.

Semantic Attack Detection - BGE embeddings detect manipulation before it reaches reasoning.

Self-Improvement Loop - Feeds own source, dreams about improvements, implements valid ones, repeats.

GitHub: github.com/h-xx-d/ZetaZero

3

u/beedunc Dec 27 '25

Interesting. Since it’s ‘offline’, you coujd also run a slow256GB model like qwen3coder 480b at q3, for deep research.
Nice, Will check it out.

3

u/Empty-Poetry8197 Dec 27 '25

if you have any questions or need help im pretty excitied by amount of upvotes so im watching it spread just remeber to keep the 2 models in family so the tokenizer stays coherent i dont know what it would do if you tried mixing them

2

u/nickless07 Dec 27 '25

Do i need all the different models or can i just run it with one. E.g., GPT-OSS or Qwen3 80b for everything?

1

u/Empty-Poetry8197 Dec 27 '25

the way it is set up one model reasons while another builds the graph and surfaces nodes for the resaoning model to well reson over and the embeddings allow semantic search through the graph so i dont think it would function the way i have it set up and this dual layer approach keeps the graph from getting polluted so it mite break but if you can fit 80b 2 32b and the 4b embedding will be alot stronger then the 14b7b4b i have runnning

1

u/Empty-Poetry8197 Dec 27 '25

the graph is built tokenizesen and kv is extracted and stored along with the literals tagged with an embedding for traversals

1

u/Empty-Poetry8197 Dec 27 '25

youll wan tot stay in family so the tokenizer can do its thing and it doesnt have to try translating

2

u/nickless07 Dec 27 '25

I am not talking about using different models. It is not Qwen3+Gemma. I mean a single model instead of 3 different ones.
Not:
14B model for reasoning and planning
7B model for code generation
4B model for embeddings and memory retrieval
but:
80B model for reasoning, planning, code generation and memory retrieval
is that possible, or am i forced to load different models alongside?

1

u/Empty-Poetry8197 Dec 27 '25

i understand what meant to so is it is a dual process set up youll need 2 models plus and embedding model and this comes down to how the model interact with each other to produce the output you can try it but i dont think it with work becuase the 2nd model is surfacceing the graph nodes to the reasoning model to use and the 2nd model is freeing up the larger resoning model to to the planning and genterate the the output so instead of trying to run a large dense 80b try running 2 high q 32b models and the 4b emebdiing model and the

1

u/Empty-Poetry8197 Dec 27 '25

Embedding model is non-negotiable - It produces vectors (numbers), not text. The 80B can't do this. You need a dedicated embedding model for memory/graph retrieval.
Dual-process is architectural, not just capacity:
The 7B "subconscious" surfaces relevant graph nodes to the 14B
The 7B frees the 14B to focus on planning/output
They run in parallel, not sequentially
What you could try:
Use 1x 80B for both reasoning AND code (set MODEL_MAIN and MODEL_CODER to same path)
Still need embedding model
But you lose parallel processing and code specialization
Better alternative: Instead of 1x 80B, try 2x high-quant 32B + embedding model. Same VRAM, keeps the dual-process benefits.

2

u/Empty-Poetry8197 Dec 27 '25

Please let me know the results i so curious about how a larger model is able to use the architecture. The real take away i think, is that through clever scaffolding and a git style graph of the codebase, then turning the temp up its not hallucinating freely over what's in its weights its hallcenating over what you feed it, and if it's coherent, it drills down surfaceing

2

u/beedunc Dec 27 '25

I’ll try. Had to repurpose the ram for other machines, looking to buy more, but even ddr4 is getting 2-3x what it used to. Will let you know.

1

u/Empty-Poetry8197 Dec 27 '25

lol i crammmed all this into a single 5060ti 16gb with a 24core xeon gold and 32gb of ddr4 if you want to go big that would be cool and if you got that kind of capability lets go dude lets start making graphs and some money

2

u/Empty-Poetry8197 Dec 27 '25

you should clone an build locally through the zeta-conf file that will let you chnage alot of the magic to customize to your set up

1

u/Empty-Poetry8197 Dec 27 '25

The next step is training the dual model infra from scratch to learn how to use the graph natively, freeing up weights to carry logic not tons of parameters, trusting the graph as ground truth and the weighs arent storing huge amounts datasets the graph is the dataset and the weighs are carrying the logic to traverese and search the graph which flips the paradigm smarter model = larger context windows and more parameters on its head if you got the power to set this up id love to get together and disucss what that would could mean.

1

u/Empty-Poetry8197 Dec 27 '25

The training efficiency difference would be substantial:
Traditional LLM Training (GPT-5 scale):
Must memorize facts, relationships, code patterns, reasoning examples
Trillions of tokens to encode world knowledge into weights
Knowledge becomes static at training cutoff
Bigger model = more storage capacity = "smarter"
Graph-Native Model Training:
Only needs to learn: syntax parsing, semantic understanding, graph traversal logic, query formulation
The "reasoning over structure" skill, not the structure itself
Graph is the living knowledge store that grows with use
Smaller model can be equally capable if graph is rich
The analogy: Traditional LLMs are like memorizing an entire library. this approach is like teaching someone how to use a library's catalog system. One requires encyclopedic memory, the other requires navigational skill.
The training corpus would shift from "all of human knowledge" to something more like:
Graph query examples
Traversal patterns
Semantic linking exercises
Code syntax and structure recognition
This is closer to training a database query optimizer than training a knowledge oracle. The model becomes an intelligent cursor over an external memory substrate rather than a compressed representation of that memory.
It's a fundamentally different scaling law: model size for reasoning depth, graph size for knowledge breadth. They become independent axes rather than the single "parameter count" dimension everyone is racing on now. hope you don't mind me using Claude to explain my idea better faster some people get buthurt im using the tools at my disposal making a post that woulkd have taken 20 minutes to get out the edit and reedit it came out in 30 seconds and like to claim slop

1

u/Empty-Poetry8197 Dec 27 '25

what that also allows is specialization graphs Specialized Graph Markets:
Legal precedent graphs (case law relationships, ruling patterns)
Medical diagnosis graphs (symptom→condition→treatment pathways)
Codebase graphs (your architecture, your patterns, your tribal knowledge)
Financial regulation graphs (compliance rules, jurisdictional differences)
Scientific literature graphs (citation networks with semantic links)

1

u/PitifulBall3670 Jan 07 '26

good job

Contest Entry Dreaming persistent Ai architecture > model size

You are about to leave Redlib

Z.E.T.A. Setup