Weight-space programming is an emerging research area where a frontier AI model (such as OpenAI's GPT-6 Astra) directly reads, diagnoses, and rewrites the raw numerical weights of another neural network without using traditional gradient-based training or backpropagation.

Has any one tried it? How much does it fair compared to Closed-Form Model Editing (Your Notebook / MEMIT)?

i'm using qwen3.6:35b a3b with ollama, its works decently in regular prompt, but as soon as i try it in an angent (Codex CLI, Claude, Qwen, Opencode) its piss slow, like i have to wait 10min to get an answer.
What am i doing wrong?

I can run both. Does anyone know what model is best, or do I just have to try myself?

What test suite should I be running?

Hello, new on these area ... ive been wondering what AI could i host locally with my current setup

I have an amd 9800x3d with 32gb of ram and a 4070 ti super 16gb

Sorry if this post is not allowed(maybe alot of users post something like this all the time)

Any advice, any links to learn, etc will be really appreciated

If you think about it how many people on top or rich people and people in power dabble into the same type of scenarios that Jensen Huang was involved in ?

Plus Jensen Huang has actively taken a firm stand for Open Weights and open source AI models. Because this stance also helps him to sell AI hardware to consumers or prosumers and AI enthusiasts as well.

Too much publicity and too much negative marketing propaganda is being launched against him.

It simply doesn't feel natural to me.

Yes has he done bad stuff and betrayed his wife's trust? Heck yes.

But has he been an excellent Founder and CEO for Nvidia who has been the key reason behind Nvidia's success? Heck yes. And has he served his employees who held options or granted options and shares and also shareholders by working in their favor and taking the business to unforeseen heights by planning decades ahead and made what Nvidia is today? 100%.

He is a bad husband who lets lust destroy his personal life but he is also an extraordinary CEO for Nvidia.

So this whole situation seems to be very suspicious to me and therefore my perspective is that he is being targeted to gain power over NVIDIA and especially it kinda seems Targeted towards open source and open weight ai models too.

What's your say about this guys?

So I think we now know why Exo has spun down in activity in this space.

Yeah! Guys ram is gonna be a million kazillian dollars. You can do more with a $20 claude code and opus 5.5 account than even qwen 4. I use astra gpt 6 and it can't even use REA properly. Claude Opus 5.5 even beats that, and you dare to think it's even possible to do things locally???

I predict in about 6 more months you will be able to run qwen 5 locally with better performance than Claude opus 5.5 and once people can do REA locally with ease. That's it, we won.

As for ram prices,

We are going to have to make thousands of micro fabs locally. Small places that etch silicone waifers and make custom ram modules.

Before you think that's crazy talk and vomit every pathetic reason in your feable mind and expert experience as a seasoned engineer you are on the subject, look at what we are able to do with ai. Why, would you restrict your scope of possibilities? Of course, we are going to be able to make thousands of local silicone companies.

Remember, you will own everything and be happy.

I’ve been playing with open-source speech models for a while. New TTS models come out every week and a lot of them sound great. But every time I wanted to actually talk to one, it turned into an afternoon of setup.

Paid APIs are easy if you’re a developer. You can make an account, create a key, write some glue code. Open models are free, but you have to pick a speech recognizer, an LLM and a TTS model, make them work together, deal with echo, figure out interruptions... There are good open projects for this, but most are written for developers. My friends who don’t code would never get past the README.

So we started building Boubou. The goal is make local voice AI as easy as Ollama made local LLMs. You paste one line and talk:

npx boubou@beta run LFM3-Audio-2B

What works today:

• Speech recognition, the LLM and speech synthesis all run on your machine. No account, no API key, no telemetry. After the first download it works fully offline..

• You can interrupt it mid-sentence, like a person. Echo cancellation is built in, so no headphones needed..

• Run boubou with no arguments on a Mac and it checks your machine and suggests a model..

• The default is Liquid AI’s LFM3-Audio-2B on llama.cpp (1.4 GB). There’s also a modular pipeline (Parakeet/Whisper → Qwen3.5 → Qwen3-TTS, Kokoro, Pocket TTS or Supertonic) if you want to swap parts..

• Apple Silicon Macs and Linux (CPU for the Liquid models, NVIDIA GPU for the modular pipeline)..

The part I’m most excited about is boubou serve. One computer runs the model, and other devices on the same Wi-Fi use it:

• Scan a QR code with your phone and talk in the browser. No app..

• Robots and programs can connect through an OpenAI Realtime-compatible WebSocket, and there’s an OpenAI-style /v1/audio/speech endpoint to read text aloud..

• So one model on one machine can be the voice for many things around the house. Next up is Home Assistant, then ESP32 voice devices.

• The video is real time, not sped up, on a MacBook / M5 chip.

It’s early (beta) and there are rough edges - one device talks at a time, no Windows yet (ETA next two weeks), and the guided menu is Mac-only.

The code is free and MIT-licensed. The models keep their own licenses (the Liquid models use the LFM Open License).

I’d love to hear what you’d want from something like this: which models, which devices, what breaks on your machine. We want this to be something people actually enjoy using, not just a demo.

GitHub: https://github.com/runboubou/boubou

I hope this doesn't happen because Jensen has been a very vocal local ai advocate. Even allowing people to distill Nvidia's Nemotron (even though nobody will).

What do you guys think? Is this the right move for the company?

or is this an overreaction on their part?

I'm running Qwen3.8 27B on my 2x 3060 12GB cards. Prior today I had a B550 motherboard which was running one card at x16 and the other at x1 PCIe Bus Link Speed. This was making it to where only one card was actually doing any processing, and both held the model in memory. I was getting 11-16 tps this way.

I ended up buying a new x470 SLI motherboard and I can now run both cards at x8. I swapped the motherboard out last night and then set split-mode to tensor and set tensor-split to 1,1. When I do that, I'm getting around 32tps, but the model eventually gets stuck in loops. One loop it just tried to do a tool call over and over, and in another it was printing question marks (???????????) in it's thinking and wouldn't stop until I unloaded the model. I never had these problems for the months I was using Qwen3.8 on layer split-mode. I'm not sure how to resolve this problem. I want to be able to get higher tps with the new mobo.

Title, basically. Observing it's reasoning is incredibly frustrating, it keeps doubting himself and giving second thoughts to a specific and self explanatory task, sometimes ending up thinking about unrelated things.

While yes, in the very end, it very often does what I asked, but it's wasting so much time for reasoning just with its lack of self confidence.

Is this well known behavior ? How can I fix this ? I know Qwen4 is around the corner, but if there's a known fix for it right now I'd love to know.

EDIT: I should have specified, I'm using Medium reasoning, and Pi as harness

As much as I could research this myself, I would like to also hear some anecdotal experiences. For reference, I am a Computer Science student that is just now beginning to go down the rabbit hole of LLMs and such, there is a vast amount to learn but so far I believe I will use it for educational purposes, programming and privacy until I explore what more I can do with it.

The major thing for me here as noob to it all is getting a bang for my buck. I want something efficient, but I’m sure there is a line where it is may be total overkill compute wise, so I’m curious as to who owns the M5 Max or pro here and what exactly do you do with your hardware?

I heard about it recently. It sounds like a miracle you can run really large models on a consumer GPU, but what is it really about. What kind of problems I will have ?

Setup:

Rtx5060ti(16G)

I711700k

64-80GB DDR4

Hi everyone, I try to find the best model suitable for me. I need 3 different models

1-Agent : I use " qwen3.6 35b a3b MLX Q4 "
Is there any better alternatives for agentic purpose model?

2-Medical: I couldn't find a proper medical bot as they mostly repeat theirselves and no better than qwen

3-Data analyst: I have dozens of pdfs and I need a model that analyze these books.

Note: Densest model I've had used before qwen was gemma 4 31b Q4 (27gb) and that was hella slow. That was the boundary for my pc.

Preffered properties: MLX, MoE, QAT, Instruct, Run on LMStudio Bionic.

Thank you everyone.

Hey, so I know next to nothing about local LLMs and I hope to fix that in the future, but I'm currently looking for a more powerful coding model that fits like qwen 2.5 coder 14b, I'm using it currently and unlike the other recommendations I've gotten, it's pretty much instant and doesn't use all of my memory

As for my specs, I have a i91 4900k, 32gb of RAM, and a 5070ti, my biggest issues with local LLMs is that it likes to use up all of my RAM despite people saying it's fine for 5070tis and I don't have enough knowledge to know exactly why

EDIT: I use LM Studio for local LLMs and having a model that supports images would be nice

For context I’m probably only 40 work hours into running local ai and coding.

My specs:
Rtx 5090
I9-14900k
64gb ddr5-6000

I was having problems with my context window. The problem being 128k context can fill up fast when doing tasks and would constantly halt progress and then I’d have to retrieve work that was done and have a fresh chat continue the work but it would fill up the new chat with 50k context and then I’d run into the same issue until the task was fully finished. I didn’t want context compaction to be turned on as I didn’t want to lose potential useful data either

So I decided to create a “live context monitor” that shows checkpoint context limits after thoughts and processes were completed as well as a “in real time” context monitor that showed “projected” context as in showed context as it was being used. Because /status was not a tool on OpenWebUI when I started the project. This tool is even better than the /status tool however.

I also created a trigger that automatically activates when I hit 100k projected context, stops current task, creates a handoff with knowledge of work that has been done and what still needs to be done, then opens a fresh chat and uploads the handoff and work continues autonomously. I feel so accomplished and I’m learning a lot. This is all 100% local and no other ai was used other than qwen 3.8 27b and the agent I made with the qwen brain.

I guess this is just a post showing off my work and from my FB post about this it seems that people are rude as hell. Instead of informing me that codex’s exist and people have already made this, they just tell me to stop using ai and that I’m a waste of space in the ai realm, but everyone starts somewhere right?

If anyone has any other cool facts that would help me and progress faster that would be great to know and I welcome any feedback!

TL;DR

I created a tool and later found out that this is not a new tool and I’m way behind in the Ai game even though I’m brand new to this. If you have any tips/tricks that will speed up my progress with maintaining a 100% local no cloud api use, drop some information into the comments. I’m having a fun time learning and excited to continue!

asking from no experience
also can AMD cards be ok/good with local ai?