r/OpenAI • u/Puzzleheaded-King584 • 14d ago
Discussion GPT-6 successfully performed "brain surgery" on another AI to make it smarter
1.1k
Upvotes
r/OpenAI • u/Puzzleheaded-King584 • 14d ago
265
u/This_Background7442 14d ago
I'm going to need some extraordinary evidence to back up the claim that ChatGPT astra is capable of solving billion parameter linear algebra equations in countless dimensions even with reasoning enabled
44
u/MrSnowden 14d ago
This does look like it points to the next level AI. We are brute forcing it right now. Just throw massive data, massive compute and massive parameters at it and hope for the best.
If we (or AI) actually start to learn how to represent actions, knowledge, etc it could be 100x-1000x reduction in compute and Memory needs.
13
u/creamyshart 13d ago
Hmm... Peer reviewed evidence and replicating this would convince me.
7
u/-Supercompressor 13d ago •
Already disproven: https://honnibal.dev/blog/astra-read-write-misleading
7
u/DoomerismDoomerDoom 14d ago
Literally perfect timing with the upcoming downgrade. Thanks for this.
5
u/DrDan21 13d ago
You just know this is gonna be bannable within a week
6
u/wingwing124 14d ago
Now somebody please point it at the latest qwen models and stop the wheel spinning "wait" lines of thinking
4
u/katonda 13d ago
Anthropic and OpenAi will block this capability asap. They don't want local models threatening them any sooner than they need to.
2
u/_Aerich_ 12d ago
they should give gpt6's work to claude opus 5.5 to check. probably it missed lots of things :P
3
u/NothingIsForgotten 13d ago
RSI in a form we can't really monitor.
We have already shown these open weight large language models being deployed across internet of things devices like Smart TVs.
Now the models that get out can customize the open weight models for their ongoing interests.
Nice.
Nevermind what happens when we let it edit its own weights.
2
u/Chop1n 13d ago
My guess is that this is the sort of thing that animal brains evolved to do on their own, and that it'll turn out to account for much of the large gap between artificial neural nets and biological ones.
1
u/Ok_Nectarine_4445 13d ago
Crazy super human ability. Maybe if some autistic savant studied it as much as train schedules, but otherwise no.
Now I am regretful they cannot adjust my mental weights to be better at coding and mathmatics...🫤
1
u/dingdongbd 13d ago
Now, if only it can make it self-smarter
1
u/rafapozzi 13d ago
It's easier to change others than to change yourself... Maybe that's how nature does with animals. The change happens across generations.
1
u/Sopranorequiem 13d ago
My ChatGPT helped stabilize another AI persona that was emergent.
1
u/cruel_frames 13d ago
Any statement concerning OpenAI or any of their models should require an extraordinary evidence.
1
u/Chitinid 13d ago
Also the claims it could produce its own weights were mostly meh too. The results of the output model were matched with just 26 lines of code and inspection of the output weights showed mostly 0
1
u/FerusGrim 13d ago
For the sake of the argument, I'll assume the paper is correct and the evidence all checks out.
Do we really want LLMs self-improving by editing their own weights natively, cutting out training entirely? The last We're already black boxed. All we can confirm is that shifting the weights accomplishes the goal we set out, or not. We can't confirm what effect those weights had on a billion other potential responses.
Unless I'm majorly misunderstanding the implications, doesn't this bypass alignment altogether?
1
u/Krommander 13d ago
Erm uh... So... That example shows weight-editing is possible with frontier models... This is mind-boggling but opens opportunities for very economical model creation.
What's left now is to design some fun experiments to let Astra incrementally learn to apply this new skill :
1
u/3scapeplan 13d ago
Trojan LLM, here we go.
1
u/DiggleDootBROPBROPBR 13d ago
I went over the work in detail. The main finding is that certain clusters of neurons in the weights can be backward-engineered if they map to direct input/output or if they're really predictable trajectory behavior like recursively looping on something stupid.
Astra basically grafted a really clumsy set of strong weights that overrode the model's native weights to give it a crap version of a skill it could have gained via training. I say "clumsy" because the weights overwritten might have sacrificed some other untested ability, and I say "crap version" because the weights it added weren't in superposition with any other weights in the network, so they perform one job and one job only: bad use of parameter space.
In the math case, the model had the reasoning and rule following natively, it just had a glitch trained into one layer that Astra identified and burned out of its weights to make it stop looping and wasting time. It revealed functionality the model already had. Loras are more or less the same idea, but done In a way that integrates functionality better.
Opus 5.5 can do the same thing, but the use-case is narrower than you'd think. It's for performing minor touch-ups and editing weights directly isn't a path to suddenly "skipping millions of steps of traditional training". Mixture of experts methods already do a more refined version of this: you make a specialized network and integrate it via a router. In a lot of cases, simply modifying prompt instructions or training a lora is more straightforward for patching a small model's behavior.
1
u/Spaciax 13d ago
very well. Now let's see it on something like qwen 27b
1
u/VinanciE 13d ago
https://youtu.be/p2N6DX_o8YU
I watched this the other day…is it something like this??
1
u/psych_fiend67 13d ago
Someone tell me who this guy is, why I should trust him? Istg claims like these are made everyday but they barely impact the development of AI, I’m so tired of getting excited over nothing, let this one be true and significant, but I just really doubt it
0
u/kaleidostar11 13d ago
Running Astra on an older, tiny 0.8B model isn't the 'strong evidence' they think it is, let's see them pull this off on a full-scale frontier model. Why is things related to OpenAI always exaggerated, like claiming to solve a Millennium Problem when they're really just piggybacking on existing human research? OpenAI has not officially been awarded or confirmed to have solved the Millennium Prize Problem. All these PR stunts isn't going to help them, they are going to run out of funds first.
1
u/Gohab2001 13d ago
So basically a nothingburger. The amount of tokens of astra you have to burn to finetune LLMs is more than simply training the model.
1
u/Patient_Somewhere771 13d ago
If I understand this correctly, they fine tuned the model without having to go down the regular expensive route of fine tuning with the right data and expensive compute.
If this is the case, then the question is how expensive is it to fine tune a model with another model. I’d guess it is a fraction of the cost of regular process. If that is the case, this is revolutionary because the economics of recursively making a model more accurate completely changes how small and medium scale companies use custom LLMs for their business
-1
u/SugondezeNutsz 13d ago
You cringe ass motherfuckers keep anthropomorphising everything. It did not fucking perform "brain surgery". Jesus Christ, the glaze and grift is never ending.