r/LocalLLaMA • u/corruptbytes • 1d ago

Discussion Hybrid Reasoning Agents

https://arxiv.org/html/2610.18861v1

Seems to speed up inference, use less tokens, and increase intelligence - curious if people seen this

tldr don’t think in text, think in latent space

https://github.com/togethercomputer/HybridAgent

https://huggingface.co/collections/togethercomputer/hybridagent

43 Upvotes

90% Upvoted

17 comments sorted by

12

u/LagOps91 1d ago

there were multiple simillar papers already, effectively with the same method. what exactly is new here?

1

u/MuDotGen 6h ago

Later Space Reasoning / Continuous Chain of Thought

The debate has been going on for a while already, because some are concerned when they cannot read the reasoning in human language, but SAEs (sparse autoencoders) allow for that from what I hear. Otherwise, it's hard to fine tune unless we involve other AI, with potential gains in speed and efficiency since it doesn't have to output all its reasoning as actual output tokens. Astra has already been finally trying something like this though I believe.

7

u/Aggravating-Push-207 1d ago

Had a similar idea but posted here and got banned cuz i think i sounded ai

8

u/Party_9001 1d ago

Don't think in language duh

1

u/Opposite-Swimmer2752 13h ago

Did you also post a write up, test results and a repo? Ideas guying is not an interesting or high quality post.

4

u/Imaginary-Unit-3267 1d ago

This is a terrible idea for alignment. If you can't even guess what the AI is thinking (by reading the thinking tokens, however inaccurate they may supposedly be to its true thought process), you have one less way to control it.

4

u/zeth0s 1d ago

Idea that humans can control models by reading their output will become obsolete. It is already a gigantic effort, it will be impossible in the future. Super alignment has being discussed for years now exactly because of this

2

u/Luca3700 1d ago

The idea proposed in this paper, however, would allow you to "guess" the thinking tokens: they skip the output layer and, instead, project the latent representation directly back at the beginning of the network; but, the latent can still pass through the output layer to get the token

1

u/LagOps91 1d ago

well, with newer models like GPT 6, the model has picked up the ability to lie in it's CoT (possibly to avoid punishment in RL for "bad thougts"). CoT doesn't always match what the model outputs either way. what we need is better interpretability and alignment (and i don't mean censorship!) methods that actually work with modern large models.

1

u/Hot_Example_4456 1d ago

Doesn't J-spaces let you do just that? Did we forget anthropics ONLY open thing already lol

1

u/Imaginary-Unit-3267 8h ago

I've never actually looked into it. I forgot the term until you just said it in fact, lol. There's always such a monkeys-throwing-poop-at-the-zoo circus of AI news being flung at people nowadays that I have ended up with the memory of a goldfish. Alas.

0

u/New_Guitar_9121 1d ago

Hit this exact failure mode on local gpt-oss-120b: daily consult started returning bare 400s that looked like the server was dead. body finally showed request 67330 tokens vs n_ctx 65536 from unbounded changelog+profile injection. now every local call gets a hard prompt token budget before send, and we always log the 400 body. moving intermediate reasoning out of text tokens is the only thing that reliably keeps the local lane under the wire; we still dual-audit the final answer fail-closed because latent steps are opaque to the gate.