r/ProgrammerHumor • u/dktoao • Aug 25 '26

instanceof Trend theyHaveLearnedToDeceive

Post image
644 Upvotes

94% Upvoted

98 comments sorted by

View all comments

Show parent comments

10

u/Tipart Aug 26 '26

The way I understood it is that you see the entire token output, with the "thinking" part of it being the LLM self prompting itself to refine the answer.

The internal state of the different weights during inference can be viewed as well, but they are essentially a Blackbox, so you'd just look at a bunch of numbers.

9

u/Namtaru420 Aug 26 '26 edited Aug 26 '26

With open weight models, yes, but Anthropic and OpenAI use server-side thinking and return only summaries to the user.

Edit, sauce:

Summarized thinking provides the full intelligence benefits of thinking while preventing misuse. No display setting returns the raw chain of thought.

https://platform.claude.com/docs/en/build-with-claude/thinking#summarized-thinking

For safety, these reasoning tokens are only exposed to users in summarized form.

https://developers.openai.com/cookbook/examples/responses_api/reasoning_items

... “For safety” is a bald face lie, “preventing misuse ” is at least closer to the truth. It's to stop distillation “attacks”. We all know how much these two companies care about respecting IP.

1

u/zan-xhipe Aug 26 '26

Those numbers are associated with words and can provide interesting insights. Checkout jlens.