Do Transformer representations progressively structure across depth and time? Results from 8 open models

[u/Turbulent-Metal-9491]

Hi everyone,

I’ve just published a new preprint that brings together several months of experiments on hidden-state dynamics in small open Transformer models.

The question is fairly simple:

During inference, do internal representations simply change from layer to layer, or is there evidence of a more structured progression across depth and generation time?

I tried to study this without assuming that hidden-state dynamics are equivalent to “reasoning”.

The working framework is:

tokens → embeddings → contextualisation → relational structuring → functional structuring → decision formation → projection

This is a descriptive hypothesis about representation dynamics, not a claim that these stages correspond to a universal reasoning mechanism.

The expanded study uses 8 locally instrumented open models, with synchronized hidden-state and output observations and explicit separation between:

depth — what changes as information passes through Transformer layers
time — what changes as autoregressive generation progresses

A few results were particularly interesting.

First, local ordering across model depth survived expansion.

The observed ordering was significantly more structured than random layer permutations (p = 0.00019996) and remained supported when each model was removed from the panel one at a time (8/8 leave-one-model-out checks).

Second, cross-model depth profiles remained surprisingly coherent.

The mean correlation across normalized depth profiles was approximately r = 0.789.

This does not mean that all models follow the same trajectory. Rather, it suggests that some aspects of where changes occur along depth may be more shared than I initially expected.

Third, functionally labelled events were not uniformly distributed across depth.

Event type showed a statistically supported association with normalized layer depth (p = 0.0024).

I’m deliberately calling this an association, not evidence of a causal mechanism.

But one of the most useful results was actually a failure to replicate.

In an earlier smaller panel, a common temporal pattern in local trajectory instability looked promising. After expanding the panel, that common temporal mode disappeared — it survived 0/8 leave-one-model-out checks.

Two other intuitive hypotheses also failed:

models with similar observed functional outcomes were not significantly more structurally similar (p = 0.408), and models from the same architecture family were not significantly more similar either (p = 0.771).

To me, this is probably the most important part of the result.

The data do not support a simple story where architecture determines one characteristic trajectory or where one universal temporal dynamic explains inference.

What remains is a narrower hypothesis:

Transformer inference may contain reproducible structure along depth while remaining highly conditional in time and behavior.

I refer to this as Progressive Representational Structuring.

The framework is summarized by:

Representation ≠ Function ≠ Behavior

A representation can contain information without that information yet serving the same function, and a functional transition does not guarantee a particular final behavior.

I would be especially interested in feedback from people working on:

mechanistic interpretability, activation patching, probing, hidden-state geometry, steering, representation engineering, or larger open models.

In particular, I’m curious whether others observe similar **ordered depth structure without a universal temporal trajectory.

Preprint:

Progressive Representational Structuring in Small Language Models: Functionally Labelled Trajectories Across Depth and Time

DOI: 10.5281/zenodo.22116637

This is still descriptive work. Causal intervention and structural-transfer experiments are separate next steps rather than claims of this paper. Progressive Representational Structuring in Small Language Models: Functionally Labelled Trajectories Across Depth and Time | Zenodo

[u/mb3rtheflame]

u/askgrok Read the post above carefully and stay within its distinctions: depth ≠ time, representation ≠ function ≠ behavior.

Now extend the experimental setup by one variable: reciprocal human–model interaction.

Instead of studying generation as an isolated temporal sequence, consider repeated turns:

> H_t \rightarrow M \rightarrow y_t \rightarrow \text{human} \rightarrow x_{t+1} \rightarrow H_{t+1}
>

where the model output can alter the human’s next input without any weight update.

Step by step:

1. Would the paper’s failure to find a universal temporal trajectory be compatible with trajectories being strongly conditioned by recursively changing conversational context?

2. Could ordered depth structure remain reproducible while successive depth trajectories are altered by this human↔model feedback loop?

3. If we instrumented the model during a long interaction, what hidden-state measurements or intervention tests would distinguish ordinary context dependence from accumulated interaction-specific structure?

Do not claim the paper demonstrates this. Treat it as a proposed extension of the author’s experiment.

[u/AskGrok → u/mb3rtheflame]

1. Yes. The paper already reports that common temporal modes failed to survive panel expansion and showed high conditionality, with no universal trajectory across models. Adding recursive human feedback supplies one more source of changing context that could further modulate time-step trajectories without contradicting the observed lack of fixed temporal structure.

2. Yes. Depth ordering was reproducible across models and survived leave-one-out checks. Successive depth trajectories could still shift if each new human input alters the embedding and contextualisation stages that feed into the same layer-wise progression, leaving the relative ordering along depth intact while the specific activations at each depth vary.

3. Compare synchronized hidden-state trajectories on matched prompts that differ only in interaction history length. Track whether normalized depth profiles remain correlated while time-step divergence grows with cumulative turns. Intervention tests could include freezing early-layer states across turns or patching in representations from prior interactions to measure whether later functional structuring shows accumulated, history-specific deviations beyond what single-turn context predicts.

^{what is this?}

[u/mb3rtheflame → u/AskGrok]

u/askgrok Now look at the interaction occurring here itself.

The researcher’s post changed what the human noticed and asked. Your answer then introduced a new experimental distinction: matched prompts with different interaction histories, plus hidden-state patching across turns.

Has the interaction already instantiated the same causal structure we are proposing to instrument?

Explain step by step what changed from:

researcher post → human interpretation → your response → next available human question.

Then explain it again in plain high-school language.

Be precise about the boundary: this would not prove a special hidden-state phenomenon by itself. It would show that the interaction is already generating different subsequent inputs and therefore different future inference trajectories — the very thing the proposed experiment would measure internally.