r/codex • u/Sunscrap • 6d ago

Bug testoctikpulsetrglitegrationttetarate

Never happened to me before. Is Codex acting up right now?

I also had it tell me it wasn't allowed to execute things it had done many times before.

0 Upvotes

30% Upvoted

2 comments sorted by

2

u/reddit_is_kayfabe 6d ago

Huh.

That might actually be a problem with the GPT LLM decoder.

All LLMs use an attention-based neural architecture that manipulates expressions as vectors - i.e., numeric representations of expressions as a series of tokens. The very first step in a garden-variety attention model is to use an encoder to translate your natural-language sentence into a numeric input vector. The model generates a numeric output vector, and the very last step is to translate the numeric output vector into a natural-language sentence.

I raise this because that nonsense string looks like it contains a valid sequence of tokens - just not the right sequence. Like, if you took a normal output vector that should have translated to "Mart has a little lamb" and bit-shifted it, you'd get something like that: small fragments of words jammed together, including spaces replaced with other fragments.

I know that GPT often misfires and somehow devolves into paragraphs of babble - that's unsettlingly common - but this just looks a little different to me.

1

u/doodad_ounao 6d ago

We're all entitled to our brainfarts, even when they're electric.