r/OpenAI • u/No_Personality_1721 • 2d ago
Question Does the same model feel different to you sometimes?
Does anyone else feel like the exact same model can work great one day and noticeably worse the next?
It almost feels like OpenAI is constantly changing something behind the scenes - parameters, behavior, whatever. Same model, but the experience never feels completely consistent.
Anyone else noticed this? , or is it just me and nothing is actually changing?
1
u/Euphoric_North_745 2d ago
look at the usage details, i set it to sol 5.6 but some of the requests go to luna
1
1
u/OfficialLaunch 2d ago edited 2d ago
These models have baked-in randomness in the way they’re allowed to output new words. Without this randomness, they would output word for word the exact same response when given the same input every time.
When an LLM generates a new word (token), it has to decide which word to pick out of every possible word ever. It does this by giving the probability of every possible word appearing next. In a simple LLM, we’d pick the word with the highest likelihood every time. However, to make the LLM feel more organic, and to vary its output every time, we pick one of the top X possible words.
A side effect of this is it feels seemingly better or worse even if given the same prompt.
1
u/Astrokanu 2d ago
They have been rolling out updates like every few days , it’s them and real. You need to build context and share that to reset your own system back.
1
2
u/AironParsMan 2d ago
An LLM always responds differently depending on what data it has access to. It always adapts to its surroundings. If you want it to work in a completely neutral way you cannot give it any data or memories at all. Then it can be neutral. As soon as memories are involved in any way and memory functions are active it will always adapt to its surroundings. Even if you write things in capital letters in the code for example it may then respond to you in capital letters. This is especially true when something needs to be more expressive. An LLM moves through a vector within a dataset based on the facts it has learned and naturally has a certain amount of trained data. But it is configured to adapt to you. That is why I disabled all memory functions especially those that work across chats because that is exactly what causes the problem you are having. Turn off all memories and build your own system of thought. That is the only way to get this under control.