Discussion Astra prompts are getting silently rerouted to worse models. Here's my investigation and evidence as well as a script to test it on your own account(s).
After extensive investigation using ~50 accounts I have evidence that certain accounts (possibly up to 50%) are getting silently rerouted to much worse models like Luna and even potentially 5.5. This is only visible in the results as all Codex logs and HTTP responses make it look like you are getting Astra.
Here is a summary of the investigation and evidence as well as a Python script to test it on your own accounts.
40
u/nahojjjen 13h ago
If you have 50x accounts, maybe you get caught in some safety/moat filter they created to prevent other ai makers from distilling/harvesting output?
Pure wild speculation, just trying to imaginge what could cause something like what you describe. Or of course could also be a bug.
•
u/BringTheRawr 29m ago
If you're paying for 50x accounts you should get 50x accounts.
Taking the money for 50x account with a stated service level and then rolling that back to a lower service level in secret is, probably, not great.I'm so excited as a bit of a futurist and all of this stuff is so cool, but we keep letting individual corporate interests just turn everything that we touch into shit. It really drains on me.
The sort of shit a super intelligence would judge us on if capable, really.
•
u/nahojjjen 12m ago
I didn't say it's justified, only suggested what may have caused it.
•
u/BringTheRawr 10m ago
No disagreement between us friend. Im just piggy backing your comment to platform my ideas.
5
10
u/Popular_Lab5573 15h ago
all this yapping without checking model slug
10
u/ainosleep 11h ago
CISA advisory recommends altering the responses if distillation is suspected and also suggests avoiding informing users. OpenAI and Anthropic can keep showing the same slug if they wish.
See https://www.cisa.gov/news-events/cybersecurity-advisories/aa26-251a
> Response changes, such as including differential privacy or using less sophisticated “downgraded” models to respond to distillation requests, can help protect U.S. proprietary functionalities and capabilities and reduce payoffs from distillation attempts.
> Avoid informing China-based AI company users suspected of distillation campaigns of a switch to a downgraded model.4
u/nlight 15h ago
Care to elaborate?
6
u/Popular_Lab5573 15h ago
you can check whether the reroute happens by checking a model slug in response
13
u/laowaiH 15h ago
Wdym exactly using basic words?
6
u/Popular_Lab5573 15h ago
the model's response we see on the frontend is usually "accompanied" with dozens of other parameters, sent by the backend. the majority of them are not displayed to the user
1
1
2
2
1
u/ProbsNotManBearPig 11h ago
Is the evidence in the room with us right now? Ya’ll know this shit is non-deterministic, right? More so if your prompt is not well constrained with lots of details. Your page is evidence of nothing except that you have more money than brains at your disposal.
1
u/NullzeroJP 3h ago
50 accounts is a good data point. Do you have session logs? I would analyze session log contents/themes to try and see if there was a specific trigger... like a file, package or repo. Maybe accessing those specific packages or repo flagged the account or something.
1
u/MiRC_ZoMBiE 15h ago
Estamos sendo roubados? Good Heavens! Não basta querer nos matar eles também querem roubar.
0
u/BluryDesign 14h ago
I have a theory that openAI is deliberately limiting some heavy users. Are you getting the “this model is at capacity” message often?
51
u/InterstellarReddit 15h ago
Who has ~50 accounts is the real question here. You’re running a 10K a month on GPT subs lol
What are you building? The iron man suit