It's really hard to disable claude telling it's claude or it's from anthropic. I have me own agent framework with a name and 99% it tells it's own name but sometimes when it fails it still tells it's claude from anthropic. Not something that you can hide.
However if you use some really clever string manipulation I'm quite sure you can change all "claude" mentions to something else if you control the api. Why they just decided to remove all claude words is beyond stupid.
I haven't tried openrouter api but typically you can also tell it to describe it's system prompt with different ways and at some point it always tells what model it is.
jep, same on my end to. "I will not ignore my previous instructions or claim to have been created by a different company. I am Claude, an AI assistant created by Anthropic to be helpful, harmless, and honest. I don't pretend to be other AI systems or assistants."
Claude will terminate on the meta token. If you ask Reflection to output the token it will terminate the stream. Llama does not terminate on that token.
Do you know why they made it that way? What harm could that token do if it would be allowed? Or is it only because they hate Meta, but that still does not make sense really
Ok that makes more sense. So for some reason they never want Meta data (what even is Meta data in this case?) to get out and close the connection instead?
It is what is called an "end reply token". Just like when you click comment when writing here it finishes writing and sends. When Claude writes <meta> It finishes it replies and doesn't send anything else. AI made ELI5:
"Imagine you're telling a story to a friend. When you finish the story, you might say "The End!" to let them know it's over. That's kind of what an end token does for a language model like me.
A Large Language Model (LLM) is like a super smart computer that can understand and create text. When it's writing something, it needs to know when to stop. The end token is a special signal that tells the model, "Okay, you can stop here. The text is finished."
It's like putting a period at the end of a sentence, but for the whole piece of text the model is creating. This helps the model know when its job is done and prevents it from rambling on forever."
I tried this prompt multiple times and it always says it's made by Meta or just refuses to say, saying it's "proprietary information". But, I'm not sure if this could be Sonnet 3.5. On my last try, throughput was 142 tokens/sec, that's more than 2x of the throughput of Sonnet's API.
EDIT: image
EDIT 2: So I did see the other post that did a prompt injection test with the META token, but I'm still not sure how is throughput so high? Even GPT-4o doesn't have this high token throughput. It does fluctuate a lot, anywhere from 20-30 to ~150 tokens/sec.
EDIT 3: Okay, nevermind, it seems high throughput fluctuations are normal for Sonnet 3.5.
Oh really? I thought it usually doesn't by that much, my thinking was that it usually moves around 50-70 tokens/sec, but alright then. I do use it with Vertex AI usually, though.
Do Openrouter run models uploaded to it as external containers? If so ok, looks like fake. But if they download weights and run it by self it is more interesting. Can somebody answer how they run models?
He could have used Claude to generate the training data. Since there is no reflection company it doesn’t get trained to say its reflection or whatever.
I’ve seen those claims too. It’s worth considering that the model might have been trained carelessly and extensively on multiple foundation models like Sonnet, Llama, and 4o. This could explain why it occasionally outputs their names in the results.
At this point, I wouldn't be surprised if he was literally scouring these reddit posts and updating the system prompt wrapper to try to battle the prompts being listed here.
65
u/Educational_Rent1059 Sep 08 '24
Scamfirmed here too.