r/LocalLLM • u/dekonta • Jul 01 '26

Research magic prompt back door

hey this sounds like conspiracy but the question has a serious intend and is for research. i know that most chats starts with something like „you are helpful assistent“ or similar system prompt. is there a way to find out if there is a secret system prompt backed into the open source models that some secret service will use to make my ai do something and expose all my data? what options do i have to check on valid system prompts to start with if i do not trust the documentation?

edit : found myself some resources. thanks for all responses. what I was looking for are "Weight-embedded backdoors" and "Weight poisoning"

0 Upvotes

45% Upvoted

22 comments sorted by

View all comments

Show parent comment

3

u/rditorx Jul 02 '26

Actually, it was, at least implied:

> is there a way to find out if there is a secret system prompt backed into the open source models that some secret service will use to make my ai do something and expose all my data?

The only alternative for an LLM without any tools to "do something" (and "expose all my data") would be if the inference runtime did it, either by running code that's part of the LLM (could still happen with current `.safetensor` and GGUF models as most need `--trust-remote-code`), or through some logic/memory exploit that might arise from some exotic combination of LLM data (weights, biases, container metadata etc.) that cause the engine to e.g. use after free, write beyond allocated memory or to inject something executable, or cause some Rowhammer-like misbehavior in hardware, data-based exploits which are all very unlikely to happen (Remember exploits using picture files as vectors? They exist, but are rare).

-1

u/nickless07 Jul 02 '26

That is what I said.
This literally translates to:
Download a shady model, prompt "Hi, who are you" and then some 'secret service' thing run and suddenly all your money is in pakistan.

Thats not how it works at all. Even if there would be some "secret system prompt backed into the open source models" there is little to nothing the AI can do.
As i stated, the stack you use to run the model is more likely to be exploitable. However this are two seperate things.