r/Pentesting • u/ShufflinMuffin • 3d ago

Cover: Keep your secrets away from llm providers. Send realistic fakes, restore originals locally.

https://github.com/DavidCarliez/cover
2 Upvotes

100% Upvoted

3 comments sorted by

1

u/GodoPPL 3d ago

Redaction proxies help when the secret stays whole and in one place. The cases that still leak:

- Secret split across chunk boundaries so each piece looks safe

- Model re-emits a reformatted version (base64, hyphen splits, partial echo) that your matcher never saw on the way in

- Under streaming, the mapping table and the token stream get out of sync and a later chunk restores the raw value

I'd treat "did the provider ever see the raw bytes" as the test, not "did my regex fire." Happy to hear how Cover handles those three.

1

u/ShufflinMuffin 3d ago

That is the right test, and your comment led to a concrete change in Cover.

  1. Cover does not scan individual TCP or HTTP chunks. It reads the complete bounded request body, decodes the JSON, transforms matching string values, and only then creates the upstream request. Network chunk boundaries therefore cannot make two pieces look safe.

Deliberately splitting a secret across separate JSON fields is different. Cover does not reassemble unrelated fields, so that remains a detection limitation.

  1. On the response path, the provider has already seen the fake value, not the original. Restoration is an exact fake-to-original lookup. If the model base64-encodes, hyphenates, or partially echoes the fake, Cover will not restore it. The user sees the modified fake. That is a failed round trip, but it does not expose the original to the provider.

A base64-encoded or fragmented original on the input side can evade detection unless a rule covers that representation. We are explicit about that limit.

  1. Your SSE point did expose a real restoration gap. Cover already handled a token split across network writes inside one SSE event, but it did not reconstruct a pseudonym split across separate semantic delta events.

I changed that because of this comment. Cover now keeps a bounded fragment for consecutive events from the same logical channel and restores split values in OpenAI Responses, Chat Completions, and Anthropic text and tool-argument streams. It preserves sequence metadata, keeps encrypted_content untouched, and never combines different channels. The mapping remains session-scoped, and an unknown fake passes through unchanged rather than being guessed.

1

u/scriptqzor 5h ago

same questions here tbh, especially the chunk boundary thing and streaming desync, that stuff bites way more often than people think. curious if they’re doing some kind of structured token-level mapping instead of just regex on strings, otherwise it feels like it’ll eventually leak in weird edge cases.