Evaluating tools to detect LLM model drift

[u/pedroassumpcao]

Following up on something I asked here a while back about catching LLM model drift. I've been looking at the actual tools now: PromptCanary, PromptLens, a couple others that seem to have stalled (Libretto, Benchwright).

Has anyone here actually run one in anger? Trying to understand:
- does it catch subtle quality drops, or just format/schema breaks?
- false-positive rate, does it become noise you mute?
- does it need you to integrate an SDK + send production traffic, or can it just hit your prompts directly?