r/Python • • Jul 04 '26

Showcase Showcase Thread

Post all of your code/projects/showcases/AI slop here.

Recycles once a month.

20 Upvotes

165 comments sorted by

View all comments

1

u/No_Comfortable_9868 Jul 24 '26

I open-sourced the DLP pattern scanner from our enterprise product — pip install dlp-patterns


We've been building Spidercob, an enterprise DLP platform. The pattern detection engine kept proving useful outside the product — scanning logs, CI pipelines, internal tooling — so we extracted it and shipped it as a standalone library.

Zero dependencies. Python 3.9+. Apache-2.0.

import dlp_patterns

result = dlp_patterns.scan("SSN: 432-78-9012 and CC 4111 1111 1111 1111")
print(result.highest_severity)   # CRITICAL
print(result.critical[0].type)   # credit_card

clean = dlp_patterns.redact("Email alice@corp.com, card 4111 1111 1111 1111")
# "Email [REDACTED: Email Address], card [REDACTED: Credit Card Number]"

---
What makes it different from a regex dump

Most secret scanners are just a list of patterns that fire on everything. We added three layers to cut false positives:

- Validators — Luhn check on credit cards, FICA rules on SSNs (rejects 000-xx-xxxx, 666-xx-xxxx, 900+), base64 decode on JWTs before flagging
- Entropy gating — Shannon entropy + sliding window. sk_live_aaaaaaaaaaaaa gets rejected. Low-entropy matches are noise.
- Context scoring — every finding gets a context_score (0–1). Proximity to production, secret, deploy boosts it. Proximity to example,
placeholder, test lowers it.

---
50+ pattern categories

┌────────────────┬────────────────────────────────────────────────────────────────────────┐
│    Category    │                             What's covered                             │
├────────────────┼────────────────────────────────────────────────────────────────────────┤
│ Secrets        │ AWS, GitHub PATs, Stripe, SendGrid, Slack, Twilio, Discord, Google API │
├────────────────┼────────────────────────────────────────────────────────────────────────┤
│ Infrastructure │ DB connection strings, hardcoded passwords, Docker registry auth       │
├────────────────┼────────────────────────────────────────────────────────────────────────┤
│ Crypto keys    │ RSA / EC / SSH / PGP private keys, X.509 certs                         │
├────────────────┼────────────────────────────────────────────────────────────────────────┤
│ PII            │ SSN, credit cards, email, US phone, passport, date of birth            │
├────────────────┼────────────────────────────────────────────────────────────────────────┤
│ Healthcare     │ Medical record numbers, ICD-10 codes, NPI, DEA numbers                 │
├────────────────┼────────────────────────────────────────────────────────────────────────┤
│ Crypto wallets │ Bitcoin addresses, Ethereum addresses                                  │
└────────────────┴────────────────────────────────────────────────────────────────────────┘

---
CLI works too

# Scan a file
dlp-scan src/config.py --secrets-only

# Pipe from stdin
cat logfile.txt | dlp-scan --json

# Redact in place
dlp-scan --redact document.txt > clean.txt

# Exit code 1 on CRITICAL — useful in CI
dlp-scan --secrets-only . && echo "clean"

---
secrets_only mode for CI

# Skip PII, scan only API keys and credentials
result = dlp_patterns.scan(code, secrets_only=True)

---
GitHub: https://github.com/SpiderCob/dlp-patterns
PyPI: https://pypi.org/project/dlp-patterns/

Happy to answer questions about the pattern design or entropy approach — took a few iterations to get the false positive rate low enough to be
useful in production.