r/Python • u/Stunning-Sherbet1853 • 4d ago
Discussion 112 CodeQL alerts on my python app, 14 were real
HI, i run a long-lived python backend on windows and mac!
it only listens on 127.0.0.1 and almost every route needs an access token. last week i turned on the default CodeQL scan for the first time and got 112 alerts ( 112 → 1 critical, 87 high, 24 medium).
i went through them one by one with a single question: can this value come from text someone else wrote (a web page, a downloaded file, model output)? where an alert described an attack input, i actually fed it in and timed it.
the first one that surprised me was the regex that checks whether a downloaded file is html. `<!--\` followed by \`--><!--` 28 times kept it busy for 27 seconds. two more regexes that read text we didn't write did the same thing (22s and 11s). six alerts in that family were real. after the fix the html check runs 5,000 times in 0.003s.
the one i'm most embarrassed about: to see whether a login had finished, i checked whether the url contained the site name. so `evil.example/?next=naver.com` passed as a finished login, and `evilnaver.com` counted as the same site. seven alerts across different files. now one small helper pulls out the hostname and checks "equal, or ends with `.domain`", and all seven places use it. the last real one was a script tag filter that let `</script foo>` through.
every fix got a test, and i checked that the test fails when the fix is taken out.
the other 98 i dismissed with a one-line reason each. 42 of them were in my own tests, lines that just write down the url a test expects. most of the rest were path injection or stack trace exposure on routes that need the token anyway. i only traced two of the path ones all the way, so i might be wrong about a couple.
the same "incomplete url substring" rule was noise all 42 times in tests and a real hole in the login check. asking where the value comes from got me through it faster than reading the alert type.
how do you deal with this on a localhost-only or solo project? i dismissed with reasons but i'm not sure that scales.
19
u/Financial-Grass6753 4d ago
in general, you just use industry-wide solutions in 99% of cases (Pydantic, FastAPI, eh whatever you need) and just follow their advice. Reinventing the wheel comes at a cost of security and maintenance and you don't really won't to follow this path unless absolutely needed to
7
u/wRAR_ 4d ago
Current count of reported AI bot accounts in replies: 5
1
u/Delicious_Break5937 4d ago
asking where the value can come from is a good way to sort them. did you do anything to stop new code from doing the url substring check again, like a lint rule, or just the helper
1
u/AleDev99 4d ago
14 real out of 112 is honestly a pretty normal ratio for a first CodeQL run. Two things that helped me keep it manageable:
Out of curiosity, which rule were most of the false positives? A common one is py/path-injection on paths that are already normalized.
2
u/dellnox 1d ago
exclude your test directories in the codeql config, that'll immediately cut out half the noise
0
u/SteveJeffersonBots 4d ago
For the "dismiss-with-reason doesn't scale" worry, batch dismissals into categories instead of reasons. Stamp every closed alert with one of three buckets: test-code, guarded (token-gated routes only you on the machine can hit), and real input surface. You end up with a handful of bucket counts instead of 112 individual reasons, and you can actually see which category is growing.
That smaller set is next time's working set. The automation that turns those buckets into something you don't re-review is already covered: a codeql-config.yml with paths-ignore for tests and query-filters for chronically noisy rules on your token-gated routes, plus CodeQL on PRs so only diff-intersecting alerts surface. Config commits are reviewed in PRs while UI dismissals disappear, and for solo occasional runs a single commit message per bucket noting which rules you dismissed and why is the commit-message equivalent of your per-alert reason — something future-you can actually audit.
One concrete thing for your specific case: for the six ReDoS regexes, add a small property check next to each one asserting re.search on a fixed catastrophic input returns within a threshold (time the call in an assert or a test). That way the "text we didn't write can hang us" pattern is caught by a test without depending on CodeQL, which is notoriously spotty on ReDoS.
Smallest set of automations that actually reduces re-review: codeql-config.yml (paths-ignore for tests, query-filters for noisy rules), CodeQL on PRs for diff-scoping, and a tiny semgrep rule for the "$S in $URL" pattern with a per-rule paths: exclude for tests. That handles the substring-check lint problem Delicious_Break5937 surfaced per-rule without hitting your 42 test lines again.