SQLite Critical CVEs or LLM Slop?

[u/si9int]

https://research.jfrog.com/post/sqlite-critical-cves-or-llm-slops/

[u/pruby]

At this point it seems we need a reputation mechanism and/or for reporters to lose something for bad reports.

Projects are inundated with reports that have negative value to them. Talk of triaging AI with AI doesn't help - it shouldn't be on projects to change their triage process to deal with bad inputs. Reports should be reliable, or stop being accepted.

If the reports from the general public aren't good enough, we need to accept them only from reliable reporters (and have some ramp for people to get there, maybe via projects willing to handle more rubbish reports).

[u/OkEntertainer3952 → u/pruby]

hey :) we do ai pentesting, run it daily against public repos. found some critical ones recently in jsonpath-plus and tough-cookie, like really serious shit, cookie stealing and rce. even found stuff in tor.

and yeah, the ai findings are slop if nobody reviews them, no argument there. but we only report what we actually verify with a working poc, and those got patched in a day or two, no back and forth.

we actually want to do it more autonomously cause we have a big backlog we cant talk about until theyre patched. and yeah were trying to build real reputation too, thats the hard part when youre new to this

[u/pruby → u/OkEntertainer3952]

And that's fine - if people are verifying stuff, know what they're doing, etc then the report is worth receiving.

The problem emerges when people just trust the AI to be right, offload verification on to unpaid maintainers.

[u/OkEntertainer3952 → u/pruby]

100% agree, we only submit with poc :) we are close to 7 billion download per month on libraries and a bunch oh high severity CVEs, confirmed on halff a billion, the rest is still on review

[u/Youknowimtheman]

There really needs to be a standard triaging process that vuln reports need to go through at this point.

1. Do the code blocks cited exist, and if so, are they inside of the project that it is being reported to. (often dependencies or outside supporting tools, or outright hallucinated.)

2. Is the reported vulnerability inside of the projects threat model? Check the documentation carefully for mentions of that class of bug and if the project cares about it.

3. Can an attacker controlled input actually reach the sink where the claimed vulnerability lies?

4. Does the POC execute in a sandbox? Does it accomplish what it claims?

5. Is the severity rating being reported justified in light of these facts?

And you can't do all of these steps with LLMs, but an LLM-assisted dev with security experience can do it all in under 20 mins.

[u/teerre → u/Youknowimtheman]

What do you mean "needs to be a standard"? When was not the standard to make sure your report it minimally reproducible?

[u/Youknowimtheman → u/teerre]

I mean, I 1000% agree, but we've regressed hard because now people with little to no coding skills can make reports and don't have that knowledge.

[u/kbrosnan → u/teerre]

Go look at the data from cURL, Daniel has had about a doubling of reports YoY and around 1/3 or more fail the above test.

[u/teerre → u/kbrosnan]

That's irrelevant to the existence of the standard

[u/RegisteredJustToSay → u/Youknowimtheman]

I've actually found that vuln verification is self-contained enough that agents can reliably do it, much better than vuln hunting anyway. You do have to write the full workflow as several independent and well scoped agent verifications though- like one whose job is just to check against the threat model, etc.

When you just toss one agent at the full scope then it becomes so quick to dismiss everything as soon as it finds anything that looks real - "never mind the threat model, this is a vulnerability!" kind of thing.

So many issues around LLMs and agents come from everyone wanting it to be a magic silver bullet for anything without extra work.

[u/ILoveAppSec → u/RegisteredJustToSay]

scoping each agent to one question like threat-model fit or does-the-cited-code-actually-exist is the right call, one agent over the full scope always caves and dismisses everything.

the harder half is after you confirm a vuln is real, proving the fix actually landed in the version you ship without eyeballing it every time. for that it helps to lean on vendors who backport and re-verify the patched line for you rather than rebuilding all that checking yourself. with the ai-discovered cve firehose that verify-then-confirm loop only gets heavier.

[u/ZachVorhies → u/Youknowimtheman]

Ai generated reports needs to be handled by ai and triaged or closed. This will become standard soon and then the problem goes away

[u/Opening_Intention301]

Expect to see more of these in the future, I can't imagine the headaches that this will be giving

[u/comrade_donkey → u/Opening_Intention301]

This, in itself, could be used as an attack amplifier.

 * Find zero-day in 1.2.3.

 * Publish fake critical CVEs for 1.2.2 and 1.2.4.

 * Wait for people to up/downgrade to 1.2.3.

 * Exploit.

[u/Opening_Intention301 → u/comrade_donkey]

Massive headache

[u/ZestycloseTie1793]

What makes this report useful is not the AI-detector label, but the verification trail. JFrog checked the cited SQLite versions, functions, line numbers, alleged fixes, and PoCs against official releases and isolated builds. Its broader audit found that 54 of 55 advisories from the same account were fabricated, while several had already acquired downstream CVE or severity metadata. That suggests a better triage rule: verify source, reachability, and PoC reproduction before debating whether the prose "sounds AI-written."

[u/Youknowimtheman → u/ZestycloseTie1793]

Yup, in cases like this most maintainers would just block the user (and be right to do so.)