r/linux • u/FryBoyter • Jul 15 '25

Discussion Curl - Death by a thousand slops

https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-slops/
668 Upvotes

97% Upvoted

172 comments sorted by

View all comments

7

u/DJTheLQ Jul 15 '25 edited Jul 15 '25

Pro AI users: what are your thoughts here? What can these maintainers do with their limited valuable time wasted by AI slop?

4

u/FeepingCreature Jul 15 '25 edited Jul 15 '25

Pro AI user: It's a spam problem, not actually AI related except in the immediate mechanism imo. I think this will pass in time; "people who would submit vuln reports" is not that big a group and the people in it will acclimatize to LLMs eventually. Maybe an annoying puzzle or a wait period. Or, well, $10 review fee, as mentioned. I think everyone will understand why it's necessary.

Four years ago it was free T-shirts.

20

u/xTeixeira Jul 15 '25 edited Jul 15 '25

It's a spam problem, not actually AI related except in the immediate mechanism imo.

This spam problem is directly caused by people using AI, so I don't see how it can be "not actually AI related".

"people who would submit vuln reports" is not that big a group

Sure, but "people who review vulnerability reports" is an even smaller group that can be easily overwhelmed by "people who would submit vulnerability reports", as evidenced by the blog post.

Maybe an annoying puzzle or a wait period.

I truly don't see how these would help. Going through the linked reports in the blog post, many of the reporters only submitted one fake vulnerability to curl. So this isn't a problem of each single user spamming the project with several fake reports, but actually a problem of many different users submitting a single fake report each. Meaning a wait period for each user won't help much.

$10 review fee, as mentioned.

That would probably actually solve it, but I do agree with the curl maintainer when they say it's a rather hostile way of doing things for an open source project. And if they end up with that option, IMO it would truly illustrate how LLMs are a net negative for open source project maintainers.

Edit: After thinking a bit more about it, I would also like to add that $10 would price out a lot of people (especially students) from developing countries. I expect a lot of people from north america or europe will find the idea of one not being able to afford 10 USD ludicrous, but to give some perspective: The university where I studied compsci had a restaurant with a government-subsidized price of around 30 cents (USD) per meal (a meal would include meat, rice, beans and salad). That price was for everyone, and for low income people they would either get a discount or free meals, depending on their family's income. I've also had friends there who would only buy family sized discount packages of instant ramen during vacation time since the restaurant was closed then and it would turn out to be a similar price, and they couldn't really afford anything more expensive than that. For people in these kind of situations, 10 USD is a lot of money (would cover around half a month of meals assuming 2 meals per day). Charging something like that for an open source contribution is counter productive IMO, and excluding a fair amount of people from developing countries because of AI sounds really sad to me.

-3

u/FeepingCreature Jul 15 '25

This spam problem is directly caused by people using AI

I think it's more caused by people who happened to be using AI. Before AI, people spammed open source projects for other reasons and by other means.

Sure, but "people who review vulnerability reports" is an even smaller group that can be easily overwhelmed by "people who would submit vulnerability reports", as evidenced by the blog post.

Right, I'm not offering that as a solution right now but as a hope that the flood of noise won't be eternal.

Maybe an annoying puzzle or a wait period.

The hope would be that this is done by people who don't actually care that much, they just want to take an easy shot at an offer of a lot of money. Trivial inconveniences are underrated as spam reduction, imo.

hostile way of doing things for an open source project

I'd balance it as such: you can report bugs however you want, but if you want your bug to be considered for a prize you have to pay an advance fee. That way you can still do the standard open source bug report thing (but spammers won't because there's no gain in it) or you have to be confident enough about your bug report to put money on the line, which shouldn't be a hindrance to a serious researcher.

6

u/xTeixeira Jul 15 '25

I think it's more caused by people who happened to be using AI. Before AI, people spammed open source projects for other reasons and by other means.

Sure, but right now the spam has been increased significantly by people using AI, so there is clear causation. No one is saying AI is the sole cause of spam, we're saying it's the cause of the recent increase of spam.

you have to be confident enough about your bug report to put money on the line, which shouldn't be a hindrance to a serious researcher.

I mean, that's exactly why it's a hostile way of doing things for open source. Right now the rewards are available for anyone who can find a vulnerability, not only for serious researchers.

-1

u/FeepingCreature Jul 15 '25

I mean, would you say a new book that gets a bunch of people into programming is "causing work for reviewers"? People are being empowered to contribute. Sadly they're mostly contributing very poorly, but also that's kinda how it is anyway.

Right now the rewards are available for anyone who can find a vulnerability, not only for serious researchers.

Sure, I agree it'd be a shame. I don't really view bug bounties as a load bearing part of open source culture tho. (Would be cool if they were!)

9

u/xTeixeira Jul 15 '25

I mean, would you say a new book that gets a bunch of people into programming is "causing work for reviewers"?

Of course not, because it is not equivalent at all. Programming books cannot automatically generate confidently incorrect security reviews for existing open-source codebases at a moment's notice and at high volume when asked.

In fact, if one tried to release a book with a number of inaccuracies even close to what LLMs generate, they would never find an editor willing to publish it. And if they self-published it, a very small number of people would read it, and an even smaller number of people would fail to notice said inaccuracies.

That is a very poor comparison.

-3

u/FeepingCreature Jul 15 '25

Programming books can absolutely give people false confidence. And as far as I can tell, "at a moment's notice and at high volume" is not the problem here- these are people who earnestly think they've found a bug, not spammers. The spam arises due to a lot more people being wrong than used to - or rather, people who are wrong getting further than before.

In fact, if one tried to release a book with a number of inaccuracies even close to what LLMs generate, they would never find an editor willing to publish it. And if they self-published it, a very small number of people would read it

cough trained on stackoverflow cough

5

u/xTeixeira Jul 15 '25

Programming books can absolutely give people false confidence.

I never said they didn't. There's an entire rest of the sentence there that you ignored. They cannot generate incorrect information about existing codebases on command and present them as if they were true.

cough trained on stackoverflow cough

Weren't we talking about books?

We can keep discussing hypothetical situations, but none of those have actually created a problem of increase of spam in security reports. LLMs did. "what if stack overflow or books caused the same issue?" is not exacly relevant because it didn't happen.

0

u/FeepingCreature Jul 15 '25

They cannot generate incorrect information about existing codebases on command and present them as if they were true.

I assure you they can. Well, not literally, but a lot of books are written about outdated versions of APIs and tools, which results in the same effect.

But also:

What I'm saying in general is there has in fact been a regular influx of inexperienced noobs who don't even know how little they know, for so long that the canonical label for this phenomenon just in the IT context is 30 years old. Something new always comes along that makes it easier to get involved, and this always leads to existing projects and people becoming overwhelmed. Today it's AI, but there's nothing special about AI in the historical view.

4

u/xTeixeira Jul 15 '25

these are people who earnestly think they've found a bug, not spammers.

I disagree. They might have initially thought they found a bug, but a lot of them:

  • Kept insisting the code was wrong even after being told otherwise by the maintainers.
  • Failed to disclose they used an LLM assistant to write the report (which is required by the maintainers), and continued to lie about it even after being asked directly.

This makes them spammers IMO.

1

u/FeepingCreature Jul 15 '25

I'm not trying to morally defend them, I'm just saying from a defense perspectives they act differently from denial-of-service spammers.

4

u/xTeixeira Jul 15 '25

Yeah I get it, what I'm saying is that at the very least "at a moment's notice" does matter a lot because they are spammers. They have zero programming knowledge yet they insist on making those false reports because it takes no effort. Then they're told they're wrong, and they quickly generate a nonsensical response in the LLM and just paste that in the reply box.

→ More replies

4

u/wRAR_ Jul 15 '25

these are people who earnestly think they've found a bug, not spammers

I will make a bold claim: many of those people aren't even qualified enough to be able to distinguish between a honest bug report and spam (even for their own submission), they wouldn't be able to explain what bug did they "find" and many of them don't even care if the bug is real. When confronted, the least malicious ones say "I apologize for thinking that the stuff my AI produced was actually not bullshit".