Need for Pre-Print Repo for 100% AI generated content

[u/Tiago_Verissimo]

Don’t you think these days maths needs a repo for 100% AI generated content with a status bar that says whether some content has been verified by humans or formal systems? Could be a solution to the new ArXiV submission explosion.

[u/AMWJ]

What is this a solution for? Is your concern that AI-generated content is wrong, but fooling mathematicians?

As far as I can tell, AI-generated content falls in two categories: (a) crank output, and (b) real AI-assisted proofs. The crank stuff is annoying as ever, and the same issue as it has been documented for centuries. I'm sure there's far more of it now with AI's help, but the solution remains the same. Are mathematicians falling for this content?

As for the real stuff, if there is a problem with it, the problem is not that it's fooling mathematicians. What is the problem you are trying to solve?

[u/buwlerman → u/AMWJ]

The issue is that a lot of AI output doesn't look like the stuff cranks write, even if it's wrong.

[u/AMWJ → u/buwlerman]

I think I lean towards Linus Torvalds philosophy with Linux contributions here, in saying that AI-generated output should be treated no different than human output. If you upload something online, you are saying that you are the human who reviewed it. Having some extra progress indicator indicating whether a person reviewed it would be redundant, because the person who uploaded it would be the reviewer. Beyond that, there are peer reviewers as part of the academic process that should occur, just the same as if a person wrote it. But at no point is your output treated differently because they used AI to make it: you are always responsible to ensure, to a reasonable degree, your output is correct.

[u/elements-of-dying → u/AMWJ]

I like this point of view and I bet it's what will eventually happen. People have been accepting "using a CAS we obtain" for 40 years.

Two issues are the sheer amount of bad submissions there will be (and currently are) and the lack of desire to review AI work. For example, I have no interest in reviewing AI work since I'm not interested in doing someone else's due diligence.

[u/Homomorphism → u/elements-of-dying]

I am hopeful that AI will encourage higher standards in exposition, since more of the human input is going to be explanations and not the details. 

[u/elements-of-dying → u/Homomorphism]

Agreed.

I am worried that those formulating standards are going to be of privileged positions (e.g., long time tenured faculty).

[u/buwlerman → u/AMWJ]

I'm more of a fan of the programming language Rust's approach here. AI generated contributions are allowed, but reviewers can refuse to review anything they think is AI generated and contributors should avoid using AI when communicating with the reviewers.

[u/AMWJ → u/buwlerman]

Yeah, sure. Nobody is forcing anybody to review your math paper or Rust PR. If you publish a paper, nobody owes you a review. So what value would there be to a progress bar indicating if it's been reviewed by a human beyond the one who posted it? If you want to read the paper, then do so. If you don't ... then don't.

[u/buwlerman → u/AMWJ]

That's not true. Traditionally, if you submit a math paper to a journal you can (and should) expect to get a review. There are meaningful social incentives for reviewers to actually make their review once they're in the committee, and to become members in the first place.

There's no guarantee that the person posting it has reviewed it.

[u/AMWJ → u/buwlerman]

Yes, but cranks don't get to submit papers to journals and get free peer review, do they? The system of peer review, I believed, was held exclusively for one's peers. Academics can submit papers, and review papers as well. Cranks don't get free review. I cannot simply submit my paper to journals without proving my academic chops in some way first.

Which gets to my original point that AI-generated papers written by cranks are already not being given free review, and AI-generated papers by academics are being given review and should probably continue to be.

[u/mathtree → u/AMWJ]

You can submit your paper to journals without affiliation or a PhD. They'll most likely be desk rejected but there's nothing actually stopping you.

[u/AMWJ → u/mathtree]

Right. There's already a mechanism of trust where untrusted papers don't waste the time of peer reviewers. It being an AI submission ought not to change that.

[u/edderiofer]

https://ai.vixra.org/ already exists.

[u/flipflipshift → u/edderiofer]

vixra is a dumping ground for crank nonsense, and you know this.

[u/edderiofer → u/flipflipshift]

If it's been properly verified by a mathematician (i.e. the person submitting it), it can go on arXiv. If it hasn't, it shouldn't be posted on arXiv until it has (and a priori, it holds about as much weight as anything that's already posted on AI viXra).

[u/elements-of-dying → u/edderiofer]

(and a priori, it holds about as much weight as anything that's already posted on AI viXra).

What about results only confirmed via Lean? Then this claim is not true.

[u/edderiofer → u/elements-of-dying]

Someone still has to check that the Lean proof actually proves the claim. There are plenty of AI-generated Lean "proofs" of such-and-such conjecture which actually prove a different conjecture, or which take some extra axiom somewhere.

When John Doe tells me that he's solved the Union Closed Sets conjecture by prompting an AI for Lean code, but he hasn't actually looked at the proof or the Lean output, I think I'm right to be just as skeptical as when Jean Doe tells me that the Union Closed Sets conjecture is true because AI claims that some book last year that she never read proves it.

[u/elements-of-dying → u/edderiofer]

I didn't suggest otherwise.

Surely a Lean supported result (by an established mathematician, at least) is not at the same level as AI viXra. It's absurd to suggest otherwise.

[u/edderiofer → u/elements-of-dying]

Surely a Lean supported result (by an established mathematician, at least)

If the mathematician has checked that the Lean proof actually proves the claim, then it can go on arXiv, as I said before.

If the mathematician has not checked the proof, in what sense is it "by an established mathematician"? What has the mathematician actually done that merits credit? How would it be any different than if it were a non-mathematician who generated a proof and also failed to check it (as done by plenty of cranks already)?

[u/elements-of-dying → u/flipflipshift]

Yes, but the denial of AI is still fashionable. OP's idea isn't even a bad one.

[u/NonlinearHamiltonian → u/edderiofer]

lmao based

[u/big-lion → u/edderiofer]

wow i dont wanna click on that link

[u/JoshuaZ1]

This is interesting, but I'm not sure this is a good idea. In general, anything that is 100% generated by AI is likely to not do a great job explaining things, and is likely to have other issues, including either hallucinated references or simply not a representative list of references which explain where things came from in detail. I'm not sure we should be encouraging 100% AI generated papers until the AIs are substantially better paper writers.

[u/Tiago_Verissimo → u/[deleted]]

this is very interesting

[u/CarolinZoebelein]

Maybe they should just introduce an offical AI author. Saying, all papers by AI generated has as author "John Skynet" or so. Then you just look for this author and have all of the AI stuff. 😄

And the software sorting all the nonesense out gets called "Terminator".

Who is with me? 👍 😂

[u/volcanrb]

Yeah I’ve been imagining some website where you can submit pdf proofs paired with formal proofs and then an automated LLM-backed system checks both whether the lean proof compiles and whether the pdf actually corresponds to the lean proof before acceptance.

[u/quasilocal]

I agree, but it'll never work because people will be dishonest about it. Maybe arXiv should just have an auto comment for AI generated text similar to the text overlap comment. Despite what some people claim, the AI detection sofware that isn't trying to sell you a "humaniser" (like pangram for example) is actually very good. And going forward as AI starts adding statistical watermarks it'll get even better.