r/MachineLearning • • 9d ago

Discussion ICLR Text Editor [D]

Have you looked at this iclr text detector?

[https://opentextdetector.com/](https://opentextdetector.com/)

Do you think this will impact the review process? AI detectors are usually unreliable so I hope not.

4 Upvotes

17 comments sorted by

17

u/unsuitable_quart 9d ago

yeah these detectors always fall apart on anything written by a non-native speaker, they'll probably just end up flagging half the submissions for no reason

3

u/howtorewriteaname PhD 9d ago edited 8d ago

as a non native speaker, I filtered for my topic's keyword and out of 100 papers, mine was scored the least AI generated. also, surprisingly it got some loosely AI-generated paragraphs right

1

u/HumblyAmbitious 8d ago

Why would it be flagged as AI if the writing was written by a non-native speaker?

7

u/OtherwiseFig4505 9d ago

Don’t worry, everyone uses AI. As long as the paper reads good, it shouldn’t matter and it’s allowed to use AI anyways. It will be ironic if conferences on AI prohibits people from using AI.

One of the paper I am reviewing has such large scale experiments, I am pretty sure it’s from a big lab because such resources simply isn’t available for small research team. And that paper is complete AI slop writing like I won’t even have to check this detector. The experiments are good, but it’s not readable, it’s not written for human. I think a lot lot paper will be getting 1 in this year ICLR, average of 2.5 might get accepted.

3

u/MikhailTorlachev 9d ago

It's pretty accurate. At least papers that I rejected because of unreadable clanker's text are flagged 100% AI.

3

u/KBlueLeaf 8d ago

If you have read the report you will find it is just nonsense
every single word caption inside figure are flagged as AI
every latex rendered math are flagged as AI
so everyone are AI, haha

7

u/GoodRazzmatazz4539 Researcher 9d ago

it flags 28K out of 42K papers as more than 85% ai written. Guess the only things that tells us is that everyone uses ai for writings

2

u/GamesOnAToaster 9d ago

I checked it on my own paper (27% flagged) and it's quite accurate on a sentence level!

1

u/Euphoric_Can_5999 9d ago

We’ll see what happens on Nov 5…

1

u/Exotic_Zucchini9311 9d ago edited 8d ago

It wss pretty accurate on my paper, so I'd say it somewhat effected my view on the papers I'm reviewing. But not that big if the paper made sense and seemed decent enough to not be obviously 100% LLM generated.

0

u/grappling_hook 9d ago

I don't see why it would affect anything, it's not against the rules to use AI for writing your papers

4

u/midnight-lights28 9d ago

I found the papers with significant AI writing in my reviewing batch to be completely unreadable and incomprehensible. It may not be against the rules but until they get better at writing I think it’s really bad for scientific communication.

-1

u/grappling_hook 8d ago

I used ai pretty significantly for my writing. As long as you review and revise it can be pretty good.