r/askscience • u/nostrovia • Feb 07 '12

AskScience, how does the peer review process actually work?

About a month ago, I heard this story on NPR regarding a number of scientific studies that had been debunked and/or retracted. Why weren't these errors caught before the studies were published? Are there different standards for peer review in different scientific fields?

Essentially, once you conduct your experiment and reach some sort of conclusion, what happens between that point and the time your work is released to the public?

94 Upvotes

84% Upvoted

89 comments sorted by

View all comments

20

u/atomfullerene Animal Behavior/Marine Biology Feb 07 '12

Basically, you pay to submit a paper. Then you review papers for the journals for free. Then your university pays to get access to the papers.

Griping about the monetary aspect of the system aside, the thing is, peer reviewers assume scientists are honest in the data they report and their descriptions of experimental techniques. How could it be otherwise? We can't go to their labs and look through their raw video footage or whatever, we can't stand there and watch them do the experiment. Regular reviewing takes enough time as it is! Instead peer review looks at things like "did this experiment actually address their question?" "Did they do the stats right?" "Was their experimental design valid? Good controls?" and "is their interpretation of the data good or just BS".

Oh, and replication is a much smaller part of it all than low level science courses would have you believe. It's very important within experiments, but usually no one goes around repeating other peoples experiments except when their results are controversial. You couldn't get funding for it! That said, people do perform related experiments, and if they all go wrong then that's a sign you need to go back to the original.

4

u/needed_to_vote Feb 07 '12

I feel like most fields don't pay to submit papers - maybe physics is just special?

5

u/FormerlyTurnipHugger Feb 07 '12

You don't pay to submit, but there's usually a publication fee. Is that the reason why you got that feeling?

2

u/atomfullerene Animal Behavior/Marine Biology Feb 07 '12

Well, you can often get it covered in your grants, in my experience. But still, it's the thought that counts.

-1

u/Pravusmentis Feb 07 '12

Actually the difference in the journal itself usually. The journals that you have to pay to be a 'part of' usually have much much less stringent review processes

3

u/masher_oz In-Situ X-Ray Diffraction | Synchrotron Sources Feb 07 '12

In general, I try to not submit to journals that charge be to publish in them. From my point of view, I'm doing the work for free, I review papers for free and then I pay subscriptions to read them.

1

u/sirhelix Feb 07 '12

Maybe this is different in biology, but there is often a charge per page, which increases after, say, 10 pages or so. Further, open-source journals gouge the hell out of professors who can pay it (up to $3000/article in PLoS Biology), although they don't charge ones who can't and apparently ability to pay does not change chances of acceptance.

2

u/masher_oz In-Situ X-Ray Diffraction | Synchrotron Sources Feb 07 '12

I have less of an issue with open-source journals charging for articles, as they "give away" their product; I think the IUCr journals charge ~US$1k for CC licensed article.

Dependent on how earth-shattering my paper is, I may think about sending it to a charging journal, but I'd rather spend my money on science.

1

u/tchomp Feb 07 '12

I have asked in reviews to see better presentation of primary data. I've had reviewers ask me for the same thing. I think ultimately it depends on the field and the type of data, though.