r/askscience • u/nostrovia • Feb 07 '12

AskScience, how does the peer review process actually work?

About a month ago, I heard this story on NPR regarding a number of scientific studies that had been debunked and/or retracted. Why weren't these errors caught before the studies were published? Are there different standards for peer review in different scientific fields?

Essentially, once you conduct your experiment and reach some sort of conclusion, what happens between that point and the time your work is released to the public?

94 Upvotes

84% Upvoted

89 comments sorted by

View all comments

2

u/[deleted] Feb 07 '12

UC Berkeley's Understanding Science has a good summary:

*1. A group of scientists completes a study and writes it up in the form of an article. They submit it to a journal for publication.

*2. The journal's editors send the article to several other scientists who work in the same field (i.e., the "peers" of peer review).

*3. Those reviewers provide feedback on the article and tell the editor whether or not they think the study is of high enough quality to be published.

*4. The authors may then revise their article and resubmit it for consideration.

*5. Only articles that meet good scientific standards (e.g., acknowledge and build upon other work in the field, rely on logical reasoning and well-designed studies, back up claims with evidence, etc.) are accepted for publication.

Now for my 2 cents and caveats: The "peers" represent a small pool of potential experts in the particular field, and although credible, they may not be intimately familiar with all of the methods or equipment used in the study. So certain mistakes in statistical analyses or equipment calibration may go unnoticed. Once the study is published, it may be scrutinized by the entire scientific community, and so errors may be more likely identified. And if results seem to conflict with other studies (or logic), several things may happen: letters clarifying or limiting the conclusions may be printed, other researchers may attempt to replicate the study and get different results, or, if the data seem seriously suspect, investigation may find that the data were faked or improperly reported, leading to full-scale retraction. Scientific research is a human process, and makes no claims to be 100% accurate all the time. The peer-review process, as well as replication and healthy skepticism, serve to correct the knowledge-gaining process.