美国国家公共电台 NPR Scientists Aim To Pull Peer Review Out Of The 17th Century

SCOTT SIMON, HOST:

The technology that drives science forward accelerates all the time, but science communication - not so much. The basic process still holds many vestiges 1 from its early days - the 17th century, that is. NPR's Richard Harris reports on efforts to modernize 2 this critical part of the scientific enterprise.

RICHARD HARRIS, BYLINE 3: When researchers studying the biology of disease make a discovery, it typically takes nine months for them to get the results published in a journal. One reason for that delay is it goes through a process of peer review that is both necessary and antiquated 4. The fate of that paper rests on just two or three scientists who have been asked to review it and decide whether it's worthy 5 of being published. Michael Eisen, a Howard Hughes Medical Institute investigator 6 at UC Berkeley, offers an online shopping analogy to illustrate 7 how crazy that approach is.

MICHAEL EISEN: If the only thing Amazon ever published were the reviews of the first three people who bought a product, then we'd have a very ineffective system for knowing what was good and bad.

HARRIS: The scientific reviews may be carried out by someone with a personal axe 8 to grind or who can't truly evaluate a study's methods or statistics. And to top it off, reviews are usually anonymous 9 and kept secret by the journal.

EISEN: If you buy pen refills on Amazon, you get far more useful feedback about the benefits and deficits 10 of a particular product than you do about a work of science that represents, you know, years and years of people's work and millions of dollars of public investment.

HARRIS: Eisen recently attended a meeting of biomedical researchers who want to find a way to modernize this process to make it more fitting for a world that now lives online and isn't so concerned about the price of paper stock for printing presses. Theo Bloom, executive editor of the British Medical Journal, BMJ, says step one is to untangle the reasons for peer review in the first place.

THEO BLOOM: Often, peer review is being asked to do two things. It's being asked to look at technical quality and to look at whether an article is suitable for a particular journal.

HARRIS: Bloom says peer review does a reasonable job of picking studies of interest to journals such as hers. But it actually does a poor job of improving the quality of the paper. Some years ago, for instance, scientists sent around papers with nine deliberate errors in them. Peer reviewers generally found just three.

Michael Lauer, a top official at the National Institutes of Health, told the meeting that one huge problem is that scientists often fail to ask a fundamental question before starting an experiment, which is how many people or animals should be included in a study. By the time a study like that is done and submitted for publication, it's too late.

MICHAEL LAUER: Peer review is not going to help. It doesn't matter how transparent 11 the peer review is. We're not going to be able to solve this problem.

HARRIS: Scientists at the meeting spent a lot of time focusing on the issue of transparency, how to make peer reviews and editors' responses available to anyone who wants to understand how a paper was evaluated and chosen for publication. Mike Eisen argues for doing far more than that. He says biomedical papers should be posted in public for anyone to see and for peer review to come later.

EISEN: What we want to see happen next is to allow the scientists who are reading papers as part of their regular process of reading the things that interest them or might be useful to them to review them.

HARRIS: And to post those reviews online, just as Amazon shoppers do today for pen refills. That would be far more informative 12 and useful. Most notably 13, it would change the incentives 15 for publication. Right now, scientists are rewarded for getting papers passed peer review in the top journals. That means attention-grabbing findings are valued over careful work. Eisen says his system of review before publication would flip 16 those priorities.

EISEN: The fact that people are going to be reviewing and assessing the work for as long as it's useful to the community should provide a very strong incentive 14 back to authors to try to produce work that is actually reproducible and durable 17 rather than work that is just flashy and gets into the right journal.

HARRIS: Eisen and his colleagues are creating a system to allow peer review for papers posted online before publication. And they hope to unveil it this summer. Richard Harris, NPR News.