No peer review crisis after all?

We’ve had a bit of discussion here at JE about potential solutions to the tragedy of the reviewer commons, so I found a recent letter in Nature (warning – it’s behind a pay wall) suggesting that there may not actually be a problem interesting. The take home message is:

At the journal Molecular Ecology, we find little evidence for the common belief that the peer-review system is overburdened by the rising tide of submissions.

and the authors base this conclusion on some basic statistics about the number of review requests required to obtain a reviewer and the average number of authors and reviewers for each paper. It’s not exactly the kind of hard, convincing data that will formally answer the question of whether there is a problem, but it’s interesting to hear that at least one journal’s editorial group isn’t particularly concerned about this supposedly impending disaster.

4 Comments on “No peer review crisis after all?

  1. Tim Vines, the EiC at Molecular Ecology, has a very sensible attitude about the putative peer review crisis. In contrast to some of his peers, he’s not so arrogant as to assume there’s obviously no problem, and could never possibly be a problem (implying that everyone who thinks there is a problem is dumb, a whiner, or has some ulterior motive). Rather, he’s sufficiently concerned about the potential for problems to be willing to analyze and present the sort of data he presented.

    I’d like to see every other journal in ecology and evolution do the same, and hopefully now that Molecular Ecology has taken the lead it will be easier to convince other journals to follow suit. Owen Petchey and I plan to do everything we can in the new year to try to make this happen. I wouldn’t be surprised to find that there are problems, but that they’re heterogeneously distributed among journals.

  2. I totally agree. I frankly find it hard to believe that there isn’t a problem for at least some decent fraction of journals based on my personal experience regarding the proportion of review requests that I turn down and conversations with AEs about how difficult it is to find reviewers. I’d love to see good cross journal data on this.

    I wonder if what this data will show is that the more well regarded the journal the easier it is for them to get reviewers. I know that I almost never say no to Science/Nature/PNAS and probably accept about 50% of requests from Ecology/Amnat/Ecology Letters, but when it comes to journals that tend to have less impact on the field and less interesting content my acceptance rate drops into the low teens or single digits.

  3. Interesting hypothesis regarding journal reputation and ease of finding reviewers. Could be true. Then again, among ecology journals it’s Ecology Letters that has expressed the most worry about the ‘tragedy of the reviewer commons’. Mark McPeek at Am Nat is modestly concerned but thinks the current system is basically robust. And the ESA EiCs disagree strongly about whether or not there are problems in the system. This divergence of views even among EiCs of leading journals highlights the need for everyone to put all their data on the table.

  4. That’s a very interesting observation about the differing views among the editors at the top tier ecology journals. Definitely time to bring some serious data to bear on these questions. Interesting covariates to include: 1) period of time allotted for the review (I’m more likely to say yes if I’m given more time); 2) method for requesting reviews (as mentioned in the Nature piece); 3) impact/eigen factor or other measures of journal quality; 4) how long the journal has been around (which might impact how it is regarded and could explain why the attitude at Ecology Letters differs from that at Ecology & Am Nat).

    Best of luck getting a big chunk of the journals to look at this. Let us know if we can advertise any efforts for data collection/analysis for you. I look forward to seeing what patterns emerge from the data.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: