Category Archives: publishing

Michael Nielsen on the importance and value of Open Science

We are pretty excited about what modern technology can do for science and in particular the potential for increasingly rapid sharing of, and collaboration on, data and ideas. It’s the big picture that explains why we like to blog, tweet, publish data and code, and we’ve benefited greatly from others who do the same. So, when we saw this great talk by Michael Nielsen about Open Science, we just had to share.

(via, appropriately enough, @gvwilson and @TEDxWaterloo on Twitter)

Oikos has a blog? [Blogrolling]

Thanks to an email from Jeremy Fox I just found out that Oikos has started a blog. It clearly isn’t on most folks radars (I represent 50% of its Google Reader subscribers), and Jeremy has been putting up some really interesting posts over there so I thought it was worth a mention. According to Jeremy:

I view the Oikos blog as a place where the Oikos editors can try to do the sort of wonderful armchair ecology that John [Lawton] used to do in his ‘View From the Park’ column. I say ‘try’ because I doubt any of us could live up to John’s high standard (I’m sure I don’t!). I’m going to try to do posts that will be thought-provoking for students in particular. Oikos used to be the place to go with interesting, provocative ideas that were well worth publishing even if they were a bit off the wall or not totally correct. It’s our hope (well, my hope anyway) that this blog will become one way for Oikos to reclaim that niche.

I think they’re doing a pretty good job of accomplishing their goal, so go check out recent posts on the importance of hand waving and synthesizing ecology, and then think about subscribing to keep up on the new provocative things they’re up to.

A GitHub of Science? [Things you should read]

There is an excellent post on open science, prestige economies, and the social web over at Marciovm’s posterous*. For those of you who aren’t insanely nerdy** GitHub is… well… let’s just call it a very impressive collaborative tool for developing and sharing software***. But don’t worry, you don’t need to spend your days tied to a computer or have any interest in writing your own software to enjoy gems like:

Evangelists for Open Science should focus on promoting new, post-publication prestige metrics that will properly incentivize scientists to focus on the utility of their work, which will allow them to start worrying less about publishing in the right journals.

Thanks to Carl Boettiger for pointing me to the post. It’s definitely worth reading in its entirety.

_______________________________________________________

*A blog I’d never heard of before, but I subscribed to it’s RSS feed before I’d even finished the entire post.

**As far as biologists go. And, yes, when I say “insanely nerdy” I do mean it as a complement.

***For those interested in slightly more detail it’s a social application wrapped around the popular distributed version control system named Git. Kind of like Sourceforge on steroids.

Putting the first back in first author

UPDATE: As of April 2012 Wiley has now changed their feeds to include the full list of authors. Thanks to Brady Allred for letting us know.

An open letter to John Wiley & Sons Inc.

Dear Wiley,

I like a lot of things that you do, but a few months ago you quietly changed your RSS feeds in a way that is both disrespectful and frankly not good for your business. You started including only the last author’s name in the RSS feed. This is bad idea for three reasons:

  1. It shows a complete lack of respect for (or understanding of) a number of scientific disciplines that do not have a strong last author tradition (including ecology; a field in which you publish a large proportion of the journals). If you do this for a paper from my field then most of the time you are publishing the name of the least significant contributor.
  2. Even in disciplines (or labs) where there is a last author tradition, not including the name of the (often junior) person who did most of the work is just disrespectful. Yes, maybe you’ll attract more click-throughs with a more senior name, but the goal in scientific fields has always been to provide credit where credit is due and you are failing to honor that tradition.
  3. Finally (and worst of all from your perspective), you are costing yourself readers. One of the considerations that I make when deciding to read a paper is based on who the authors are. At least in fields like mine I will rarely see a name associated with a paper that is meaningful to me since the last author may well be an undergraduate or a tech.

In case you think this is just one person’s opinion we took a quick informal poll a little while ago. Of 37 respondents 100% agree that if you are going to list a single author’s name with a paper it should be the first author.

So please, either switch back to using the first author’s name or, better yet, actually list the entire author line. Seeing someone’s name whose work we respect will encourage us to click-through to the paper regardless of where that name occurs in the author line.

Regards,

Ethan White (and the readers of Jabberwocky Ecology)

P.S. Also, we know that the RSS feed includes the abstract. We don’t need it in large, bold, capitalized letters at the top of every feed.

Senior authorship in ecology? Maybe.

A couple of months ago we took a poll to see what the attitudes of ecologists were towards the concept of senior authorship. Twenty-seven folks (including the two of us) weighed in (thanks everyone!) and here are the results:

Of those who where confident that the field was in one state or the other, twice as many thought that the concept of senior authorship did not apply in ecology. However, when including the not sure results it became a tie between those who thought senior authorship applied and those who didn’t. This is unfortunately the worst possible situation that the field could be in since it leads to inconsistent perceptions regarding the meaning of author lines and therefore inconsistent evaluation of scientific contributions.

So, what should we do? One thing that we could do  to help clarify things is to follow the advice of Tscharntke et al. 2007 to include (in the Acknowledgements section of the paper) a description of which of the methods of author ordering where used for each paper. This won’t change the impression of someone giving a CV a quick skim, but it provides a clear record of contribution that can be looked at in cases where a more precise understanding of contributions is warranted (e.g., tenure decisions, evaluating job packets once the committee has narrowed things down to a short list, etc.). Beyond that I’m not sure what can be done. I’d love to here suggestions if you have them.

I’d also like to take one more poll to help understand our reaction as a field to Wiley-Blackwell’s recent decision to include the last author as the only named author in their RSS feeds.

Thanks again to everyone who filled out the first poll and commented on the previous post. It’s great that we are starting to be able to have conversations like this with folks outside of our own labs and departments.

Science 2.0 [Things you should read]

I’ve read two great posts in the last couple of days that highlight what the recent debate over the the possibility of ‘arsenic based life’ has shown about how scientists are leveraging the modern web to quickly evaluate, discuss and improve science.

Marc Cadotte, Nicholas Mirotchnick and Caroline Tucker have a great post over at EEB & flow that will fill in the background for you. They use this example as a rallying cry to encourage the use of this new technology to improve the scientific process:

Academics should be the first, not the last, to adopt new communication tools. We are no longer limited by the postal service, email or PDFs; the web has gone 2.0 and we should follow suite. So go forth, young researchers, and blog, edit and share. And then go tweet about it all so your eight year-old kid knows how hip you are.

RealClimate’s piece is more focused on what this recent debate proves about the self-correcting nature of science and thus the inherent lack of vast scientific conspiracies related to things like climate change:

The arseno-DNA episode has displayed this process in full public view. If anything, this incident has demonstrated the credibility of scientists, and should promote public confidence in the scientific establishment.

It ends with a pleasantly sophisticated take* on the complexities of doing science in a world of immediate responses that can occur from across all corners of the web.

They are both definitely worth the read and may well inspire you to run out and join the online dialog.

*In stark contrast to this piece in Nature.

No peer review crisis after all?

We’ve had a bit of discussion here at JE about potential solutions to the tragedy of the reviewer commons, so I found a recent letter in Nature (warning – it’s behind a pay wall) suggesting that there may not actually be a problem interesting. The take home message is:

At the journal Molecular Ecology, we find little evidence for the common belief that the peer-review system is overburdened by the rising tide of submissions.

and the authors base this conclusion on some basic statistics about the number of review requests required to obtain a reviewer and the average number of authors and reviewers for each paper. It’s not exactly the kind of hard, convincing data that will formally answer the question of whether there is a problem, but it’s interesting to hear that at least one journal’s editorial group isn’t particularly concerned about this supposedly impending disaster.

Has the era of senior authorship come to ecology?

Senior authorship is the practice whereby the last position on an author line is occupied by the leader of the lab in which the project was conducted (i.e., the P.I., the advisor, whatever terminology you prefer). Being the senior author on a paper is considered a sign of leadership on the project and is arguably at least as prestigious as being the first author. This practice is commonplace (i.e., practically required) in the cellular, molecular & biomedical fields, and is becoming increasingly prevalent in ecology.

Nearly two years ago I suggested that the idea of using the last position on an author line to indicate the “senior author” was bad for collaborative, interdisciplinary, fields such as ecology. While I still believe this to be true I’m wondering if this is a battle that has already been quietly fought and lost. I’ve seen more and more examples of labs that are using this senior authorship model (i.e., the advisor is always in last place on the author line and presumably not because they always make the smallest contribution) and just in the last few weeks I’ve noticed that Wiley’s RSS feeds no longer even list the first author of the paper, just the last author. So, I thought I’d ask you (and any of your friends you’d like to forward this to) what you thought so that folks starting their own labs (including me) can get a feel for what the field’s take on last authorship is.

Feel free to discuss further in the comments.

UPDATE: Corrected Freudian slip in the title.

Why PubCred is both easy to implement and better than real money [Guest post]

After posting about PubCreds I emailed the authors of the original article to invite a response because: 1) it’s only fair if you’re going to criticize someone’s idea to give them a chance to defend it; and 2) I think that the blogosphere is actually the ideal place to have these kinds of discussions because unlike journals it is actually designed to allow for… well… discussions. Below follows a guest post by Jeremy Fox & Owen Petchey. My thanks to Jeremy and Owen for taking the time to respond. Enjoy.

First, thanks to Ethan for a very thoughtful post on Pub Creds. This kind of constructive criticism is actually more welcome and valuable than unreserved praise. Thanks also to Ethan for inviting Owen and I to respond. Owen and I have chatted about our response, and I’ve taken the lead on actually writing it.
Read the rest of this entry

PubCreds: Paying for journal submissions with reviews… or maybe we should just use money

The peer review system has recently been under increasing pressure as the number of papers submitted has been skyrocketing. Jeremy Fox and Owen Petchey have recently proposed a new system for fixing this, so called, “tragedy of the reviewer commons.” The crux of the argument is that for every paper a researcher submits they must review three papers in exchange (thus balancing the review load imposed by each submitted paper). A centralized PubCred bank would keep track of reviews, submissions and the balance of credits for each researcher.

At first look this seems like kind of a cool idea and I’ve seen a recent surge of interest in it via email and an enthusiatic post over at i’m a chordata urochordata. However, there are, as I see it, two major challenges for this type of system. The first is that in order to make it function properly there have to be a bunch of detailed rules in place for special circumstances. The authors of the proposal address some of these and acknowledge that there will be others†. But who should make these rules? Certainly we won’t all agree on the best solutions (e.g., I think that forcing reviewers to rereview manuscripts without additional credit, as proposed, is dangerous and likely to lead to increasingly poor editorial practice*), so who decides. I,for one, would be loathe to hand this responsibility off to the publishers‡, so I guess we’ll need some sort of council of researchers, from across a breadth of disciplines and countries, preferably elected in some sort of democratic process and then they can meet and vote on the rules. That sounds difficult to setup and organize, but we are talking about a group that is going to control a major aspect of the scientific process, so we’d better do it right.

The other major challenge is setting up the actual technical aspects of the system. Fox & Petchey suggest that given currently available web technology that the basic setup should be no more than three person-months, and that sounds about right to me for the basic site. But it ignores some important complexities. The most serious of these is the lack of a universal author identification system. There are tens (if not hundreds) of thousands of individuals contributing to the writing and review of papers across disciplines and this will lead to numerous instances where authors/reviewers have similar/identical names. There are initiatives underway to address this problem (primarily motivated by search issues), but none of them is complete and we are probably years away from an agreed upon standard within disciplines (let alone among them). Until such a system becomes established it is difficult to understand how PubCreds could properly operate. I suppose the PubCred system could try to take on this responsibility itself, but I suspect that the political realities of numerous groups competing to provide this service will make that complicated.  In addition we would presumably need to consider a secure solution for validating payments from non-lead authors (in the proposal authors are allowed to split up the “cost” of review however they deem appropriate). If this is really going to be such a valuable currency as to solve the reviewer issue then it will be valuable enough to generate unseemly behavior. Maybe a simple email confirmation process would suffice, but we need something to prevent the lead author from unilaterally deciding on how to divide up the cost. Regardless, my point is simply that while the basic system is easy, if we are going to use this to literally govern whether or not a (potentially important) scientific paper can be submitted, then the system needs to be about as robust as a banking system, and accounting for complex contingencies and putting together appropriate security makes this quite a bit more than a 3 person-month job.

So I guess we’d better get to work, because to do this properly is going to take a lot of organization and some serious effort. Or, we could just “privatize the reviewer commons” in exactly the same way we “privatize” everything else. We could use money. This has already been proposed quite eloquently in an editorial by Lonnie Aarssen (and he even implemented this idea for a while at his new journal – IEE; see also the follow up editorial) that we’ve discussed here before. The current proposal discounts this possibility because:

…a fee-to-submit system would disadvantage authors who lack the means to pay, might require exorbitant payments in order to attract referees who would not otherwise agree to serve, likely would cause authors to avoid journals charging submission fees, and would require frequent currency exchange due to the international nature of science.

I consider the first three points to be logically flawed relative to the proposed system for the following reasons (in the same order as the original objections):

  1. We’re all broke in PubCred land. We all start with zero credits and have to earn enough to submit manuscripts. If we replace credits with a fixed payment – fixed fee system where each reviewer is paid one third of the cost of a submission for each review then this is exactly the same situation as if I have $0 in my bank account. I have to review 3 papers to have earned enough money to submit one.
  2. It doesn’t matter how high these numbers have to get because they are offset by the cost of submitting a paper.
  3. Just like the proposed PubCred system, this only works if a large number of powerful journals are involved in a coordinated manner. Clearly having a small number of journals implement either system will lead to authors simply avoiding those journals (as happened to IEE when it tried implementing the money based system).

So, it seems to me that a simple logical substitution of dollars for credits negates all but one of the supposed objections to a monetarily based system. The final point related to currency exchange simply seems inconsequential.

In addition to being more straightforward than implementing a new PubCred system, I think that a monetary approach has an additional advantage. It allows the market to operate on the peer review system. I’m sure that I haven’t even begun to imagine all of the ways that the market could influence peer review, but here’s a short list of things that come to mind:

  1. Great reviewers could be rewarded more than mediocre reviewers. PubCred treats reviews dichotomously. They are either good enough for credit or not. But we all know that reviews and reviewers don’t just fall into two groups, so why not reward reviewers on a sliding scale. Each journal can keep track of the quality of reviews and use that information to decide how much to offer a reviewer to entice them to review.
  2. Journals that want papers reviewed faster can offer higher payments to entice reviewers.
  3. Top journals that want more reviewers can charge higher submission fees to cover the expense.
  4. Down the road this potentially provides an avenue for appropriately charging for-profit journals for the massive amount of… free… labor upon which they rely to make large profits.
  5. Funding agencies and universities could potentially stop funding publication costs.

In conclusion I should say that I am super impressed with Fox & Petchey for being some of the first folks out there to actually put forth a serious suggestion for fixing the current problems with peer review and I think that they have (with an appropriately long lead-time and substantial up-front investment) come up with a system that would actual work. It’s just that overall it seems like there is a much simpler approach available. Take their approach, replace each credit with a fixed number of dollars (to start), and as a result get rid of all of the decision making and infrastructure.

UPDATE: Owen Petchey’s name is now spelled correctly. Sorry Owen.

——————————————————————————–

Even the most basic rule of a 3:1 ratio of reviews to submissions seems like it should be a topic of discussion. What about journals like Science and Nature that due to an abundance of caution often get 3-5 reviews on a manuscript instead of the standard 2. Because the current proposal does not allow different journals to charge different numbers of credits for submissions or provide less credit for reviews, journals that utilize more reviewers will put a burden on the system (NB: editors also receive credit for managing manuscripts so the 3:1 ratio is really appropriate for a standard 2 reviewer system). So, should the ratio be increased to 4:1 or 5:1 or should journals be given flexibility with regards to credits and/or payments?

*We here at JE have noticed an increasing trend of late in the number of re-reviews requested and an apparent unwillingness on the part of some editors to take the time to evaluate whether the changes recommended by the reviewer have been provided. Instead they simply keep sending the paper back to the original reviewers until they have no comments left. This slows down the system, wastes reviewer time and motivation, frustrates authors, and under the proposed system there is no disincentive to stop editors from doing this ad nauseum – the reviewer has no recourse because if they don’t complete the potentially never ending re-reviews they receive no credit.

‡Who are in most cases motivated more by profit margins than the good of science. This is perfectly reasonable given that in the vast majority of cases they are private corporations, but it means that we don’t want them being the ones who are making critical decisions that would have large impacts on the scientific process.

Follow

Get every new post delivered to your Inbox.

Join 575 other followers