Laying the Groundwork for Change [Quote]
How then is it possible to modify and improve upon an academic culture populated by smart, creative individuals who are motivated by ideals more than by money, who have deep, intense interests, value substance over form, have little patience for conformity, think for themselves, do not defer to authority, and see their work not as a job but as a calling? Clearly the challenge is to find the incentives and rewards that will motivate this unique workforce to buy into desired changes and work willingly toward implementing them. But the first step is to explain clearly why change is nececessary and, even more important, why change does not mean abandoning core academic values. To win the hearts of academics, one first has to educate them.
– James C. Garland, Saving Alma Mater
This is just one of many brilliantly reasoned (and worded) arguments from Saving Alma Mater. If you are an academic, or an administrator at an academic institution, you really should read this book.
Another opportunity to change the way we collaborate – G o o g l e W a v e I n v i t e s for ecologists
The current model of writing up collaborative research in science is that a single individual “takes the lead” and writes a complete draft of the manuscript, which is then sent on to coauthors for comments, corrections, etc. This means that even when the development of the ideas and the work of research and analysis has been conducted in a truly collaborative manner (which, I suspect is actually not all that common, at least for the research and analysis parts) that the writing is really more of a one writer – multiple critics system.
This is in part due to the legacy of technology. Up until a few years ago most people simply couldn’t easily work on the same document together. In sophisticated environments the official copy of the manuscript could have been stored on a central server and individuals could “check it out” to work on it for a while, upload it when they had finished, someone else could check it out, lather, rinse, repeat. This required enough coordination that I’m not sure it was much better than just emailing the manuscript from one person to another. There was of course better tech, but scientists didn’t typically know about it, let alone use it.
These technological limitations have now been largely overcome (though there are certainly still some kinks to work out). Wikis represent the first, easiest, step to move beyond these constraints. Most wikis still only allow one person to work on a document (in this case a page) at any given time (but see this recent announcement by weecology’s prefered wiki host, PBworks), but by having the document stored on the web and editable via the browser there is no dead time for the document. You are either working on it actively or you’ve saved it and it’s available to others to work on. This reduces the size of the steps that need to be made on a paper because it wasn’t “you’re turn” to make progress and you didn’t commit to making a real contribution by checking out the paper. Even better than wikis are online collaborative editors like Google Docs or Zoho. These allow multiple individuals to work on a single document at one time. This might seem extravagant, but if two people happen to have a spare 30 minutes at the same time of day, not getting in each others way can make a big difference. Better yet, it allows you to intentionally work simultaneously. At weecology we will actually schedule writing meetings, where two or three of us will sit down in the same room with separate laptops (or an equivalent remote setup) and go to work on a paper. It’s amazing how much easier it can be to sit down and write when your whole team is doing it at the same time and it facilitates active interaction on the paper – “Hey, what do you think we should do about this part of the section I’m working on right now?”
And now, there is a new exciting set of technologies that provide yet another opportunity to move beyond the old system of collaboration – Google Wave. You can think of Wave as combining email, instant messaging, and collaborative document editing. This combination is cool enough on it’s own, but the seamlessness of the collaborative editing goes beyond anything currently available, thus removing some of the hiccups and frustrations of the current options (NB: don’t expect wave to run this smoothly yet as they are still scaling up the system).
This post was motivated by the fact that I just received my long awaited invite to try out Google Wave, which is still in private Beta. I logged in last night and did a quick search for public waves with the word ecology in them. There weren’t any. So, I am happy to announce that I have five invites to give away to practicing ecologists (that’s scientists who study ecology, not environmentalists) who want to give Google Wave a try. Leave a comment with an email and (ideally) something (like a link to a website) that demonstrates that you are an ecologist in case the swarm of folks looking for invites finds JE.
UPDATE: Two things. First, I started a public wave – Any ecologists on wave yet? – which you can find with a quick search of either ‘ecologists with:public’ or ‘ecology with:public’. Second, I forgot to warn the folks who get invited that according to Google “Invitations will not be sent immediately. We have a lot of stamps to lick.”. So it might take a while for your invitations to show up. When they do, stop by and leave a comment so that others know about how long they can expect it to take.
UPDATE 2: I’ve been reloaded with invites, so there are plenty to go around. Just leave a comment if you want one.
[Quote] You can recognize a pioneer by…
You can recognize a pioneer by the arrows in his back.
– Beverly Rubik
You can recognize a pioneer by the arrows in his back
[Quote] Nilsson on credit for scientific work
Some years ago, someone wrote a book called “The Seven Laws of Money.” One of the “laws” went something like this: “Do good work and don’t worry about money; it will come along as a side effect.” Whether or not that’s true of money, I don’t know, but in my experience, it’s true of credit for scientific work. Just make sure you keep working at important problems, enjoying a life of science, and don’t worry so much about credit. You will probably get what you deserve — as a side effect.
Nils Nilsson (via Vladimir Lifschitz)
Frequency distributions for ecologists V: Don’t let the lack of a perfect tool prevent you from asking interesting questions
I had an interesting conversation with someone the other day that made me think I needed one last frequency distribution post in order to avoid causing some people to not move forward with addressing interesting questions.
As a quantitative ecologist I spent a fair amount of time trying to figure out the best way to do things. In other words, I often want to know what the best method is available for answering a particular question. When I think I’ve figured this out I (sometimes, if I have the energy) try to communicate the best methodology more broadly to encourage good practice and accurate answers to questions of interest to ecologists. In some cases finding the best approach is fairly easy. For example, likelihood based methods for fitting and comparing simple frequency distributions are often straightforward and can be easily looked up online. However, in many cases the methodological challenges are more substantial, or the question being asked is not general enough that the methods have been worked out and clearly presented. This happens in the case of frequency distributions when one needs non-standard minimum and maximum values (a common case in ecological studies) or when one needs discrete analogs of traditionally continuous distributions. It’s not that these cases can’t be addressed, it’s just that you can’t look the solutions up on Wikipedia.
So, what is someone without a sufficient background to do (and, btw, that might be all of us if the problem is really hard or even… intractable). First, I’d recommend trying to ask for help. Talk to a statistician at your university or a quantitative colleague and see if they can help you figure things out. I am always pleased to try to help out because I always learn something in the process. Then, if that fails, just do something. Morgan and I will probably write more about this later, but please, please, please don’t let the questions you ask as an ecologists be defined by the availability of an ideal statistical methodology that is easy to implement. In the context of the current series of posts, if you are trying to do something with a more complex frequency distribution and you can’t find a solution to your problem using likelihood then use something else. If it was me I’d go with either normalized logarithmic binning or something based on the CDF as these methods can behave reasonably well. Sure, people like me may complain, but that’s fine. Just make clear that you are aware of the potential weaknesses and that you did what you did because you couldn’t figure out an appropriate alternative approach. That way you still get to make progress on the question of interest and you may motivate people to help work on developing better methods. Sure, you might be the presenting the “right” answer, but then I very much doubt that we ever are when studying ecological systems anyway.
The broken peer-review system: a reviewer’s opinion
Many of us have had the feeling that something is not right these days with the peer-review system in science. Whenever I chat with colleagues about the peer review system, two issues consistently crop up: an increasing number of review requests that we cannot possibly keep up with and/or reviews that seem to indicate a reviewer did not spend much time with the manuscript they were reviewing. So, when Ecology Letters published an article in 2008 (Hochberg et al), written by a group of its editors, titled “The tragedy of the reviewer commons”, I read with great interest. However, I was dismayed to see that apparently the entire fault for the current sad state of affairs lay with people like me: reviewers and authors. I was slightly peeved at the tone of the article that implied that things would improve if only reviewers/authors behaved better. Where was the responsibility of the journals/editors in this mess? I thought, “I really need to write a blog post on this”. I never got around to it. Since then, at conferences and in additional publications (e.g., McPeek et al 2008), I have heard the same refrains: Scientists need to review faster, better, smarter. I began to wonder if I was alone in this world in my feelings that reviewers/authors are only half of the equation. Then I read a blog article over at the Chronicle for Higher Education. This article was also about the problems with the peer-review system, but from the perspective of a reviewer/author. And I realized not only was I not alone, but that we needed more voices demanding real dialogue on this issue. So here we go: a reviewer/author’s take on how journals/editors can help reviewers/authors make journal/editors happier.
1) Better reviewer databases: I say no a lot to reviews because I say yes a lot to reviews, not because I lack a sense of scientific responsibility. The Chronicle blog (by a sociologist) points out that the number of members in the American Sociological Association is more than enough to support a reasonable number of reviews/person. However, a much smaller number of people seem to be shouldering the load. I suspect the same is true for ecology. So why is this? Undoubtedly the journals are right that there are curmudgeons who simply refuse to review. But I also suspect that editors are busy people like the rest of us and when we are busy we go with the names of people who come to mind quickly; these “go-to” people are “the most obvious people” to review a paper or give a talk. However, those go-to people are often the same for many people – resulting in the smaller number of people getting a higher load of review requests. As a reviewer I try to help with this situation by recommending people I think are not yet “in the system” (post-docs, young assistant professors, etc), but I might humbly suggest that journals invest in better reviewer databases to help editors come up with a better diversity of names.
2) More editorial control: My next two suggestions are not going to make me popular with either authors or editors. And I know (if they got implemented) I would occasionally get hoisted in my own petard, but I strongly believe that with the demands journals are making on reviewers theses days (thorough reviews, lots of reviews, quick reviews) journals have a responsibility to protect reviewers from superfluous reviews (i.e. unnecessary review requests).
a) Better pre-review vetting. Many authors will hate this because this means one person is probably deciding whether or not to send something out for review. A bad draw on an editor (who has a strong personal opinion on the validity/novelty of your work) can kill your submission. However, I am not alone in having received manuscripts for review that are so poorly written that they are in effect incomprehensible or so far from the journal’s standard that clearly no editor looked at the manuscript before sending it on to me. I’m not talking about borderline cases but manuscripts so bad I barely know how to review them. As a reviewer this just makes me mad and takes up valuable time that could have been dedicated to a manuscript that actually deserved consideration. As the Chronicle post, points out: manuscripts do not have a fundamental right to be reviewed.
b) Stop looking for reviewer consensus. I have noticed a trend at certain journals: manuscripts keep being sent back to the reviewer until the reviewer “signs off” on the manuscript. This is consistent with the idea in the Ecology Letters article that authors are needlessly lengthening the review process by ignoring reviewer comments. As much as we may all wish otherwise, not all reviewer comments reflect absolute truth. We all have our opinions on things that (if we’re being honest with ourselves) actually are in gray areas. Sometimes reviewers just flub things. And, journals are right, sometimes reviewers give shoddy reviews. As both a reviewer and an author I recognize this. As a reviewer, I assume the editor will read my review (and the paper) and decide for his or herself whether they agree with my opinion. As an author, I assume that the editor will read my response to a reviewer and decide whether my objections to a certain critique have merit. As a reviewer, the only time I want to re-review a paper is if I have labeled my concern as “fatal” and the editor is uncertain whether the authors have either dealt with that concern or have a valid argument for why it is not a concern. In a world where reviewers are scarce, manuscripts should only go back to reviewers when absolutely necessary. This requires editors to insert themselves more into the process than perhaps they have been accustomed.
Maybe journals and editors already feel like they do these things. I don’t know. I do know I feel like I already do the things they want me as a reviewer to do! However, given how widespread concern over the strain on the peer-review process is, it seems to me that perhaps it’s time for a real dialogue – and that involves both sides talking about their perspectives and making suggestions about how to improve things. Anyone out there have additional ideas for things that could be done?
Why you should use a feed reader to monitor journal table of contents
A couple of weeks ago we made it possible for folks to subscribe to JE using email. We did this because we realized that many scientists, even those who are otherwise computationally savvy, really haven’t embraced feed readers as a method of tracking information. When I wrote that post I promised to return with an argument for why you should start using a feed reader instead – so here it is. If anyone is interested in a more instructional post about how to do this then let us know in the comments.
The main argument
I’m going to base my argument on something that pretty much all practicing scientists do – keeping track of the current scientific literature by reading Tables of Contents (TOCs). Back in the dark ages the only way to get these TOCs was to either have a personal subscription to the journal or to leave the office and walk the two blocks to the library (I wonder if anyone has done a study on scientists getting fatter now that they don’t have to go to the library anymore). About a decade ago (I’m not really sure when, but this seems like it’s in the right ballpark) journals started offering email subscriptions to their TOCs. Every time a new issue was published you’d receive an email that included the titles and authors of each contribution and links to the papers (once the journal actually had the papers online of course). This made it much easier to keep track of the papers being published in a wide variety of journals by speeding up the process of determining if there was anything of interest in a given issue. While the increase in convenience of using a feed reader may not be on quite the same scale as that generated by the email TOCs, it is still fairly substantial.
The nice thing about feed readers is that they operate one item at a time. So, instead of receiving one email with 10-100 articles in it, you receive 10-100 items in your feed reader. This leads to the largest single advantage of feeds over email for tracking TOCs. You only need to process one article at a time. Just think about the last time you had 5 minutes before lunch and you decided to try to clear an email or two out of your inbox. You probably opened up a TOC email and started going through it top to bottom. If you were really lucky then maybe there were only a dozen papers and none of them were of interest and you could finish going through the email and delete it. Most of the time however there are either too many articles or you want to look at at least one so you go to the website, read the abstract, maybe download the paper, and the next thing you know it’s time for lunch and you haven’t finished going through the table so it continues to sit in your inbox. Then, of course, by the time you get back to it you probably don’t even remember where you left off and you basically have to start back at the beginning again. I don’t know about you but this process typically resulted in my having dozens of emailed TOCs lying around my inbox at any one time.
With a feed reader it’s totally different. If you have five minutes you start going through the posts for individual articles one at a time. If you have five minutes you can often clear out 5 or 10 articles (or even 50 if the feed is well tagged like PNAS’s feed), which means that you can use your small chunks of free time much more effectively for keeping up with the literature. In addition, all major feed readers allow you to ‘star’ posts – in other words you can mark them in such a way that you can go back to them later and look at them in more detail. So, instead of the old system where if you were interested in looking at a paper you had to stop going through the table of contents, go to the website, decide from the abstract if you wanted to actually look at the paper, and then either download or print a copy of the paper to look at later, with a feed reader you achieve the same thing with a one second click. This means that you can often go through a fairly large TOCs in less than 10 minutes.
Of course much of this utility depends on the journals actually providing feeds that include all of the relevent information.
Other benefits
Keeping your TOCs and other feeds outside of your email allows for greater separation of different aspects of online communication. If you monitor your email fairly continuously, the last thing you need is to receive multiple TOC emails each day that could distract you from actually getting work done. Having a separate feed reader let’s you actually decide when you want to look at this information (like in those 5 minutes gaps before lunch or at the end of the day when you’re too brain dead to do anything else).
Now that journals post many of their articles online as soon as the proofs stage is complete, it can be advantageous to know about these articles as soon as they are available. Most journal feeds do exactly this, posting a few papers at a time as they are uploaded to the online-early site.
Sharing – want to tell your friends about a cool paper you just read. You could copy the link, open a new email, paste the link and then send it on to them. Or, you could accomplish this with a single click (NB: this technology is still developing and varies among feed readers).
And then of course there are blogs
I’ve attempted to appeal to our non-feedreader-readers by focusing on a topic that they can clearly identify with. That said, the world of academic communication is rapidly expanding beyond the walls of the journal article. Blogs play an increasingly important role in scientific discourse and if you’re going to follow blogs you really need a feed reader. Why? Because while some blogs update daily (e.g., most of the blogs over at ScienceBlogs) many good blogs update at an average rate of once a week, or once a month. You don’t want to have to check the webpage of one of these blogs every day just to see if something new has been posted, so subscribe to its feed, kick back, and let the computer tell you what’s going on in the world.
A message to journal editors/managers about RSS feeds
- If you don’t have an easily accessible RSS feed available (and by easily accessible I mean in the browser’s address bar on your journal’s main page) for your journal’s Table of Contents (TOCs), there is a certain class of readers who will not keep track of you TOCs. This is because receiving this information via email is outdated and inefficient and if you are in the business of content delivery it is, at this point, incompetent for you to not have this option (it’s kind of like not having a website 10 years ago).
- If, for some technophobic reason, you refuse to have an RSS feed, then please, pretty please with suger on top, don’t hide the ability to subscribe to the TOCs behind a username/password wall. All you need is a box for people to add their email addresses to for subscribing and a prominent unsubscribe link in the emails (if you are really paranoid you can add a confirmation email with a link that needs to be followed to confirm the subscription).
- Most importantly. Please, for the love of all that is good and right in the world, DO NOT START AN RSS FEED AND THEN STOP UPDATING IT. Those individuals who track a large number of feeds in their feed readers will not notice that you stopped updating your feed for quite some time. You are losing readers when you do this.
- If you have an RSS feed that is easily accessible (congratulations, you’re ahead of many Elsevier journals) please try to maximize the amount of information it provides. There are three critical pieces of information that should be included in every TOCs feed:
- The title (you all manage to do this one OK)
- All of the authors’ names. Not just the first author. Not just the first and last author. All of the authors. Seriously, part of the decision making process when it comes to choosing whether or not to take a closer look at a paper is who the authors are. So, if you want to maximize the readership of papers, include all of the authors’ names in the RSS feed.
- The abstract. I cannot fathom why you would exclude the abstract from your feed, other than to generate click throughs to your website. Since those of you doing this (yes, Ecology, I’m talking about you) aren’t running advertising, this isn’t a good reason, since you can communicate the information just as well in the feed (and if you’re using website visits as some kind of metric, don’t worry, you can easily track how many people are subscribed to your feed as well).
If this seems a bit harsh, whiny, etc., then keep this in mind. In the last month I had over 1000 new publications come through my feed reader and another 100 or so in email tables of contents. This is an incredible amount of material just to process, let alone read. If journals want readers to pay attention to their papers it is incumbent upon them to make it as easy as possible to sort through this deluge of information and allow their readership to quickly and easily identify papers of interest. Journals that don’t do this are hurting themselves as well as their readers.
A quote and some random thoughts on plagarism
During the course of this long volume I have undoubtedly plagiarized from many sources–to use the ugly term that did not bother Shakespeare’s age. I doubt whether any criticism or cultural history has ever been written without such plagiary, which inevitably results from assimilating the contributions of your countless fellow-workers, past and present. The true function of scholarship as a society is not to stake out claims on which others must not trespass, but to provide a community of knowledge in which others may share.
-F. O. Matthiessen, American Renaissance 1941.
Thanks to academhack for pointing me to this great quote. Given the spirit of the quote I don’t think he’ll mind me reposting it. I find this to be a particularly relevant in light of recent discussion about tracking down self-plagarism and how grave an offense it may be. I’m not saying that self, and regular, plagarism aren’t serious issues. I’ve been involved in reporting a case of self-plagarism myself and it’s disturbing when you see it. It’s also clearly bad for science. It clutters the already crowded literature, has a negative influence on broader perceptions regarding the ethics of scientists, and results in undue credit (which presumably influences funding and promotion). That said, I think it’s important to keep the bigger picture in mind. We are, after all, in the business of ideas. Words are important for communicating those ideas, but the ideas themselves are the currency of interest. Our ideas are influenced by everyone we talk to, colored by every paper we’ve ever read and talk we’ve ever seen. The goal of science is (or at least should be) to progress our knowledge as rapidly as possible. I think that conversations surrounding plagarism, and what it means in this new era, should start from this core goal and proceed from there.
Amen brothers: why stimulus funding for science was a good idea
We just read this great piece from the Huffington Post by Todd Palmer and Rob Pringle on why including funds for NSF and NIH in the stimulus bill was a good idea (thanks to Ecotone for pointing us to the article). The great thing about the piece is that it doesn’t just make a cogent argument for the stimulus funds, but for why funding basic science is economically beneficial in general. Probably the high point of the article was this little gem:
Truthfully, the return on our relatively modest investment in basic research over the last half-century is so astronomical that it’s impossible to calculate. Science hasn’t just stimulated the economy; it has revolutionized the economy, and our lives along with it.
which seems like it must be hyperbole, but at least from our perspective it certainly is not. However, if we had to pick our favorite moment in the article it would definitely be the paraphrase of Paul Baskin’s concern about the utility of this funding:
Aren’t we just subsidizing a bunch of nerds who already have cushy academic jobs and buy fancy Japanese-made instruments? No.
This is definitely one of the clearest, best, and funniest explanations of why funding basic science is critical to the economy and to society in general. Go check it out.
Jabberwocky Ecology 