This form does not yet contain any fields.


    Peer-Review: Please, Tell Me It's Worth the Time and Effort To Do It Right

    Many thanks to Dave Brodbeck for inviting me to vent where he vents.

    I have more than a few ruminations to share, but only one today, and I promise not to be cynical. Really. Instead, I want to try to do something positive whenever I write for public consumption, and that includes when I'm complaining. My goal is to raise issues that affect science students and the community at large, and hopefully inspire at least a few students or researchers who read my comments to have similar discussions so that they might somehow contribute to solutions -- or at least avoid being part of some problem or another. Oh, and of course by that I mean problems, in my view.

    Today, the problem I want to discuss has to do with the peer-review process in science and research. Before I start railing, however, I want to be clear about something: I believe peer-review is currently the best way we have to apply quality control to what counts as knowledge, useful information, or sound ideas. It is not a perfect process, however, and academic folks have been debating its various problems for decades. I will make no attempt here to contribute to an academic discussion about any of the oft-mentioned limitations and imperfections of the peer-review process. Instead, I'm going to gripe a bit about something I encounter, from time to time, when I serve as a reviewer for some science journal editors.

    As a university professor with an active research program, I spend a lot of time participating in the peer-review process. I am often asked to review research reports that have been submitted for publication in scientific journals, or grant applications that other researchers have submitted to some funding agency or another, in either Canada, the U.S., or the U.K. Serving as a peer-reviewer is something that one normally does on a purely voluntary basis (although I recently received a small honorarium for reviewing a grant application for a special award competition -- that was the first time, and it wasn't necessary, but I took it without any hesitation or guilt). I don't agree to serve as a reviewer every time I am asked. I receive about 3 or 4 requests from journal editors each month, and I accept about two-thirds of them; I decline to review when I am unable to give a manuscript the time and attention it deserves because of other obligations. I accept all requests to review research grant applications.

    I consider service as a peer-reviewer to be an extremely important part of my job, even though it is all voluntary, and I receive absolutely no tangible reward or recognition for this service. Except in rare cases, reviewers are anonymous and only the journal editor knows who provided the reviews. In most academic departments within a large university, no one really has any idea how much time a colleague is spending on peer review. It is not normally listed in a c.v. (although it could be, perhaps should be, and I am sure some people include it). Deans and department heads normally have no idea how much a particular faculty member contributes to peer review, and most of them probably do not care very much, as this activity does not bring any obvious returns to the department or the university. Only my wife and I really know how much time I spend on it. So, peer-review is kind of like making an anonymous donation to some charity -- you do it because you want to help, you know its the right thing to do, and you don't need any recognition or payback for doing it.

    My guess is that a lot of my colleagues get about the same number of requests to review as I do, and that some get more and some get fewer. I would also guess that, like me, many of my colleagues refuse some, but accept most invitations to review work submitted for publication in their areas of expertise. There is probably a lot of variability, though. It's also my guess that, like me, most of my colleagues also readily agree to review grant applications whenever they are asked to do so. I'm just guessing, but I'd bet that most of them also take the responsibility of peer-review seriously, and that the majority of them do an excellent job on a consistent basis.

    Today, my rant is about those other peers of mine who agree to review, but then do a lousy job of it, primarily because they just aren't trying hard enough and putting in the time needed to be thorough. In my opinion, the only good reason for wanting to be a reviewer is because you have intrinsic motivation to help assure quality and integrity of the data, and the scientific soundness of the arguments, that get published. What other justification is there? Yet, sometimes I get the feeling that some people have different reasons for agreeing to review for science journal editors. I mean, if you really care about something enough to repeatedly do it voluntarily and anonymously, then wouldn't you want to spend as much time on it as needed to get it done properly?

    Here is what set me off, and made me decide to write about this topic: A couple of weeks ago, I spent the better part of two days working on my evaluation and review of a manuscript that had been submitted for publication in the highly-regarded journal, Brain Research. The manuscript was about 40 pages (near the average length), and it was a general review of previous research on an interesting and somewhat important subject. Two days was not an unusual amount of the time for me to spend evaluating and writing my recommendations for a paper like this one. I take the responsibility very seriously -- after all, once a paper is published, it becomes part of the public domain of knowledge for posterity. Millions upon millions of taxpayer dollars are spent to support scientific research, and this is one of the main reasons why I care about the integrity of the "facts" that are reported by other scientists, and the soundness of the interpretations and theories they put forth. I even feel somewhat honoured that I have this privilege to contribute to something that I think is important, and I want to do my best job of it at all times, using my most unbiased and objective scrutiny. Editors typically give reviewers two or three weeks to evaluate a manuscript and submit a report that indicates the reviewer's disposition and concerns, requests for changes, etc, concerning the manuscript. I almost always submit my review a few days after the deadline. I seem to need the sense of urgency I get from the courtesy email reminder from the editor that the deadline is looming. I procrastinate, but when I eventually start the task, I put most everything else aside until it is done. Some times of year are a bit busier than others (grant review season), but averaged over the whole year, I estimate that I spend about 8 - 10 hours on peer-review in a typical week. That's a little more than one full workday, for people with a regular 9-5 job. I think its worth it. I have no doubt that authors and editors are happy to know someone is willing to put in the time.

    For this manuscript, the editor sought three reviewers. As is usually the case, after the editor received all three reviews, a decision was made about the fate of the manuscript, and there was an opportunity for each of the three reviewers to see the decision letter, and to view the other two reviews.

    To be blunt, I was pissed when I saw the other two reviews. Both consisted of a point-form list of comments, none of which reflected any kind of deep analysis. Most of the comments pointed out minor errors, or asked for clarification on some tangential point or issue. Both seemed like a list of comments that one might make while reading through a manuscript for the first time; you know, short notes on all the obvious shortcomings that jump out at you, like inconsistent arguments, errors of logic, certain typos, or bad grammar.

    Its not that any of the comments from the other reviewers were off-base -- in fact, I didn't see any that I disagreed with. Both reviewers also gave pretty clear directions about what the authors needed to do to address their concerns. (That last part is really important. It can sometimes be frustrating for authors when a reviewer has a problem with something in a manuscript, but does not say what he or she believes should be done to fix it). But I really doubt that either of the other two reviewers read through the manuscript a second time, let alone a third time, or that either of them checked the references to see if there were any obvious omissions, or checked to see that important previous work was being cited properly, or stepped back to consider whether the paper actually accomplished the objectives it stated at the outset, or whether it was an original analysis versus a run-of-the mill literature review. These bases take a lot of time to cover, but if the peer-review process does not do it, they will not be covered. There are many other big-picture analyses or beneath-the-surface assessments that a reviewer can try to accomplish when evaluating a manuscript. No individual reviewer can be expected to undertake more than a few of them, but at least a few should be expected and delivered.

    Again, I want to be clear -- I'm not disrespecting the intelligence or abilities of my peer experts. I don't think very many of them are actually unqualified to evaluate the scientific work of others. Frankly, I think the problem is just a combination of laziness, lack of pride in good work, and in some cases, a dishonourable reason for wanting to do the job in the first place. I wonder…, Does anyone ever agree to help editors with the review process with the hope that it will make that editor more likely to be positively disposed toward their work if they submit it to the same journal? Do people ever agree to review so they can be in a position to quash any data or arguments that challenge their own conclusions? Can any of them actually have such contempt for the noble pursuit of scientific knowledge? Absolutely, and without a doubt, some of my "peers" are just that dastardly in their thoughts and deeds. I know there are jerks and bastards out there. I have met many of them over the years.

    So, what can be done? I sure don't know. But, by talking about these kinds of things with my graduate students, I hope to instill in them a sense of responsibility and a desire to keep their integrity while their careers develop and evolve. This way, I hope, they are likely to do a good job of providing peer-review when the time comes for them to contribute. Other than that, I just wish more editors had the balls to give reviewers frank feedback on the quality of the reviews they submit. Maybe some of the editors don't care enough? You know, I have some thoughts about that one.

    If enough people want (enough is only 2 or 3), I'll write again, next time about why I believe science and research in the academic world has systemic flaws that encourage certain bad behaviors by scientists and researchers. Some of the flaws in the system can even explain the half-assed or disingenuous peer-review that I've been complaining about. But even worse, the problems in the system account for a great deal of squandering of taxpayers money -- millions of tax dollars that are allocated to support scientific research. The industry is such that there is an incredible amount of waste and inefficient expenditure, and a lot of money is used in ways that aim to benefit some peoples' research careers more than they aim to benefit science.

    Actually, I think I'll just keep writing about this stuff whether or not anyone else is reading it. It's like therapy.

    Dave Mumby is behavioral neuroscientist and a professor in the psychology department at Concordia University in Montréal. He and his students study memory and brain functions. Dave is an academic advisor, and the author of Graduate School: Winning Strategies for Getting In. He also has a blog and is a frequent contributor to, a website that helps undergraduate students prepare and apply successfully to graduate school.


    Some Myths About University Professors....

    The school year is set to start here in North America, and students are gearing up for another year.  So are support staff and faculty.  I have not been a student for quite a while, and I have never been a member of the support staff, so I will talk about what it is like from our side of the podium.

    There are a number of myths about profs and our work, so I thought why not write a post about that?  Plus, who the hell are you to stop me from posting on my own blog?  OK, you weren't, sorry....

    There are no stupid questions

    You might hear one of us say this in a classroom, usually on the first day, to encourage participation.  You know though, there certainly are stupid questions.  Most of us, however, will not make you feel like you asked a stupid question, so ask away.  I have had to, once in my whole career, ask someone (after class, privately) not to ask a question every time a thought came to his head.

    Profs don't care about teaching

    OK, I hear this one a lot.  "They don't care about us, they just want to be in the lab/library or whatever".  Yeah sure, there are some people like this.  But, in my experience we all like teaching.  Research takes a long time, and the personal sense of satisfaction that comes from it takes even longer.  The sense of satisfaction that comes with telling someone something they don't know, that is an amazing feeling.

    Their teacher ratings don't matter

    This one is complete bullshit.  I have sat on tenure and promotion committees at two universities, and we always look at evaluations.  We look at student evaluations of teaching along with research contributions.  Teaching will have more or less weight than research depending on the school, but, if you cannot teach, you will have trouble getting promoted and getting tenure.  The same holds true for research.  

    He only teaches six hours a week, boy a job where I was paid for six hours would be sweet...

    This one shows a fundamental misunderstanding of what we do for a living.  As I mentioned above, we teach, and most of us love it (hell, some people hate their jobs, no matter what they are).  However, we also have to do research (most of us love that too).  We have to do original stuff, sort of like big frakking term papers, but, more involved, and usually more original.  Finally, we have to do soul destroying committee work.  This is a cool job, and one of the things we do is help run where we work, hence the committees.  There are people that like committee work, I don't understand these people (and I'm a psychologist) but, they do exist....

    Tenure makes people lazy

    Tenure is simply permanence.  We have a probationary period in our jobs, just like any other job.  A typical probationary period at a company might be say 90 days.  Ours is five or six YEARS.  We come up for review every year or two and are told what we are doing right, and what we are doing wrong.  Most of us get tenure, but it is not a sure thing.  Oh, and tenured faculty can be fired.  It has to be for cause, but it happens.  That said, the weeding out process during the probationary period usually ensures that we are all pretty decent at our jobs.  Now are there people that do sweet dick all after they are tenured, hell yes.  They are, however, quite rare.

    Anyway, those are the ones I could think of now, ok, back to making up course outlines....


    Spit and Twitches: The Animal Cognition Podcast - Pilot Episode

    Well here it is, the much anticipated, ok, not anticipated much, but well you know, the pilot episode is here!  I sat down this afternoon with Laurie Bloomfield of the Psychology Department at Algoma University here in Sault Ste. Marie.  We talked about her specialty, bird communication. Photo credit: Oregon State University ArchivesLaurie studies chickadees mostly, but of course, as we all know, it is not the species that is important, it is the problem.  (Or, as my PhD supervisor once said to an ornithologist, "I don't study species, I study problems").  Well, Laurie and I talked about a lot of stuff, her career, how she started out in animal cognition, and about fist fights between eminent researchers.  (That last part came in the final bit, where I asked her four questions, an idea I have stolen borrowed from FourCast).  

    I would really like feedback on the show, what do you think?  Feel free to comment here, on facebook, google + or whatever.  

    I really appreciate Laurie sitting down with me and talking today.  I had a really good time.  

    The music in this episode is by Battery Life and is called "Beer Rights"

    download mp3




    Words I Hate

    I understand that languages evolve, and that new words get invented to deal with new stuff, and I have no problem with that.  What I do have a problem with is imprecise or innaccurate language.

    As a psychologist, I truly despise when new 'disorders' are invented that are not disorders at all, but just being an asshole.  So, this post is inspired by those two things.

    I guess the two current buzzwords that I hate the most are homophobia and islamaphobia.  Now, I know what both of these mean, as do you.  What pisses me off is the use of the term 'phobia'.  Homophobia is hatred of LGBT people, and Islamaphobia is hatred of Muslims.  THESE ARE NOT PHOBIAS.  A phobia is an irrational fear, a real psychological problem, and they can be debilitating.  

    Islamaphobia and homophobia are simply bigotry.  They are simply hatred.  These are not disorders, they are being a bigoted asshole.  That is not a disorder.  Why in the hell can't we just use the word 'bigot'?  Perhaps we could come up with a better term, anti-Semtisim is not called 'Jewphobia' or something. 

    Now, I don't think (or I certainly don't hope) that most people thinnk of these two things as disorders, but the words themselves imply that they are disorders.  I also am not naive enough to believe those words will change.  Anyway, as usual, I am starting to ramble.....


    Lecture Notes for Fall 2011 Available At

    While my home for my lectures is of course still active, I thought I would also post the lecture notes/powerpoint presentations here.  So, you can see both Brain and Behaviour and Animal Behaviour tabs at the top of the site.  Even if you are not in the class, feel free to download the lecture notes, and the lecture audio when they come out in the fall.