Casting a Jaundiced Eye on Peer Review
Casting a Jaundiced Eye on Peer Review
Recently a colleague who had read one of my blogs on Cancer Network asked me why I was “wasting my time” writing pieces that are not peer-reviewed.* He followed up by reciting for me the many purported benefits of peer-reviewed publication, including maintenance of scientific rigor and academic credit.
On reflection, I believe he asked a reasonable question, deserving of an answer. As background, I should comment that I have a good amount of experience on both sides of the peer review divide, having published approximately 75 peer-reviewed articles and having served as an editorial reviewer for about the same number of manuscripts written by others. This experience has jaundiced my view of the peer review process.
The first obvious problem with peer review is delay. I have spent as long as 2 years working to get a manuscript through editorial review and into print. Part of the delay is with reviewers who fail to meet deadlines, but this appears to be improving in recent years as editors have gotten tougher in demanding timely review. (I confess that I have not always served as a role model in this respect.) Delays are problematic. A first report might lose priority. Patients might not get the benefit of a new treatment until it is too late.
If delays resulted in higher quality and objectivity of review, it might represent an adequate trade-off, but my experience tells me that this is seldom the case. On more than one occasion, questions and objections from reviewers have mystified me. Had they even read the manuscript? On the same and other occasions, I observed what appeared to be strong and persistent bias on the part of reviewers that ultimately led to rejection or intrusive revision of the manuscript. Reviewer bias is obviously much more of a problem when the author is presenting new information that challenges current paradigms. On other occasions, reviewers did not have strong objections to the content of the manuscript but complained that questions they were interested in were not addressed. In such circumstances, the author can either revise the manuscript to write the article the reviewer wants to read, or send the manuscript elsewhere. My former colleague Rod Schwarz advises never accepting rejection. Just send the manuscript on to the next journal in line, and so on, ad infinitum, until it is ultimately accepted. Each such submission, however, requires time and effort in revisions in style and construction, as well as time-consuming communication with co-authors.
In the worst circumstances, review processes appear to be improper. We have had a manuscript rejected from a society journal by an editor who did not submit it to reviewers, because he personally disagreed with the conclusion—despite the fact that the material had been selected for an oral report to the same society’s annual meeting. A letter of complaint to the editor-in-chief stating that we had not been provided the level of editorial review promised in the submission guidelines went unanswered.
My most frustrating experience to date involved a reviewer for a European journal who raised 10 major objections and advised rejection of the article. The other reviewer had questions and critiques but would accept a revised manuscript. I wrote to the editor and explained that I could easily identify the reviewer from the tenor of his comments, since I had acrimonious debate in print with him on prior occasions. I suggested that it was improper for him not to have revealed this prior conflict to the editor—as I had done when requested to review his work. I then wrote a rebuttal to his objections that was approximately three times the length of the original manuscript and returned it with a letter to the editor, explaining that I was certain that this reviewer would next provide a list of 10 new objections, and that I would continue to respond to each such rejection response seriatim. My prediction proved accurate. After a second round of objections, and even more lengthy responses, the editor finally made the decision to publish over the reviewer’s objections. This was a satisfying but Pyrrhic victory, considering the many months and enormous amounts of effort and frustration involved. I am quite certain that a younger, less stubborn author would have been defeated by such tactics.