| Nature 422, 259 - 261 (20 March 2003); doi:10.1038/422259a |
|
PETER A. LAWRENCE
Authors, reviewers and editors must act to protect the quality of research.
Listen. All over the world scientists are fretting. It is night in London and Deborah Dormouse is unable to sleep. She can't decide whether, after four weeks of anxious waiting, it would be counterproductive to call a Nature editor about her manuscript. In the sunlight in Sydney, Wayne Wombat is furious that his student's article was rejected by Science and is taking revenge on similar work he is reviewing for Cell. In San Diego, Melissa Mariposa reads that her article submitted to Current Biology will be reconsidered, but only if it is cut in half. Against her better judgement, she steels herself to throw out some key data and oversimplify the conclusions — her postdoc needs this journal on his CV or he will lose a point in the Spanish league, and that job in Madrid will go instead to Mar Maradona.
The decision about publication of a paper is the result of interaction between authors, editors and reviewers. Scientists are increasingly desperate to publish in a few top journals and are wasting time and energy manipulating their manuscripts and courting editors. As a result, the objective presentation of work, the accessibility of articles and the quality of research itself are being compromised.
One main cause
These trends are fuelled by the increasing pressure in biomedical science to publish in the leading journals. Even our language reflects this obsession — we say that Jim Jargon did well as a graduate student because he published a "Cell paper", illustrating that we now consider the journal to be more important than the scientific message. If we publish in a top journal we have arrived, if we don't we haven't.
Why has this happened? It is partly because, rather than assessing the research itself, those who distribute the money and positions now evaluate scientists by performance indicators (it is much easier to tot up some figures than to think seriously about what a person has achieved). Managers are stealing power from scientists and building an accountability culture that "aims at ever more perfect administrative control of institutional and professional life"1. The result is an "audit society"2, in which each indicator is invested with a specious accuracy and becomes an end in itself.
Evaluations of scientists depend on numbers of papers, positions in lists of authors, and journals' impact factors. In Japan, Spain and elsewhere, such assessments have reached formulaic precision. But bureaucrats are not wholly responsible for these changes — we scientists have enthusiastically colluded. What began as someone else's measure has become our (own) goal. Although there are good reasons for publishing papers where they are more likely to be read, when we give the journal priority over the science we turn ourselves into philistines in our own world.
Some scientists realize this, but why have most taken up the journal chase so enthusiastically? It has to do with both psychology and careerism. Young researchers see a paper in a good journal as their initiation into the scientific élite. The established seek publication in leading journals to certify their high opinion of themselves. All are learning that building capital in the hard currency of the audit society can be safer and easier than founding a reputation on discoveries. Another factor is that contemporary society has a craze for publicity, to which scientists are not immune. Many are gratified to find themselves or their work reported (accurately or not) in the media, and leading journals provide a route through press releases. El País, for example, usually features articles about any work by Spanish scientists published in Nature, Cell or Science.
Consequences
There are consequences for authors, editors and reviewers.
Authors have to decide when and how to write up their work. The ideal time is when a piece of research is finished and can carry a convincing message, but in reality it is often submitted at the earliest possible moment (two papers count for twice as much as one, never mind if the second paper mainly corrects errors in the first). Findings are sliced as thin as salami and submitted to different journals to produce more papers.
Work must be rushed out to minimize the danger of being scooped — top journals will not consider a paper if a similar result has appeared in a competing journal, even if the experiments have taken years and there is only a week or two of disparity. Yet it can be advantageous if rival papers are submitted at the same time, as each author can use the other paper to tempt editors into concluding that the topic is a hot one. This practice has led to many dangerous liaisons between competing groups. It is no wonder that agonizing over presentation as well as the timing of submission keeps many scientists awake at night.
Authors need to decide how to get their paper into a top journal. Can the results be hyped to make them look more topical? Are there some trendy stock phrases that can be used3? Would oversimplification add to the appeal? Could a lofty take-home message be made to fit? Can even a tenuous link to a human disease be found? (Mention of a human disease boosts the number of subsequent references to the paper and can make it more attractive to a journal.) Can the results be squeezed into a shorter format than they require? For example, can they be submitted as a brief Letter to Nature — even though a longer paper in a more specialized journal would be of greater service to readers? Letters to Nature and Reports in Science are often presented in such a compressed form with such minuscule figures that they can be hard to decipher. Supplementary online material may alleviate this problem, although readers of print editions may not find it convenient to look at, and people have concerns about the length of its electronic shelf-life.
Increasingly, such a high premium is put on presentation that the leader of a group (who has not done the experiments) writes the paper reporting work done by a junior scientist (who has). The team leader is more experienced and more able to present the work in the best possible light — and for this, a lack of knowledge of the details can be advantageous! The student or postdoc is released to go back to the bench, increasing productivity. However, she or he does not get taught how to write up results4.
Editors. It is no surprise that editors of élite journals receive many submissions. For example, Nature now receives around 9,000 manuscripts a year (double that of 10 years ago) and has to reject about 95% of biomedical papers. Development, a quality specialist journal, now rejects roughly 70%, compared with 50% in 1990. In leading journals there are too many submissions to send most out for peer review, so the editor's decision has become, quantitatively, much more important than the judgement of reviewers. Consequently, editors are courted by authors who resort to tactics such as charm offensives during "presubmission enquiries", networking at conferences and wheedling telephone calls — or pulling rank, using contacts, threatening and bullying. Group leaders can justify spending time and ingenuity on these stratagems — editors can be swayed and the rewards for success are high. Furthermore, impact factors and finance have joined forces to build up competition between top journals (Cell Press was recently sold for a great deal of money). One result is that editors are sent out to woo star scientists for their trendiest papers. These forces all combine to create an antiscientific culture in which pushiness and political skills are rewarded too much, and imaginative approaches, high-quality results and logical argument, too little.
Even experienced editors are on uncertain ground — sifting through a mass of diverse papers objectively and hurriedly is almost impossible. The advent of the Medline search and other Internet-based services has helped them, but it is still difficult to see clearly into the dark corners of specialization. Understandably insecure, editors play safe and favour the fashionable, familiar and expected over the flaky and unexpected — or original. Inevitably, mistakes are made. The original paper by Michael Berridge and Robin Irvine on phosphoinositol and signalling, which became the second most quoted article throughout the 1980s, was originally turned down by Nature. The authors fought back and it was accepted5. But when Berridge synthesized the information and added new ideas in another paper, it was rejected again by Nature, eventually published by the Biochemical Journal6 and became the fifth most quoted paper of the 1980s7.
Reviewers are, of course, authors wearing a different hat. There can be conflicts — for example, does the reviewer favour the work of a competitor and thereby endanger his or her student's career? Such opposing interests can explain why two reviewers of similar expertise sometimes present vastly different opinions about the same paper. It does not help that top journals are increasingly giving reviewers an extra task. Apart from the traditional technical and scientific assessments, where objective criteria are paramount, reviewers are now being asked to judge whether a manuscript constitutes a "Science" paper — is it sufficiently exciting to interest the "general reader"? This participation in editorial decisions gives reviewers opportunities to punish authors they do not like, settle old scores and hold up competitors. From many years of editing experience, I am persuaded that a minority of reviewers take advantage of these opportunities. Some bounce the same paper from more than one journal, making it more difficult for a less politically adept scientist to present his or her work, especially if it goes against the current grain. Objectivity is also threatened by a tacit understanding between some leading scientists: they invite each other onto committees, to conferences, nominate each other for prizes and awards, and support publication of each other's papers.
Another relatively recent phenomenon is the practice of sending papers to three reviewers. Although this is partly to ensure that at least two reviews are received, I think it is also so that the advice received cannot be a tie. Decision by vote can encourage rejected authors to make empty appeals, praising favourable reviewers, denigrating negative ones and asking for new reviewers — in the hope of getting another plus. Rejection is easier to accept if there is a thoughtful reason for it from which one can learn.
Hard-pressed editors take power from authors and hand it to reviewers in other ways. Reviewers often ask for changes and new experiments, even though they may be rather ignorant of the details and may have formed an opinion of the paper in half an hour. Nevertheless, the easiest and most commonly chosen course for the editor is to ask the authors to "satisfy" all of the reviewers, then send the revised manuscript for reassessment. If authors have well-founded disagreements with a reviewer, they find themselves in a dilemma: do they invest time in experiments they do not believe will help, do controls that few other informed people would find important, or even draw conclusions that are not theirs? If they do not, reviewer X may not be appeased and the editor would be unswayable. In former days, these authors could have solved their problems by sending their papers elsewhere, but now that the journal itself has become so important to their careers, they feel forced to comply. In this situation the reviewers can become more like censors than assessors. I have seen many examples of this and, sometimes, months of research time have been misspent, even to the extent that an author can be scooped in the interim.
Faster publication times, materials-transfer agreements and threats of legal action to force journals to identify reviewers have added to the pressure. In the case of faster publication times, journals can offer chosen authors fast-track treatment and advanced online publication, helping them to steal a march on competitors. A reviewer can use information and may have time to modify his or her own manuscript and even publish it elsewhere first. Temptation and suspicion have heated up enough to melt the wall of confidentiality that reviewers owe to authors. Still, I believe there is genuine confusion about the level of confidentiality that reviewers should adopt. Is the reviewer obliged not to reveal even the existence of a submitted manuscript to anyone? I think so, but do we all concur? Should a reviewer agree to assess a paper that he or she has already advised another journal to reject? I think not, but this happens frequently.
Cures?
It is no wonder that authors are becoming paranoid. Roughly half of the submission letters I now receive request me not to use certain reviewers, often because of "conflicts of interest". Behind this phrase lurks the fear of misuse of the information in the paper — although admittedly it is sometimes a ploy to avoid the sharp-eyed and critical.
My main purpose here is consciousness raising. But we can all start to improve things by toning down our obsession with the journal. The most effective change by far would be if the organizations that award grants and manage research programmes were to place much less trust in a quantitative audit that reeks of false precision. Such organizations have the big advantage of hindsight — unlike editors and reviewers at the time of submission, they can ask themselves if key papers published by the candidate are illuminating, have proved influential and whether their main results have been confirmed by others.
Authors can help to break up the cult of the journal. One way is to set up mutually supportive alliances, as has been done for the field of cell signalling (http://www.signaling-gateway.org). If established authors start to publish selectively in open-access websites and in specialized journals when appropriate, a better example would be set for younger scientists. This would reduce the enormous pressure on the leading journals, which then could again begin to publish more comprehensible papers that tell a complete story, perhaps even bringing the 'general reader' back to life.
I am not suggesting sweeping changes to the review process. For example, I don't think a change to open peer-reviewing (as discussed in ref. 8) will help, mainly because younger reviewers would be intimidated and the political power of the established would be increased. One change which would now be feasible through online submission of two forms of the manuscript, would be to deny the reviewers authors' names. It is crucial that the responsibilities and duties of reviewers are clarified and made more public. For example, they could be better educated about confidentiality by the journal, along the lines of Nature's advice (http://www.nature.com/nature/submit/policies/index.html#8).
Professional editors need to be more aware of these dangers. They now have to make difficult decisions that are of vital importance to authors, far beyond the publication or not of a particular paper, as well as meeting rejection rates as high as 95%. They have — perhaps understandably — been relinquishing too many of their responsibilities to reviewers. It does not help that editors may not have had enough experience of research and lack hands-on knowledge, particularly outside one narrow subject area. They need to act now to reinstate authors' rights. Once a decision has been made to publish in principle, they should never simply demand in a blanket sense that authors satisfy reviewers X, Y and Z, but should interpret referees' advice and be willing to accept reasoned discussion about aspects of the referees' criticisms. Editors should then be in a decision to adjudicate among themselves or to seek further opinion from an expert who is given both sides of the argument. Editors should appreciate that, unlike the authors whose names are out there, anonymous reviewers will not be held to account if they make a mistake. It should always be remembered that the proper role of the reviewer is to advise the editor, not to gain control over the author's paper.
Editors should also take a more long-term and broader view about what is of interest, and act positively to encourage new approaches and topics in an affirmative action against fashion.It is fashion that makes looking for new members of signalling pathways into the hottest of current topics, which can lead to unnecessary duplication. Just one example — no less than four independent studies on the same new gene (pygopus), each describing years of careful and hard work by several people, have just been published (see ref. 9 and references therein).
As authors, we have abandoned the attempt to make our experimental papers accessible or comprehensible to the nonspecialist, often writing undiluted mixtures of hype and jargon. This is partly because we are writing in shorthand to fit our papers into a small space, and partly because we are trying to con the editors. But why not write papers that are readable, reduce the number of acronyms and gobbledeegook, and put methodological details in supplementary material on the web?
It is we older, well-established scientists who have to act to change things. We should make these points on committees for grants and jobs, and should not be so desperate to push our papers into the leading journals. We cannot expect younger scientists to endanger their future by making sacrifices for the common good, at least not before we do.
References | 1. | O'Neill, O. A Question of Trust (Cambridge Univ. Press, 2002). |
| 2. | Power, M. The Audit Society: Rituals of Verification (Oxford Univ. Press, 1997). |
| 3. | Lawrence, P. A. Nature Rev. Genetics 2, 139-142 (2001). | Article | ISI | ChemPort | |
| 4. | Lawrence, P. A. Nature 415, 835-836 (2000). | Article | ChemPort | |
| 5. | Berridge, M. J. & Irvine, R. F. Nature 312, 315-321 (1984). | Article | PubMed | ISI | ChemPort | |
| 6. | Berridge, M. J. Biochem. J. 220, 345-360 (1984). | PubMed | ISI | ChemPort | |
| 7. | Pendlebury, D. The Scientist 4, 18-21 (1990). |
| 8. | Gura, T. Nature 416, 258-260 (2002). | Article | PubMed | ISI | ChemPort | |
| 9. | Belenkaya, T. et al. Development 129, 4089-4101 (2002). | PubMed | ISI | ChemPort | |
No comments:
Post a Comment