<The Decline of Peer Review in Science: A Critical Examination>
Written on
# The Decline of Peer Review in Science: A Critical Examination
Introduction
The term enshittification was recently introduced by author Cory Doctorow to describe a pattern observed in online platforms. According to him:
> "Here is how platforms die: first, they are good to their users; then they abuse their users to make things better for their business customers; finally, they abuse those business customers to claw back all the value for themselves. Then, they die."
I propose that this concept can also be applied to the realm of science, particularly in relation to the peer-review process. Is it possible that the peer review system is undergoing a similar decline?
Understanding Peer Review
Peer review is a fundamental aspect of modern scientific research, often regarded as sacred within the academic community. Wikipedia defines it as:
> Peer review is the evaluation of work by one or more people with similar competencies as the producers of the work (peers). It functions as a form of self-regulation by qualified members of a profession within the relevant field. Peer review methods are used to maintain quality standards, improve performance, and provide credibility. In academia, scholarly peer review is often used to determine an academic paper’s suitability for publication.
However, my personal experience with the peer-review process in recent years has been troubling. While delays and issues have always existed, I sense that reviewers, editors, and publishers once treated this critical task with more seriousness.
Currently, reviews seem to take longer, their quality is often questionable, and major academic publishers appear to prioritize profit over the integrity of science and respect for scientists.
For instance, I recently submitted a paper to a well-regarded journal and encountered significant delays. Ideally, an associate editor should be assigned within days after submission, with the paper sent for review shortly thereafter. However, three and a half months later, I had yet to receive any reviews or even see the status updated to "Under Review."
This situation reflects a troubling lack of regard for the effort that scientists put into their work.
On another occasion, after waiting in vain for an update from a different journal, I decided to withdraw my submission to pursue opportunities elsewhere. Despite several attempts to communicate my intent, I received no response, and eventually submitted my paper to another venue, likely leaving my original submission in limbo.
I suspect that many new journals are established by academics primarily to enhance their CVs in pursuit of promotions and grants, leading to a lack of genuine interest in the journal's operations. Unfortunately, this attitude seems to be echoed by many publishers, whose focus appears to be on financial outcomes rather than scientific quality.
The Issue of Retractions
Retractions present another significant concern in academic publishing:
> In academic publishing, a retraction is a mechanism by which a published paper in an academic journal is flagged for being seriously flawed to the extent that their results and conclusions can no longer be relied upon.
Mistakes are an inherent part of research, but a recent report by Nature indicates alarming trends:
> "The number of retractions issued for research articles in 2023 has passed 10,000 — smashing annual records — as publishers struggle to clean up a slew of sham papers and peer-review fraud."
This raises serious questions about the reliability of scientific research. The infamous 1998 paper that incorrectly linked vaccines to autism was not retracted until 2010, leading to widespread vaccine skepticism that continues to affect public health today.
Scientific Conferences
Attending peer-reviewed conferences is a common practice among scientists. Many conferences are categorized as "A*" based on a ranking system that includes four levels: A*, A, B, and C. While conferences provide deadlines for submissions, the variability in review quality can be frustrating.
In recent years, the volume of submissions to top conferences has surged, leading to a decline in review quality and an increase in the likelihood of receiving subpar feedback. A colleague who recently served as an Area Chair for a major conference shared that many reviewers simply failed to submit their reviews, leaving a backlog of papers unassessed.
Factors Contributing to Decline
Several factors contribute to the deterioration of the peer-review process:
- An unprecedented volume of research papers.
- Significant papers are often those that introduce new concepts, making evaluation more challenging.
- Researchers face immense pressure to publish, leading to a "publish or perish" mentality.
The Rise of Preprints
The emergence of preprints—early versions of scholarly papers that have not yet undergone formal peer review—has been notable in recent years. These can be found on dedicated platforms such as arXiv for mathematics and computer science or bioRxiv for biological sciences.
While preprints facilitate rapid dissemination of research, they are not peer-reviewed, raising concerns about their reliability. However, many researchers opt for this route to share their findings quickly, often achieving high quality despite the absence of formal review.
In fields such as medicine, the implications of preprints can be more complicated, especially when large data sets are involved. There is a pressing need for clearer guidelines regarding retractions, particularly as they pertain to preprint servers.
Conclusion
The question remains: Is peer review evolving into "pee review"? The integrity of scientific validation appears to be under threat. While the problems are evident, effective solutions are not readily apparent.
Some suggest that compensating reviewers might improve the situation, but this faces resistance from publishers. It remains uncertain whether such measures would yield positive results.
As an AI researcher, I ponder the potential for artificial intelligence to take over the peer-review process in the future.
Ultimately, the future of peer review remains uncertain.
(P.S. I previously created cartoons, including one on the topic of "pee review.")