Papers almost always had the same conclusions as their pre-published versions, but preprints still had the power to mislead.
Four in five preprint research papers were eventually published in peer-reviewed journals, and the conclusions remained largely the same upon publication, a new analysis shows.
The scientific process was turned on its head over the pandemic, when the rush to understand the new virus led to the ascendence of preprint research.
But questions about the quality of this research prompted Assistant Professor Tony Bai, an infectious diseases expert at Queens University in Canada, and his colleagues to analyse over 150 preprints investigating the treatment or prevention of covid posted in 2021.
They found that close to 80% of all preprints went on to be published in peer-reviewed scientific journals, and that the main conclusion remained the same in the journal article for all but two preprints, despite over half of the articles having different outcomes, analyses, and results from their original versions. Nearly 30% of the studies’ primary outcomes had changed between preprint and publication.
Nevertheless, Professor Bai was hesitant to put too much faith in preprint research.
“I think we should always be cautious and apply critical appraisal when reading preprints, because they have not been through the peer review process yet,” Professor Bai told TMR.
“Our study findings support this, [showing] that preprints may have a higher risk of bias.”
For the 33 preprints that were not published within the 470-day follow-up period, the authors found that small sample sizes and high risks of bias were the biggest barriers to publication.
Compared to studies with less than 200 participants, preprints of studies with samples over 1000 were more than twice as likely to be published, while preprints identified as having a low risk of bias were three times more likely to be published than those with a high risk of bias.
They also found the median time between preprint posting and journal publication was more than six months, a significant delay considering the preprints were posted during the height of the pandemic.
According to the authors, this lag was most likely the result of lengthy peer-review processes for scientific journals, particularly given manuscripts are revised before publication.
The authors highlighted the advantage of preprints in making information more easily accessible but emphasised the need for caution before interpreting or applying any findings.
“There is an impetus to make covid-19 randomised controlled trial findings publicly available as soon as possible so that they can be applied in clinical settings,” they wrote.
“However, preprints may not meet the required standards for scientific publication without peer review, they may be subject to change because the manuscript is not yet finalised, and they may contain errors or false information. As a result, preprints can mislead the public.”
