Paula A. Rochon, MD, MPH; Lisa A. Bero, PhD; Ari M. Bay, BSc; et al

(https://jamanetwork.com/journals/jama/fullarticle/194974)

Abstract

Context To compare the quality, presentation, readability, and clinical relevance of review articles published in peer-reviewed and "throwaway" journals.

Methods We reviewed articles that focused on the diagnosis or treatment of a medical condition published between January 1 and December 31, 1998, in the 5 leading peer-reviewed general medical journals and high-circulation throwaway journals. Reviewers independently assessed the methodologic and reporting quality, and evaluated each article's presentation and readability. Clinical relevance was evaluated independently by 6 physicians.

Results Of the 394 articles in our sample, 16 (4.1%) were peer-reviewed systematic reviews, 135 (34.3%) were peer-reviewed nonsystematic reviews, and 243 (61.7%) were nonsystematic reviews published in throwaway journals. The mean (SD) quality scores were highest for peer-reviewed articles (0.94 [0.09] for systematic reviews and 0.30 [0.19] for nonsystematic reviews) compared with throwaway journal articles (0.23 [0.03], F_2,391 = 280.8, P<.001). Throwaway journal articles used more tables (P = .02), figures (P = .01), photographs (P<.001), color (P<.001), and larger font sizes (P<.001) compared with peer-reviewed articles. Readability scores were more often in the college or higher range for peer-reviewed journals compared with the throwaway journal articles (104 [77.0%] vs 156 [64.2%]; P = .01). Peer-reviewed article titles were judged less relevant to clinical practice than throwaway journal article titles (P<.001).

Conclusions Although lower in methodologic and reporting quality, review articles published in throwaway journals have characteristics that appeal to physician readers.

Throwaway" journals are characterized as journals that contain no original investigations, are provided free of charge, have a high advertisement-to-text ratio, and are nonsociety publications.¹ Large circulations¹ and readership polls² suggest that throwaway journals are more widely read than some peer-reviewed journals in the same subject areas. Despite their popularity, throwaway journals are judged disparagingly as a source of "instant cookbook medicine"³ and journals that are given away. Indeed, throwaway journal articles¹ are seldom peer reviewed and are almost never cited in the medical literature. They are considered to be of poor quality compared with peer-reviewed journal articles, despite the lack of formal quality comparisons.¹ Given the success of throwaway publications, we sought to understand why so many physicians read them. We assessed the quality, presentation, readability, and clinical relevance of review articles published in a sample of peer-reviewed journals compared with those published in a sample of throwaway journals.

Findings

We found that review articles published in throwaway journals were easier to read than review articles published in peer-reviewed medical journals. Review articles published in throwaway journals were rated consistently better than articles published in peer-reviewed journals on virtually all measures of presentation, readability, and the clinical relevance of the message. As expected, peer-reviewed journal articles were of superior methodologic and reporting quality relative to articles published in throwaway journals. These findings are consistent with the large body of evidence showing that peer-reviewed medical journals produce articles of superior quality compared with those published in non–peer-reviewed journals.

The simplest way of writing is not always the best. Complex messages may require complex writing to convey accurate information. Through the use of color, larger font size, and the incorporation of more graphics, many peer-reviewed journals have attempted to improve the appeal of the scientific material they publish to their readership. Despite these efforts, our findings suggest that peer-reviewed journal articles lag behind the throwaway journal articles in these communication techniques.

Our study has several limitations. First, review article quality scoring instruments reward articles that are systematic reviews. Many articles in our sample were not intended to be systematic reviews. Nonsystematic reviews can provide valuable information. However, systematic review articles are the only type of review that has been shown to minimize bias. Second, our physician reviewers may not be representative of all physicians; all had a clinical focus and were recent graduates. Third, titles may not be the best way to judge clinical relevance but play an important role in attracting readers' attention and influence the decision of whether to read an article.

A balance needs to be achieved between presenting high-quality information and communicating the message. Throwaway journals do not serve the same markets as peer-reviewed journals and are largely supported by advertising; therefore, their editors may choose to publish articles for which there are enthusiastic sponsors. In contrast, peer-reviewed journals may be more likely to tackle difficult and sometimes less popular topics. Although lower in methodologic and reporting quality, review articles published in throwaway journals possess characteristics that are appealing to physician readers.