The characteristics of peer reviewers who produce good-quality reviews

Abstract

Objective: To determine the characteristics of good peer reviewers.Design: Cross-sectional analysis of data gathered during a randomized controlled trial. Setting: The Journal of General Internal Medicine. Participants: 226 reviewers of 131 consecutively submitted manuscripts of original research. 201 (91%) completed the review and submitted a curriculum vitae. Measurements and main results: The quality of each review was judged on a scale from 1 to 5 by an editor who was blinded to the identity of the reviewer. Reviewer characteristics were taken from the curricula vitae. 86 of the 201 reviewers (43%) produced good reviews (a grade of 4 or 5). Using logistic regression, the authors found that when a reviewer was less than 40 years old, from a top academic institution, well known to the editor choosing the reviewer, and blinded to the identity of the manuscript’s authors, the probability that he or she would produce a good review was 87%, whereas a reviewer without any of these characteristics had a 7% probability of producing a good review. Other characteristics that were significant only on bivariate analysis included previous clinical research training, additional postgraduate degrees, and more time spent on the review. There was a negative but statistically nonsignificant association between academic rank and review quality: 37% of full professors, 39% of associate professors, and 51% of assistant professors or fellows produced good reviews (p=0.11). Conclusions: Good peer reviewers for this journal tended to be young, from strong academic institutions, well known to the editors, and blinded to the identity of the manuscript’s authors.