CONTEXT: Peer review should evaluate the merit and quality of abstracts but may be biased by geographic location or institutional prestige. The effectiveness of blinded peer review at reducing bias is unknown. OBJECTIVE: To evaluate the effect of blinded review on the association between abstract characteristics and likelihood of abstract acceptance at a national research meeting. DESIGN AND SETTING: All abstracts submitted to the American Heart Association's annual Scientific Sessions research meeting from 2000-2004. Abstract review included the author's name and institution (open review) from 2000-2001, and this information was concealed (blinded review) from 2002-2004. Abstracts were categorized by country, primary language, institution prestige, author sex, and government and industry status. MAIN OUTCOME MEASURE: Likelihood of abstract acceptance during open and blinded review, by abstract characteristics. RESULTS: The mean number of abstracts submitted each year for evaluation was 13,455 and 28.5% were accepted. During open review, 40.8% of US and 22.6% of non-US abstracts were accepted (relative risk [RR], 1.81; 95% confidence interval [CI], 1.75-1.88), whereas during blinded review, 33.4% of US and 23.7% of non-US abstracts were accepted (RR, 1.41; 95% CI, 1.37-1.45; P<.001 for comparison between peer review periods). Among non-US abstracts, during open review, 31.1% from English- speaking countries and 20.9% from non-English-speaking countries were accepted (RR, 1.49; 95% CI, 1.39-1.59), whereas during blinded review, 28.8% and 22.8% of abstracts were accepted, respectively (RR, 1.26; 95% CI, 1.19-1.34; P<.001). Among abstracts from US academic institutions, during open review, 51.3% from highly prestigious and 32.6% from nonprestigious institutions were accepted (RR, 1.57; 95% CI, 1.48-1.67), whereas during blinded review, 38.8% and 29.0% of abstracts were accepted, respectively (RR, 1.34; 95% CI, 1.26-1.41; P<.001). CONCLUSIONS: This study provides evidence of bias in the open review of abstracts, favoring authors from the United States, English-speaking countries outside the United States, and prestigious academic institutions. Moreover, blinded review at least partially reduced reviewer bias.
CONTEXT: Peer review should evaluate the merit and quality of abstracts but may be biased by geographic location or institutional prestige. The effectiveness of blinded peer review at reducing bias is unknown. OBJECTIVE: To evaluate the effect of blinded review on the association between abstract characteristics and likelihood of abstract acceptance at a national research meeting. DESIGN AND SETTING: All abstracts submitted to the American Heart Association's annual Scientific Sessions research meeting from 2000-2004. Abstract review included the author's name and institution (open review) from 2000-2001, and this information was concealed (blinded review) from 2002-2004. Abstracts were categorized by country, primary language, institution prestige, author sex, and government and industry status. MAIN OUTCOME MEASURE: Likelihood of abstract acceptance during open and blinded review, by abstract characteristics. RESULTS: The mean number of abstracts submitted each year for evaluation was 13,455 and 28.5% were accepted. During open review, 40.8% of US and 22.6% of non-US abstracts were accepted (relative risk [RR], 1.81; 95% confidence interval [CI], 1.75-1.88), whereas during blinded review, 33.4% of US and 23.7% of non-US abstracts were accepted (RR, 1.41; 95% CI, 1.37-1.45; P<.001 for comparison between peer review periods). Among non-US abstracts, during open review, 31.1% from English- speaking countries and 20.9% from non-English-speaking countries were accepted (RR, 1.49; 95% CI, 1.39-1.59), whereas during blinded review, 28.8% and 22.8% of abstracts were accepted, respectively (RR, 1.26; 95% CI, 1.19-1.34; P<.001). Among abstracts from US academic institutions, during open review, 51.3% from highly prestigious and 32.6% from nonprestigious institutions were accepted (RR, 1.57; 95% CI, 1.48-1.67), whereas during blinded review, 38.8% and 29.0% of abstracts were accepted, respectively (RR, 1.34; 95% CI, 1.26-1.41; P<.001). CONCLUSIONS: This study provides evidence of bias in the open review of abstracts, favoring authors from the United States, English-speaking countries outside the United States, and prestigious academic institutions. Moreover, blinded review at least partially reduced reviewer bias.
Authors: Kanu Okike; Mininder S Kocher; Charles T Mehlman; James D Heckman; Mohit Bhandari Journal: J Bone Joint Surg Am Date: 2008-11 Impact factor: 5.284
Authors: Jonathan P Tennant; Jonathan M Dugan; Daniel Graziotin; Damien C Jacques; François Waldner; Daniel Mietchen; Yehia Elkhatib; Lauren B Collister; Christina K Pikas; Tom Crick; Paola Masuzzo; Anthony Caravaggi; Devin R Berg; Kyle E Niemeyer; Tony Ross-Hellauer; Sara Mannheimer; Lillian Rigling; Daniel S Katz; Bastian Greshake Tzovaras; Josmel Pacheco-Mendoza; Nazeefa Fatima; Marta Poblet; Marios Isaakidis; Dasapta Erwin Irawan; Sébastien Renaut; Christopher R Madan; Lisa Matthias; Jesper Nørgaard Kjær; Daniel Paul O'Donnell; Cameron Neylon; Sarah Kearns; Manojkumar Selvaraju; Julien Colomb Journal: F1000Res Date: 2017-07-20