Anna Kaatz1, You-Geon Lee, Aaron Potvien, Wairimu Magua, Amarette Filut, Anupama Bhattacharya, Renee Leatherberry, Xiaojin Zhu, Molly Carnes. 1. A. Kaatz is director of computational sciences, Center for Women's Health Research, University of Wisconsin-Madison, Madison, Wisconsin. Y.G. Lee is associate researcher, Wisconsin Center for Education Research, University of Wisconsin-Madison, Madison, Wisconsin. A. Potvien is a doctoral candidate, Department of Statistics, and researcher, Health Innovation Program, University of Wisconsin-Madison, Madison, Wisconsin. W. Magua is postdoctoral research associate, Center for Women's Health Research, University of Wisconsin-Madison, Madison, Wisconsin. A. Filut is research assistant, Center for Women's Health Research, University of Wisconsin-Madison, Madison, Wisconsin. A. Bhattacharya is an undergraduate student and data science scholar, Center for Women's Health Research, University of Wisconsin-Madison, Madison, Wisconsin. R. Leatherberry is staff researcher, Center for Women's Health Research, University of Wisconsin-Madison, Madison, Wisconsin. X. Zhu is associate professor, Department of Computer Science, University of Wisconsin-Madison, Madison, Wisconsin. M. Carnes is director, Center for Women's Health Research, professor in the Departments of Medicine, Psychiatry, and Industrial and Systems Engineering, University of Wisconsin-Madison, and part-time physician, William S. Middleton Veterans Hospital, Madison, Wisconsin.
Abstract
PURPOSE: Prior text analysis of R01 critiques suggested that female applicants may be disadvantaged in National Institutes of Health (NIH) peer review, particularly for renewals. NIH altered its review format in 2009. The authors examined R01 critiques and scoring in the new format for differences due to principal investigator (PI) sex. METHOD: The authors analyzed 739 critiques-268 from 88 unfunded and 471 from 153 funded applications for grants awarded to 125 PIs (76 males, 49 females) at the University of Wisconsin-Madison between 2010 and 2014. The authors used seven word categories for text analysis: ability, achievement, agentic, negative evaluation, positive evaluation, research, and standout adjectives. The authors used regression models to compare priority and criteria scores, and results from text analysis for differences due to PI sex and whether the application was for a new (Type 1) or renewal (Type 2) R01. RESULTS: Approach scores predicted priority scores for all PIs' applications (P < .001), but scores and critiques differed significantly for male and female PIs' Type 2 applications. Reviewers assigned significantly worse priority, approach, and significance scores to female than male PIs' Type 2 applications, despite using standout adjectives (e.g., "outstanding," "excellent") and making references to ability in more critiques (P < .05 for all comparisons). CONCLUSIONS: The authors' analyses suggest that subtle gender bias may continue to operate in the post-2009 NIH review format in ways that could lead reviewers to implicitly hold male and female applicants to different standards of evaluation, particularly for R01 renewals.
PURPOSE: Prior text analysis of R01 critiques suggested that female applicants may be disadvantaged in National Institutes of Health (NIH) peer review, particularly for renewals. NIH altered its review format in 2009. The authors examined R01 critiques and scoring in the new format for differences due to principal investigator (PI) sex. METHOD: The authors analyzed 739 critiques-268 from 88 unfunded and 471 from 153 funded applications for grants awarded to 125 PIs (76 males, 49 females) at the University of Wisconsin-Madison between 2010 and 2014. The authors used seven word categories for text analysis: ability, achievement, agentic, negative evaluation, positive evaluation, research, and standout adjectives. The authors used regression models to compare priority and criteria scores, and results from text analysis for differences due to PI sex and whether the application was for a new (Type 1) or renewal (Type 2) R01. RESULTS: Approach scores predicted priority scores for all PIs' applications (P < .001), but scores and critiques differed significantly for male and female PIs' Type 2 applications. Reviewers assigned significantly worse priority, approach, and significance scores to female than male PIs' Type 2 applications, despite using standout adjectives (e.g., "outstanding," "excellent") and making references to ability in more critiques (P < .05 for all comparisons). CONCLUSIONS: The authors' analyses suggest that subtle gender bias may continue to operate in the post-2009 NIH review format in ways that could lead reviewers to implicitly hold male and female applicants to different standards of evaluation, particularly for R01 renewals.
Authors: Molly Carnes; Patricia G Devine; Linda Baier Manwell; Angela Byars-Winston; Eve Fine; Cecilia E Ford; Patrick Forscher; Carol Isaac; Anna Kaatz; Wairimu Magua; Mari Palta; Jennifer Sheridan Journal: Acad Med Date: 2015-02 Impact factor: 6.893
Authors: Carol Isaac; Angela Byars-Winston; Rebecca McSorley; Alexandra Schultz; Anna Kaatz; Mary L Carnes Journal: Adv Health Sci Educ Theory Pract Date: 2013-04-20 Impact factor: 3.853
Authors: Susan E Waisbren; Hannah Bowles; Tayaba Hasan; Kelly H Zou; S Jean Emans; Carole Goldberg; Sandra Gould; Deborah Levine; Ellice Lieberman; Mary Loeken; Janina Longtine; Carol Nadelson; Andrea Farkas Patenaude; Deborah Quinn; Adrienne G Randolph; Jo M Solet; Nicole Ullrich; Rochelle Walensky; Patricia Weitzman; Helen Christou Journal: J Womens Health (Larchmt) Date: 2008-03 Impact factor: 2.681
Authors: Daniel M Blumenthal; Andrew R Olenski; Robert W Yeh; Doreen DeFaria Yeh; Amy Sarma; Ada C Stefanescu Schmidt; Malissa J Wood; Anupam B Jena Journal: Circulation Date: 2017-02-07 Impact factor: 29.690
Authors: Erica T Warner; René Carapinha; Griffin M Weber; Emorcia V Hill; Joan Y Reede Journal: J Womens Health (Larchmt) Date: 2017-08-03 Impact factor: 2.681