Knowledge syntheses in medical education: Meta-research examining author gender, geographic location, and institutional affiliation.

INTRODUCTION: Authors of knowledge syntheses make many subjective decisions during their review process. Those decisions, which are guided in part by author characteristics, can impact the conduct and conclusions of knowledge syntheses, which assimilate much of the evidence base in medical education. To better understand the evidence base, this study describes the characteristics of knowledge synthesis authors, focusing on gender, geography, and institution.
METHODS: In 2020, the authors conducted meta-research to examine authors of 963 knowledge syntheses published between 1999 and 2019 in 14 core medical education journals.
RESULTS: The authors identified 4,110 manuscript authors across all authorship positions. On average there were 4.3 authors per knowledge synthesis (SD = 2.51, Median = 4, Range = 1-22); 79 knowledge syntheses (8%) were single-author publications. Over time, the average number of authors per synthesis increased (M = 1.80 in 1999; M = 5.34 in 2019). Knowledge syntheses were authored by slightly more females (n = 2047; 50.5%) than males (n = 2005; 49.5%) across all author positions. Authors listed affiliations in 58 countries, and 58 knowledge syntheses (6%) included authors from low- or middle-income countries. Authors from the United States (n = 366; 38%), Canada (n = 233; 24%), and the United Kingdom (n = 180; 19%) published the most knowledge syntheses. Authors listed affiliation at 617 unique institutions, and first authors represented 362 unique institutions with greatest representation from University of Toronto (n = 55, 6%). Across all authorship positions, the large majority of knowledge syntheses (n = 753; 78%) included authors from institutions ranked in the top 200 globally.
CONCLUSION: Knowledge synthesis author teams have grown over the past 20 years, and while there is near gender parity across all author positions, authorship has been dominated by North American researchers located at highly ranked institutions. This suggests a potential overrepresentation of certain authors with particular characteristics, which may impact the conduct and conclusions of medical education knowledge syntheses.

Introduction

Medical education is postsecondary education related to the practice of medicine; it typically includes: (a) initial training to become a physician (i.e., medical school and internship), (b) follow-on graduate medical education (i.e, residency and fellowship), and (c) continuing professional development. In medical education, researchers have been encouraged to publish knowledge syntheses and educators to act as evidence-informed practitioners in their application of these reviews toward the education of medical trainees and practicing physicians [1,2]. As a result, the recent proliferation of knowledge syntheses published in core medical education journals is unsurprising [3]. Knowledge syntheses, which often form the evidence base for implementing curricular innovations and determining how a field defines its key terminology, can have immense impact on a field’s discourse and future directions [4,5].

The Canadian Institutes of Health Research (CIHR) defines knowledge syntheses (aka, reviews) as: “the contextualization and integration of research findings of individual research studies within the larger body of knowledge on the topic.” [6] When conducting a knowledge synthesis, author teams are required to make multiple decisions, many of which can be subjective. For example, authors must decide which populations to include and which to exclude; which contexts matter and which do not; which factors are important to extract from the primary studies and which are not; and even which languages to review and which to exclude. Such decisions are shaped by author characteristics, backgrounds, and even the power structures and cultural norms in which the authors operate. Therefore, best practices in knowledge syntheses encourage scholars to assemble a diverse author team with representation from a variety of backgrounds and perspectives [7-9]. And these factors have the potential to impact the conduct of any given knowledge synthesis, as well as the conclusions drawn from the analysis [10]. In short, author characteristics have important implications for a field’s evidence base.

Broadly speaking, researchers have raised concerns that author characteristics, such as gender [11-14], geographical location [15-17], and institutional affiliation [18,19], can bias publications, including knowledge syntheses, and inadvertently reinforce dominant power structures. To this point, the Cochrane Collaboration, a major supporter and publisher of systematic reviews, has flagged the lack of international representation and diversity in published reviews as a significant problem and reports that more diverse author teams generate more relevant reviews with less research waste and fewer errors [20]. In 2020, based in part on these findings, the Cochrane Collaboration advocated for wide participation from a variety of stakeholders in conducting reviews as one of its key strategic initiatives [21].

In medical education, we know little about the characteristics of the authors who write knowledge syntheses, and so we lack a clear understanding of which author voices dominate and which are absent from the evidence base created through these reviews. This gap in our understanding means we risk inadvertently prioritizing certain views while diminishing others, and potentially creating an evidence base that is irrelevant to some people in some contexts. For example, a review written by a United States author team on providing student feedback may resonate well with North American readers. However, due to a series of author decisions (e.g., inclusion criteria) and interpretations that likely vary based on cultural norms, such a review may be less useful to an audience outside of North America where cultural norms and educational systems differ.

Medical education researchers have just begun examining authorship characteristics. For example, a recent study explored author gender from articles published in four medical education journals [22], and another investigated authors’ geographic location in papers indexed as medical education [23]. While both of these recent studies are valuable, they do not specifically examine knowledge syntheses, which have become a critical part of the evidence base in medical education.

The purpose of this study is to describe and examine the characteristics of the authors of knowledge syntheses with a focus on gender, geographical location, and institutional affiliation. In doing so, we hope to raise medical educators’ awareness of author characteristics that may have bearing on the current state of the field’s evidence base. We also hope to foster a healthy skepticism in the medical education evidence, which has been and continues to be synthesized through review articles.

Materials and methods

We conducted meta-research to examine the authors of knowledge syntheses published in 14 core medical education journals between 1999–2019.

Data collection

To undertake this meta-research, we utilized a publicly accessible data set that members of this author team created in 2020 and is licensed under a CC BY 4.0 license [24]. The data set includes citations and related PubMed and Web of Science (WoS) metadata for 963 knowledge syntheses published in 14 core medical education journals between 2009–2019 (See S1 Appendix for complete journal list and search strategies). The 14 journals were selected based on previous bibliometric studies that had identified these titles as “core” based on their presence in WoS and their perceived relevance to medical education by members of the field [25,26]. Additionally, all journals are indexed in PubMed/MEDLINE and have been publishing medical education research for over a decade. In the data set, we identified knowledge syntheses by screening the titles and abstracts of 2,210 articles published in these journals for articles that met the above CIHR definition of a knowledge synthesis. Full details on how we created the data set are published in Maggio et al., 2020 [3]. We utilized this existing data set because data reuse has been associated with reduced research waste, faster translation of research findings into practice, and enhanced reproducibility and transparency of science [27,28]. We chose this data set because it is the only existing, up-to-date data set of knowledge syntheses in medical education. All data were downloaded and managed in GoogleSheets [29].

To predict author gender, we extracted the first names of all authors from the data set. In cases where authors used initials only (e.g., D.A.D.C Jaarsma, aka Debbie Jaarsma), we conducted a web search to identify their first name. All first names were then submitted to the gender prediction tool Genderize.io [30]. Genderize.io predicts whether a name is male or female based on a database of over 20,000 names and provides a probability that the name is either male or female. This tool has been used in multiple publications with similar aims to the current study (e.g., Hart and Perlis; [31] Bagga et al. [32]). We accepted the tool’s designation for a name if the probability was over 70%. For each name that Genderize.io reported with <70% certainty (n = 151) and those 90 first names that the tool was unable to identify, we looked up the authors’ online presence and cross referenced the authors’ names with their publication and online profiles at their academic institutions and social media sites (e.g., LinkedIn, Academia.edu, ResearchGate) relying on author photos and the pronouns used in institutional bios. We recognize that our effort to predict gender is an oversimplification of a complicated social construct, especially because an individual’s gender is best described by that individual. However, we believe this approach is a reasonable starting point to begin providing a sense of the field; it also follows the protocol of similar papers recently published on this topic [11,33].

Geographical location

For each knowledge synthesis, we extracted from the WoS metadata the country of all author institutions. Due to the structure of the metadata, we were only able to accurately identify the location of the first author at the level of the individual author. Therefore, for non-first authors we report for each knowledge synthesis the countries represented in aggregate without regard to an individual’s placement in author order. Thus, we can only make limited claims about a non-first author’s geographical attribution. Countries were described using the World Bank’s 2021 world region classification system [34], which includes four-levels of countries (low, lower middle, upper middle, and high income). Country levels are based on a country’s gross national income [34].

Institutions

For each knowledge synthesis, we identified the first author’s institutional affiliation. Similar to geographical location, using WoS metadata we also identified for each knowledge synthesis a listing of all institutions that contributed to the study. To characterize institutional affiliation, we used the Times Higher Education (THE) World University Rankings for 2020 [35]. We selected this ranking due to its broad coverage of over 1,400 universities from 92 countries. THE ranking is based on 13 metrics (e.g., teaching, research, international outlook, industry outcomes). This ranking groups institutions starting with the top 200 institutions individually and then reports the remaining institutions in groups of 50, 100, or 200 depending on their position. In a minority of the cases examined in this study, authors provided multiple institutional affiliations (e.g., universities). In such cases, we included the first affiliation listed. Additionally, authors listed non-academic affiliations (e.g. professional associations, government entities, community hospitals), which we coded as organizations. Organizations are accounted for in our results, but unranked in relation to the THE rankings.

Analysis

Descriptive statistics were calculated using GoogleSheets [29], and data were visualized using Tableau v.2020.04 [36].

Results

We identified 4,110 authors listed across all authorship positions, and of those 3,200 were unique authors (See S2 Appendix for a listing of the most prolific authors). The number of authors per knowledge synthesis ranged from 1–22 with an average of 4.27 authors (SD = 2.51, Median = 4). Seventy-nine knowledge syntheses (8.2%) were single-author publications. Over the 20-year time period examined, the average number of authors per knowledge synthesis increased (M = 1.80 in 1999; M = 5.34 in 2019; See Fig 1).

Fig 1

The median number of authors per knowledge synthesis (10th and 90th percentiles also shown) published in 14 core medical education journals published between 1999–2019.

Gender

We identified the gender for 4,052 authors. We were unable to make a confident prediction of gender identification for 59 authors (even after a web search), all of whom only appeared once in the dataset (See S3 Appendix for listing of author names and corresponding gender). Knowledge syntheses were authored by slightly more females (n = 2047; 50.5%) than males (n = 2005; 49.5%) across all author positions. In addition, more females were listed as first authors (n = 494; 51.9%) and second authors (n = 483; 55.4%). On the other hand, the last author position was held by more males (n = 404; 56.0%). See Fig 2 for author order by gender.

Fig 2

Author order by gender for knowledge syntheses in 14 core medical education journals published between 1999–2019.

We were unable to determine the gender of 59 author names, which are excluded from this figure.

Most author teams were a combination of genders (n = 683 teams; 70.9%), but 280 teams included authors of a single gender (117 all female; 163 all male). For single authored papers, 52 were written by males and 26 by females. Over the time period examined, the ratio of female authors in all positions has increased (See Fig 3).

Fig 3

The ratio of female authors in all authorship positions for knowledge syntheses published in 14 core medical education journals published between 1999–2019.

Geography

Across all authorship positions, authors listed affiliations in 58 countries, including 22 LMIC (see Fig 4; See S4 Appendix for listing of all countries). By number of knowledge syntheses, the countries most represented were the United States (US) (n = 366; 38%), Canada (n = 233; 24%), and the United Kingdom (UK) (n = 180; 19%). Fifty-eight (6%) knowledge syntheses included at least one author listing an affiliation based in a LMIC; of these, 39 (4%) were first authored by an author with a LMIC affiliation. Of the 58 knowledge syntheses including LMIC authors, authors based in China were most prolific, publishing 22 knowledge syntheses of which 15 were written by authors who were all based in China. First authors represented 42 countries, including 13 LMIC. The most represented countries for first authors were the US (n = 312; 25%), Canada (n = 183; 15%), and the UK (n = 151; 12%). Twenty-seven (3%) knowledge syntheses were exclusively authored by individuals based in a LMIC. The most countries represented on a single team were seven, in a study featuring authors from France, Ireland, UK, Italy, Belgium, Croatia, and Germany [37]. Eighty percent (n = 767) of knowledge syntheses included authors from a single country only. Of those representing a single country, authors were predominantly located in the US (n = 271; 22%), Canada (n = 149; 12%), and the UK (n = 122; 10%).

Fig 4

Map highlighting the 42 countries listed as affiliations of first authors of knowledge syntheses published in 14 core medical education journals published between 1999–2019.

Contains information from OpenStreetMap and OpenStreetMap Foundation, which is made available under the Open Database License.

Across all authorship positions, we identified 617 unique institutions (See S5 Appendix for complete list of institutions). Institutions most often represented were the University of Toronto (n = 212 authors), the Mayo Clinic (n = 110 authors), and Maastricht University (n = 89 authors). See Table 1 for the top 10 institutions by frequency. Nearly half (n = 451; 47%) of knowledge synthesis listed authors from a single institution. The most institutions represented on a knowledge synthesis was 14 [38].

Table 1

Top 10 institutional affiliations by count of first authors of knowledge syntheses published in a core set of medical education journals 1999–2019.

Institution (country)	Count of first authors (%)	Times Higher Education Ranking	Count of authors across all authorship positions (%)
University of Toronto (Canada)	56 (5.7)	18	212 (5.2)
Mayo Clinic (United States)	32 (3.3)	ORG	110 (2.7)
McMaster University (Canada)	18 (1.9)	72	64 (1.6)
Monash University (Australia)	18 (1.9)	75*	72 (1.8)
University of Ottawa (Canada)	18 (1.9)	141*	72 (1.8)
National Health Services (United Kingdom)	17 (1.8)	ORG	70 (1.7)
University of British Columbia (Canada)	15 (1.6)	34	52 (1.3)
University of Utrecht (Netherlands)	14 (1.5)	75*	76 (1.8)
McGill University (Canada)	13 (1.3)	42	52 (1.3)
University of Alberta (Canada)	13 (1.3)	136*	54 (1.3)
University of Calgary (Canada)	13 (1.3)	201–250	47 (1.1)
Maastricht University (Netherlands)	13 (1.3)	127	89 (2.2)

Rankings retrieved from Times Higher Education (THE) World University Rankings 2020.

* indicates a tie in THE rankings.

ORG = organizations (e.g., health system).

For all authorship positions, 78% (n = 753) of knowledge syntheses included authors from institutions ranked in the THE top 200. The remaining knowledge syntheses included 368 with authors based at institutions ranked between 200–1000 and 217 at institutions ranked below 1000 or unranked. Of the 458 unique academic institutions represented, a total of 154 were unranked. Two hundred and twenty-nine authors (6%) represented non-university affiliated medical centers or hospitals and 229 (6%) listed affiliations at professional organizations.

First authors represented 362 unique institutions with the most first authors based at the University of Toronto (n = 56, 15%) and the Mayo Clinic (n = 31, 9%). The third most frequently represented institution was a tie between Monash University, McMaster University, University of Ottawa, and the National Health Service in the United Kingdom (n = 18; 5%). Of the top 200 institutions, only 110 (30%) institutions were represented in the first author position, yet the top 200 institutions accounted for 486 (50%) publications in our sample. Beyond the top 200, first authors represented 101 (28%) institutions ranked between 200–1000 and another 89 (25%) institutions were beyond 1000 or unranked. Thirty-four (9%) first authors represented professional organizations and 25 (7%) non-university affiliated medical centers or hospitals.

Discussion

In this meta-research, we described the author characteristics of 963 knowledge syntheses in medical education published between 1999–2019. We observed that the size of author teams has grown over the past 20 years, and while there is near gender parity across all author positions, authorship has been dominated by North American researchers located at highly ranked institutions.

The majority of knowledge syntheses examined here were multi-authored manuscripts, which aligns with the description of medical education researchers as inherently collaborative [39] and may reflect the significant time and effort required to publish a rigorous knowledge synthesis [40]. Notably, over the 20 years examined here, the average size of the author team increased. This growth mirrors a similar trend in science more broadly [41] and in some medical specialties more specifically [42,43]. Researchers have attributed this growth to multiple factors, including increased ease of collaboration between scholars as a result of computer and internet technology; utilization of research methodologies that require a variety of expertise and skills; the growing complexity of topics/research questions addressed; the overall increase of systematic reviews and meta-analyses; and the availability of guidelines for production of systematic reviews that require the inclusion of multiple authors [3,44,45]. In addition, findings from a study of research practices in medical education suggest that some of this growth in author team size may be related to questionable research practices like honorary authorship [46,47]. As the field of medical education continues to mature and its literature base grows, strategic selection of team members, including the rightsizing of the knowledge synthesis team, will become increasingly important. Thus, future research may be needed to further examine the size and composition of knowledge synthesis teams in order to provide evidence-based practical guidance on team construction.

The majority of author teams included both genders, suggesting that both male and female voices are present across the evidence being synthesized by these reviews. Over the study period, the ratio of females to males has increased. Researchers have attributed this growth to the increasing number of females entering medical school [48]. Additionally, gender parity among authorship teams has been achieved in recent years, which aligns with Madden’s recent analysis of four medical education journals across a variety of publication types [22]. Furthermore, similar to the findings from Madden et al examining other medical specialities [49-51], we identified significantly more males in the last author position. In biomedicine, the last position is traditionally occupied by the “senior author” who takes on a leadership role in the study [52] or is often the principal investigator of the research laboratory conducting the work. Similarly, in medical education, the last author position is often held by the senior author. While further investigations are necessary, we speculate that this finding may be related to the under-representation of women in leadership positions in academic medicine [53]. Overall, however, the gender results observed in this study are encouraging; nonetheless, future work should continue to track author gender in medical education to monitor for additional changes that might occur over time. For example, recent research related to the impact of the COVID-19 pandemic on science raises concern that female investigators, especially those with younger children, have had less time for research [54] and writing and, as a result, may be publishing fewer papers during the pandemic than their male counterparts [55].

Although we identified author representation from 58 countries, 80% of author teams were based in a single country. Additionally, authorship was dominated by individuals based in the US, Canada, and the UK, suggesting a heavy influence from English-speaking countries. This may have implications for the inclusion or potential exclusion of non-English language articles from reviews, as multinational teams are more likely to include non-English studies in their knowledge syntheses [56]. The exclusion of non-English articles is a known issue in the conduct of knowledge syntheses and has been labeled the “Tower of Babel Bias”; [57] this bias has implications for the accuracy and generalizability of research findings [58]. While examining the language inclusion criteria of each article is beyond the scope of this study, our findings suggest further investigation is warranted to better understand if the Tower of Babel Bias is an important issue in medical education knowledge syntheses.

Few authors listed affiliations in LMIC, and there was even less representation from LMIC in the first authorship position. This finding indicates geographical diversity is lacking in medical education knowledge syntheses, which has implications for the relevance of these reviews. In a 2019 study with similar findings to the present investigation, Thomas concluded that medical education research, more than any other field, is conducted by authors in the English-speaking Western countries, which he referred to as the “realm of the rich” [59]. This dominance of authors based in Western countries may limit the utility of these findings for non-Western health and education systems. To address some of this imbalance, the Cochrane Collaboration suggests that knowledge syntheses authors “take account of the needs of resource-poor countries and regions in the review process and invite appropriate input on the scope of the review and the questions it will address.” [60]

Researchers have identified that knowledge syntheses conducted across multiple institutions can improve the quality and visibility of a publication, as well as help to avoid a “silo effect” [61,62]. Our findings demonstrate that just under half of the knowledge syntheses examined were multi-institutional investigations. Moreover, across all authorship positions, 78% of knowledge syntheses included authors affiliated with institutions ranked in the THE Top 200. As other medical education researchers have noted “The big players moreover are in a position to influence the global discourse more than others” [63]. As such, the field would do well to consider growing the number of multi-institutional collaborations that not only perform original research, but also collaborate to conduct knowledge syntheses.

Limitations and future directions

There are a number of important limitations in the present work that suggest some fruitful areas of future research. First, our data set is composed of 14 core journals, which did not include journals from specific world regions, such as the African Journal of Health Professions Education. This limitation is particularly important, especially because we wanted to understand who does and does not have a voice in the development of knowledge syntheses. Had the data set examined other journals, we would likely have attained different results. That said, these 14 journals have been defined earlier as core medical education publications [25,26].

Second, we used a gender prediction tool to determine whether a first name was characterized as male or female. We recognize that this binary approach is an important limitation of our study and that, as noted above, an individual’s gender is best described by that individual. Based on a survey of the literature for similar studies [11,33], we believe a critical need for future research is work that aims to more accurately ascertain investigator gender.

Third, we did not review the full texts of the knowledge syntheses we analyzed. Therefore, we are unable to make any claims about how author characteristics may have impacted the formulation of their research question, conduct of the knowledge synthesis, or their conclusions. Future work should consider a more in-depth examination of the full text to examine components like inclusion and exclusion criteria and the stated rationale behind the authors’ decisions. Additionally, researchers might consider investigating the full text of reviews to determine whether there is a difference between knowledge syntheses written, for example, by a global team of researchers in comparison to reviews conducted at a single institution. This follow-on work might also include qualitative inquiry to better understand how authors approached their review, including reflections on how their backgrounds may have impacted the conduct of the review. In relation to THE ranking, not all universities submit data for ranking and thus institutions may have been missed. Additionally, we identified authors from associations and organizations, which would not have been ranked, but that may have influence (e.g., the Association of American Medical Colleges).

Conclusion

The production of knowledge syntheses, like all knowledge production, can be influenced by the authors’ characteristics, backgrounds, and the power structures and cultural norms from which they operate [10,20]. In this study, we identified and critically examined the characteristics of the authors of knowledge syntheses to better understand the voices present–and those that may be missing–in the medical education evidence base. While gender parity has improved in recent years, knowledge synthesis authors predominantly work in elite institutions from high-income countries. Although more research is needed to truly understand the impact of these and other author characteristics, we suspect that some of the imbalances observed herein may have negative implications for medical education’s evidence base and its global relevance.

List of core journals and search strategy.

(DOCX)

Click here for additional data file.

Most prolific authors.

(DOCX)

Click here for additional data file.

Author gender.

(XLSX)

Click here for additional data file.

All countries.

(DOCX)

Click here for additional data file.

Institutions.

(DOCX)

Click here for additional data file.

6 Jul 2021

PONE-D-21-18264

Knowledge syntheses in medical education: Examining author gender, geographic location, and institutional affiliation

PLOS ONE

Dear Dr. Maggio,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The manuscript presents interesting data and is of potential interest. However, there are several important limitations that preclude to accept the manuscript in its current form. We invite you to submit a revised version of the manuscript that addresses the items indicated by the reviewers, with a point-by-point responses to their comments. In addition, I also have several suggestions as the Academic Editor:

1. Please provide more details about the manual classification names that Genderize.io reported with <70% certainty (n=151) and failed to identify (n=90) - which gender suggested Genderize (based on the highest probability) and which you identified.

2. Please extend the description of Institutions identification in the "Methods". Particularly, include the information on how many Institutions we not in the Times Higher Education World University Rankings. One of the reviewers pointed out a very important considerations about management of Institutions information by PubMed before and after 2013, this certainly should be described in the "Methods" and considered in the manuscript.

3. Please report the exact PubMed search strategy (in Appendix) by which the dataset has been obtained.

4. Please clarify the phrase "Therefore, for authors other than the first author, we report for each knowledge 172 synthesis all countries represented without regard to an individual’s placement in author 173 order." and explain how this influenced the geographic attribution for the non-first authors.

5. The statistically significant difference you reported in the phrase "Knowledge syntheses were authored by slightly more females (n=2047; 50.5%) than males (n=2005; 49.5%) across all author positions (Pearson Χ2=22.02, p<.001)." seems rather surprising taking into account the small absolute differences. In the "Methods" you only reported the tools used, but have not mentioned the statistical test applied. Please report statistical tests in details. Considering the use of p-value, it is allowed in the PLoS, but please report also the measurement of uncertainty where needed.

6. Please explain more in details which limitations could be related to the fact that your "research team is made up of individuals from the US, Canada, and the  Netherlands", why it is ironic and privileged, and which "various world perspectives that may be relevant in understanding important author characteristics" are reported in the world literature from other regions.

7. At the Figure 1 you reported "Average" number of authors, indicating the mean. Apart of the terminology, the parametric description does not fit well with the highly skewed distribution of number of authors. Please use instead the median and percentiles (the exact percentile is upon your choice) to present these findings.

8. Please provide more information about the Core medical education journals listed in the Supplemental Appendix A, with the major aim to justify their selection for the analysis.

9. Considering that the gender is one of the major topics of your manuscript, please present full first names of the authors included in the analysis and listed in the Suppl B.

10. You have performed an excellent and very elegant work. The vast majority of classifications have no doubts. However, there are several cases that looks to be classified in a different way. In the Appendix C you have provided full list of names and classifications (and I congratulate you with the adherence to the open research and full reporting!), and there a person having the name "Ashley" is classified as "No data on gender". There are oriental names that you classified as one-gender, while they used for both male and female name - for example Xiao (google the "Xiao male or female name") or Su (google the same or see https://genderchecker.com/pages/search-engine). I understand that identification of the gender could be made with knowing additional details you explained in the "Methods" (Linkedin search, etc) and that you can identify the exact gender for a given author with unisex name. Considering this, could you please introduce another column and indicate in the Appendix C for each author whether the gender was identified by Genderize.io or by manual search.

11. Please provide some explanations in the tables' footers to make the tables easier to interpret, for example the table in the Appendix E has a column "Count of KS across all author positions" that requires some brief explanations in the appendix (without a need to look at the main manuscript).

12. The figure 3 is very nice, and represents interesting findings. It would be worth to consider in the "Discussion" the possible reasons for the steep increase in female:male ratio during the short period from 2009 to 2012, and indicate in the "Results" whether these increase has been detected in all groups of countries, or limited only to high-income countries.

Please submit your revised manuscript by Aug 20 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Boris Bikbov

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We noted in your submission details that a portion of your manuscript may have been presented or published elsewhere.

[To undertake this study, we utilized a publicly accessible data set (http://doi.org/10.5281/zenodo.3990481) The data set includes citations and related PubMed metadata for 963 knowledge syntheses published in 14 core medical education journals between 2009-2019. We utilized this existing data set because data reuse has been associated with reduced research waste, faster translation of research findings into practice, and enhanced reproducibility and transparency of science.

In this present study, we used this publicly available data set in an earlier published study, which broadly described the included knowledge syntheses, but did not delve into authorship. We have attached this earlier manuscript to our submission. In this present study, we further enriched the existing data set by linking it with other resources, including genderize.io, the Times Higher Education Rankings, and the World Bank's World Region classification.

We do not believe this present submission constitutes dual publication because we designed this study with different research aims and enriched the existing data set to meet those aims. ]

Please clarify whether this publication was peer-reviewed and formally published. If this work was previously peer-reviewed and published, in the cover letter please provide the reason that this work does not constitute dual publication and should be included in the current manuscript.

3. We note that Figure 4 in your submission contain map images which may be copyrighted. All PLOS content is published under the Creative Commons Attribution License (CC BY 4.0), which means that the manuscript, images, and Supporting Information files will be freely available online, and any third party is permitted to access, download, copy, distribute, and use these materials in any way, even commercially, with proper attribution. For these reasons, we cannot publish previously copyrighted maps or satellite images created using proprietary data, such as Google software (Google Maps, Street View, and Earth). For more information, see our copyright guidelines: http://journals.plos.org/plosone/s/licenses-and-copyright.

We require you to either (1) present written permission from the copyright holder to publish these figures specifically under the CC BY 4.0 license, or (2) remove the figures from your submission:

1.    You may seek permission from the original copyright holder of Figure 4 to publish the content specifically under the CC BY 4.0 license.

We recommend that you contact the original copyright holder with the Content Permission Form (http://journals.plos.org/plosone/s/file?id=7c09/content-permission-form.pdf) and the following text:

“I request permission for the open-access journal PLOS ONE to publish XXX under the Creative Commons Attribution License (CCAL) CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). Please be aware that this license allows unrestricted use and distribution, even commercially, by third parties. Please reply and provide explicit written permission to publish XXX under a CC BY license and complete the attached form.”

Please upload the completed Content Permission Form or other proof of granted permissions as an "Other" file with your submission.

In the figure caption of the copyrighted figure, please include the following text: “Reprinted from [ref] under a CC BY license, with permission from [name of publisher], original copyright [original copyright year].”

2.    If you are unable to obtain permission from the original copyright holder to publish these figures under the CC BY 4.0 license or if the copyright holder’s requirements are incompatible with the CC BY 4.0 license, please either i) remove the figure or ii) supply a replacement figure that complies with the CC BY 4.0 license. Please check copyright information on all replacement figures and update the figure caption with source information. If applicable, please specify in the figure caption text when a figure is similar but not identical to the original image and is therefore for illustrative purposes only.

The following resources for replacing copyrighted map figures may be helpful:

USGS National Map Viewer (public domain): http://viewer.nationalmap.gov/viewer/

The Gateway to Astronaut Photography of Earth (public domain): http://eol.jsc.nasa.gov/sseop/clickmap/

Maps at the CIA (public domain): https://www.cia.gov/library/publications/the-world-factbook/index.html and https://www.cia.gov/library/publications/cia-maps-publications/index.html

NASA Earth Observatory (public domain): http://earthobservatory.nasa.gov/

Landsat: http://landsat.visibleearth.nasa.gov/

USGS EROS (Earth Resources Observatory and Science (EROS) Center) (public domain): http://eros.usgs.gov/#

Natural Earth (public domain): http://www.naturalearthdata.com/

4. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: No

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: I have been invited to review the manuscript entitled “Knowledge syntheses in medical education: Examining author gender, geographic location, and institutional affiliation”. I appreciate the opportunity and thank the editors for considering me.

Overall, I found the article interesting, well-written, and well-informed. Therefore, to start with, I do recommend its publication. Please consider my following comments as an opportunity to improve the manuscript and introduce more clarity where I believe it needs it.

TITLE

I find the study design lacking in the title, which had me second-guessing for quite a while what was done. To avoid this, I suggest including the term “meta-research”. My suggestion, then, would be as follows:

“Knowledge syntheses in medical education: A meta-research examining author gender, geographic location, and institutional affiliation”

I don’t think it necessary to reference the prolific output by John Ioannidis on the topic of meta-research, but if the authors wish to follow up on this, there are several good articles published in Plos One by him.

METHODS

The authors say they have done a case study. While this is not incorrect, I think a better description of the study design is meta-research or research on research.

Lines 131-132

I think you should state clearly from the start that the 14 core journals you used are all indexed in PubMed/MEDLINE, making the “PubMed metadata” phrase a bit less cryptic for readers less versed in bibliometrics.

Lines 143-145

The sentence “Specifically,…” is not supported. Please delete. Additionally, it is repetitive.

Line 147

“Data enrichment” sounds strange to my ears. Why not “variables”?

Lines 158-161

Please consolidate the information provided for the 151 uncertain genders with the 90 not identified, as the same approach was used.

Lines 171-173

I did not understand this sentence.

Lines 186-187

When you use the expression “multiple affiliations,” please specify that you refer to multiple institutional affiliations (as in universities).

Lines 193

An analysis plan is lacking. In my view, this is a strictly descriptive study. You did not do a pre hoc sample size calculation. You used the whole population of knowledge synthesis articles for a set of journals, which is fine. However, you cannot make any generalizations from your data. You should only use descriptive statistics. You have no hypothesis to test.

RESULTS

Please remove any p-values from your results.

A comment on Geography. You say in methods that you will use the World Bank world region classification system based on the country’s gross national income. I think this indicator is profoundly misleading. Lacking a better one to offer, I understand that you would use it. But maybe consider including in the discussion that many HICs are profoundly different from the countries most represented in your findings (not unsurprisingly, US, Canada, and the UK). Many HICs are more like your LMIC. I consider Global South and Global North a more meaningful description of these differences; although inaccurate, it does place a key emphasis on power relations and asymmetries.

Line 284

Please specify the country of the National Health Service mentioned.

DISCUSSION

Please include a paragraph summing up the main findings before beginning the discussion.

Line 339

Should say “author” not “authors”.

Line 356

Reference [20] is to the Cochrane handbook and seems out of place here. I would suggest deleting the whole sentence on waste and bias as it distracts readers from the main thrust of the discussion: the concentration of knowledge synthesis in medical education in some (few) countries (that have in common not only English but also wealth).

Lines 394-396

Author characteristics not only impact conduct and conclusions but, first, and most importantly, the research questions.

Reviewer #2: Your article is fluent and well-organized. Your figures are clear and helpful. However, in their current form, your manuscript and the supporting appendices do not supply adequate information about your dataset. I am hoping that you checked all the affiliation data with the article webpages and just failed to make this clear in the manuscript. Based solely on the information you have supplied, I would believe that the author affiliation information in your dataset, for most of the journals studied, was drawn entirely from PubMed. You do not comment on the fact that MEDLINE indexing of authors' affiliations changed in 2013 or describe how you dealt with this change (see "Author, Corporate Author, and Collaborator Affiliation Display Changes." NLM Tech Bull. 2013 Nov-Dec;(395):e9.) . As you probably know, records indexed prior to 2013 were edited by the indexer to include the first author's affiliation only. For example, -- Cook DA, Bordage G, Schmidt HG. "Description, justification and clarification: a framework for classifying the purposes of research in medical education." Med Educ. 2008 Feb;42(2):128-33. doi: 10.1111/j.1365-2923.2007.02974.x. Epub 2008 Jan 8. PMID: 18194162. -- is indexed with Cook's Mayo Clinic affiliation only. Bordage's University of Illinois at Chicago affiliation and Schmidt's Erasmus University affiliation are not reflected in the PubMed record. If your affiliation data for most journals studied were derived solely from PubMed records, your data collection for 1999-2012 articles is seriously flawed. Even PubMed affiliation data for late 2013 - 2019 publications are not entirely trustworthy. The affiliation data are entered by the publisher and are not edited by PubMed's indexers. If the publisher enters the first author's affiliation only, that is the only affiliation reflected in PubMed. For your study, use of records from a literature database that focuses on affiliation indexing (like Scopus) would have made more sense. For an article about the effect of the indexing change on affiliation studies, see Ibarra, M.E., Ferreira, J.P., Torrents, M. et al. "Changes in PubMed affiliation indexing improved publication identification by country." Scientometrics 115, 1365–1370 (2018). https://doi.org/10.1007/s11192-018-2714-x

My other concerns are minor and/or philosophical and would not be sufficient to prevent publication. You frequently refer to the Cochrane Handbook. I feel that the Cochrane Handbook would be a more appropriate source of guidance if you were discussing reviews of therapeutic interventions and diagnostic techniques, the types of reviews published in the Cochrane Database of Systematic Reviews. Representation of research from around the world is relevant to reviews of therapeutic interventions and diagnostic techniques. As you note, educational techniques may be more culture-specific. You mentioned the fact that appropriate feedback methods vary depending on the culture. Is this, perhaps, a point in favor of production of culture-specific syntheses rather than a point in favor of including all voices in a given article? Authors and readers want to focus on information that is relevant in their setting. If author/reader time and article length were unlimited, many educators would want to know everything about all cultures and educational systems, but sadly both time and article length are typically limited. This brings me to my next point. The Cochrane Handbook would be a more appropriate guide if you were discussing articles in journals with essentially no article length limits. As articles in the Cochrane Database of Systematic Reviews are often very lengthy, there is room in these reviews to explore the delivery of care in nations with different economic resource levels. I do agree that information and voices from around the world are relevant to many educational research topics. I also think that many reviews must be focused on educational systems and/or cultures similar to the authors' to accomplish the authors' objectives within many journals' word-number limits ( even when awareness of other cultures and systems would be helpful to educators who deal with students who have come from other cultures).

You mention the growth in number of authors per article and provide some possible reasons for this growth. You haven't mentioned the rapid growth in the publication of systematic reviews and meta-analyses as one reason for this change. Guidelines for production of systematic reviews/meta-analyses require the participation of a multi-author team.

Your citation 20 on line 97 refers to the entire Cochrane Handbook. It would be better to cite specific chapters, as you have in other in-text citations. Your reader should not have to comb the entire Handbook to find the source of your statement.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Vivienne C. Bachelet

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

14 Sep 2021

Please see the attached revision table.

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

11 Oct 2021

Knowledge syntheses in medical education: Examining author gender, geographic location, and institutional affiliation

PONE-D-21-18264R1

Dear Dr. Maggio,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Boris Bikbov

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: In my opinion, the authors have taken into account all suggestions provided by the reviewers, including mine. They have also adequately explained when choosing not to.

I recommend this version por publication in the journal.

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Vivienne C. Bachelet

Reviewer #2: No

13 Oct 2021

PONE-D-21-18264R1

Knowledge syntheses in medical education: Meta-research examining author gender, geographic location, and institutional affiliation

Dear Dr. Maggio:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Boris Bikbov

Academic Editor

PLOS ONE