Analysis of the distribution and scholarly output from National Institute of Academic Anaesthesia (NIAA) research grants.

The National Institute of Academic Anaesthesia (NIAA) was founded in 2008 to lead a UK strategy for developing academic anaesthesia. We aimed to assess the distribution of applications and quantify the academic returns of NIAA-supported research grants, as this has hitherto not been analysed. We sought data on the baseline characteristics of all grant applicants and recipients. Every grant recipient from 2008 to 2015 was contacted to ascertain the status of their supported research projects. We also examined Google Scholar, Scopus® database and InCites Journal Citation Reports for citation, author and journal metrics, respectively. In total, 495 research project applications were made, with 150 grants being awarded. Data on 121 out of 150 (80.7%) grant awards, accounting for £3.5 million, were collected, of which 91 completed studies resulted in 140 publications and 2759 citations. The median (IQR [range]) time to first or only publication was 3 (2-4 [0-9]) years. The overall cost per publication was £14,970 (£7457-£24,998 [£2212-£73,755]) and the cost per citation was £1515 (£323-£3785 [£70-£36,182]), with 1 (0-2 [0-8]) publication and 4 (0-25 [0-265]) citations resulting per grant. The impact factor of journals in which publications arose was 4.7 (2.5-6.2 [0-47.8]), with the highest impact arising from clinical and basic science studies, particularly in the fields of pain and peri-operative medicine. Grants were most frequently awarded to clinical and basic science categories of study, but in terms of specialty, critical care medicine and peri-operative medicine received the greatest number of grants. Superficially, there seemed a geographical disparity, with 123 (82%) grants being awarded to researchers in England, London receiving 48 (32%) of these. However, this was in proportion to the number of grant applications received by country or city of application, such that there was no significant difference in overall success rates. There was no significant difference in productivity in terms of publications and citations from grants awarded to each city. The 150 grants were awarded to 107 recipients (identified as the most senior applicant for each grant), 27 of whom received ≥ two grants. Recipients had a median career total of 21 (8-76 [0-254]) publications and 302 (44-1320 [0-8167]) citations, with an h-index of 8 (3-22 [0-54]). We conclude that a key determinant of grant success is simply applying. This is the first study to report the distribution and scholarly output of individual anaesthesia research grants, particularly from a collaborative body such as the NIAA, and can be used as a benchmark to further develop academic anaesthesia in the UK and beyond.

Introduction

The National Institute of Academic Anaesthesia (NIAA) was set up in 2008 to address the perceived crisis in academic anaesthesia by leading a national strategy to “promote, support and deliver world class biomedical and health research in anaesthesia” 1, 2, 3. One of the strategic aims of the NIAA is to support high‐quality researchers and research by awarding bi‐annual competitive grant funding on behalf of the four founding partners and 11 funding partners 4 (Table 1). To date, nearly £7 million has been awarded in the nine years since the inception of the NIAA 4. Most awards are to research project grants, although the NIAA has awarded 23 undergraduate, six doctoral and seven senior academic grants worth a total of £2.3 million. Following research grant awards, successful applicants are requested to submit a first year and interim or final report for review and publication on the NIAA website, however details of grant‐supported publications are not mandated 5. Although the NIAA is addressing the crisis it initially set out to resolve, a systematic and quantitative assessment of the scholarly productivity of NIAA‐facilitated research grants has not been performed to date.

Table 1

Founding and funding partners of the NIAA, and the year they joined the NIAA

Partner	Year joined NIAA
Founding partners
Anaesthesia	2008
Association of Anaesthetists of Great Britain and Ireland (AAGBI)	2008
British Journal of Anaesthesia (BJA)	2008
Royal College of Anaesthetists (RCoA)	2008
Funding partners
Association for Cardiothoracic Anaesthesia (ACTA)	2008
Obstetric Anaesthetists’ Association (OAA)	2008
Anaesthesia Research Society (ARS)	2009
Difficult Airway Society (DAS)	2009
Neuro Anaesthesia and Critical Care Society of Great Britain and Ireland (NACCSGBI)	2009
Society for Education in Anaesthesia UK (SEA UK)	2009
Association of Paediatric Anaesthetists of Great Britain and Ireland (APAGBI)	2010
Vascular Anaesthesia Society of Great Britain and Ireland (VASGBI)	2010
Regional Anaesthesia UK (RA UK)	2011
British Society of Orthopaedic Anaesthetists (BSOA)	2013

We therefore aimed to assess the distribution of grant applications and quantify the academic returns of all NIAA‐supported research project grants, excluding undergraduate, doctoral and senior academic grants, awarded since inception. We also sought to explore the association between geography, subject of study and researcher baseline characteristics on the success of grant applications and academic output of NIAA research grants.

Methods

Details of all NIAA research grants applied for and awarded between 9 December 2008 and 03 December 2015 were obtained from the NIAA, including: the description of the grant award; the applicant name; the country, city and institution to which the grant was awarded; the year the grant was awarded; the title of the project; and funding applied for and awarded. Undergraduate, doctoral and senior academic grants were excluded. There is no expectation of publication outputs with undergraduate grants; the primary marker of success of a doctoral grant is successful award of a PhD, rather than publications; and senior academic grant awards contribute to more than individual research projects. Moreover, only seven individuals received senior academic grants, thus the generalisability of this last grant stream is more limited.

Information on project status was sought by directly contacting every grant recipient using a standardised e‐mail sent by two of the authors (KE, AMD). Three rounds of e‐mails were undertaken to maximise response rates, the final round ending on 31 May 2017, and a call to action was printed in the NIAA monthly newsletter (February 2017). Where no responses were obtained, alternate contact details were pursued by examining other publications by each author or contact information on institutional web pages. Grant recipients were asked to provide details of project outputs for each grant awarded. Projects were defined as ongoing if stated as such by recipients. Project outputs were defined as peer‐reviewed publications that were directly as a result of NIAA grants awarded; correspondence and abstracts were excluded. The date of publication, journal title and digital object identifier (DOI) were ascertained for each publication. If the DOI was not directly provided by grant recipients, they were determined by directly querying Google Scholar using provided publication details. Using the DOI, Google Scholar was again queried to determine number of citations each manuscript received up to and including 1 September 2017. Publication‐ and citation‐related metrics were only examined for NIAA‐funded studies that had been completed. We did not seek data on presentations as the global reach and quantification of different forums for these is heterogeneous.

The categories of research were determined by examining the methodology of each grant applied for and awarded. These were categorised into one of the following categories:

Clinical on patients, such as a clinical trial

Human study on non‐patient volunteers (e.g. physiology or pharmacology study)

Basic science study on animals, cells or tissue

Bench study (e.g. device, technology or equipment study)

Process/system study (e.g. data set analysis, surveys)

Simulation or manikin study (e.g. clinician performance)

Meta‐analysis/mathematical analyses

Grants were then further sub‐categorised into the specialty to which they pertained (e.g. airway, cardiac, critical care, regional). The specialty could encompass any of the categories (e.g. ‘cardiac — basic science’ or ‘regional — bench’). The geographical location was assessed by determining the country, city and individual institution to which the grants were awarded. For quantitative analysis based on population, this was determined from an electronic search, and the number of consultant anaesthetists in each nation was determined by the most recent census data available and publically available reports 6, 7.

The scholarly productivity of the most senior grant recipient was examined, both directly from NIAA research grant funding and overall. The senior applicants were determined by review of each applicants’ credentials in analysing h‐index and academic grade. The total grant money, publications and citations each recipient produced directly related to NIAA research grant funding was determined. Using the Scopus® author database 8, the career total number of publications and citations achieved by each grant recipient up to and including 1 September 2017 was determined. The h‐index, which is an indicator of scholarly output (publications and citations) 9, was also determined for each primary NIAA research grant recipient using the Scopus author database.

The 2016 impact factor for each journal in which NIAA‐supported publications arose was determined by querying the InCites Journal Citation Reports 2.0 (Web of Science™, Thomson Reuters®, NY, USA) using title. For context, the 2016 journal impact factor for Anaesthesia was 4.7, Anesthesia and Analgesia 4.0, Anesthesiology 5.8 and British Journal of Anaesthesia 6.2. The ‘total impact score’ was calculated for each grant by multiplying the total number of publications by the sum of all impact factors of each journal in which a publication arose per grant 10. The aim of this method was to quantify the total impact factor from all publications related to an individual grant.

All data obtained were inputted onto a standardised and anonymised Microsoft Excel 2016 (Microsoft Corp, Redmond, WA, USA) spreadsheet. SPSS® for Mac version 23.0 (SPSS Inc., Chicago, IL, USA) was used for statistical analysis. Sex‐ and location‐related success was compared using the Chi‐square test. Correlations were analysed using Spearman's correlation (r), with p < 0.05 considered statistically significant.

Results

The NIAA received 495 research project applications for a sum total of £14,116,565 requested during the study period. The median (IQR [range]) sum applied for per grant was £17,429 (£10,326–£46,719 [£500–£157,438]). A total of 150 grants applications (30.3%) were successfully awarded to a total sum of £4,220,149 over the study period, with £15,000 (£10,050–£48,486 [£1038–£143,419]) awarded per grant. The largest total grant sums were awarded in the years 2010, 2014 and 2015 (Table 2).

Table 2

Total grants awarded as well as all grants where data were available by year and category of award. Productivity metrics in terms of publications and citations are reported. Data arranged by year or sum awarded

	Applications	Awarded					Outputsb
	All studies		Missing	Completed	Ongoing
	n	n	£	n	n	n	Publications	Citations	Citations/publication	Publications/grant	Citations/grant	£/publication	£/citation
Year
2008	57	14	553,029	2	12	0	23	649	28	2	54	19,697	698
2009a	44	15a	351,001	0	12	1	33	942	29	3	79	9450	331
2010	71	22	672,842	3	19	0	33	636	19	2	33	19,329	1003
2011	64	22	467,862	6	11	5	17	275	16	2	25	20,025	1238
2012a	62	19a	411,042	2	16	0	11	101	9	1	6	30,818	3356
2013	61	20	370,851	7	8	5	9	38	4	1	5	15,723	3724
2014	68	18	699,970	5	10	3	12	118	10	1	12	35,359	3596
2015	68	20	693,551	4	3	13	2	0	0	1	0	59,776	–
Category
Basic sciencea	158	62a	2430,429	9	47	4	84	1637	19	2	35	20,322	1043
Clinical	261	64	1,443,974	14	34	16	50	1083	22	1	32	17,729	819
Bencha	31	14a	236,133	3	6	4	4	36	9	1	6	29,486	3276
Simulation	20	7	72,088	2	3	2	2	3	2	1	1	17,092	11,394
Process/ system	18	3	37,525	1	1	1	0	0	0	0	0	–	–
Mathematical analysis	7	0	0	–	–	–	–	–	–	–	–	–	–

Discrepancies in grants with data are due to two cancelled basic science studies and one cancelled bench study in 2009 and 2012.

All outputs are reported for completed studies.

Of the awarded grants (n = 150), no responses were received to account for data from 29 (19.3%) grants (Appendix S1). We obtained responses confirming the project status of 121 out of 150 awarded grants (80.7%), accounting for £3,531,216 (83.7%) of awarded funding. There were 91 completed studies, 27 studies ongoing at various stages and three grants for studies that were never conducted (Fig. 1). The monies from two of these latter three grants were returned to the funder, and one grant was awarded to a researcher who subsequently passed away. Out of the 27 ongoing studies, one was from a grant awarded in 2009, five were from grants awarded in 2011, five were from 2013, five from 2014 and 13 from 2015.

Figure 1

Flowchart of NIAA research grant applications and awards.

Grant funding arose from a combination of 17 different partners, but the majority of awards and money came from the AAGBI/Anaesthesia and the Royal College of Anaesthestists/British Journal of Anaesthesia, providing 53 grants worth £1,100,513 and 46 grants worth £2,3368,848, respectively (Table 3). There was a similar success rate across each funding partner.

Table 3

Total grants awarded as well as all grants where data was available from each funding partner. Productivity metrics in terms of publications and citations are reported. Data arranged by sum awarded

Funding partner(s)	Applications	Awarded					Outputsb
		All studies		Missing	Completed	Ongoing
	n	n	£	n	n	n	Publications	Citations	Citation/ publication	Publications/ grant	Citations/ grant	£/ publication	£/ citation
BJA/RCOAa	115	46	2,368,848	8	31	6	64	1306	20	2	42	23,945	1173
AAGBI/Anaesthesia a	200	53	1,100,513	10	32	9	50	1166	23	2	36	15,834	679
OAA	30	13	281,174	3	8	2	7	81	12	1	10	27,407	2368
AAGBI/Anaesthesia & BJA/RCoA	25	5	73,749	1	2	2	3	140	47	2	70	9995	214
APAGBI	15	4	73,290	2	0	2	0	0	0	0	0	–	–
ARS	14	7	66,958	2	5	0	2	28	14	0	6	24,458	1747
ACTA	27	4	54,232	0	4	0	5	20	4	1	5	10,846	2712
NACCSGBI	13	3	39,836	1	1	1	1	10	10	1	10	9834	983
ACTA/VASGBI/BJA/RCoA	7	1	39,552	0	1	0	1	0	0	1	0	39,552	–
DAS	10	4	36,526	1	1	2	1	1	1	1	1	3855	3855
SEA UK	13	4	18,046	0	2	2	1	1	1	1	1	4713	4713
APAGBI/BJA/RCoA	4	1	18,005	0	1	0	2	3	2	2	3	9003	6002
VASGBI	5	1	17,415	0	1	0	1	1	1	1	1	17,415	17,415
Anaesthesia/Wiley	2	1	15,000	0	1	0	1	0	0	1	0	15,000	–
RA UK	11	2	13,514	0	1	1	1	2	2	1	2	10,000	5000
BSOA	4	1	3500	1	–	–	–	–	–	–	–	–	–

Discrepancies in grants with data are due to two cancelled AAGBI/Anaesthesia studies and one cancelled BJA/RCOA study in

All outputs are reported for completed studies

Grants were most commonly awarded to clinical (n = 64), basic science (n = 62) and bench (n = 14) research studies (Table 2), but this was in proportion to applications from these categories, with success rates of 25%, 39% and 45%, respectively. There was a trend of reducing basic science research funding, with increasing funding for clinical research (Appendix S2), but again in line with applications from these categories. Meta‐analysis/mathematical analyses received no funding despite seven applications, and only 17% of process/system grant applications were successful. Studies in critical care (n = 28), pain (n = 24) and peri‐operative medicine (n = 20) received the largest number of NIAA research grants and funding (Appendix S3).

There was a significant correlation between the number of grant applications submitted and success rate (r = 0.413, p = 0.001). 80.4% of applications were from England (n = 398), 12.3% from Scotland (n = 61), 4.8% from Wales (n = 24), 1.4% from Ireland (n = 7), 0.4% from Northern Ireland (n = 2), and 0.2% from Netherlands (n = 1), New Zealand (n = 1) and USA (n = 1). Correspondingly, 82% of all grants (n = 123) were awarded to researchers in England, with the NIAA awarding £6 per thousand population. A total of £13 was awarded per thousand population in Scotland, £2.6 in Wales, £1 in Ireland, whereas no grants were awarded to Northern Ireland (Appendix S4). However, there was no significant difference in grant application success between countries (p = 0.503). In terms of cities, London accounted for both the largest number (n = 48) and the largest total grant funding applied for and awarded (Fig. 2, Appendix S5). However, the success rate of applications from London was no different to that of all other cities combined (36.9%, p = 0.137).

Figure 2

Geographical location of NIAA grant applications from the UK (a) and London (b) as well as grants awards in the UK (c) and London (d). The size of the dots represents the amount of money applied for, and the colour of the dots reflects the number of applications (a and b) or the success rate (c and d) Because London had >80% of grant applications and awards, it has been plotted separately.

Three hundred and seventy‐six applications (76%) were from male applicants, and 119 (24%) were from female applicants. However, sex had no effect on the likelihood of grant application success (31.1% vs. 27.7%, respectively, p = 0.484). There was a trend of an increasing proportion of female grant applicants since 2008 (Fig. 3).

Figure 3

The annual number of male (blue line) and female (red line) grant applicants from 2008 to 2015.

Of the 107 senior grant recipients, the median (IQR [range]) number of NIAA research grants awarded was 1 (1–1 [1-6]), and the total award was £18,867 (£11,256–£50,642 [£1039–£279,144]) per recipient. This produced 1 (0–2 [0-8]) publications and 8 (0–38 [0-265]) citations attributable to NIAA grants. Analysis of the scholarly background of the senior grant recipient revealed a total number of publications of 21 (8–76 [0-254]) and citations of 302 (44–1320 [0-8167]), with an h‐index of 8 (3–22 [0-54]) (Appendix S6). Twenty‐seven grant applicants were awarded ≥ 2 grants, with multiple‐grant recipients receiving 71 of the 150 grants (47.3%). Grant recipients applied 1 (1–2 [1-7]) time per awarded grant, but there was no correlation between h‐index and success rate of grant applications. Nineteen applicants received two grants, four received three grants, one applicant received four, one received five and two received six grants (Table 4).

Table 4

Recipients of single and multiple grants, with the overall awards per recipient per grant. £/grant/recipient is reported as median (IQR [range])

Awarded grants	Number of recipients	Total grants awarded (£)	£/grant/recipient
6	2	485,949	35,839 (16,697–64,483 [9918–87,087])
5	1	71,209	7825 (7428–11,045 [5956–25,000])
4	1	98,476	18,097 (13,506–36,547 [7800–51,221])
3	4	318,761	14,970 (9491–31,905 [3000–72,301])
2	19	1,451,445	45,820 (13,773–63,745 [3855–106,713])
1	80	1,791,369	14,820 (9992–23,968 [1039–143,419])

Out of the 91 completed studies, accounting for £2,767,525, 77 (84.6%) grants directly contributed to ≥ 1 peer‐reviewed publications, the median (IQR [range]) time to first or only publication was 3 (2–4 [0-9]) years and the time to the last publication (where > 1 paper, n = 31) was 5 (4–7 [1-9]) years after grants were awarded. A total of 140 publications and 2759 citations were attributable to NIAA‐facilitated grants with 0 (0–0 [0-2]) publications per year and 1 (0–5 [0-38]) citation per year since publication. Up to 1 September 2017, there was 1 (0–2 [0-8]) publication and 4 (0–25 [0-265]) citations per grant awarded, with the overall cost per publication of completed studies being £14,970 (£7457–£24,998 [£2212–£73,755]) with a mean (SD) of £21,031 (£19,170), whereas the cost per citation was £1515 (£323–£3785 [£70–£36,182]), with a mean (SD) of £5107 (£8600). Bench and simulation studies represented the greatest cost per publication and citation (Table 2). There was a significant correlation between the sum of the grant awarded and the number of publications and citations (r = 0.294, p = 0.005 and r = 0.210, p = 0.045, respectively) (Table 5, Appendix S7).

Table 5

Grant outcome data for completed projects stratified by different value grants

Grant award (£)	Grants with data	Completed projects	Total awarded (£)	Publications	Citations	Publications/grant	Citations/grant	£/publication	£/citation
0–9999	24	16	97,908	10	177	1	11	9791	553
10,000–19,999	43	31	451,448	45	716	1	23	10,032	631
20,00–29,999	14	11	263,257	19	626	2	57	13,856	421
30,000–49,999	14	13	558,195	24	476	2	37	23,258	1173
50,000–99,999	24	18	1,146,585	32	457	2	25	35,831	2509
≥ 100,000	2	2	250,132	10	307	5	154	25,013	815

The impact factor of journals in which NIAA‐supported publications arose was 4.7 (2.5–6.2 [0–47.8]). Six out of the 140 publications were published in journals with a 2016 impact factor of 10–20 (American Journal of Respiratory and Critical Care Medicine, Brain, JAMA Neurology, Journal of Hepatology, Journal of Pineal Research and Science Translational Medicine) and one was published in a journal with an impact factor of 47.8 (The Lancet). Forty‐seven (33.6%) were published in anaesthetic literature, and 93 (66.4%) were published in non‐anaesthesia journals. The total impact score for each grant with publications was 5.1 (2.4–38.9 [0–310.6]). There was no correlation between the sum of the grant awarded and the impact factors of journals in which publications arose (r = 0.014, p = 0.866). There were 6 (4–9 [1-108]) authors or collaborators per publication.

Discussion

This study provides the first systematic analysis and benchmarking of anaesthesia grant activity in the UK. Our data demonstrate significant variation in geography, subjects and baseline characteristics of grant applicants. Based on these data, a vital determinant of grant success is simply applying.

The NIAA continues to develop its role in nurturing scholarly productivity in UK anaesthesia. It follows a pattern of increased research funding of UK university institutions 11, 12, 13, which has seen greater impact and quality of research being produced over the last two decades 14, 15. However, compared with other specialty research, anaesthesia receives significantly less investment. The Wellcome Trust funded £822 million across health research areas in 2015–2016 but funded on average one anaesthesia‐related application per annum compared with an average 1200 annual awards 16. Anaesthesia research accounts for < 2% of National Institute for Health Research (NIHR) studies 17, yet mental and public health and neurological disorders comprise 15% of studies. The Association of Medical Royal Colleges report that cancer, cardiovascular and neurological research account for 70% of charity‐funded research 18. The range of annual research project funding by individual funding charities or organisations in the UK is from £40,000 to £400 million, with the NIAA sitting at the lower end of this spectrum at around £0.5 million per annum 18.

There are several academic anaesthesia institutions internationally that serve a similar role to the NIAA. The International Anesthesia Research Society in the USA has contributed more than £10.6 million ($14 million USD) to more than 200 research grants since 1983, and the Canadian Anesthesia Research Fund (CARF) delivered £2.24 million ($3.685 million CAD) to 159 projects between 1985 and 2005 19, 20. Since 2007, CARF has disseminated a total £1.1 million ($1.91 million CAD) to 97 separate project grants (personal communication). The European Society of Anaesthesiology runs a competitive grants programme that issues seven grants annually up to a sum of £184,367 (€205,000) 21, having delivered a total of 80 grants in the last 15 years 22, which is dwarfed by the £1 million (A$1.7 million) that is awarded annually by the Australian and New Zealand College of Anaesthetists 23. Thus, in the financial climate of modern medicine, the NIAA should be celebrated in its role in funding UK anaesthetic research when compared with global counterparts.

The cost per publication is one potential function of the cost of productive research. To contextualise anaesthesia research, costs per publication ranges from £13,000 to £21,000 in surgical research, but is as low as £395 for respiratory research 12, 13, 24, 25. Across medical specialties, the Association of Medical research Charities reports the cost per publication as £55,000 18. However, when compared with UK data from infectious diseases research, the NIAA delivers superior costing metrics, with the cost per publication reaching up to £100,000 for HIV, TB and malaria research 26.

Academic outputs in the form of publications and citations within the same specialty could therefore be viewed as a function of productivity. The scholarly productivity of NIAA grants are therefore best compared with awards from similar organisations. Although 36 years older and awarding research grants since 1987 27, the Foundation for Anesthesia Education and Research (FAER) has delivered a total of £343.17 million ($448.44 million USD) through 391 grants in North America. The FAER grant recipients had a career total median of 33 publications and 724 citations per grant, which exceeds our results of 21 and 302, respectively 28. Of course, the longer the lag following grant awarding, the greater the possibility that more publications and citations (and undoubtedly clinical uptake) will follow 29. Although total grant sums awarded per application are greater in the USA, there are only 24 professors of anaesthesia in the UK compared with more than 130 academic institutions in the USA 30. Moreover, a trend for a reducing proportion of anaesthesia‐related publications has been demonstrated over a sustained period of time in UK anaesthesia research 31, particularly before the introduction of the NIAA 32, 33, 34, and there may be room for further development in the quality of clinical anaesthesia studies 35. Notably, this is the first study to quantify the output of individual awards, as there are few data available on the cost per publication and citation arising from anaesthesia grants. Our results can therefore be used as a benchmark to develop academic anaesthesia in the UK and beyond.

Our data reveal that the NIAA appears impartial to the baseline demographics of grant applications; grants are awarded to those who apply most. Although English‐based, in particular London, researchers received the greatest funding, this is likely a function of more applications arising from those locations. This indicates that there are pockets of established researchers where applications frequently come from, which further supports previous evidence of a handful of departments contributing most of the academic outputs 36. It may be that researchers outside of these locations require further support to apply for NIAA funding. There was also a clear difference in the sex of applicants, but not in success rates. This sex discrepancy is seen throughout UK medical research, but is likely a reflection of the differences in anaesthesia, with just 32% of the consultant workforce being women 37.

The NIAA has conducted two research priority settings for anaesthesia and peri‐operative care 37, 38. Clinical outcome‐related research is heavily weighted in importance in both proposals, which is reflected in our data by a recent increase in funding for this category of research (Appendix S2).

This study has several limitations. First, we were unable to obtain data on research output from a fifth of NIAA grants. A response of rate of > 75% has been shown to validly reduce the impact of response bias 39, yet data from nearly £690,000 worth of funding was not available to us. Second, we found the median time from grant awarding to publication was 3 years, but 27 studies were still ongoing and completed studies may yet lead to publications. This phenomenon is an unavoidable limitation irrespective of when this study was conducted. This might unfairly disadvantage the representative productivity from certain centres that have several large ongoing studies, and might suggest improved productivity from centres who received small grants for smaller projects that can be completed in a relatively short period of time. A third limitation is that there are often multiple grant applicants for each grant, but we analysed data from the most senior researcher on each awarded grant. This was to ensure the highest scholarly backing for individual projects was demonstrated. However, this might limit the applicability of researcher‐specific data in the more junior grant recipients. Fourthly, we applied the h‐index to assess scholarly productivity throughout the career of researchers, but this instrument has some drawbacks and might have different metrics in specialty research compared with general medical research 30, 40. Despite this, it remains the most validated measure for academic output from researchers to date 36, 41, 42. In all, 27 researchers received more than one NIAA grant, and there was a lack of clarity at times as to which grant contributed to which publication. Nonetheless, the total number of publications and citations for each researcher, location, and research subject should not be affected by this discrepancy. Many grant recipients may have received support from other sources, be it financial, resource or time, that could have contributed to the productivity of each NIAA grant 43. This is challenging to quantify and follow‐up, and was beyond the scope of this study. The financial and time‐costs of grant application and publication preparation are also challenging to account for. Moreover, we only assessed the productivity arising from research grants, and career‐development grants, undergraduate and doctoral grants were not assessed, despite accounting for more than a third of the NIAA‐directed funding. This was specifically chosen in order to focus on individual research studies, although the data might have provided further insight. In addition, the geographical breakdown of data could be limited in generalisability to a certain extent due to the small number of grants awarded to certain locations. Furthermore, not all funding partners joined the NIAA at the same time; some became partners more recently which could to some extent explain funding discrepancies. Finally, we have not assessed the impact of NIAA‐supported research on clinical outcomes, but scholarly surrogates. Although this is the ultimate marker of research impact, determining whether the academic impact is reflected in patient outcome improvements is challenging to quantify.

In conclusion, this is the first study to benchmark output from individual grants in the anaesthesia literature. Further investment and planning of anaesthesia research will help the NIAA to achieve the goal of delivering “world class biomedical and health research in anaesthesia” 1, 2.

Appendix S1. Baseline characteristics for grants awarded with no response from recipients. Data arranged by year of award.

Appendix S2. The annual number of grants awarded to each category of research. Data arranged by sum awarded.

Appendix S3. Grants awarded to each specialty.

Appendix S4. Grants awarded for every country from which each recipient was based. Data arranged by sum applied for.

Appendix S5. Grants awarded for every city from which senior or primary grant recipient was based, arranged by sum of money applied for. The percentage of completed studies accounts for the fraction of studies that have been completed from those that data were available for.

Appendix S6. Coded grant recipients’ baseline characteristics, NIAA research grant applications and awards, NIAA research grant‐supported outputs and total career outputs. Data arranged by city. The h‐index was determined using Scopus® database, searched on 1 September 2017.

Appendix S7. Correlation between the grant sum awarded (£) and the number of publications (blue dots, blue line) and citations (red crosses, red line).

Click here for additional data file.