Comparative Performance of Adult Social Care Research, 1996-2011: A Bibliometric Assessment.

Decision makers in adult social care are increasingly interested in using evidence from research to support or shape their decisions. The scope and nature of the current landscape of adult social care research (ASCR) need to be better understood. This paper provides a bibliometric assessment of ASCR outputs from 1996 to 2011. ASCR papers were retrieved using three strategies: from key journals; using keywords and noun phrases; and from additional papers preferentially citing or being cited by other ASCR papers. Overall, 195,829 ASCR papers were identified in the bibliographic database Scopus, of which 16 per cent involved at least one author from the UK. The UK output increased 2.45-fold between 1996 and 2011. Among selected countries, those with greater research intensity in ASCR generally had higher citation impact, such as the USA, UK, Canada and the Netherlands. The top five UK institutions in terms of volume of papers in the UK accounted for 26 per cent of total output. We conclude by noting the limitations to bibliometric analysis of ASCR and examine how such analysis can support the strategic development of the field.

Introduction

It is hard to pinpoint the first research studies in adult social care but there is no doubt that most decision makers in the field—whether national or local, public or private, commissioners or providers, people who use services or carers—have become increasingly interested in using evidence from research to support or shape their decisions. Growth in the demand for adult social care evidence has both stimulated and been stimulated by apparent growth in the supply of research. There have also been investments in the infrastructure, including reviews on the state of social care research (e.g. Sharland, 2009; Reilly ), and establishment in England of the National Institute of Health Research's (NIHR) School for Social Care Research (SSCR) in 2009 (Knapp ), and also of a national Social Care Research Ethics Committee in the same year. Social care has increasingly come to be recognised as a ‘research field’ (Sharland, 2009). The scope and nature of the current landscape of adult social care research (ASCR) needs to be better understood, and approaches to measure its impact explored, in order to shape its future. The objective of this paper is to provide a bibliometric assessment of ASCR outputs from 1996 to 2011 within the UK and compared to other countries, and to discuss the implications of using bibliometrics in describing and assessing patterns of ASCR. Bibliometrics is the quantitative analysis of scientific publications and their citations, typically focusing on journal papers in the peer-reviewed literature (De Bellis, 2009). It is one of a set of evaluation methods that may be used to help assess research (Ismail ), and has been used in comparative analysis of other fields, typically in the medical sciences (Lewison ; Patel and Sumathipala, 2006), but also in mental health (Larivière ).

Bibliometrics has been used occasionally in the ASCR field, each time with more specific aims than we have here. Urquhart and Dunn (2013), for example, analysed use of a particular data-set, McFadden and Best reviewed the quality of a number of different data-sets and web search engines for social work research, and Holden looked at social work articles appearing in a specific journal. Similarly, McVicar charted the use of specific research methods and Holden explored bibliometrics as a method for social work. To our knowledge, no studies have attempted a comprehensive bibliometric assessment of ASCR, which is the purpose of our paper. There are three reasons for doing so. The first is to provide research funders with an evidence base to inform decisions on operations, policy and strategy: bibliometrics contributes to that evidence base by providing those responsible for research management with data-capturing trends in research activity and impact, by institute, country and field. The second is to demonstrate to the field of ASCR where its major producers are and how their roles have evolved overtime. The third reason is, through the provision of such knowledge, to identify strengths in the ASCR field and thence develop means of addressing limitations.

We examine trends in research outputs in the UK over time, compare output and impact (as measured by citations) by country, and examine patterns of output, impact and collaboration for different centres within the UK. We begin by explaining how we defined and identified ASCR papers. In the discussion, we highlight the limitations of the analysis and use of bibliometrics to analyse ASCR impact, and then draw out policy observations.

Methods

The key to any bibliometric analysis is to accurately define and identify a data-set of papers that, in this case, constitute ASCR output. In this section, we describe the selection of the underlying bibliographic database, the process we used to identify and retrieve ASCR papers and the indicators we used to analyse those papers. (We have posted a detailed account of the methods for those interested in replicating this study in the future; see ).

Selection of database

Worldwide, there are two major bibliographic databases that are suitable to compute statistics on the scientific impact of peer-reviewed research outputs based on citation analyses, namely the Web of Science (WoS) by Thomson Reuters and Scopus by Elsevier. These databases index the cited references of each document they contain (e.g. articles, conference papers, letters, reviews, notes and press releases). We chose Scopus as our source database, as it had a broader coverage of the relevant literature. For instance, about 70 per cent of the specialist journals retained in the data-set of ASCR are indexed in the WoS; the papers from these journals represent roughly 80 per cent of the papers in the full set of journals retained in Scopus (see below). Similarly, the portion of the query based on keyword searches (see below) would, in the WoS, have retrieved approximately 65 per cent of the papers retrieved from Scopus using exactly the same keyword set.

Construction and development of the ASCR data-set

The bibliometric analysis was commissioned by the Personal Social Services Research Unit (PSSRU) at LSE and SSCR as part of an exploration of methods to assess impact from research, and to ascertain whether bibliometrics could be used in future to assess SSCR-funded studies. To provide a starting point, and because the core of SSCR (in its first phase) comprised six research groups that existed prior to its establishment and were awarded core membership because of their ASCR track records, the names or abbreviations of these units as well as those of SSCR were searched for in the author addresses of papers in Scopus to retrieve papers that were highly likely to be representative of the research to be relevant to ASCR. The six units were: PSSRU, London School of Economics and Political Science (LSE); PSSRU, University of Manchester; PSSRU, University of Kent; Social Policy Research Unit, University of York; Social Care Workforce Research Unit, King's College London; and Tizard Centre, University of Kent. This set of papers (referred to below as the SSCR papers) were used to develop and validate a broader data-set of ASCR outputs.

The data-set of ASCR papers was developed by examining the population of 22 million papers published between 1996 and 2011 and indexed on Scopus, and extracting papers that we defined as relevant to adult social care. We began by identifying papers with the term ‘social care’ in their titles, abstracts or author keywords, but found that 71 per cent of such papers had a UK address, suggesting it was a highly specific term used in the UK context. (In fact, the usage of this term is negligible outside the UK; the next countries with the largest shares are the USA and Germany, each with a share of about 3 per cent of such papers.)

Social care research is often labelled ‘social work’ and it appeared that the term ‘social care’ might be equivalent to ‘social work’ in some countries. However, slight differences might exist in the subjects covered under the ‘social care’ label in the UK and the ‘social work’ label in other countries. To limit any bias towards the UK while providing a representative coverage of ASCR in building the data-set, a semantic analysis of the papers using the term ‘social care’ as well as of the SSCR papers was performed to better grasp the diversity of terms used under the ‘social care’ label. This analysis consisted of the creation of maps based on the co-occurrence patterns of MeSH headings and noun phrases (extracted from the title, author keywords and abstract of papers) appearing in the above papers using VOSviewer (Van Eck and Waltman, 2009) as well as on the computation of the specificity of these terms to the set of analysed papers. (The papers were matched with Medline using Science-Metrix' algorithm to enable the extraction of Medline's controlled vocabulary (i.e. Medical Subject Headings, MeSH) for indexing journal articles by subject in the life sciences as well as for the subsequent extraction of Scopus papers using this vocabulary.) Specificity was assessed using a statistical measure (term frequency–inverse document frequency, or TD-IDF) commonly used in information retrieval to evaluate how important a term is to a corpus (i.e. the above set of papers) relative to a reference collection (i.e. Scopus). This led to the conclusion that ‘social care’ is indeed somewhat synonymous of ‘social work’; this was in fact the MeSH term of highest specificity to the set of analysed papers.

We therefore decided to use a set of specialist journals in social work as a seed in creating the ASCR data-set and subsequently expanded it using various strategies to encompass the full spectrum of subjects covered under the ‘social care’ label as revealed by the semantic analysis. Science-Metrix has developed a journal-based classification of science (Archambault ), and one of the classification's subfields is Social Work. This journal set contains sixty-seven journals. The list of journals was shared with SSCR's Executive Group, who reviewed the list—removing eleven journals and adding fifty-six. A number of the journals that were removed focused on children's social work (a field outside the scope of this study). This resulted in a list of 112 journals (see methodological report available online at ). Note that only 102 of these journals had papers of the proper document types in the database, namely peer-reviewed documents that include references to and are cited by other academic documents, mostly articles, conference papers and reviews. All 51,783 papers published in this set of journals were included in our analysis (Phase 1 in Figure 1).

Figure 1

Composition of ASCR bibliometric data-set

Once journals making up the seed data-set were identified, an intermediate phase was used to expand it. This consisted of the extraction of additional papers from Scopus using MeSH terms highly specific to the seed data-set (i.e. their identification was aided by computing the TF-IDF of major MeSH terms appearing in it). To ensure that the final data-set would encompass the full spectrum of subjects covered under the ‘social care’ label, major MeSH terms specific to the SSCR papers as well as those using the ‘social care’ term in their titles, abstracts or author keywords were also used. Similarly, additional papers were retrieved from Scopus searching for noun phrases highly specific to the above three groups of papers.

The resulting query was reviewed by SSCR's Executive Group and contained about 240 expressions (i.e. MeSH terms and noun phrases as listed in the methodological report available from the authors). The query was applied to the whole of Scopus, allowing for the identification of 135,095 additional papers using highly specific terms/words to the field of adult social care (Phase 2 in Figure 1).

Finally, papers preferentially ‘citing’ or ‘cited by’ the interim data-set (i.e. the journal data-set after expansion using the MeSH term/noun phrase query) were retrieved. More specifically, papers having received at least four citations, 60 per cent of which are from the interim data-set, were included as well as papers having referenced at least five papers, 60 per cent of which are from the interim data-set. The 60 per cent thresholds as well as the minimum number of citations received (four) or given (five) are based on previous (unpublished) work; a random sample of papers was used to manually validate the quality of the results in terms of the relevance of the retrieved papers for adult social care. This resulted in an additional 8,951 papers being identified (Phase 3 in Figure 1).

The reliability of the ASCR data-set

As illustrated in Figure 1, this three-phased approach resulted in the identification of 195,829 ASCR papers published between 1996 and 2011. To test the reliability of this data-set, we examined the 131 papers initially identified as SSCR papers (17 per cent of total) not retrieved by our query. We found that 36 per cent of missed papers relate to social care for young people, which is out of scope in the context of ASCR, and another 21 per cent relate to the economics of health and social care. In building the data-set, it was decided not to cover exhaustively the economics of adult social care since this would have resulted in the inclusion of a significant amount of papers on the economics of health care in general, broadening the data-set too much. This is not surprising, as many of the economic terms that would have been required to retrieve such papers are less specialised with application in a broader set of fields. Furthermore, we were satisfied with the 83 per cent recall of the initial ASCR data-set. We also took a random sample of thirty papers from the resulting data-set and examined those qualitatively to see how precise the query was. Of those thirty, two could be described as false positive—that is, they were not relevant to ASCR but were picked up by the query, four were peripheral to ASCR and the remaining twenty-four were clearly relevant to the field. Thus, we concluded that overall the query we developed to identify ASCR papers provided sufficient recall and precision for the comparative bibliometric analysis.

Indicators

We used the following bibliometric indicators in our analysis: We also performed a social network analysis to study collaboration between institutions. Based on matrix cross-linking entities based on their number of co-publications (a symmetric matrix in full or sparse format), the software programme GEPHI is used to produce a visual representation of the strength of the relationships between the selected entities. More specifically, a ‘force atlas’ algorithm was used to establish the relative locations of the entities in the graphic representation.

Number of papers: this is the number of scientific papers by authors from a given country, as found in authors' addresses appearing on papers. Papers are attributed using the ‘full’ counting method, which means that each country appearing on a paper gets one ‘contribution’. In other words, if there are three authors from the USA and one author from the UK, both the USA and the UK get a publication count of one.

Average relative citation (ARC): this is an indicator of the scientific impact of the papers produced by an entity based on the number of citations received by its papers; it is therefore said to be a direct measure of impact. The number of citations to each paper was counted for the year in which it was published and all subsequent years. This results in a variable citation window across publication years. To compensate for this inequality across papers published in different years (older papers have had more time to accumulate citations) as well as the inequality arising from differences in citation practices across scientific fields (Gingras and Larivière, 2011; Moed ; Opthof and Leydesdorff, 2010; Schubert and Braun, 1986), the citation count of papers was normalised by the average citation count for all papers in their subfield for the year in which they were published to obtain a relative citation count (RC). When the ARC is greater than 1, it means that a paper or a group of papers scores better than the world average for its research area; when it is below 1, those publications are cited less often than the world average for the research area.

Average of relative impact factors (ARIF): this is a field-normalised measure of the scientific impact of publications produced by a given entity based on the impact factors (IFs) of the journals in which they were published (also taking the publication year of scientific contributions into account in the normalisation process). As such, the ARIF is an indirect impact metric reflecting the average citation rate of the publication venue instead of the actual publications. As a result, this indicator may serve as a proxy for the ‘quality’ of the research performed by a given entity. Indeed, the more cited a journal, the more researchers will seek to publish in it and the more the editors will be in a position to select the best papers. Its main advantage is that, compared to the ARC, it can be computed up to the latest available year. However, because citations take longer to accumulate in the social sciences and humanities, IFs based on a five-year window were used (i.e. the IF of a journal in 2001 is based on 1996–2000). Consequently, IFs could not be computed prior to 2001, since there are no data prior to 1996 in Scopus. When the ARIF is above 1, it means that an entity scores better than the world average; when it is below 1, it means that, on average, an entity publishes in journals that are not cited as often as the world level.

Specialisation Index (SI): this indicates the relative proportion of publications of a given country in ASCR relative to the proportion of the world's publications in ASCR. An SI value above 1 means that an observed group of researchers publishes more in the domain than would be expected based on the reference entity (i.e. the world in Scopus), while an index value below 1 indicates the opposite.

Growth Index (GI): this indicator represents the ratio in the output of a given entity (e.g. a researcher) between the first and second halves of a study period. In other words, it measures the increase in the number of publications. The increase for a given entity can be compared to the increase calculated for the world in the same research area in order to ascertain whether the increase experienced by the entity has kept pace with the world increase in this research area.

Results

We first report our analyses of the UK's total research output and impact (as measured through citations) between 1996 and 2011, and then compare this with other countries producing ASCR papers. We go on to examine the output, growth in output, impact and collaboration of the leading fifty researching institutions in the UK involved in ASCR.

The UK's total research output and impact

Table 1 presents trends in ASCR in the UK and as a contribution to total world output since 1996. Overall, 195,829 ASCR publications were indexed in the Scopus database from 1996 to 2011. Of these, 16 per cent (33,894) involved at least one author from the UK. The UK's output grew steadily between 1996 and 2011, with a 2.45-fold increase in publications. However, this was not sufficient to keep pace with world output, which increased 2.86-fold over the same time frame. As such, the UK's total contribution to the world output slightly declined from 18 per cent in 1996 to 16 per cent in 2011. However, despite its decreasing contribution to world output, the UK is even more specialised in ASCR in 2011 than it was in 1996 (the SI increased from 2.09 to 2.38). This is due to an increase in the share of the UK's research that is devoted to ASCR relative to the same share at the world level. For instance, the share of the UK's research that is devoted to ASCR is more than twice as large as that observed globally in 2011. The scientific impact of UK research in adult social care is slightly above the world average for the 1996–2011 period. The paper-based ARC indicator is 1.07 but, as illustrated in Table 1, has increased from 0.96 in 1996 to 1.13 in 2009. (Due to the lag between publication and citation of a paper, it is not possible to estimate beyond this point.) Likewise, the journal-based ARIF is 1.05, and increased from 0.99 in 2001 to 1.05 in 2011.

Table 1

The UK's scientific output, contribution to world production, degree of specialisation and impact of adult social care research (ASCR), 1996–2011

Year	Papers		UK's contribution to world output (%)		Impact
	UK	World		SI	ARC	ARIF
1996	1,265	6,860	18	2.09	0.96	n.c.
1997	1,430	7,567	19	2.14	0.91	n.c.
1998	1,493	7,736	19	2.18	1.05	n.c.
1999	1,443	8,014	18	2.07	0.98	n.c.
2000	1,719	8,799	20	2.24	1.06	n.c.
2001	1,660	9,122	18	2.27	1.04	0.99
2002	1,741	9,694	18	2.26	0.95	0.99
2003	1,914	10,638	18	2.20	1.03	1.02
2004	2,037	11,501	18	2.25	1.12	1.01
2005	2,297	13,040	18	2.30	1.19	1.07
2006	2,478	14,621	17	2.24	1.13	1.06
2007	2,875	16,046	18	2.38	1.12	1.11
2008	2,720	16,610	16	2.28	1.12	1.08
2009	2,819	17,807	16	2.26	1.13	1.08
2010	2,895	18,103	16	2.33	n.c.	1.07
2011	3,108	19,671	16	2.38	n.c.	1.05
Total	33,894	195,829	16	2.25	1.07	1.05

Note: The contribution to world output indicates the percentage of world papers with at least one author from the UK.

Source: Computed by Science-Metrix using Scopus.

The comparative performance of the UK and other countries in ASCR

Figure 2 presents the SI and the scientific impact (ARC) on a scatter plot of the twenty-four highest-publishing countries. This figure is divided into four quadrants. Countries in the upper-right quadrant (ARC > 1 and SI > 1) have a scientific impact above the world average and have a higher specialisation in ASCR compared to the world average. Those in the lower-right quadrant (ARC < 1 and SI > 1) are specialised but have a scientific impact lower than the world average, while countries in the higher-left quadrant (ARC > 1 and SI < 1) have a scientific impact above the world average but are not specialised. Countries in the lower-left quadrant (ARC < 1 and SI < 1) are below the world average in terms of impact and specialisation. Finally, the size of the dots is proportional to the number of papers published by a given country in ASCR.

Figure 2

Positional analysis of the most-publishing countries in ASCR, 1996–2011.

Among the highest-publishing countries in ASCR, the UK is the most specialised and it has the second largest production after the USA. It is followed by other Anglo-Saxon countries (e.g. Australia, Ireland, New Zealand and Canada). The UK is the last (twelfth) among the countries with an impact score (ARC) above the world average. However, between 1996 and 2009, the ARC has increased from 0.96 (below world average) to 1.13. It is interesting to note that, among the most-publishing countries, those with greater research intensity in ASCR are generally those with higher scientific impact (upper-right quadrant) and include Anglo-Saxon (USA, UK, Canada, Australia) and Scandinavian countries (Finland, Norway and Sweden).

The comparative performance of the UK's fifty most-publishing institutes

Table 2 lists the fifty highest-publishing institutions in the UK between 2003 and 2011. We focus on the second half of our time period for this analysis to present relatively contemporary data but also to examine the growth in output between 1996–2002 and 2003–11, through the GI. The final column of data provides the impact score (ARC) for each institution.

Table 2

The fifty most-publishing UK institutions in adult social care research (ASCR), 2003–11

Institution	Papers	GI	ARC
King's College London	2,002	1.39	1.39
University College London	1,514	1.46	1.57
University of Manchester	1,086	1.27	1.42
University of Birmingham	726	1.35	1.17
University of Sheffield	699	0.99	1.45
University of Oxford	682	1.28	1.57
University of Nottingham	648	1.44	1.30
London School of Hygiene and Tropical Medicine	638	1.49	1.92
University of Glasgow	589	1.13	1.20
University of Bristol	580	1.03	1.41
University of York	577	1.25	1.61
University of Leeds	529	1.34	1.30
Imperial College London	527	1.08	1.23
Newcastle University (UK)	523	1.09	1.32
University of Cambridge	511	1.96	1.88
Cardiff University	504	1.21	1.42
University of Edinburgh	502	1.71	1.32
University of Southampton	496	1.36	1.54
Medical Research Council (MRC)	440	1.30	1.96
University of Liverpool	419	1.03	1.22
London School of Economics (LSE)	415	1.41	1.42
University of Leicester	399	1.18	1.28
University of Kent	375	1.52	1.13
University of Warwick	351	1.32	1.39
Lancaster University	351	1.37	1.52
Queen Mary	348	1.18	1.53
University of East Anglia	325	1.78	1.39
St George's	307	1.06	1.28
Queen's University Belfast	297	1.51	1.20
University of Aberdeen	296	1.18	1.25
University of Ulster	277	1.18	1.32
Sheffield Hallam University	266	1.52	1.04
University of Durham	253	2.43	1.20
Brunel University	241	1.49	1.10
South London and Maudsley NHS Foundation Trust	240	1.69	0.86
Bangor University	240	1.40	1.44
NHS Greater Glasgow and Clyde	239	1.07	1.18
City University London	236	0.96	1.39
University of Exeter	232	1.61	1.63
University of Stirling	229	1.33	1.29
Swansea University	228	2.53	1.21
University of Dundee	219	1.44	1.25
Guy's and St Thomas NHS Trust	205	0.95	1.63
Nottingham University Hospitals NHS Trust	202	1.33	1.17
University of Surrey	199	2.16	1.03
University of Plymouth	199	1.19	1.03
University of Central Lancashire (UCLAN)	183	3.08	1.01
Keele University	179	1.00	1.09
De Montfort University	177	1.37	1.33
University of Bath	176	2.23	1.46

Source: Computed by Science-Metrix using Scopus.

A number of observations can be made from Table 2, although it should be noted that collaboration between institutions means there will be double counting in this table. The top institution—King's College London—contributes 9 per cent of the UK's ASCR output, followed by University College London (6 per cent) and the University of Manchester (5 per cent). As is typical of bibliometric analysis, the distribution seems to follow a power law relationship (Lotka, 1926), with the top five institutions accounting for 26 per cent of the total output, the top ten accounting for 37 per cent and the top twenty-five accounting for 70 per cent.

The second set of observations is around the growth of output between two periods (i.e. comparing 1996–2002 with 2003–11). Within the top fifty, the University of Central Lancashire ranks first in this respect (GI = 3.08) with a score well above the world level. It is followed closely by Swansea University (2.53) and the University of Durham (2.43). Of the top five institutions in terms of total output, none has seen their output increase by a greater margin than the world in ASCR (GIs ranging from 0.99 to 1.46 compared to 1.45 for the world). Despite this, all have maintained or increased their output (GI ≥ 1.00).

An ARC greater than 1.00 indicates above-average citation impact for the field of ASCR globally. The highest ARCs were for the Medical Research Council (MRC) and the University of Cambridge, with ARC scores of 1.96 and 1.88, respectively. Of the top five most-publishing institutions, the ARC ranged between 1.17 (University of Birmingham) and 1.57 (University College London).

Research is increasingly a collaborative activity (Larivière ; Wuchty ) and this is certainly evidenced for ASCR. Figure 3 illustrates the collaboration networks of the fifty highest-publishing UK institutions in ASCR. Each institution is represented by a node in the network and identified using a label. The size of nodes is proportional to the size of an institution's publication output. The width of links is proportional to the number of co-publications between any pair of institutions. Only links representing at least eighteen co-publications (i.e. two per year on average) are shown. Nodes are coloured based on their clustering into communities. The nodes of institutions at which SSCR's first-phase units were located have a thick dotted rim.

Figure 3

Collaboration network of the fifty most-publishing UK institutions in adult social care research (ASCR, 2003–11).

As can be seen from Figure 3, the clustering pattern of the highest-publishing institutions in the UK reflects, to some extent, their geographical distribution. For example, those in the red cluster are mostly from the south-east, those in the green cluster are mostly from central and western areas, and those in the blue cluster are mostly from northern areas.

Discussion

The purpose of this paper is to map global and UK ASCR activity since the mid-1990 s and explore the potential for bibliometrics to provide a method for describing patterns of activity and to assess its impact. Bibliometrics provides an analytical basis for benchmarking ASCR trends for future assessments. In this section, we highlight the limitations of our analysis and draw out some observations that have policy implications.

Caveats and limitations

Before drawing conclusions for the conduct of ASCR, it is important to highlight a number of important qualifications when assessing the validity of bibliometric analysis (Moed, 2005). Only journals included in Scopus form part of our analysis. We manage this in two ways. First, we note that the potential exclusion of relevant cited studies is likely to be small, as Scopus includes the most visible (and thus cited) share of researchers' scientific output. Second, we triangulated three methods of paper identification: key journals, MeSH terms combined with noun phrases, and commonly cited/referenced papers.

Bibliometrics provides a useful means of summarising some aspects of research, including describing and comparing patterns of activity, whether over time or across countries or between universities or indeed individual researchers (not analysed here). But the benchmarking of social care research is limited by the evidence available for analysis, which currently means it is limited by what is published in academics journals. It is important to recognise that ASCR is not solely the preserve of universities or NHS Trusts, as Table 2 might suggest. A proportion of ASCR is carried out by local authorities, independent sector providers, central government, consultancy companies and think tanks. Such organisations are more likely to label some of these activities as ‘audit’ or ‘management information reporting’, less likely to publish their findings in peer-reviewed academic journals and more likely either never to make their research publicly available or to publish openly accessible reports. For these reasons, we cannot judge how well bibliometric analysis can represent the volume of the ASCR in its entirety. In the past, university-based researchers in the UK might also have favoured non-journal media for publicising their research findings, but pressure from research assessment exercises and other sources to publish peer-review papers has changed the approach taken by many researchers. Our bibliometric analysis therefore underestimates the volume of ASCR.

Another caveat is what a citation actually represents. Citation analysis is predicated on the notion that the reference practices of researchers can reveal high-performing academics, papers and institutions, as well as popular and influential areas of research. Unfortunately, there is no theory underpinning this empirical observation (Vinkler, 2002). Moreover, citations measure only one dimension of research impact, while the impact of social care research in the real world is multifarious, such as through the generation of highly qualified personnel or the improvement of care and support, and in turn the improvement of people's lives. However, at present, data on these other dimensions of research impact are not widely available. So, while the analysis presented is of only one type of quantitative analysis (bibliometrics), which is focused on a particular type of impact (academic), it provides perhaps the best current proxy for the overall academic impact of ASCR.

Finally, countries whose researchers publish their work in languages other than English are placed at a disadvantage (Archambault ). The comparative advantage that the English language confers on the research base in the UK, the USA and other English-speaking countries may diminish in the future as English is also used for teaching in countries where it is not the first language. In addition, while national and institutional capacity in English, especially in the sciences, may provide a comparative advantage in the current state of the world, this may change as other languages gain in importance. The comparative advantage conferred to English does not detract from the results presented here, but is an important consideration in their interpretation.

Observations

Bibliometric analyses can provide a helpful platform for understanding and developing evidence on the state of ASCR and thus support its development as a distinct discipline. Based on the analysis presented in this paper, we can draw a number of observations, first on the need for a better definition of ASCR and second on the state of ASCR evidence and its potential to achieve impact.

One immediate observation to make from this analysis is that it is difficult clearly to define ‘social care’ research, as has been noted previously by other reviewers of the field (e.g. Sharland, 2009). A major contribution to the adult social care field comes from social work research, but it is often hard to distinguish the smaller, profession-driven body of research from the wider, more service-driven body, and therefore also hard to conduct bibliometric analyses of either. Does this matter? For the purposes of marshalling evidence so as to provide decision makers with broad information to support their commissioning, provision, regulation or strategic planning, the inability to make these distinctions may not unduly matter. But it does matter if the distinctive contributions of particular professions or approaches are to be identified and acted upon in commissioning, provision and so on. And it clearly matters for the purposes of workforce planning and investment, including for the research workforce. It also matters if the funding of research—including by overarching bodies such as the Higher Education Research Council for England (HEFCE) or by Research Councils—is influenced by the performance of individual researchers or institutions as rated using bibliometrics. The statistics generated by bibliometric analysis, in the context of this study, therefore describe measurable outputs, and do not describe research productivity (outputs relative to resources), nor research potential (the latent ability of an institution or individual), nor the more relevant impact of research (in the ASCR case, this might include changing the lives of people with social care needs). Performance indicators of any kind should be interpreted and employed in cognisance of the context in which they are generated, the data used to generate them and the set of factors that may influence them, particularly those factors that are beyond the immediate or even longer-term control of the entities or individuals being performance-assessed. Uninformed utilisation of bibliometric measures could do more harm than good.

One interesting finding from our analysis was the appearance in the top fifty highest-publishing institutions in the UK of universities not generally known for their activity in ASCR. For example, sixteen of the fifty institutions listed in Table 2 did not make submissions under the ‘Social work and social policy’ unit of assessment in the 2014 Research Excellence Framework, and some are clearly NHS Trusts. One reason for their appearance in our search is that they could be carrying out health sciences research that has implications for ASCR, such as studies of people with learning disabilities or mental health problems, or older people with dementia or long-term conditions who have both health and social care needs. Recognition of the contributions that these universities and other institutions make to ASCR—and of the key messages for social care emerging from their studies—could be very helpful to the field.

A second key finding was that the UK was found to be the most specialised for ASCR. To what extent was this as a result of terms used for the analyses? It is difficult to accurately reflect international approaches to social care, and the equivalent terms used in other countries. For example, long-term care is possibly the term most used to describe what is termed social care in the UK. We included both terms, and social work, for the purposes of our analyses but a complication arises, as not only are terms different internationally, but approaches in social care policy and practice, and funding for ASCR, differ too. This makes it difficult to learn from the comparative performance across countries.

We found a 2.45-fold increase in ASCR publications between 1996 and 2011, which is very welcome growth (on the untested assumption that the relevance of research has not got worse over the same period), although a slower rate of growth than we found globally (a 2.86-fold increase). Analysis of UK mental health research by Larivière indicates a 2.23 increase, also noting a faster rate than medical disciplines altogether. Comparatively, ASCR has grown at a similar rate. However, our analysis was not able to look at possible reasons for the increase in ASCR publications, such as increases in ASCR funding, or compare this growth with other disciplines in more depth. Indeed, in the absence of reliable data on the volume of research funding (within, let alone between, countries), it is not possible to know, for example, whether the most-publishing organisations or countries have higher levels of or better access to research funding, or whether they are more productive. Nor can we comment on whether less-publishing organisations or countries are giving lower priority to peer-reviewed journal papers and higher priority to other media (including ‘grey’ literature) for communicating research or whether they are less productive. Similarly, we cannot—at least currently—use the analyses to comment on whether peer-reviewed journal outputs have a more direct or significant impact on front line service delivery or on individuals' lives than other media.

Concluding remark

Our use of bibliometrics to profile ASCR is exploratory, and it is not possible to draw definitive conclusions about research volume, quality or impact. Consequently, it would not be appropriate to use the results obtained to date as the basis for allocating research funding. The aims of ASCR include making contributions to the achievement of academic, social, economic and quality-of-life benefits for individuals, families, communities and societies. Further development of bibliometric methods, such as through integration of high-quality statistics on research inputs, will help to illuminate the strengths and weaknesses of ASCR in this pursuit. Just as research is the effort to discover and increase human understanding of society works, research policy should also include the effort to seek to understand how research works.