Citation context and impact of 'sleeping beauties' in paediatric research.

Objectives 'Sleeping beauties', i.e. publications that are not cited for a long while, present interesting findings in science. This study analysed the citation trends of sleeping beauties in paediatric research. Methods The study used bibliometric software to analyse the papers citing sleeping beauties in paediatric research, to understand the context in which paediatric sleeping beauties were finally cited and the impact of these sleeping beauties on paediatric research. Results Two paediatric sleeping beauties, addressing medical homes and the transition from paediatric to adult health care, respectively, awakened in response to organizational needs. Both presented novel concepts of paediatric service organization that became important because of an increased need for optimization of services. Conclusion All sleeping beauties bring new knowledge that becomes important only after several years. Paediatric sleeping beauties exhibited unique characteristics; however, their presence in paediatric research shows that knowledge acquisition in paediatrics resembles that in other disciplines.

Introduction

Scientific publications are received by the research community in various ways.[1] Some articles are quickly cited, while others are discovered only after a longer period of time.[2] The term ‘sleeping beauty’ (SB) is used to describe a publication that went unnoticed (asleep) for a long while and then, almost suddenly, became interesting and highly cited.[3] SBs have also been termed ‘premature discoveries’, ‘resisted discoveries’, or ‘delayed recognitions’.[4] SBs present interesting findings in science, and their specific characteristics make their discovery ‘not just an exotic whim, but a necessity to have an answer to Mendel-like claims.’[3] SBs might assist the understanding of citation dynamics in general;[5] additionally, they may help us understand delays in innovation.[6]

Garfield[3,7] and Stent[8] pioneered the study of SBs. Following this, Lange[9] identified specific SBs in the physiology research literature, and van Raan[6] did the same for physics and engineering. Ke et al.[10] also identified a number of SBs, from different disciplines, using a score-based approach; however, Nobel laureate Einstein together with two famed physicists Podolsky and Rosen contributed the best-known work, in which they questions if the quantum-mechanical description of physical reality can be considered complete.[11] Despite these efforts, little is known about the citation trajectories of SBs or about the length of time SBs may remain dormant, only that this is variable and may be long – one extreme case of a SB is that of Peirce’s[3] short note published in Science in 1884,[12] which lay asleep for 116 years.[2] There is also little known about the context in which BSs begin to be cited or about the papers, referred to as ‘princes’, that first begin to cite the SBs and awaken interest in these works.

In the field of paediatrics, Završnik and Kokol[13] identified five SBs in the paediatric literature from the Web of Science database (Thomson Reuters, New York City, New York, USA), using their own SB analysis software and the van Raan criteria (‘sleeping period’ of at least 5 years; ‘depth of sleep’ of an average less than one citation per year; and ‘awaking intensity’ of more than five citations per year on average for at least 10 years)[6]. The earlier study by Završnik and Kokol[13] aimed only to identify paediatric SBs. The present study aimed to expand on this work and to answer the following research questions:

What was the context in which paediatric SBs began to be cited?;

Who were the princes?;

What impact did the paediatric SB have on paediatric research?

Methods

To answer the research questions, the titles and abstracts of the searched documents were analysed using bibliometric software (VOSviever version 1.6.4; University of Leiden, Leiden, The Netherlands).[14] The software used text data mining techniques to analyse the body of searched documents, producing bibliometric maps or ‘science landscapes’. A science landscape is a two-dimensional map in which the proximity of terms reflects their relatedness, where terms positioned closer by are considered more related than terms positioned farther away.

The position of terms was calculated on the basis of co-occurrences of terms in the searched documents. Landscape induction proceeded through four steps,[15] including: (I) identification of noun phrases, and conversion of plural to singular noun phrases; (II) selection of the most relevant noun phrases, followed by clustering of the phrases, based on the distribution of noun co-occurrences over all the noun phrases (associated noun phrases were grouped into clusters denoted by same colour); (III) mapping and clustering of terms, which was done using unified algorithms for both tasks; and (IV) visualization with LinLog/modularity normalization.[15]

Common terms, including ‘study’ and ‘significance’, as well as country names were excluded from the analysis. Different parameters in the software were tuned to define the final cluster configuration – the full counting method and default two landscapes were created, one to visualize the clustering of terms and the other to present the timeline of term appearance. Thus, landscapes were created and visualized in various ways to emphasize different aspects of the searched documents.[16]

SB citation context and princes

A cluster landscape was generated and used to compare the cognitive concepts presented in SBs with those in their citing papers. Additionally, a timeline landscape was generated to analyse the type of research that was triggered by the SB awakening. In the timeline landscape, terms were colour coded according to their time stamp, which was calculated as the average of the publication years in which the terms appeared.

The princes were identified with the help of the Web of Science citation report service, which shows citations by year. For each SB, the authors analysed all papers citing the SB in the year of awakening. While in each year there were many citing papers, many possible princes can be identified, however we were interested in the most important one. Thus, only the most cited paper was selected as the prince.

SB impact

To quantify the impact of a paediatric SB on paediatric and other research, the study compared the total number of citations (TNC) of the SB with the number of papers published (NPP) in the scientific area defined by the keywords given in the SB. The TNC was obtained with the Web of Science citation analysis service. The NPP was acquired using the Web of Science search function, applying the SB keywords as search attributes. SBs with a TNC larger than the NPP were proved to have an impact outside their main research theme.

The qualitative impact of an SB was assessed with the timeline landscape of its citing papers. The timeline landscape identified the state of the art research themes for each SB cluster.

Results

The five paediatric SBs identified by Završnik and Kokol[13] were named the Low Birth Weight SB, the Transition SB, the Medical Home SB, the Vesicoureteral Reflux SB and the German Children SB. The Low Birth Weight SB[17] described the voluntary collaborative research network research addressing outcomes and medical interventions for very low birth weight infants at participating centres in the USA and Canada in 1990. The Transition SB[18] presented the impact of significant advances in medical science and technology on the survival of children with chronic diseases – namely, a dramatic increase in survival rate (85%), which resulted in a much greater need for adolescent transitional care. In the Medical Home SB,[19] the American Academy of Pediatrics Task Force on Medical Homes defined characteristics of the medical home, as family centred, accessible, continuous and compassionate, and specified that delivery of care should be by well-trained physicians known to the child and family. The Vesicoureteral Reflux SB[20] reported (then) current controversies in the management of vesicoureteral reflux, while the German Children SB[21] investigated growth (height and weight) rates in healthy 15- to 18-year-old children.

The SB citation histories are shown in Figure 1. Two different citation trajectories were identified. The first, seen for the Transition SB, was a curve exhibiting a continuous rise in the number of citations, while the second, seen in the other SBs, was a bell-shaped curve exhibiting a rise in the number of citations in the first few years after awakening, followed by a slow decline.

Figure 1.

The citation trajectories of paediatric ‘sleeping beauties’.

SB = sleeping beauty; SP = sleeping period; DoS = depth of sleep; AI = awakening intensity

The scientific landscape of the SB citing papers is presented in Figure 2. Five clusters emerged, one for each SB. The research publications triggered by the Low Birth Weight and Vesicoureteral Reflux SBs were clearly separated; however, both research areas were linked through the German Children SB. The clusters of papers citing the Transition and Medical Home SBs were, not only closely positioned, but also partially intertwined, meaning that both SBs triggered related research. The main cognitive content of the citing papers was associated with the cognitive content of the research presented in the SBs.

Figure 2.

The cognitive context of the research catalysed by paediatric ‘sleeping beauties’ (SBs).

The figure indicates the five SBs: Low birth weight SB (blue); German children SB (yellow); Transition SB (green); Vesicoureteral reflux SB (red); and Medical home SB (violet)

As shown in Figure 3, the research presented in the Low Weight SB became important as different problems related to low weight of premature infants (for example, infection, poor motor response and cerebral palsy) and their associated therapies, together with therapies for reducing mortality and morbidity (e.g., antenatal corticosteroid therapy) became more intensively researched and used. Thus, the paper by Jobe and Ikegami[22] on mechanisms initiating lung injury in the preterm neonate was the awakening prince. Similarly, the Transition SB awakened when the shift from institutional to community-based, family-centred care increased in importance and became operational. It was awakened by a paper by Oeffinger et al.[23] on programs for adult survivors of childhood cancer. The Medical Home SB resurged when vaccination, immunization, child abuse and autism became associated with the medical home framework. The workforce statement by the American Academy of Pediatrics Committee on Pediatric Workforce[24] was the prince. The Vesicoureteral Reflux SB awakened with an increase in research on genetics and new therapeutic (e.g. endoscopy) and diagnostic (e.g. ultrasonography) technologies. Its prince was a paper reporting a study of medical vs surgical treatment in children with findings of severe vesicoureteral reflux of dimercaptosuccinic acid.[25] The German Children SB became popular with more intensive study of puberty and the discovery of unknown syndromes, such as congenital adrenal hyperplasia and idiopathic pulmonary artery hypertension. It was awakened by a publication on the factors influencing growth in height and weight in children with diabetes.[26] Four of the five paediatric SBs were awakened in 1998.

Figure 3.

The timeline for terms found in papers citing paediatric ‘sleeping beauties’.

SBs impact

Table 1 shows the impact of the paediatric SBs. The most influential of the SBs was the Transition SB, with 395 citations; after awakening, only 212 publications were published in the paediatric fields associated with the concepts presented in the Transition SB, indicating that the impact of the Transition SB impact extended into wider paediatric research. A similar trend was observed for the Vesicoureteral Reflux SB. Despite that the Transition and Medical Home SBs triggered related research, they had very different impact. The Transition SB had a widespread impact on paediatric research; conversely, the Medical Home SB triggered fewer citations, despite that its concepts were researched and discussed in a number of publications in related journals.

Table 1.

The impact of ‘sleeping beauties’ on paediatric research

SB	TNC	NPP
Low birth weight
	107	1719
Transition
	395	212
Medical home
	139	2382
Vesicoureteral reflux
	166	140
German children
	119	692

SB = sleeping beauty; TNC = total number of citations for the sleeping beauty; NPP = total number of papers published in the scientific area of the sleeping beauty

As seen in Figure 3, the Low birth weight SB and the Transition SB have had the greatest impact on later research. The Low Birth Weight SB has been cited in later publications on prostatic receptors, gliosis, hydrocephalus and germinal matrix /intraventricular haemorrhage in premature infants. Meanwhile, the Transition SB was cited in publications on adult medical care, health care transition, inflammatory bowel disease, phenylketonuria and perinatally acquired HIV.

Discussion

Principal results

The analysis showed that the paediatric SB citations followed either of two trajectories, exhibiting: (I) a continuous rise in the number of citations or (II) a bell-shaped increase followed by decrease in the number of citations. The science landscape clusters showed that the main cognitive content of the citing papers was closely associated with that of the research presented in the SBs. Three of the SBs had an impact that was limited to their area of research, while the others had an impact on the global paediatric research community.

Among the five identified SBs, the Transition and Medical Home SBs appeared to have been awakened more for organizational than scientific concerns. Both presented innovations in paediatric health service organization, which became important because of an increasing need for optimization of services in different segments of health care, especially at the level of primary care. Later papers citing these SBs have addressed the issue of work organization in primary care, including the lowering of costs through reduced referral to secondary care.[27,28]

Comparison with the related research

Van Raan[3] characterized SBs along a spectrum: at one end of the spectrum were the ‘mild’ SBs, with shorter sleeping periods and lower awakening intensities; and at the other end were the ‘extreme’ SBs, with very long sleeping periods and higher awakening intensities. According to this schema, the Vesicoureteral Reflux SB represented an extreme SB, and the other paediatric SBs could be seen as mild SBs.

Ke et al.[10] observed ‘… many examples of papers achieving delayed yet exceptional importance in disciplines different from those where they were originally published.’ This was true for the Transition SB and partially so for the Vesicoureteral Reflux SB, which were also cited outside their context; however, paediatric the SBs were also somewhat unique: They were five to ten times rarer in number than, for example, those in physics[6] or international economics and business.[29] As well, their citation trajectories differed from the ‘sticky’ (continues to be cited) or ‘transient’ (early peak followed by a steep decline in the number of citations) trajectories identified by Baumgartner and Leydesdorff.[30]

Limitations and strengths

The SBs and their citing papers were identified in the Web of Science bibliographic database, and search in other databases might have revealed different SBs and their princes; however, currently, the Web of Science likely possesses the most complete citation information. As well, the interpretation of the science landscapes was subjective and might be interpreted differently by other researches. The strength of this study is that it provides a novel insight to the SB phenomenon in paediatric research.

Conclusion

SBs present important medical, clinical, or social innovation. Understanding the reasons for delayed recognition and awakening mechanisms may bring earlier recognition of innovations and ultimately, reduced time to integrate innovation into clinical practice.

The present study showed that in paediatric research, SBs exhibit some unique characteristics; however, their presence in the paediatric research literature sets paediatrics on par with the natural, economic and statistical sciences. The presence of paediatric SBs in the paediatric research literature shows that the processes for knowledge accumulation[31] in paediatrics can be compared with those in other academic disciplines.

Ethics

The study did not involve human or animal subjects, so ethical permission was not required.