Inviting parents to take part in paediatric palliative care research: a mixed-methods examination of selection bias.

BACKGROUND: Recruitment to paediatric palliative care research is challenging, with high rates of non-invitation of eligible families by clinicians. The impact on sample characteristics is unknown. AIM: To investigate, using mixed methods, non-invitation of eligible families and ensuing selection bias in an interview study about parents' experiences of advance care planning (ACP).
DESIGN: We examined differences between eligible families invited and not invited to participate by clinicians using (1) field notes of discussions with clinicians during the invitation phase and (2) anonymised information from the service's clinical database.
SETTING: Families were eligible for the ACP study if their child was receiving care from a UK-based tertiary palliative care service (Group A; N = 519) or had died 6-10 months previously having received care from the service (Group B; N = 73).
RESULTS: Rates of non-invitation to the ACP study were high. A total of 28 (5.4%) Group A families and 21 (28.8%) Group B families (p < 0.0005) were invited. Family-clinician relationship appeared to be a key factor associated qualitatively with invitation in both groups. In Group A, out-of-hours contact with family was statistically associated with invitation (adjusted odds ratio 5.46 (95% confidence interval 2.13-14.00); p < 0.0005). Qualitative findings also indicated that clinicians' perceptions of families' wellbeing, circumstances, characteristics, engagement with clinicians and anticipated reaction to invitation influenced invitation.
CONCLUSION: We found evidence of selective invitation practices that could bias research findings. Non-invitation and selection bias should be considered, assessed and reported in palliative care studies.

What is already known about the topic?

Recruitment is challenging in palliative care research, in part due to professional gate-keeping.

This may result in selection bias, which can influence the external validity of study findings.

Despite these potential consequences, the nature and degree of selection bias is rarely examined or reported.

What this paper adds?

Identifies key factors which may give rise to low invitation rates and selection bias in paediatric palliative care research, and provides a model for how these factors interact.

Demonstrates the utility and efficacy of a mixed-method approach to investigating selection bias due to non-invitation.

Implications for practice, theory or policy

Researchers should assess and report non-invitation rates and selection bias wherever possible.

Anonymised, routinely collected clinical data combined with qualitative study of invitation practices is an effective method for detecting non-random invitation to participate.

Introduction

High-quality research is needed to inform best practice in palliative care.[1] However, research quality and applicability may be limited by challenges to successful recruitment, not only due to moderate response rates, but also significant rates of non-invitation to participate.[2-4]

Non-invitation of some patients who meet the eligibility criteria described in a study protocol is sometimes referred to as ‘gate-keeping’.[5] Gate-keeping, while sometimes unavoidable, may result in selection bias, where invited patients differ systematically from non-invited eligible patients. The external validity of the findings may therefore be compromised.[6,7] Furthermore, gate-keeping may reduce the sample size or increase the time required for recruitment.[6] These effects can compromise the value and applicability of the research to policy and practice.

While non-response bias in paediatric palliative care is beginning to be investigated,[8] knowledge of non-response bias is of limited use if the sampling frame itself is biased by selective invitation. Despite concern about this potential selection bias,[3] the nature and degree of such bias are largely unknown.

In this article, we examine the selective invitation of eligible families to participate in paediatric palliative care research using mixed methods. Our case study is a pilot interview study about advance care planning (ACP) for children with a life-limiting condition (the ‘ACP study’), in which the sampling frame was large, yet invitation rates were low. The findings from the ACP study itself will be reported elsewhere. We demonstrate the use of routinely collected clinical data to examine statistical differences between invited families and non-invited families. We define ‘invitation rate’ and ‘non-invitation rate’ as the proportion of eligible patients approached and not approached about the study, respectively.

Background information about the ACP study

The ACP study sample was drawn from the caseload of a UK tertiary referral centre for children’s palliative care, comprising patients with a diverse range of malignant and non-malignant conditions, ethnic and socioeconomic backgrounds, and ages (0–19 years). As a specialist referral service with an extensive outreach programme, the team of clinicians works collaboratively with multiple services and other institutions to support children and their families in various care settings (home, hospice, tertiary children’s hospital and local hospitals). The ACP study was designed with input from the clinicians in the service.

Potential participants eligible for inclusion were parents with a living child receiving palliative care services from the clinical team (Group A), and bereaved parents of a child who had received care from the clinical team and died 6–10 months previously (Group B). Parents were ineligible if they were (1) participating in another research study or had completed participation within the last 6 months (this was later revised to include only psychosocial research) or (2) unable to give informed consent.

A total of 14 clinicians, including medical and nursing members of the palliative care service, were available to approach eligible families between December 2011 and December 2012 to invite them to participate in the ACP study. This was the first time many members of the service had been asked to approach families for a research study. Adhering to the processes approved by the responsible ethics committee, initially clinicians approached Group A families during routine contact (e.g. telephone call or face-to-face visit). They were asked to give families a brief verbal introduction to the ACP study and offer them an information pack. Parents could then contact the research team for further information. Due to the lack of regular contact with families beyond 6 months’ bereavement, clinicians were asked to mail information packs to bereaved families with a personal covering letter. From August 2012 (9 months into recruitment), clinicians could also approach Group A families via mail if they preferred. Invitations were registered on the clinical team’s electronic database. Invitations continued until at least six families had agreed to take part in each group: the invitation period spanned December 2011–December 2012 for Group A and January–June 2012 for Group B.

All clinicians were trained at the start of the recruitment period via an interactive presentation led by the research team, who were already known to them. Following this, the research team arranged regular meetings with clinicians to discuss their experiences and any concerns, updated clinicians on recruitment progress and thanked them for any invitations during weekly team meetings, displayed reminder posters in the clinical team office and provided pocket guides to approaching families. The clinical data manager (A.D.) created an automatic pop-up message which appeared when clinicians opened an eligible living patient’s electronic record, and identified and printed a list of eligible bereaved families each month, which was displayed in the clinical team office. In response to low invitation rates, an ‘opt-out’ policy was promoted by the lead nurse (J.H.) from 3 months into recruitment, to encourage clinicians to approach all eligible families; this was supported by introducing protected time during clinical team meetings to discuss approaching families identified as eligible.

Methods

Data collection and analysis

Two datasets were utilised to investigate differences between families invited and not invited to the ACP study. Dataset 1 consisted of ethnographic field notes recorded prospectively by two researchers (J.C. and E.B.) during the invitation period, including 88 individual conversations between researchers and clinicians, 29 clinical team meetings and 3 research seminars (held jointly with clinical and research teams). The anonymised field notes included clinicians’ views on and experiences of inviting families to take part in the ACP study.

Field notes were analysed thematically: (1) A researcher (J.C.) coded them inductively with respect to factors potentially associated with invitation and non-invitation of families. (2) The coding framework was discussed, revised and re-implemented by the research team (J.C., E.B. and P.K.). (3) A second researcher (E.B.) independently coded 20% of the field notes; comparison with the original coding led to further refinement of the framework and re-coding of the dataset. (4) A researcher (J.C.) also indexed the dataset for references to Group A versus Group B families, blind to the previous coding. (5) Two researchers (J.C. and E.B.) compared Group A and B within each category of the coding framework, identifying similarities and differences between the two groups. All coding and indexing were carried out using QSR NVivo 10 software.

Dataset 2 consisted of anonymised, individual-level quantitative information extracted from the clinical team’s electronic database about the families who met the inclusion criteria for Group A (N = 519) and Group B (N = 73) during the invitation period. The data were de-identified by a member of the clinical team (A.D.) in accordance with the Information Commissioner’s Office code of practice for data anonymisation.[9]

The variables extracted from the database are shown in Table 1. These variables were selected because they were routinely recorded with relatively little missing data and were thought both from the literature review and preliminary analysis of Dataset 1 to be possibly associated with invitation.

Table 1.

Variables extracted from the palliative care team database.

Variable name	Description	Type/format and response categories	Reason for inclusion
invitation status	Whether or not the patient’s parent(s) was invited to take part	Binary variable (‘invited’ or ‘not invited’)	Outcome of interest
Age	Patient’s age at the start of recruitment period (Group A) or death (Group B)	Continuous variable, rounded to the nearest month if under 1 year or to the nearest year if over 1 year (to protect patient identity)	Basic demographic information; possible confounding factor
Gender	Patient’s gender	Binary variable (‘male’ or ‘female’)	Basic demographic information; possible confounding factor
Ethnicity	Patient’s ethnicity	Binary variable (‘White British/UK’ or ‘Other’)	Basic demographic information; possible confounding factor. Quantitative research revealed an association between ethnicity and participation in a paediatric palliative care study.[8] Due to considerations of data quality and patient privacy, we were unable to break down ‘Other’ into meaningful categories
Diagnosis	Patient’s diagnosis	Binary variable (‘malignant’ or ‘non-malignant’)	Patients with malignant and non-malignant disease are referred to the palliative care service via different routes and are managed differently by the service. Due to considerations of data quality and patient privacy, we were unable to break down these categories into meaningful sub-groups
Time between referral to the service and study eligibility (Group A) or death (Group B)	In months	Continuous variable, rounded to the nearest month (to protect patient identity)	Qualitative research suggests clinician’s knowledge of and/or relationship with patients influences invitation to clinical trials, paediatric and palliative care research.[10–12]
Total family contact time with the palliative care service during eligibility period (Group A)	In hours, including face-to-face visits and telephone calls	Continuous variable	In the ACP study, contact during eligibility periods constituted a direct opportunity for invitation
Number of days of contact with the palliative care service during eligibility period (Group A)	Including face-to-face visits and telephone calls	Continuous variable
Total family contact time with the palliative care service 12 months before patient eligibility (Group A)	In hours, including face-to-face visits and telephone calls	Continuous variable
Number of days of contact with the palliative care service 12 months before patient eligibility (Group A)	Including face-to-face visits and telephone calls	Continuous variable
Total family OOH telephone contact time with the palliative care service during patient’s eligibility period (Group A)	In hours (OOH = out-of-hours)[a]	Continuous variable
Number of days of OOH telephone contact with the palliative care service during patient’s eligibility period (Group A)	OOH = out-of-hours[a]	Continuous variable
Total family OOH telephone contact time with the palliative care service 12 months before patient eligibility (Group A)	In hours (OOH = out-of-hours)[a]	Continuous variable	As above
Number of days of OOH telephone contact with the palliative care service 12 months before patient eligibility (Group A)	OOH = out-of-hours[a]	Continuous variable
Total family contact time with the palliative care service 12 months before patient death (Group B)	In hours, including face-to-face visits and telephone calls	Continuous variable
Number of days of contact with the palliative care service 12 months before patient death (Group B)	Including face-to-face visits and telephone calls	Continuous variable
Total family OOH telephone contact time with the palliative care service 12 months before patient death (Group B)	In hours (OOH = out-of-hours)[a]	Continuous variable
Number of days of OOH telephone contact with the palliative care service 12 months before patient death (Group B)	OOH = out-of-hours[a]	Continuous variable
Total family contact time with the palliative care team 0 < 6 months post death (Group B)	In hours, including face-to-face visits and telephone calls	Continuous variable
Total family contact time with the palliative care team 6–10 months post death, that is, during eligibility period (Group B)	In hours, including face-to-face visits and telephone calls	Continuous variable

Out-of-hours contact with the palliative care service is initiated by parents during weekday nights (6 p.m.–8 a.m.) and weekends. In this dataset, it constituted 4.5% and 4.3% of the total contact between families and the palliative care service during eligibility period and 12 months prior, respectively.

Within each group (A and B), families invited were statistically compared with families not invited on each of these variables. Variables significantly associated with invitation status at the 25% level[13] on two-tailed univariate analysis were entered into a logistic regression model. Continuous variables not linear in the logit of invitation status were transformed into categorical variables. To minimise the risk of re-identifying individuals by data combination, the individual-level data were provided by the clinical team initially in univariate form (i.e. each variable linked only to invitation status and not to the other variables). Only variables which qualified for multivariable analysis were then provided in multivariable form (i.e. linked to each other). Variables significant at the 5% level in the logistic regression model using a two-tailed likelihood ratio test were considered independently associated with invitation status. All analyses were carried out using IBM SPSS Statistics 21.

We used a convergent parallel mixed-method design, based on a pragmatist philosophy and giving equal priority to qualitative and quantitative data.[14] Dataset 1 was collected during the invitation period (December 2011–December 2012), whereas Dataset 2 was extracted after recruitment had finished (September–December 2013). The two datasets were then analysed concurrently and independently, before the results were combined for interpretation.

Regulatory approvals

A favourable opinion for this study was received from the London Bloomsbury National Health Service (NHS) Research Ethics Committee (11/LO/0710) and Great Ormond Street Hospital NHS Foundation Trust (09NS06).

Results

Invitation rates

During the invitation period, 519 living patients and 73 deceased patients met the inclusion criteria for Groups A and B, respectively, according to the clinical team database. Of these patients, clinicians invited the parents of 28 (5.4%) in Group A and 21 (28.8%) in Group B (p < 0.0005). Each clinician (N = 14) invited 1–31 (median 2) families. The Group A invitation rate did not increase after introduction of a mail option for approaching families (4.2% before vs 2.7% after; p = 0.2).

Factors affecting invitation and non-invitation

Thematic analysis of Dataset 1 revealed three sets of factors which influenced clinicians’ decisions to invite or not invite families to the ACP study: (1) family factors, relating to clinicians’ perceptions of families and families’ worlds; (2) family–clinician contact and relationship, relating to clinicians’ perceptions of their interactions with families; and (3) clinician factors, relating to clinicians’ perceptions of themselves and their own worlds. These were interrelated (Figure 1).

Figure 1.

Factors influencing invitation and non-invitation of families by clinicians.

Family factors

In Dataset 1, several family factors were associated qualitatively with invitation and non-invitation: wellbeing, circumstances, characteristics, engagement with healthcare professionals and anticipated reaction to invitation (Table 2). In addition, on three occasions, families were deemed ineligible to take part because they were participating in other research. In Dataset 2, there was no significant difference between families invited and not invited in terms of the patient’s age, gender, ethnicity, diagnosis or time since referral to the service (Tables 3 and 5), and none of these factors appeared influential in Dataset 1.

Table 2.

Perceived family factors associated with invitation or non-invitation in Group A and Group B (Dataset 1).

Factor	Description	Excerpt from field notes
Wellbeing and circumstances	• Parent’s emotional, mental or physical condition	‘She [clinician] does not want to approach one family because she did not know parent well and remembers they were very stressed’ (Group B)
	• Patient stability/instability and proximity to death (Group A only)	‘She [clinician] will consider inviting them [parents] next week when they will come back to have patient’s line taken out. It depends on the results of the scan which are due before then and may be distressing for the parents’ (Group A)
	• Extraneous family circumstances
	• Availability and adequacy of psychological support
Characteristics	• Persona, for example, ‘lovely’, ‘difficult’	‘[Clinician] says the parent would be great as she is “very articulate” and would be very good at explaining why she made a decision’ (Group B)
	• Language and communication skills	‘[Clinician] does not want to invite one family as they cannot read’ (Group A)
	• Literacy	‘… [family] would be a good candidate as parent has been involved in a lot of planning …’ (Group A)
	• Experience relevant to study
	• Previously expressed willingness to take part in research/help others
	• Location within/outside service catchment area
Engagement and communication with healthcare professionals	• Willingness to engage with healthcare professionals	‘.. [family] have asked for palliative care involvement and emergency care planning, so [clinician] thinks they would be good for ACP project’ (Group A)
	• Responsiveness to attempts to contact family	‘parent … does not want any more contact with [hospital] professionals’ (Group B)
Anticipated reaction to invitation	• Distressed/upset	‘[Clinician] says today will not be a good time to invite them [family] as she will be discussing the patient’s Emergency Care Plan – this is likely to be difficult for the family and she thinks they would probably just throw the information pack in the bin’ (Group A)
	• Annoyed
	• Not interested

Table 3.

Univariate analyses of parent invitation to Group A (N = 519).

	Invited (N = 28)	Not invited (N = 491)	p value	Missing data
Patient’s age at start of recruitment period (years) – median (IQR)	4.5 (0.7–13)	4 (0.6–10)	0.46	0 (0.0%)
Patient’s ethnicity
White British/UK	7/17 (41.2%)	125/375 (33.3%)	0.50	127 (24.5%)
Other	10/17 (58.8%)	250/375 (66.7%)
Patient’s gender
Male	14/28 (50.0%)	245/490 (50.0%)	>0.99	1 (0.2%)
Female	14/28 (50.0%)	245/490 (50.0%)
Patient’s diagnosis
Malignant	[a]	38/491 (7.7%)	0.48	0 (0.0%)
Non-malignant	[a]	453/491 (92.3%)
Time between referral to service and start of eligibility period (months)[b] – median (IQR)	2 (0–16)	4 (0–21)	0.92	0 (0.0%)
Total family contact during eligibility period (hours)[b] – median (IQR)	7.5 (3.0–16.7)	1.0 (0.0–3.4)	<0.0005	0 (0.0%)
Total family contact 12 months before eligible (hours)[b] – median (IQR)	2.1 (0.0–6.6)	0.0 (0.0–1.5)	<0.0005	0 (0.0%)
Total OOH family contact during eligibility period (hours)[b] – median (IQR)	0.2 (0.0–0.7)	0.0 (0.0–0.0)	<0.0005	0 (0.0%)
Total OOH family contact 12 months before eligible (hours)[b] – median (IQR)	0.0 (0.0–0.2)	0.0 (0.0–0.0)	0.001	0 (0.0%)

IQR: interquartile range; OOH: out-of-hours.

Due to there being fewer than 5 patients per cell in the malignant group, these numbers have been suppressed to preserve patient anonymity.

The number of days of contact and out-of-hours contact during eligibility and 12 months prior were also included in the univariate analyses, but due to their strong correlation with the equivalent total contact time variables (Spearman’s r > 0.96; p < 0.001), these variables were excluded from the multivariable analysis in favour of the more precise contact time.

Table 5.

Univariate analyses of parent invitation to Group B (N = 73).

	Invited (N = 21)	Not invited (N = 52)	p value	Missing data
Patient’s age at death (years) – median (IQR)	5 (0.8–11)	3 (0.8–11)	0.80	0 (0.0%)
Patient’s ethnicity
White British/UK	7/15 (46.7%)	18/42 (42.9%)	0.80	16 (21.9%)
Other	8/15 (53.3%)	24/42 (57.1%)
Patient’s gender
Male	11/21 (52.4%)	34/52 (65.4%)	0.30	0 (0.0%)
Female	10/21 (47.6%)	18/52 (34.6%)
Patient’s diagnosis
Malignant	6/21 (28.6%)	19/52 (36.5%)	0.52	0 (0.0%)
Non-malignant	15/21 (71.4%)	33/52 (63.5%)
Time between referral to service and patient death (months) – median (IQR)	1 (0.5–9.5)	5 (1–12.5)	0.27	0 (0.0%)
Total family contact during eligibility period (6–10 months post death) (hours) – median (IQR)	0.0 (0.0–0.0)	0.0 (0.0–0.0)	0.83	0 (0.0%)
Total family contact 0 < 6 months post death (hours) – median (IQR)	0.2 (0.0–0.7)	0.0 (0.0–1.5)	0.99	0 (0.0%)
Total family contact 12 months before death (hours) – median (IQR)	5.7 (2.4–18.9)	3.9 (1.3–12.7)	0.13	0 (0.0%)
Total OOH family contact 12 months before death (hours) – median (IQR)	0.0 (0.0–1.5)	0.0 (0.0–0.6)	0.39	0 (0.0%)

IQR: interquartile range; OOH: out-of-hours.

Family–clinician contact and relationship

Contact between the clinical team and families was a key factor associated with invitation to Group A in Dataset 1, even after the mail option for approaching families had been introduced. For example, one clinician commented that ‘although [research ethics committee] have approved inviting Group A by post … she would never want to send out a cold letter’. This desire for contact before inviting parents of living children was sometimes associated with the following: (1) deferring/waiting until the next contact, which was not necessarily predictable; (2) forgetting to introduce the study; (3) coincidence with a ‘difficult conversation’ (both actual and anticipated) or a period of patient instability or crisis, such that invitation was perceived to be ‘inappropriate’; (4) aborting the invitation when the subject of research was broached, due to perceived ‘negative signals’ from the family.

Another related factor to emerge from the field notes (Dataset 1) was how the clinicians characterised their relationships with families. Clinicians appeared reluctant to invite families they had a strained or new relationship with, preferring to invite families that they had a ‘good’ relationship with and/or knew well. For example, one clinician ‘seemed happy to post out a couple of packs to families she felt she had a good relationship with’. Another deferred inviting a family because ‘she has only met the family once and “needs to build a relationship” with them before introducing the study’. Accordingly, invited families had had more contact with the palliative care team before becoming eligible (Group A) or before patient death (Group B) than non-invited families (Tables 3 and 5), although in Group B this association did not attain statistical significance.

Consistent with these observations, on multivariable analysis, there was a strong statistical association between out-of-hours family–clinician contact while families were eligible and invitation in Group A (Table 4). The total amount of family–clinician contact during eligibility was also positively associated with Group A invitation, although this did not attain statistical significance (Table 4). In Group B, none of the contact variables were statistically significantly associated with invitation on univariate analysis, and only one qualified for multivariable analysis (Table 5). Group B families were much less likely to have had contact with the palliative care team while eligible for invitation compared to Group A families (5.5% vs 63.6%, respectively; p < 0.0005).

Table 4.

Multivariable analysis of parent invitation to Group A (N = 519).

Variable in model	Crude odds ratio (95% CI)	Adjusted odds ratio (95% CI)	p value
Total family contact during eligibility period (hours)	1.11 (1.06–1.17)	1.05 (1.00–1.10)	0.07
Total family contact 12 months before eligible (hours)	1.08 (1.04–1.12)	1.04 (0.99–1.10)	0.15
Some OOH contact during eligibility period (yes/no)	9.45 (4.25–21.04)	5.46 (2.13–14.00)	<0.0005
Some OOH contact 12 months before eligible (yes/no)	4.21 (1.68–10.57)	0.86 (0.22–3.46)	0.84

OOH: out-of-hours; CI: confidence interval.

Nagelkerke R2 = 0.21.

Clinician factors

Clinician factors that appeared to influence invitation in Dataset 1 included the following: available time, forgetting/remembering to introduce study, shared or changing responsibility for patient care, comfort/discomfort with postal mode of invitation, perceived benefit of study to patient/family and confidence in inviting families (Figure 1). For example, with regard to inviting parents of living children, one clinician spoke of her fear that ‘inviting families at the wrong time could jeopardise her clinical relationship with them, undoing everything that has been built so far’. Clinicians’ time and confidence were issues in inviting bereaved families too. One clinician preferred to call bereaved families before posting invitations because ‘she feels she needs to talk to them rather than just inviting them to take part’. However, this took time (‘about half an hour’) and sometimes delayed invitations. For others, there was discomfort and hence delay when they had not been in contact with the family for some time.

Discussion

Our report increases understanding of the nature, effects and complexity of issues surrounding the recruitment of participants to research in paediatric palliative care. In our exemplar, the ACP study, the proportion of eligible families invited to participate was unexpectedly low, particularly among families of living children (Group A) compared to deceased children (Group B).

Our findings suggest that family–clinician relationship was a key factor influencing invitation. Families with whom clinicians had out-of-hours contact, knew well and/or felt they had a ‘good’ relationship appeared more likely to be invited. These relationships may have seemed more robust and therefore less likely to be jeopardised by an invitation to take part in research. One reason for the higher invitation rate in Group B could be the cessation of the therapeutic relationship following patient death, leading to a perception that invitation was less risky.

Clinicians’ perception of families’ wellbeing and circumstances also appeared to play an important role; within this category, patient instability and proximity to death were identified as barriers to invitation unique to Group A and could further explain the lower invitation rate in this group. Furthermore, family experience relevant to the study was identified as a facilitator to invitation; this may have led to some Group A families being excluded due to a perceived lack of experience of ACP.

Our qualitative findings also indicate strong differences between clinicians in their approach to invitation, suggesting that clinician characteristics may be important predictors of invitation independent of family characteristics. We were unable to investigate this quantitatively due to the overlapping and frequently changing nature of clinicians’ caseloads.

Our findings, if replicated in similar projects, have implications for the validity and applicability of research. In quantitative research, such differences between those invited and not invited would indicate an unrepresentative sample and might limit the generalisability of the findings. Qualitative studies, by contrast, often benefit from purposive or theoretical sampling of ‘information-rich’ cases,[15] and bias can be reduced by incorporating a wide range of different perspectives.[16] In the ACP study, the clinicians’ selective invitation of families perceived to have good communication skills, for example, may have provided rich data at the individual participant level; however, the exclusion of families perceived to lack these skills may have led to an absence of diverse perspectives.

Many studies in palliative care are hampered by low rates of invitation. Hinds et al.[2] reported that in prospective studies about end-of-life decision making for children with cancer, up to 27% of eligible families were placed in a ‘do-not-approach’ category by clinicians, and a ‘missed opportunity’ rate of 54.9% was reported. The proportion of families placed in a ‘do-not-approach’ category was also higher in prospective studies where the child was still living compared to retrospective studies where the child was deceased.[2]

Similar factors contributing to clinician gate-keeping in research have been reported internationally, including concerns about impact on family- or patient-professional relationships,[10,11,17-20] patient/family wellbeing or burden,[2,3,11,12,19-23] family ‘type’,[12] disease prognosis[18] or proximity to death,[3] anticipated refusal,[10,12,21] time constr-aints,[11,12,19,21,24] forgetting to ask,[2,17] and doubts about participant benefit.[10,25,26] Other factors identified in the literature (e.g. clinicians’ research experience and gender[26]) could not be assessed, given the characteristics of our cohort of clinicians.

A primary strength of our article is the use of mixed methods. By using an ethnographic prospective approach, we could identify factors influencing invitation and consider how they interact. This was complemented by quantitative data on invitation practice using anonymised, routinely collected clinical data. However, we were unable to study some potentially important variables such as parent demographics, first language and education, as these were unavailable. Also lacking was information regarding families’ participation in other research; we could not, therefore, identify and exclude all ineligible patients. Our field notes suggest that such patients constituted a small proportion of the dataset and therefore would have had minimal impact on our analyses. Another limitation was the small number of invited families (particularly in Group B) so that we could detect only large differences between invited and non-invited families and cannot conclude that no associations exist between invitation and the other variables we examined. Finally, without opportunities to speak to eligible non-invited families, we could not pursue the impact of selective invitation on the findings of the ACP study.

In conclusion, our findings highlight the potential for selection bias in paediatric palliative care research. The nature and degree of selection bias are likely to vary across studies, according to research design and local context – an issue which warrants further study. We recommend that when designing studies, researchers consider how the method and mode of invitation might impact on non-invitation rates and selection bias, and how these could be minimised. We would suggest a mixed-method assessment of the invitation and recruitment process, including observation of practice, prospective interviews with clinicians and examination of anonymised data about the sampling frame. While routine clinical data are rarely used for this purpose, they can be a valuable resource. Finally, we encourage researchers to report non-invitation rates and selection bias wherever possible, to aid interpretation of research findings.