Establishment of consensus on content and learning objectives for an interprofessional education in childhood cancer: a Delphi survey.

BACKGROUND: Complex treatment, care and rehabilitation require continuous healthcare professional development and maintenance of competencies in collaboration with other professionals. Interprofessional education in childhood cancer involves several groups of healthcare professionals with both general and specific knowledge and skills.
OBJECTIVE: To establish consensus on content and interprofessional learning objectives for an interprofessional education in childhood cancer.
DESIGN: A three-round Delphi survey in Scandinavian childhood cancer departments. PARTICIPANTS: Healthcare professionals appointed by their head of departments and head nurses based on their profession and their involvement in continuing professional development. MAIN OUTCOME MEASURES: A prioritised list of interprofessional learning objectives with a mean score of ≥3 on a five-point scale (1=not relevant, 5=extremely relevant).
RESULTS: 12 childhood cancer departments participated with 30 healthcare professionals: 11 nurses, 10 medical doctors, 5 social workers, 2 physiotherapists and 2 pedagogues. In total, 28 (93%), 25 (83%) and 22 (73%) completed the first, second and third round, respectively. In the first round, we asked open-ended questions and used directed content analysis to analyse 386 statements. We formulated 170 interprofessional learning objectives in six categories: (1) acute life-threatening situations, (2) gastrointestinal toxicities and side effects, (3) pain, (4) palliation, (5) play and activity, and (6) prescription and administration of medicine. The second round resulted in 168 interprofessional learning objectives receiving a mean score of ≥3 on a five-point scale. Final agreement in the third round resulted in a prioritised list of 168 learning objectives.
CONCLUSIONS: Consensus on content and interprofessional learning objectives for an interprofessional education in childhood cancer was established across five groups of healthcare professionals in three countries. Some learning objectives are generic and can be applied in settings other than childhood cancer, where healthcare professionals collaborate to provide patients and families optimal treatment and care. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Well-structured and evaluated, interprofessional education in childhood cancer is lacking, as is consensus on the content and learning objectives of interprofessional education.

Interprofessional learning objectives were clearly formulated for a childhood cancer setting. Some of the identified interprofessional learning objectives are generic. They can be applied in clinical settings other than childhood cancer where healthcare professionals collaborate to provide patients and families optimal treatment and care.

Introduction

Complex treatment, care and rehabilitation require continuous professional development and maintenance of healthcare professionals’ competencies, both in their own professional fields and in collaboration with other professionals.1 Interprofessional education in childhood cancer is a multifaceted field involving multiple healthcare professionals with general and specific knowledge and skills who provide the best clinical management of children and adolescents with cancer.

The continuing professional development of medical doctors in childhood cancer is well documented, as demonstrated by well-structured fellowship and residency programmes that have been professionally evaluated.2–7 Continuing professional development of other groups, such as nurses and psychologists in childhood cancer, is less documented.8–10 Despite working alongside each other daily, healthcare professionals in childhood cancer are not formally trained in how to collaborate or possess knowledge about each other’s competencies.2 Consequently, well-structured, evaluated interprofessional education in childhood cancer is lacking.2 There is no agreement on the content or which learning objectives are relevant for an interprofessional education in childhood cancer.2

Interprofessional education can be defined as ‘Occasions when two or more professionals learn with, from and about each other to improve collaboration and the quality of care’.11 12 This implies that an education intervention can improve how healthcare professionals work together, which in turn contributes to improved patient outcomes.12 13 However, interprofessional education outcomes are not easily monitored, and research addressing interprofessional education is inherently complex as definitions of what constitutes interprofessional education and evaluation methods vary.13 14 There is no gold standard.

No standards exist in terms of defining, designing or evaluating educational activities in interprofessional childhood cancer.2 It is unknown which educational activities and outcomes should be included in an interprofessional education in childhood cancer. This is important because specific and measurable learning objectives are essential to evaluating educational activities.15 Learning objectives are defined as ‘an end towards which an effort is directed’.15 In the development of a new education for interprofessional childhood cancer, defining learning objectives based on group consensus can increase content dependability and credibility.16 To reach transferability, this Delphi survey invited childhood cancer departments in three countries, Denmark, Norway and Sweden, to participate.

When there is no consensus in a specific area, such as defining, designing or evaluating interprofessional education in childhood cancer, the Delphi method is suitable to synthesising expert opinions and enhancing decision-making.16 17

The objective of this Delphi survey is to establish consensus on the content and interprofessional learning objectives of an interprofessional education in childhood cancer.

Material and methods

We used a three-round Delphi survey (flowchart in figure 1). The Delphi method is a consensus method where experts are solicited to give their expert opinion. The method is used to synthesise opinions and enhance decision-making in areas where there is no established consensus. The consensus method typically includes three rounds that are predefined, structured and most importantly anonymous to avoid dominance by certain groups. The Delphi method is widely used in healthcare education for developing curricula components such as learning objectives.18 19 To ensure methodological rigour, this Delphi survey follows the 11 recommendations of Humphrey-Murto et al,16 as illustrated in online supplementary file 1.

Figure 1

Flow diagram of the Delphi process.

Selection of Delphi participants

The selection of Delphi participants comprised a pre-Delphi step to ensure a transparent and inclusive process (see flowchart in figure 1). We emailed the heads of department and head nurses of all 14 childhood cancer departments in Denmark, Norway and Sweden, to invite them to select Delphi participants.

We asked them to select at least two healthcare professionals to participate in the Delphi survey, one with the same and one with a different professional background as themselves. The individuals selected had to be members of a team treating, caring or rehabilitating children and adolescents with cancer and they had to be involved in continuing professional development.

There were no restrictions on the number of years of employment or educational background. The heads of department and head nurses were permitted to name themselves.

Denmark, Norway and Sweden share childhood cancer treatment protocols and have similar social, healthcare and educational systems. The Nordic languages are mutually intelligible, which greatly facilitated the first round of the Delphi survey, as healthcare professionals could write comments in their own language. To ensure completeness, a forward translation from Danish to Swedish and Norwegian was done of the questions and text for the first Delphi round and then a back translation was carried out from Swedish and Norwegian into Danish with the assistance of native speakers. The context of the translation and cultural aspects were taken into consideration in the translation.20

Content of questionnaire and consensus

The scientific evidence for the first round of the questionnaire was selected from the findings of a systematic literature search conducted in 2018.2 The findings were supplemented by existing programmes and guidelines from the Danish Paediatric Haematology and Oncology Group (DAPHO).21 This generated 17 categories relating to interprofessional issues in childhood cancer (online supplementary file 2). The questionnaire for the first Delphi round was pilot tested on a small group of healthcare professionals that matched the inclusion criteria.

In the first round, open-ended questions were formulated in accordance with Murphy et al 22 to avoid biassing the participants. An example of an open-ended question was: “In your work with children and adolescents with cancer, which competences do you find important to: Communicate interprofessional with colleagues” (see online supplementary file 2 for all questions).

In the second and third round, the participants were asked to rate each learning objective on a five-point scale (1=not relevant, 2=less relevant, 3=relevant, 4=very relevant and 5=extremely relevant). Figure 2 illustrates the structure of the learning objectives.

Figure 2

Flowchart of content analysis process

In the first two rounds, the participants were encouraged to suggest additional categories and comments.

In the final third round, participants could re-rate the learning objectives or maintain their answers from the second round.

As the aim of this Delphi survey was to establish consensus on learning objectives, the criteria for inclusion of a learning objective was set at a mean score of ≥3 across the participants.

Reporting between rounds

We reported to the participants how their answers were distributed between the first and the second round.

Design and administration of questionnaire

The Delphi survey was carried out using a REDCap electronic questionnaire.23 Figure 1 illustrates the three rounds of the Delphi process. REDCap has a feature for required fields that makes it impossible to have missing data because participants have to reply to all questions before continuing.

Data analysis

To analyse the participants’ statements from the first round, we applied a directed approach to content analysis.24 25 Five authors (MKT, MW, HBL, MHH, JLS), representing different healthcare backgrounds, were involved in the reading of individual statements in the first round. Figure 3 presents a flow chart of the data analysis process.

Figure 3

Structure of learning objectives.

Based on Thomas’15 structure for formulating learning objectives, Blooms taxonomy was applied26 to ensure action verbs in the psychomotor and affective domains, emphasising ‘doing’ rather than ‘knowing’.

The rating of the interprofessional learning objectives was assessed by means. The variance of response is presented as the distribution of ratings in a range for each objective (see tables 1 and 2). Data were compiled and assessed in Excel for Office 365 (Microsoft, Redmond, Washington, USA).

Ethics

The Delphi participants were informed that by clicking on the link to the questionnaire they authorised the use of their answer in the survey. All contact information was treated with confidentiality and all answers were anonymised. The Danish Data Protection Agency, an independent authority that supervises compliance with the rules on protection of personal data approved the survey (P-2019-76). In the final round, participants were asked if they wanted to be acknowledged by name in the publication.

Patient and public involvement

Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.

Results

Of the 14 childhood cancer departments in Denmark, Norway and Sweden invited to participate, 12 departments participated with 30 healthcare professionals. Each centre participated with one to four healthcare professionals. Table 3 describes the profession of participants. The departments’ participation was comparable in number of participants. There were five different professional groups represented in this survey. The largest group of participants was nurses (n=11), followed by doctors (n=10), social workers (n=5), pedagogues (n=2) and physiotherapists (n=2).

Table 3

Professions of the participants (n=30)

Sex, n (%)
Women	26 (87)
Men	4 (13)
Profession
Doctor	10 (33)
Nurse	11 (36)
Social worker	5 (17)
Pedagogue	2 (7)
Physiotherapist	2 (7)

The distribution of participants who responded was similar in the three rounds. Of the 30 healthcare professionals chosen to participate, 28 took part in the first round (93%), 25 in the second (83%) and 22 (73%) in the third and final round.

The distribution of non-respondents was evenly distributed across countries and mainly affected the two largest represented groups, doctors and nurses. In round two non-responders were: 2 doctors, 2 nurses and one social worker. In round three this increased to: 4 doctors, 3 nurses and one social worker. All three countries were represented in the last round by ten centres.

The 17 categories from the first round generated 386 statements in the first Delphi round. The statements varied markedly in length and content, from short remarks to long descriptive sentences. Five authors (MKT, MW, HBL, MHH, JLS) analysed the statements and formulated 170 interprofessional learning objectives divided into six new categories on three hierarchical levels (see online supplementary file 3).

The three hierarchical levels are (1) all professionals working with children or adolescents with cancer, (2) all doctors and nurses, and (3) specialised paediatricians and experienced or specialised paediatric nurses.

Dispersed across the three hierarchical levels, the 170 interprofessional learning objectives build on each other; hence, level 1 learning objectives also apply to healthcare professionals on levels 2 and 3.

The six categories are (1) acute life-threatening situations, (2) gastrointestinal toxicities and side effects, (3) pain, (4) palliation, (5) play and activity, and (6) prescription and administration of medicine.

Only one interprofessional learning objective was removed in the second round due to a mean score ≤3 on the five-point scale. The objective was: “Be part of the team treating the child or adolescent with cancer”, which appeared twice and was removed from these two categories: “Gastrointestinal toxicities” and “Prescription and administration of medicine”.

In the third round, participants maintained their previous answers, resulting in the inclusion of 168 interprofessional learning objectives in the interprofessional education in childhood cancer.

Two tables illustrate the distribution of the final 168 interprofessional learning objectives, with table 1 listing the generic interprofessional learning objectives, and table 2 the specific interprofessional learning objectives.

Table 1

Generic interprofessional learning objectives

	Generic interprofessional learning objectives	Acute life-threatening situations	Gastrointestinal toxicities	Pain	Palliation	Play and activity	Prescription and administration of medicine
		Mean (range)	Mean (range)	Mean (range)	Mean (range)	Mean (range)	Mean (range)
ALL PROFESSIONALS (teachers. pedagogues. social workers. physiotherapists. all medical doctors. nurses and other professionals work-worn with children and adolescents with cancer)	Identify one’s own professional limitations and ask for help	4.7 (3–5)	4.6 (3–5)	4.5 (3–5)	5.0 (3–5)	4.4 (3–5)	4.5 (3–5)
	Identify need for communication with the nurse and/or doctor in charge of patients	4.7 (3–5)	4.4 (3–5)	4.4 (3–5)	5.0 (3–5)	4.4 (2–5)	4.5 (3–5)
	Understand and respect the perspectives of colleagues	4.4 (3–5)	4.4 (3–5)	4.5 (3–5)	5.0 (3–5)	4.6 (3–5)	4.4 (3–5)
	Bring up problems or tough issues	4.4 (3–5)	4.2 (3–5)	4.1 (3–5)	5.0 (3–5)	4.3 (3–5)	4.1 (2–5)
	Communicate in an age-appropriate manner, adjusted to the situation and the patient’s need for and right to information	4.4 (2–5)	4.1 (1–5)	4.1 (2–5)	4.5 (2–5)	4.4 (3–5)	3.6 (1–5)
	Listen, appreciate and enquire regarding issues beyond one’s own knowledge and profession	4.1 (2–5)	4.2 (3–5)	4.1 (3–5)	5.0 (3–5)	4.3 (3–5)	4.2 (3–5)
	Apply appropriate and targeted communication, taking into account the patient’s mental development and resources	4.0 (2–5)	3.9 (1–5)		4.5 (3–5)	4.4 (3–5)	3.5 (1–5)
	Assess the patients’ need for information	3.9 (2–5)	3.9 (1–5)	3.9 (3–5)	4.0 (3–5)	4.2 (2–5)	3.6 (1–5)
	Convey information between and to and from relevant healthcare professionals and patients and their families	3.9 (2–5)	3.7 (1–5)	4.0 (2–5)	4.5 (3–5)	3.9 (3–5)	3.5 (3–5)
	Identify dilemmas regarding patient autonomy, involvement and respecting patient confidentiality at ages 15–17, and act according to national legislation	3.8 (2–5)	4.1 (1–5)	4.0 (2–5)	4.5 (3–5)	4.3 (3–5)	3.3 (1–5)
	Be part of the team treating the child or adolescent (…)	3.2 (1–5)	2.9 (1–5)	3.8 (2–5)	3.5 (2–5)	4 (2–5)	2.9 (1–5)
ALL DOCTORS AND NURSES	Identify complex management issues and involve other relevant staff, including paediatric oncologists	4.3 (1–5)	4.2 (1–5)	4.2 (1–5)	4.2 (1–5)	4.1 (1–5)
	Initiate, maintain and/or evaluate pharmacological and non-pharmacological treatment	4.3 (1–5)	3.9 (1–5)	4.3 (1–5)	4.2 (1–5)
	Apply knowledge of relevant local/national /international guidelines/standard operating procedure	4.1 (1–5)	4.1 (1–5)	4.2 (1–5)	4.2 (1–5)	3.8 (1–5)	4.2 (1–5)
	Observe and communicate the need to initiate, maintain and evaluate treatment, pharmacological and non-pharmacological	4.5 (1–5)	4 (1–5)	4.4 (1–5)	4.2 (1–5)
	Collaborate with children and adolescents and their families and significant others in assessment, treatment and evaluation	4.3 (1–5)	3.9 (1–5)	4.4 (1–5)	4.2 (1–5)
SPECIALISED PAEDIATRICIANS (not trainee doctors) and EXPERIENCED/SPECIALISED PAEDIATRIC NURSES	Identify need for communication with other relevant staff, including the paediatric oncologist	4.4 (1–5)	4.3 (1–5)	4.3 (1–5)	4.3 (1–5)	4.1 (1–5)	4.2 (1–5)
	Communicate strategies to healthcare professionals as well as patients and families	4.4 (1–5)	4.2 (1–5)	4.3 (1–5)	4.2 (1–5)	3.9 (1–5)
	Assume team leadership, within one's own professional capacity	4.2 (1–5)	4.0 (1–5)	4.2 (1–5)	4.3 (1–5)	3.5 (1–5)
	Coordinate and delegate tasks	4.4 (1–5)	4.1 (1–5)	4.2 (1–5)	4.3 (1–5)	3.7 (1–5)
	Assist in identification and resolution of current and potential disagreements between healthcare professionals or between healthcare professional and families	3.8 (1–5)	4.0 (1–5)	4.2 (1–5)	4.2 (1–5)	3.8 (1–5)

Table 2

Specific interprofessional learning objectives

	Specific interprofessional learning objectives	Acute life-threatening situations	Gastrointestinal toxicities	Pain	Palliation	Play and activity	Prescription and administration of medicine
		Mean (range)	Mean (range)	Mean(range)	Mean (range)	Mean (range)	Mean (range)
ALL PROFESSIONALS (teachers, pedagogues. social workers, physiotherapists, all medical doctors, nurses and other professionals working with children and adolescents with cancer	Identify a child or adolescent in urgent need of help	4.6 (3–5)
	Apply basic paediatric resuscitation	3.7 (1–5)
	Identify how to handle waste material, such as vomit, urine or faeces		3.7 (1–5)
	Identify a child or adolescent with nausea and/or vomiting		3.3 (1–5)
	Identify a child or adolescent with gastrointestinal problems such as mucositis		3.0 (1–5)
	Identify a child or adolescent with pain			4.1 (2–5)
	Be forthcoming to wishes from the patient and their families				5.0 (3–5)
	Apply communication with patients or families in crisis				4.5 (3–5)
	Apply communication with patients in palliative care and their families				4.5 (3–5)
	Identify ethical dilemmas				4.5 (3–5)
	Alleviate physical, psychological and spiritual issues				4.5 (3–5)
	Identify a child or adolescent in need of palliative care				4.0 (2–5)
	Identify barriers of communication, such as grieving and crisis, and how these may culturally different				4.0 (3–5)
	Ensure access to psychological support for both patients and families and healthcare professionals				3.5 (2–5)
	Assist in procuring necessary aids to ease everyday life at home or at the department				3.5 (2–5)
	Encourage participation of siblings and friends in daily routines				3.0 (2–5)
	Limit isolation					4.6 (3–5)
	Identify the importance of play and activities in respect to physical and mental development					4.6 (3–5)
	Encourage participation in normal social relations such as school and social activities					4.5 (3–5)
	Include siblings and friends from school in the treatment whenever possible					4.2(1–5)
	Include play and activity in treatment and procedures					4.0 (1–5)
	Inform children. adolescents and their families about age appropriate play and activities					3.9 (1–5)
	identify a child or adolescent in need of play and activity					3.9 (1–5)
	Assist children and adolescents in planning and performing age appropriate play and activities					3.9 (1–5)
	Identify how various cancer diseases affect the musculoskeletal function					3.5 (1–5)
	Respect non-disturbance zones at the department						4.3 (3–5)

As a single number 168 learning objective seems vast, this is because the learning objectives are repeated across the six categories.

For example, there are 23 learning objectives for category 2 (gastrointestinal side effects). The category with the most learning objectives is category 4 with 33 learning objectives. We have included the relevant, very relevant and extremely relevant objectives in this study which makes it possible for educators to choose all the objectives or only objectives at certain levels of relevance.

We identified 21 generic interprofessional learning objectives that were repeated in all six categories, which accounts for the high number of learning objectives. The following two generic interprofessional learning objectives, for example, appeared in all six categories: (1) identify one’s own professional limitations and ask for help and (2) identify need for communication with the nurse and/or doctor in charge of patients. These two objectives received the highest rating across the six categories (mean 4.7 (3–5)).

We also identified 57 specific interprofessional learning objectives that only pertained to one of the six categories. Categorised under ‘Palliation’, the one with the highest ranking was: “Be forthcoming regarding the wishes of the patient and their families” (mean 5.0 (3–5)).

As indicted by the ranges, the interprofessional learning objectives were rated consistently high.

Discussion

In this Delphi survey we established consensus on content and interprofessional learning objectives by identifying 168 interprofessional learning objectives in six categories for an interprofessional education in childhood cancer. Most of the interprofessional learning objectives identified are generic and can be applied in clinical settings other than childhood cancer.

Integration with prior work

The interprofessional education studies that we identified in a scoping review2 predominantly consider doctors and nurses. However, this Delphi survey illustrates that it is possible to formulate interprofessional learning objectives that also include healthcare professionals such as social workers, physiotherapists and pedagogues. Their presence, in addition to doctors and nurses, is warranted in interprofessional education in childhood cancer as there is a need for interprofessional teamwork due to the increasing complexity of skills and knowledge required to provide the best treatment, care and rehabilitation.1

Thistlethwaite et al 27 proposed that learning objectives for interprofessional education can be divided into three categories: profession specific objectives, generic objectives for two or more professions, and generic objectives to be met by all professionals. This was also reflected in our survey. Our content analysis revealed three hierarchical levels in interprofessional learning objectives that suggest connections between healthcare professionals while simultaneously indicating clear boundaries between professional roles and responsibilities. In this Delphi survey, the highest rated learning objectives indicate that it is crucial for healthcare professionals to know who to contact when they realise that their competences are inadequate in terms of providing optimal treatment and care for children and adolescents with cancer. Learning objectives like these are not specific to childhood cancer and suggest the importance of being aware of professional limitations when collaborating interprofessionally, as Hammick states.28

Previous studies have also shown that collaboration, understanding and respecting roles and responsibilities are key interprofessional learning objectives that transgress professional boundaries.27 29 These values were also rated highly in our learning objectives. In accordance with Hammick et al,28 this indicates that the best clinical management depends on the collaboration of healthcare professionals who are knowledgeable about each other’s skills, roles and responsibilities.

Implications for practice and future work

To derive the most benefit from medical education we need to apply gold standards similar to any other healthcare technology. This Delphi survey contributes with specific and measurable learning objectives that are instrumental in the evaluation of interprofessional education. Managers such as heads of department and head nurses interested in continuous professional development and maintenance of their employees’ knowledge, skills and attitudes can apply the identified learning objectives to structure and evaluate educational activities. These educational activities can, in turn, contribute to achieving improvements in the clinical health outcomes of the patients.15 30 31 Leadership buy-in is crucial for setting aside time and establishing safe learning spaces, where healthcare professionals from different backgrounds can exchange knowledge, skills and attitudes to improve collaboration and quality of care.

Strengths and limitations

The 30 participants in this Delphi survey were selected by the heads of department and the head nurses in 12 Scandinavian childhood cancer departments representing five groups of healthcare professionals. However, we cannot assume that the opinions of these 30 healthcare professionals are representative of all healthcare professionals working in childhood cancer. There are evidently more groups of healthcare professional working with children and adolescents, such as occupational therapists, dieticians and psychologists. Groups like these might have different views on which competences are relevant when working with children and adolescents with cancer. In future studies, greater effort should be made to ensure representation of all groups. In case of anonymity and dominance concerns, different methods could be applied such as focus group interviews with the same group of healthcare professionals.

Content analysis can be a challenging process with a risk of losing information and the presence of analytical bias.32 To avoid bias, five authors read and discussed the 386 statements in three steps before formulating the interprofessional learning objectives.

The formulation of learning objectives can be challenging. One the one side, interprofessional learning objectives should be all encompassing and on the other side, they should be operational. Some of the interprofessional learning objectives may appear wide, leaving a task to the educators to unfold, target and adapt the learning objectives to the specific setting and interprofessional group.

One of the benefits of conducting a Delphi survey is the opportunity to have smaller yet significant groups to voice their opinions. In childhood cancer, doctors and nurses are the largest group of healthcare professionals represented. However, in this Delphi survey, the opinions of small groups of healthcare professionals were assigned the same significance as professions with a larger in volume.

The high response rate in the first round (98%) could be attributed to the option healthcare professionals had to write their responses in their native tongue as opposed to English. It also suggests that participants found the topic relevant, which coupled with frequent reminders to participate, are important factors in attrition rates, which is an issue that ultimately affects the validity of Delphi study findings.17

Conclusion

Consensus on content and interprofessional learning objectives for an interprofessional education in childhood cancer was achieved across the three countries and the five groups of healthcare professionals. Some learning objectives are generic and can be applied in settings other than childhood cancer, where healthcare professionals collaborate with each other, the patients and the relatives about optimal treatment and care.

A randomised controlled trial with the objective to study the effect of interprofessional versus monoprofessional case-based learning on healthcare professionals’ attitudes on interprofessional learning and collaboration will be tested on 14 groups of healthcare professionals including allied health professionals at the childhood cancer department in Copenhagen, Denmark. The topic of the case-based learning will the gastrointestinal side effects (Clinical Trials.gov: NCT04204109).33