A tool to improve the performance of multidisciplinary teams in cancer care.

Introduction: While multidisciplinary teams (MDTs) are well established in many healthcare institutions, both how they function and their role in decision-making vary widely. This study adopted an innovative methodology to assess multidisciplinary team performance and engage teams in performance improvement strategies.
Methods: The study comprised a survey to evaluate MDT members' perceptions of their team's performance before the implementation of the programme and annually thereafter, and a maturity matrix designed as a self-assessment tool. Each MDT used the matrix to collectively assess its performance and identify areas for improvement.
Results: In the first cycle, 180 member surveys from 19 MDTs were completed. This provided insights into team members' perceptions of performance. 12 of these teams continued with the study and all 12 completed the matrix. Most teams rated themselves at level one or two (low) on a scale of five for most items. Conclusions: The MDT survey and maturity matrix have the potential to be useful for cancer care teams to identify their strengths and weaknesses and monitor performance over time and also for management to review its performance against standard criteria and to identify priority areas for improvement and further support.

Background

In many countries, multidisciplinary teams (MDTs) are considered the preferred model of care for patients with cancer. Their functioning and effectiveness have been studied extensively. MDT meetings allow optimal patient management to be determined. Improved outcomes from teamwork have been recognised in some non-cancer disciplines1 and from MDTs managing patients with cancer.1–5

Several reviews, while noting the difficulty of excluding extraneous variables, found improvements in decision-making and care processes. Taplin et al, in a comprehensive literature review, found that discussion of cases within MDTs improved the planning of therapy, adherence to recommended preoperative assessment, pain control and compliance with medications.6 Freeman et al found, among patients managed by a multidisciplinary model, significantly improved timeliness, cost of care and adherence to national guidelines for staging and treatment.7

Surveys and reviews of how MDTs function have identified many opportunities for improvement.8–12 The key attributes of effective MDTs and the main barriers to team effectiveness have been widely studied and discussed.13–17 The National Health Service (NHS) National Cancer Action Team provides clear guidance on how a highly performing MDT should function.14 Examples of high performance in isolated cancer programmes, often championed by an effective team leader, have also been described.1 2 There is, however, little research on methodologies to assess MDT performance against defined guidelines nor on how to help an MDT progress to a higher level of performance.

Over the last 20 years, the Sydney West Cancer Network (SWCN) has progressively introduced MDTs for its tumour programmes (eg, thoracic oncology, melanoma). They are now well accepted and attended. While the organisation and composition of teams is similar to that found in many countries, how they function varies widely, as does the role they play in decisions about patient management.

Both an implementation science study18 and quality improvement reviews (including performance data, process mapping and semistructured interviews) of three SWCN MDTs demonstrated many common constraints.

In view of this, our cancer network established a programme to optimise how our MDTs function, the tumour programme strengthening initiative (TPSI). Two core components of this programme were the MDT member survey and the administration of what we termed a ‘maturity matrix’, both of which monitor perceptions of performance while, at the same time, educating teams on how they might perform better. Here, we discuss the two tools and early perceptions of their value.

Methodology

This study comprised an annual member survey and a maturity matrix completed by each team. The survey provides a snapshot of a team’s perceptions of their performance, while the matrix provides a framework for improvement over a period of time. For many desired outcomes, the matrix outlines several steps along the pathway towards maturity.

The SWCN has three university teaching hospitals within it, each with cancer MDTs. All MDTs were asked to complete the survey. The 12 teams that had 10 or more responses, and one with 8 responses, were invited to participate in the study. One team withdrew. The remaining 12 (colorectal, gynaecological oncology, head and neck, hepatocellular, lymphoma, melanoma, neuro-oncology, sarcoma, upper gastrointestinal, uro-oncology and two for lung cancer) agreed to continue.

Patient and public involvement

This study did not have public or patient involvement. It will be included as the study evolves.

MDT member survey

In 2017, a 43-question online SurveyMonkey questionnaire, adapted from one previously described by Rankin et al,10 was used to assess members’ perceptions of their team’s performance over multiple domains. Members participating in more than one MDT were asked to complete a survey for each one.

The responses were analysed using the SurveyMonkey data analysis software, which allows various filters to be put in place. Filters were used to isolate the results for each MDT. The authors analysed all 43 responses for each MDT and compared these with the combined results for the whole network. A summary of the findings was presented to each team. As presenting all 43 questions was onerous for team members, the study team selected results of interest to the team, generally where the MDT performed well or poorly compared with the combined result for the network.

The authors undertook more detailed analysis for six key criteria, which were commonly listed in the literature as important for team performance, and which ranged in the level of maturity required for a team to comply.13–15 When analysing the combined results for the six key criteria, the authors used filters to remove the teams that did not continue with the study so that only the results for the 12 teams were included. A comparison of all the teams was undertaken for these six criteria and the results presented to each team.

After feedback on the 2017 survey, a revised and simplified version was repeated in 2018. The six key questions were unchanged and preliminary results for the 12 teams, comparing the 2 years, have been analysed. For this, on the advice of a statistician, cluster analysis was used.

The statistical software IBM SPSS Statistics V.24 was used to analyse the data. Two-tailed tests with a significance level of 5% were used throughout. In each year, members of the same team tended to have responses which were more similar to one another than to responses from members of different teams. For each of the six key questions, each participating team was considered as a ‘cluster’. Generalised estimating equations with a robust covariance matrix and exchangeable correlation structure were used to fit logit models and test for a difference between 2017 and 2018 in the proportion of ‘positive’ responses to each question adjusted for team clustering. The cluster-adjusted Odds Ratio (OR) of a ‘positive’ response in 2018 versus 2017 and its 95% Confidence Interval (CI) were used to quantify the change and the associated cluster-adjusted p value is reported.

To gain insights into how the programmes could be improved, the survey had three open-ended questions for respondents to provide positive (Q 41) or negative (Q 42) reflections on their MDT, and general comments. After review, the responses for questions 41 and 42 were analysed by the study team and grouped into similar themes. Where a response contained several themes, these were separated. The research team incorporated activities addressing the key barriers into the broader tumour programme strengthening process, which accompanied the introduction of these tools. Where appropriate, the findings were also incorporated into the maturity matrix discussed below.

MDT Maturity matrix

The maturity matrix is a self-assessment tool for monitoring team performance. In November 2016, a planning workshop identified strategies to improve the functioning of MDTs in the SWCN. The authors incorporated the themes emerging from the workshop, the qualitative data from the MDT member survey and key themes from the literature into a matrix containing 20 domains (five components, each with four subcomponents) spread over five performance levels. The framework is shown in table 1.

Table 1

MDT maturity matrix framework

Components	Subcomponents	Maturity levels
		1	2	3	4	5
1. Governance and leadership	Leadership  Obligations of team members  Decision-making  Risk management
2. Meeting organisation and logistics	Logistics and representation  Pre-meeting  At meeting  Post-meeting
3. Linkages and communication with GPs and patients	Access for GPs  Communication with GPs (patients)  Information and education (general)  Patients
4. Data collection, analysis and research	Data collection  Monitoring and evaluation  Data quality and system integration  Research
5. Infrastructure and human resources	Facilities and equipment  MDT co-ordination  Care co-ordination  Data management

GP, general practitioner; MDT, multidisciplinary team.

The total number of squares in the matrix is 100 and each square contains one or more statements. Each team was asked to consider the statements in the matrix and to indicate, by highlighting, whether it considered it had complied with each one. The matrix was designed so that it was more difficult to comply with statements at higher levels.

To quantify the results for each MDT, we allocated two points if all the statements in a square were highlighted, one point if some but not all were and no points if nothing was highlighted. A final score was calculated for each team and for the network. MDTs that scored highly were considered to be more mature. It was envisaged that these results would be compared and monitored over time.

All 12 teams participating in the programme agreed to continue with this tool.

Results

A total of 180 surveys were completed across the network reflecting the work of 19 MDTs. Seventy-seven percent of all responses were from 14 MDTs at hospital A, 9% from hospital B (2 MDTs) and 14% from hospital C (3 MDTs). A total of 129 responses, comprising 82 (64%) treating specialists, 19 (15%) nursing staff, 18 (14%) diagnostic specialists, 4 (3%) research staff, 3 (2%) allied health and 3 (2%) administrative staff, were received from the 12 teams that continued with the study. The responses for these 12 teams for the first 40 questions are shown in online supplementary file 1.

The terms ‘core’ and ‘extended’ members have been used14 to distinguish between those members who are important for a full multidisciplinary discussion and are expected to attend most meetings and those who are welcome to attend but are not integral to the functioning of the team. Although core and extended membership had not yet been established for SWCN MDTs, the responses were estimated to represent more than 70% of core membership.

Table 2 shows the percentage of positive responses by MDT and the combined results for the 12 teams. Responses varied greatly. Some teams, on the left, scored highly and others rated a much lower score (to the right).

Table 2

Percentage of positive responses by MDT for the six selected criteria (2017 MDT member survey)

C=Combined results from 12 teams		Multidisciplinary teams
	C	7	10	12	9	8	1	3	5	6	4	2	11
Does the MDT have TORs or guidelines to guide the conduct of the meetings? (yes/no)	16	30	43	8	10	9	36	25	20	0	0	0	0
Are there established criteria for referral of patients to MDT meetings? (yes/no)	26	70	43	38	50	36	18	25	0	8	0	10	10
Is there a follow-up process to check whether referrals from the MDT are actually made? (yes/no)	14	30	57	8	30	18	0	0	10	0	0	0	0
Does the MDT use clinical practice guidelines or standard treatment protocols relevant to the diagnosis, treatment and care of patients with cancer? (always or usually)	63	80	65	69	80	81	100	63	30	58	50	40	30
Does the MDT routinely collect the time from diagnosis to definitive treatment? (yes/no)	10	20	14	31	33	0	0	25	0	0	0	0	0
Are internal audits conducted to confirm that treatment decisions match current best practice? (yes/no)	7	20	14	8	0	9	9	0	10	0	10	0	0

MDT, multidisciplinary team; TOR, Terms of Reference.

When the results were presented to teams, interest was high. Several MDTs immediately decided to improve in one or more respects. Most wished to improve in the following year; some have selected a core leadership group to initiate change. Online supplementary file 2 is an example of a presentation to a team (note that while the initial presentations compared the teams with the whole network, for all further analyses only results from the 12 participating teams have been used).

A summary of the responses from the open-ended questions (Q 41 and Q 42) can be seen in table 3 with more detail included in online supplementary file 3. The most common positive attributes related to cooperation, collaboration, participation of team members and the quality of discussion. The most common barriers were the lack of administrative and nursing resources, time constraints, poor attendance, and IT and data collection issues.

Table 3

Summary of responses to open-ended questions*

Q 41: What do you believe are the strengths of multidisciplinary care in your facility? Please list them.		Q 42: Do you think there are any barriers to the implementation or improvement of multidisciplinary care in your facility? If so, please list these barriers.
Responded with answers	122	Responded with answers	96
Answered ‘none’ or ‘N/A’	2	Answered ‘no’ or ‘not sure’	27
Did not respond	5	Did not respond	6
Total	129	Total	129
Themes	Number	Themes	Number
Participation and attendance	42	Lack of a dedicated administrative co-ordinator	24
Cooperation and collaboration	38	Poor attendance	20
Discussion	22	IT support and infrastructure	20
Decision-making	18	Meeting organisation and logistics	18
Administration/logistics	17	Time constraints and workload	18
Research and education	13	Database and data collection	18
Care delivery	12	Live data entry and documentation of outcomes/decisions	15
Pathology/imaging review	12	Lack of a care co-ordinator	13
Expertise	8	Communication with patients and GPs	6
Other	12	Research	6
Total	194	Finances (unspecified)	5
Respondents reporting barriers	5	Other	24
Total	199	Total	187

*Where responses included several themes, these have been counted separately.

GP, general practitioner; IT, Information Technology.

These results were integrated into the TPSI by the TPSI team, which has proceeded to address many of the issues raised. Where relevant, items were included in the matrix. Following comments from team members relating to the length of the survey, prior to distribution in 2018 it was simplified and shortened, without compromising the core content.

Table 4 shows the comparison between 2017 and 2018 for the six key criteria. The percentage of respondents answering ‘yes’ for the two questions “Does the MDT have TORs or guidelines to guide the conduct of the meeting?” and “Does the MDT have established criteria for referral of patients to MDT meetings?” increased from 16% to 46% and 26% to 53%, respectively. Both of these were significant at the 0.001 level. No significant change was seen for the other four criteria.

Table 4

Number of ‘positive’ responses, total number of responses, raw percentage of ‘positive’ responses, cluster (team) adjusted OR and 95% CI for ‘positive’ responses in 2018 vs 2017 and associated p value for the six key questions

Question	2017 Responses			2018 Responses			OR	95% CI for OR		P value†
	n‡	N§	%*	n‡	N§	%*		Lower	Upper
1. Does the MDT have TORs or guidelines to guide the conduct of the meetings? (yes)	20	129	15.5	54	117	46.2	4.8	2.5	9.3	<0.001
2. Are there established criteria for referral of patients to MDT meetings? (yes)	34	129	26.4	62	117	53.0	2.9	2.0	4.4	<0.001
3. Is there a follow-up process to check whether referrals from the MDT are actually made? (yes)	18	129	14.0	22	117	18.8	1.6	0.7	3.5	0.262
4. Does the MDT use clinical practice guidelines/treatment protocols relevant to patients with cancer? (always or usually)	81	129	62.8	72	117	61.5	0.9	0.6	1.3	0.532
5. Does the MDT routinely collect the time from diagnosis to definitive treatment? (yes)	13	128	10.2	22	117	18.8	2.0	0.9	4.1	0.078
6. Are internal audits conducted to confirm that treatment decisions match current best practice? (yes)	9	129	7.0	9	117	7.7	1.2	0.7	1.9	0.543

*Percentage of ‘positive’ responses.

†P value adjusted for team clustering.

‡Number of ‘positive’ responses.

§Total number of responses.

CI, confidence interval; MDT, multidisciplinary team; OR, odds ratio; TOR, Terms of Reference.

MDT maturity matrix

While all teams completed the matrix, only four did it collaboratively as proposed. Members in three teams completed it individually and the study team collated their results, while for the remainder, an individual or a small core group filled it in for the team. In all cases, the results were presented back to the whole MDT. Despite the methodological difficulties, there was broad agreement between the perceived performance from the survey and that from the matrix. Figure 1 shows the score assigned to each team.

Figure 1

Scores allocated to each multidisciplinary team (MDT) following completion of the 2017 maturity matrix. A higher score indicates a higher level of maturity.

Discussion

This study showed that an annual MDT member survey and completion of a maturity matrix by each team provided insights into a team’s performance, helped engage MDT members in improvement and provided a framework for strengthening performance.

The key insights from both the survey and the matrix were that the level of maturity of MDTs, even within the same institution, varied greatly and that, as practising clinicians, many MDT members had not previously been engaged in process improvement and were not aware of the emerging, more complex, functions being asked of MDTs around the world. Finally, team members were supportive of a programme to improve performance while seeking guidance to support its implementation.

A high level of engagement from team members was indicated by their willingness to complete the survey and matrix, their eagerness to have the results presented to them and by independent initiatives to address their identified weaknesses. The high level of completion maintained in 2018 reinforces this.

The survey was useful in several respects. First, presenting the 2017 results to the teams raised awareness and interest among team members as demonstrated by the independent initiatives some teams embarked on such as establishing a core leadership group. Second, the open-ended question about barriers to performance allowed the TPSI team to prioritise improvement activities. Finally, the early comparisons for the six key criteria between 2017 and 2018 gave the team confidence that the interventions had the potential to generate change.

Self-assessment ensured both that the responses reflected the opinion of most team members and that it was not overly burdensome for MDT members or for management.

Weaknesses were identified. Core and extended membership has been clarified and the survey has been revised to address issues with respect to its length. Some questions were amalgamated to make responding simpler and several, which did not require annual review, have been removed.

In introducing the matrix, it became evident that the mode of delivery was a constraint to standardising its completion. Many teams commented that the matrix was a good initiative while noting that adaptations were needed. Most suggested wording changes to reduce ambiguity. Some felt that a software solution was needed to improve the mechanism of delivery. Many team members suggested that the matrix be adapted to have a more educational focus. These adaptations will be introduced before the next version is finalised.

The importance of the two instruments was that they reinforced each other both in outlining how a highly functioning MDT should perform and in providing guidance on the practical steps that are required to become a highly performing team. The repeated reinforcement of the concepts in the survey and matrix served to educate team members on this.

This fed into the wider goal of the TPSI programme which is not simply to monitor performance but to inspire major and sustainable organisational change over time. While the foundations have been laid, the programme will be continually adapted. Although the initial significant results were pleasing, it is long-term sustainable improvement in team performance that is the ultimate goal.

Many articles describe the functioning of MDTs,10 11 the characteristics of well-performing teams4–6 19 or examples of improved performance in individual teams led by motivated individuals.1 20 Lamb et al outlined several strategies to improve the efficiency and use of multidisciplinary teams.21

A few groups have outlined tools—both electronic and paper based—to strengthen MDTs. Soukup et al provide an excellent summary of the recent literature.17 The Western and Central Melbourne integrated cancer service provided paper-based tools to help with organising the MDT.19 Patkar et al have demonstrated benefits from a decision support system,22 and Nouraei et al have shown that a database aimed at ensuring that patients were presented after all their relevant investigation had been completed, demonstrably improved efficiency.23 Several teams have developed and validated tools for teams to measure their performance.24–26

This study filled a gap by describing, from an institutional perspective, a means of improving, and measuring improvement in performance of MDTs over time. It differed from others in several important ways. First, it targeted all tumour streams within the SWCN, regardless of their level of maturity. Second, while allowing considerable flexibility in the activities each team chose to address, it provided a comprehensive framework by which MDTs could determine areas for improvement. Finally, and importantly, it provided management with insights about team performance so that poorly performing teams could be assisted and monitored on an ongoing basis.

Our use of the survey of Rankin et al differs from theirs in several ways.10 We collected the opinions of a cohort of members of the team (at least eight) rather than just that of the chair or delegate. We compared the results of each team with the collated results from the SWCN and presented these comparisons back to the teams. Finally, the survey was used in conjunction with the maturity matrix.

The maturity matrix describes steps along the path from a basic to a highly performing team, thus providing a framework for teams to strengthen their performance in incremental stages. This progression advances the concepts outlined by the NHS National Action Cancer Team in 2010.14

Conclusions

This is a practical methodology to engage MDTs at various stages of maturity. Moreover, it has potential as a means for teams to identify their own strengths and weaknesses and for management to review all MDTs against standardised criteria and to determine further support required.

The significance of this initiative is that overall programme improvement should reflect strengthening of the weakest teams and not just further improvement in already highly performing MDTs.