Quality indicators for the diagnosis and surgical management of breast cancer in South Africa.

INTRODUCTION: Quality indicators (QIs) for breast cancer care have been developed and applied in high-income countries and contributed to improved quality of care and patient outcomes over time.
MATERIALS AND METHODS: A modified Delphi process was used to derive expert consensus. Potential QIs were rated by a panel of 17 breast cancer experts from various subspecialties and across South African provinces. Each QI was rated according to importance to measure, scientific acceptability and feasibility. Scoring ranged from 1 (no agreement) to 5 (strong agreement). Inclusion thresholds were set a priori at mean ratings ≥4 with a coefficient variation of ≥25%. Levels of evidence were determined for each indicator.
RESULTS: The literature review identified 790 potential QIs. After categorisation and removal of duplicates, 52 remained for panel review. There was strong consensus for 47 which were merged to 30 QIs by exclusion of similar indicators and indicator grouping. The final set included eight QIs with level I or II evidence and two QIs with level III evidence which were deemed "mandatory" due to clinical priority and impact on care. The remaining QIs with lower-level evidence were grouped as eight "recommended" QIs (regarded as standard of care) and twelve "optional" QIs (not regarded as standard of care).
CONCLUSION: A regional set of QIs was developed to facilitate standardised treatment and auditing of surgical care for breast cancer patients in South Africa. Routine monitoring of the ten mandatory QIs, which were selected to have the most substantial impact on patient outcome, is proposed.

Introduction

Breast cancer is the most common malignancy among women in Sub-Saharan Africa [1]. Patient outcomes remain poorer in low-to middle-income countries (LMICs) compared to high-income countries (HICs), and reasons include difficult access, late presentation, decreased awareness, and limited treatment resources [2]. Robust data on the quality of surgical breast cancer care in South Africa is non-existent to date, but considerable variations depending on the geographical location and socioeconomic status of the patient are likely. A recent report on timely delivery of chemotherapy, radiation, and endocrine treatment showed marked non-adherence and the need for focused quality improvement in South Africa [3]. Although South Africa is regarded as an upper-middle-income country (UMIC) by the World Bank, it has a high index of inequality. Socioeconomic disparities characterise its heterogeneous society which, together with other factors, affect health status and patient outcomes; a divide that is particularly obvious in the dual public and private health care system [4].

Breast cancer is a heterogeneous disease and is best treated by a multidisciplinary specialist team. Case volume and expertise clearly affect outcomes and women treated by specialist surgeons are more likely to receive guideline-adherent therapy with better oncological outcomes [[5], [6], [7]]. Despite these clear benefits and increasing breast cancer incidence, there are very few dedicated surgical breast units in the South African public sector that offer standardised care, and these are often burdened with high patient volumes and inadequate resource allocation.

Specific breast cancer QIs have been developed and applied in various HICs [[8], [9], [10]] with China being the only LMIC to have published dedicated QIs to date [11]. Implementation of QIs is feasible in various settings and tends to improve quality over time [[12], [13]]. The process of audit facilitates awareness and adherence to guidelines [14]. QIs are not always readily transferable and need to be adjusted to each setting to allow for variations in clinical practice and health care systems [15].

Non-communicable diseases including cancer were recently declared a priority by the South African National Department of Health and National Breast Cancer Policy Guidelines were developed and published. However, they remain to be implemented [16]. The onus therefore still remains with clinicians to find solutions to standardise and improve breast cancer care in South Africa. This study aimed to develop a regional set of QIs for the diagnosis and surgery of breast cancer in South Africa, through a clinician-driven process using a modified Delphi method.

Materials and Methods

Systematic review

The literature was reviewed on databases including PubMed, Springer, and MEDLINE, as well as QIs published by various international institutes and societies including the American Society of Breast Surgeons (ASBrS), American Society of Clinical Oncology/National Comprehensive Cancer Network (ASCO/NCCN), American Society of Clinical Oncology/National Initiative for Cancer Care Quality (ASCO/NICCQ), Commission on Cancer of the American College of Surgeons (CoC/ACS), European Society of Breast Cancer Specialists (EUSOMA), German Cancer Society (DKG), National Breast Cancer Organisation Netherlands (NABON), National Institute for Health and Care Excellence (NICE), National Quality Forum (NQF) and Scottish Cancer Taskforce. Search terms included “quality indicator”, “quality measures”, “quality parameters”, “breast cancer”, “breast surgery” and “breast pathology”. National and international clinical guidelines and breast centre accreditation criteria, such as EUSOMA criteria and National Accreditation Program for Breast Centres (NAPBC) were considered [[17], [18]].

A national expert panel

A panel of experts was selected according to: (i) provincial representativeness, (ii) subspecialties and expertise in breast cancer diagnosis and treatment, (iii) knowledge of the local health care system, and (iv) clinical and academic seniority. The final panel consisted of 17 experts: ten surgeons, two medical oncologists, two radiation oncologists, two pathologists, and one radiologist (Fig. 1). Each participant was consented and received a study participant number and an electronic link to each Delphi rating round by email. All ratings were anonymous with controlled feedback following each rating round. In October 2018, a face-to-face meeting at the Breast Interest Group of Southern Africa (BIGOSA) conference in Durban, was attended by all members of the expert panel. The potential QIs, their definitions, and the process of selecting the QIs were presented, discussed, and accepted as valid and further locally relevant QIs were invited for inclusion.

Delphi rounds

The Delphi process assumes that group judgements are more valid than individual judgements and is a widely recognized method to reach expert consensus in a structured manner. Key principles include statistical group response and iteration with controlled feedback. The strengths of the Delphi process are the effective use of a committee and the anonymity of equally-weighted responses which reduces the risk of opinion shifts and peer pressure. Limitations of the Delphi process include the lack of clear methodological guidelines and limited reliability and validity [19]. The Nominal Group Technique (NGT) was considered as an alternative which is based on structured meetings with facilitated discussion but the Delphi process was selected for its anonymity of ratings. However, the Delphi process was modified to include a face-to-face meeting, essentially merging one component from the NGT to allow for critical discussion while preserving anonymous rating as the key strength of the Delphi process. The inclusion of a meeting is a commonly used modification in Delphi projects for healthcare quality indicators [20].

Three rounds took place whereby each QI was rated according to: a) importance to measure (relevance and priority), b) scientific acceptability (reliability and validity), c) appropriateness to our local setting (feasibility). Scoring ranged from 1 (no agreement) to 5 (strong agreement). Inclusion thresholds were set a priori at mean ratings ≥4 with a coefficient variation of ≥25%. The first Delphi rating round took place during the face-to-face meeting by anonymous electronic rating. The subsequent two rating rounds were conducted electronically by emailed invitation following the meeting. Study participant numbers were not identifiable to any of the authors. Study data were collected and managed using the REDCap electronic data capture tool and processed by an independent analyst [16].

Level of evidence

Lastly, the respective level of evidence (LoE) for each indicator was determined to allow for practical prioritisation of QIs. Levels of evidence were adopted from previously published guidelines or determined by literature review. For ease of application, the LoE was determined by the United States Agency for Healthcare Research and Quality (AHRQ, www.ahrq.org) classification [21].

Level 1 – Requires good-quality randomised controlled trials or systematic reviews

Level 2 – Requires well designed quasi-experimental studies

Level 3 – Requires well designed descriptive studies

Level 4 – Expert judgement

Results

The literature review revealed 920 potential QIs. Those which did not address preoperative diagnosis or surgical management were removed, leaving 129 potential QIs. Duplicate QIs were also removed, and 52 potential QIs remained for expert panel consideration (Fig. 2).

The first rating round excluded three QIs that failed to achieve a consensus of importance (supplementary figure 1). These included “intraoperative use of ultrasound by the surgeon”, “documentation of contralateral breast cancer risk” and “intraoperative pathology assessment of sentinel lymph nodes”. “Use of preoperative magnetic resonance imaging (MRI)” and “appropriate axillary surgery in ductal carcinoma in situ (DCIS)” did not reach consensus in the initial rating round but were rescued after discussion and revote. All 17 experts were present and completed the rating.

The second rating round excluded one indicator on the “use of preoperative MRI” due to a coefficient variation of >25% (supplementary figure 2); 13 expert responses were received, of which 12 were complete and used for analysis. One indicator was deemed not feasible in the third rating round with a coefficient variation of >25% (“appropriate clinical assessment of the axilla with sonar and fine needle aspiration”). The third round had 11 complete responses (supplementary figure 3). There was no dissension in round two and three and no revote was required.

47 QIs had expert consensus following the Delphi process. Three indicators showed similarity including “single operation rate for invasive breast cancer” and “single operation rate for DCIS” compared to “reoperation rate”, as well as “pathology assessment of lymph nodes” which was included in “complete histopathological characterisation” and the indicators with higher overall panel ratings were retained. A further 22 parameters were merged into eight grouped QIs which resulted in a final set of 30 QIs. Of these, eight QIs had strong LoEs (level 1 or 2). Two additional indicators with lower LoE (unit and surgeon case volumes and multidisciplinary discussion) were regarded as critical for quality improvement and were incorporated into the mandatory set (Tables 1 and 2). A further eight QIs were regarded as standard of clinical care but had lower LoE (level 3 or 4). These were deemed recommended (Table 3). A further twelve QIs were not regarded as clinical standard of care with lower LoE and should be regarded as optional non-standard QIs (Table 4).

Discussion

The first set of quality indicators for the diagnosis and surgical management of breast cancer in South Africa was developed.

Consensus was strong amongst the panel, which affirmed the foundation work and preselection of potential QIs. Nevertheless, such a large number of QIs is not practical for clinical application and a three-tiered set is therefore proposed with mandatory, recommended and optional QIs based on LoE and clinical priority.

Evidential strength and clinical priority

Table 1 shows the mandatory QIs and their definitions: eight have high LoE (I or II) and two have lower LoE (III or IV) which we regard as clinically essential for quality improvement. Complete histopathological evaluation of tumours is critical for appropriate treatment planning. An incomplete evaluation can lead to repeated biopsies, unnecessary surgery and other treatments, all of which carry potential morbidity and mortality for the patient [22]. The standard evaluation should include the histological type, tumour grading, oestrogen receptor (ER), progesterone receptor (PR), human epidermal growth factor receptor-2 (HER-2) and Ki-67 proliferation marker on the preoperative biopsy. In addition to these parameters, the surgical specimen requires pathological staging, with the size of the invasive component and nodal assessment, lymphovascular invasion and distance to the nearest margin.

Two trials on breast-conserving surgery (BCS) in conjunction with radiation therapy (RT) were published in the early 80’s and demonstrated equivalent survival rates despite increased local recurrence rate (LRR) after BCS and RT [[23], [24]]. 20-year follow-up of these trials confirmed the equivalent survival and paved the way for BCS as the preferred treatment option for early-stage breast cancer [[25], [26]]. Newer data shows lower LRRs and suggests that it has become a rare event with contemporary treatment and is probably rather a reflection of tumour biology than the extent of surgery [27].

Reoperations should generally be kept to a minimum as they increase morbidity and health care expenses. However, they are significantly higher with breast cancer surgery than in other surgical fields. This is due to the increased burden of reoperations for positive margins after BCS and completion axillary lymph node dissection (ALND) after a positive sentinel lymph node biopsy (SLNB), particularly after mastectomy [[28], [29], [30]]. One of the confounding factors for increased reoperations has been the variable definition of a positive margin. An adequate margin for invasive breast cancer has now been defined as “no ink on tumour” for invasive disease [31,32] and 2 mm for non-invasive disease [33] which should reduce reoperation rates in future, especially in specialised centres. The rate of BCS, reoperations, and positive margins are a sound reflection of the quality of surgery and are influenced by multiple factors such as quality of preoperative assessment, tumour localisation, use of neoadjuvant systemic therapy, surgical expertise and oncoplastic skill.

Similarly to the shift towards BCS within the breast, there has also been a major shift towards conservative approaches in the axilla and appropriate use of axillary surgery as well as the number of nodes excised should be measured on a mandatory basis. SLNB allows for accurate assessment of the axilla with lesser morbidity compared to ALND, which is inappropriate in clinically node-negative patients with invasive breast cancer or patients with non-invasive breast cancer [[34], [35]]. Morbidity, especially lymphoedema, is related to the number of nodes excised during axillary surgery and should not exceed five for a SLNB; but should also not be less than ten for an ALND in patients undergoing primary surgery [[36], [37]].

Breast cancer treatment is multidisciplinary and the modern surgeon has to understand oncological principles and neoadjuvant and adjuvant therapies. Radiotherapy (RT) reduces local recurrence risk and improves progression-free survival following BCS. A postoperative referral is standard of care with the exception of select patient subgroups [38]. Equally, inflammatory or other non-operable locally advanced breast cancer (LABC) should always be treated with neoadjuvant systemic therapy [39]. Mandatory QIs, therefore, include the rate of RT post BCS and rate of neoadjuvant therapy for non-operable LABC.

Although case volumes and MDT discussions do not have high LoE, these two indicators are deemed important to improve the quality of care in South Africa. Critical case volumes for units and surgeons are well described and a breast unit should treat at least 150 newly diagnosed patients per annum. A surgeon should personally operate on no less than 50 cases per annum [40]. Patients treated by specialised surgeons and units are more likely to have BCS, undergo sentinel lymph node biopsy and are more likely to receive appropriate adjuvant therapies resulting in better outcomes in terms of local recurrence and survival. In addition, treatment is more cost-effective and litigation decreases with treatment in specialised centres adhering to benchmarking and quality control [[41], [42], [43], [44], [45], [46], [47], [48], [49]]. This obviously needs to be seen in the context of accessibility which is extremely varied in South Africa. Ideally, further breast units should be established in areas of need with training of subspecialists and support from existing units in a hub and spoke model. Nevertheless, decentralisation with care by general surgeons is critical in very remote areas, to avoid further barriers to care. MDTs lead to improved, evidence-based clinical decision-making, quality of treatment, provide guidance in non-standard treatment decisions and offer the opportunity to establish a complete treatment plan for each patient [50]. Ideally, the patient should be discussed at diagnosis before initiation of any therapy, postoperatively and at any point of disease progression.

The second group of recommended QIs are listed in table 3 and should be regarded as standard of care in any modern practice. Prior to any breast cancer surgery, there should be pathological confirmation of diagnosis, appropriate clinical examination and breast imaging with bilateral mammography and ultrasound. The routine use of breast imaging reporting and data system (BIRADS) should be applied to imaging reports, all breast specimens should be orientated, surgical clips should always be placed after BCS to aid RT and a tissue marker clip should be placed for patients who are planned for BCS after neoadjuvant systemic therapy. Clinical and pathological staging, locoregional recurrence, progression-free survival and overall survival should all be part of standard follow-up documentation.

The third group of QIs are optional and listed in table 4. The panel deemed them valuable, but they do not have a high LoE and were not regarded as standard of care.

Five QIs were excluded by the panel. “Intraoperative use of ultrasound by the surgeon”, “documentation of contralateral breast cancer risk” and “intraoperative pathology assessment of sentinel lymph nodes” were deemed to have low relevance or clinical priority in our setting. Intraoperative ultrasound, although widely used in European breast centres, is rarely available in South Africa. The intraoperative evaluation of lymph nodes is often inaccurate and controversial, with the current omission of ALND in patients with micrometastases and selected patients with macrometastases undergoing BCS and RT [51]. “Use of preoperative MRI” and “appropriate clinical assessment of the axilla with sonar and fine needle aspiration” are applied in HICs [9] but were excluded due to concerns over reliability and validity, as well as feasibility in our constrained health care system.

Implications and implementation

This set of QIs has been endorsed by the Breast Interest Group of Southern Africa (BIGOSA). Although implementation within surgical practices will be voluntary, BIGOSA will circulate the mandatory set with a strong recommendation that they should be applied and regularly audited by any practice performing surgery on breast cancer patients (Table 1). Rapid implementation can be expected by established breast units and high-volume surgical practices, but these are still very scarce in South Africa. Many breast surgeries are performed by general surgeons and the greatest improvement of quality is anticipated after application of the QI set in the non-specialised setting. The QI set will therefore also be published locally as BIGOSA clinical guidelines and audit results of participating units will be presented at general surgical conferences in order to reach a broader surgical community. However, clinical regulation is historically difficult in South Africa with poorly resourced clinical governance, a divided health care system and a very high degree of autonomy given to the individual practitioner. At present, there is no breast surgical clinical regulation, no directed audit and no accreditation processes for units or MDT meetings. This project therefore should be seen as a collective starting point to initiate and direct quality improvements in breast cancer care. It will be the first attempt to measure current regional quality and offer a gauge for practices with more standardised treatment over time.

The critical ten indicators contain core data and their application appears feasible even in lower resourced settings. Even though the indicators with high evidential strength are likely to have the biggest impact, this remains to be proven in South Africa. Evidence-based medicine has limitations and “high level of evidence studies” with randomisation is often not feasible. Therefore, clinical experience and reasoning may have to be applied; two QIs with lower evidence were incorporated into the mandatory group because of their overwhelming clinical priority. Although the quality indicators were described with the classic division into process, structural or outcome markers, it is important to note that many indicators are not solely dependent on the quality of a surgical practice but interlinked with various patient and system factors. In a complex health care scenario such as the multidisciplinary treatment of breast cancer within a resource-constrained health care system, as in South Africa, shortcomings of other factors such as access to care and screening or the availability of adjuvant therapies will invariably have a more substantial impact than in well-resourced health care systems. This will for instance decrease the achievable rate of BCS which should be regarded as an outcome rather than a pure process metric. It is for this reason, that no benchmarks were allocated to the indicators in this project; benchmarks from HICs are likely unachievable and reasonable benchmarks within a resource-constrained system will need to be determined through future application of the QIs by established local breast centres.

It is likely that this QI set will evolve over time to further accommodate to our local needs. Regional variations may exist, for instance time delays to treatment do not show strong evidence in HICs but decreased overall and disease-specific survival have been described [52,53]. In the case of much more extensive delays in constrained health care systems, these effects may be amplified.

There are several pitfalls anticipated in the implementation of this QI set. Firstly, the most obvious is that a set with too many parameters could overwhelm and discourage use. Even though the indicators in the recommended group are standard of care and practice, most should be already routinely applied and measurements are less likely to yield major effects. The mandatory set was therefore condensed to only include essential QIs with the most substantial anticipated impact. Secondly, data may not be immediately available in all practices and will require incremental, prospective data collection. Differences in resources for additional data management are expected between academic and non-academic units, as well as provincial and private practices. Thirdly, any practice change will initially increase workload and evoke resistance with implementation varying according to clinician knowledge and attitude. Nevertheless, successful implementation over time will not only improve quality of care but also streamline processes, close clinical gaps, strengthen health care systems and improve practice workflow.

There are several other ways to improve quality of care besides measuring QIs. Access to rapid, high-quality diagnosis and multidisciplinary high-quality units is needed as well as advocacy, policy change, innovation, as well as strengthening of local research and training of specialists and subspecialists. Health care professionals need to overcome their resistance to change and innovation, whether this is due to adherence to usual routines, or to protect personal interests. The most systematic approach is the establishment and accreditation of specialised breast centres. Although validated in HICs, some of the established EUSOMA or NAPBC criteria may not be feasible in lower resourced settings. These include routine pathology review of biopsies performed outside of the respective breast centre or breast imaging accreditation criteria that do not exist in South Africa. This initial effort for quality improvement will hopefully form the basis of establishing reasonably adjusted local accreditation criteria in future. Overall, there are promising developments for breast cancer care in South Africa: breast cancer policy and guidelines were issued by the National Department of Health, BIGOSA is gaining traction, and applications for the implementation of a breast and endocrine surgery subspecialty are in process.

Future research

In a future study, the entire QI set will be applied to the surgical breast units participating in the South African Breast Cancer and HIV Outcomes (SABCHO) study [54] to allow for an initial audit, validation and reasonable benchmarking within our resource-constrained health care system. Furthermore, we will attempt to evaluate if individual QI non-adherence results in worse intermediate patient outcomes. These will include potential locally relevant factors with lower evidence, such as timeliness to care.

Limitations

This study has several limitations in keeping with the limitations of a Delphi process. In addition, the modification to include a face-to-face meeting facilitated essential discussion but may have compromised the methodology of a pure Delphi process despite keeping all ratings anonymous. Furthermore, results will always be bound by the quality and knowledge base of the panel. Our study is limited to diagnostics and surgical therapy and does not include all aspects of multidisciplinary care. However, a broad panel including various subspecialties represented the interlinked multidisciplinary approach during every step of a contemporary treatment pathway. In addition, the surgical part of the panel was diverse, representing all but one established breast unit within the South African public sector, as well as surgeons from more remote areas and high-volume surgeons from the private sector. In South Africa, most patients will present with symptomatic disease and will be seen in surgical clinics and practices first. It is crucial to improve the quality of the first point of contact as well as surgical therapy which remains a critical component in the care of breast cancer.

Conclusions

A three-tiered set of QIs was developed based on locally obtained consensus to improve the quality of surgical breast cancer care. Application of the ten critical QIs should be mandatory for all practices treating breast cancer patients. The proposed QIs await implementation but are a pragmatic first step towards improved quality of breast cancer care.

Declaration of competing interest

None.

Table 1

Top ten mandatory quality indicators.

#	Top ten critical quality indicators		Definition	Numerator	Denominator
1	Complete histopathological characterisation:	a) invasive breast cancer	Proportion of cases for which the following prognostic/predictive parameters have been recorded: Histological type; Grading; ER; PR; HER-2; Proliferation index (Ki67).In addition to the above parameters, the following parameters must be recorded after surgery: Pathological stage; Size in mm for the invasive component; Lymphovascular invasion; Distance to nearest radial margin.	Invasive breast cancer patients for which all prognostic/predictive parameters have been recorded.	All invasive breast cancer patients.
		b) non-invasive breast cancer	Proportion of cases for which the following prognostic/predictive parameters have been recorded: Grading; Dominant histological pattern; Size in mm; Distance to nearest radial margin; ER.	Non-invasive breast cancer patients for which all prognostic/predictive parameters have been recorded.	All non-invasive breast cancer patients.
2	Breast conserving surgery (BCS) rate		Proportion of patients undergoing BCS among all patients who receive surgery for the primary breast cancer.	Patients with breast cancer who undergo BCS.	All patients with breast cancer who undergo surgery for the breast primary.
3	Rate of positive margins after breast BCS		Proportion of patients with positive margins after BCS among all patients who underwent BCS.	Patients with positive margins after BCS.	All patients who undergo BCS.
4	Reoperation rate		Proportion of patients with breast cancer who required a reoperation for the breast primary within six months of the initial surgery.	Patients who required a reoperation for the primary tumour within six months of initial surgery.	All patients who underwent surgery for the primary tumour.
5	Appropriate axillary surgery:	a) rate of surgery in invasive breast cancer	Proportion of invasive breast cancer patients who underwent axillary surgery.	Patients with invasive breast cancer who underwent axillary surgery.	All patients with invasive breast cancer.
		b) sentinel node biopsy (SLNB) only in clinically node-negative disease	Proportion of patients with invasive cancer and clinically negative axilla who underwent SLNB only (excluding patients who received primary systemic therapy).	Patients with invasive cancer and clinically negative axilla who underwent sentinel lymph node biopsy only.	All patients with invasive cancer and clinically negative axilla who underwent axillary surgery
		c) no axillary lymph node dissection (ALND) in patients with ductal carcinoma in-situ (DCIS) only	Proportion of patients with DCIS only undergo ALND.	Patients with DCIS only who undergo axillary lymph node dissection.	All patients with DCIS.
6	Number of nodes excised during axillary surgery:	a) ALND ≥10 nodes	Proportion of breast cancer patients undergoing ALND who have at least 10 nodes examined (excluding patients who received primary systemic therapy).	Patients undergoing ALND with at least 10 nodes examined.	All patients who underwent ALND.
		b) SLNB ≤5 nodes	Proportion of patients with invasive cancer who underwent SLNB with no more than 5 nodes excised.	Patients with invasive cancer who underwent SLNB with no more than 5 nodes excised.	All patients with invasive cancer who underwent SLNB.
7	Receipt of radiotherapy after BCS		Proportion of invasive breast cancer patients who received radiotherapy after BCS.	Patients who received radiotherapy after BCS.	All patients BCS for invasive breast cancer.
8	Appropriate treatment sequencing in inoperable locally advanced breast cancer (LABC)		Proportion of patients with inflammatory or non-operable LABC who undergo primary systemic treatment before surgery.	Patients with inflammatory or non-operable LABC who undergo primary systemic treatment before surgery.	All patients undergoing surgical treatment for inflammatory or non-operable LABC.
9	Case Volume:	a) breast unit ≥150 newly diagnosed cases per annum	Each unit needs to see a set minimum number of 150 patients with newly diagnosed breast cancer per annum.	N/A.	N/A.
		b) surgeon volume ≥50 breast cancer cases per annum	Each surgeon needs to perform a set minimum number of surgeries on patients with newly diagnosed breast cancer per annum. In units which train surgeons this would include supervised (scrubbed in) surgeries.	N/A.	N/A.
10	Multidisciplinary team discussion		Proportion of patients with breast cancer who are discussed in a MDT meeting (ideally pre- and postoperatively).	Patients with breast cancer who are discussed in the MDT meeting.	All patients with breast cancer.

Table 2

Mandatory quality indicators with evidence levels and results from the Delphi rounds, SD = standard deviation, CV = coefficient variation.

#	Quality Indicator Title		Level of Evidence	Round 1: Mean Score(SD); CV	Round 2: Mean Score(SD); CV	Round 3: Mean Score(SD); CV	Mean Sum Score (SD)
1	Complete histopathological characterisation:	a) invasive breast cancer	2.0	5.0 (0.0); 0.00	5.0 (0.0); 0.00	5.0 (0.0); 0.00	15.0 (0.0)
		b) non-invasive breast cancer	2.0	5.0 (0.5); 0.10	5.0 (0.5); 0.10	5.0 (0.5); 0.10	15.0 (1.5)
2	Breast conserving surgery (BCS) rate		1.0	4.0 (1.0); 0.25	5.0 (0.5); 0.10	4.5 (0.5); 0.11	13.5 (2.0)
3	Rate of positive margins after breast conserving surgery (BCS)		2.0	4.5 (1.0); 0.22	5.0 (0.5); 0.10	4.5 (0.5); 0.11	14.0 (2.0)
4	Reoperation rate		2.0	4.5 (1.0); 0.22	5.0 (0.5); 0.10	4.5 (1.0); 0.22	14.0 (2.5)
5	Appropriate axillary surgery:	a) rate of axillary surgery in invasive breast cancer	1.0	5.0 (1.0); 0.20	5.0 (0.5); 0.10	4.5 (0.5); 0.11	14.5 (2.0)
		b) sentinel node biopsy only in clinically node-negative disease	1.0	5.0 (0.5); 0.10	5.0 (0.5); 0.10	5.0 (0.5); 0.10	15.0 (1.5)
		c) no ALND in patients with DCIS only	2.0	4.0 (1.5); 0.38	4.5 (0.5); 0.11	4.5 (0.5); 0.11	13.0 (2.5)
6	Number of nodes excised during axillary surgery:	a) lymph node dissection ≥10 nodes	2.0	5.0 (1.0); 0.20	5.0 (0.5); 0.10	5.0 (0.5); 0.10	15.0 (2.0)
		b) sentinel node biopsy ≤5 nodes	1.0	4.0 (1.0); 0.25	5.0 (0.5); 0.10	4.5 (0.5); 0.11	13.5 (2.0)
7	Receipt of radiotherapy after breast conserving surgery		1.0	5.0 (0.5); 0.10	5.0 (0.5); 0.10	4.5 (0.5); 0.11	14.5 (1.5)
8	Appropriate treatment sequencing in inoperable LABC		2.0	5.0 (0.5); 0.10	5.0 (0.5); 0.10	4.5 (0.5); 0.11	14.5 (1.5)
9	Case Volume:	a) breast unit ≥150 newly diagnosed cases per annum	3.0	4.5 (1.0); 0.22	5.0 (0.5); 0.10	4.5 (0.5); 0.11	14.0 (2.0)
		b) surgeon volume ≥50 breast cancer cases per annum	3.0	4.5 (1.0); 0.22	4.5 (0.5); 0.11	4.5 (0.5); 0.11	13.5 (2.0)
10	Multidisciplinary team discussion		3.0	5.0 (0.5); 0.10	5.0 (0.5); 0.10	4.0 (1.0); 0.25	14.0 (2.0)

Table 3

Recommended standard of care quality indicators with evidence levels and results from the Delphi rounds.

#	Quality Indicator Title		Level of Evidence	Round 1: Mean Score(SD); CV	Round 2: Mean Score(SD); CV	Round 3: Mean Score(SD); CV	Mean Sum Score (SD)
1	Definitive diagnosis prior to treatment		3.0	5.0 (0.0); 0.00	5.0 (0.5); 0.10	5.0 (0.5); 0.10	15.0 (1.0)
2	Appropriate clinical assessment and breast imaging		3.0	5.0 (0.0); 0.00	5.0 (0.5); 0.10	4.5 (0.5); 0.11	14.5 (1.0)
3	Use of BIRADS classification in imaging reports		3.0	4.5 (0.5); 0.11	5.0 (0.5); 0.10	4.5 (0.5); 0.11	14.0 (1.5)
4	Specimen orientation		4.0	5.0 (0.5); 0.10	5.0 (0.5); 0.10	4.5 (0.5); 0.11	14.5 (1.5)
5	Use of marker clips to aid radiation		3.0	5.0 (0.5); 0.10	5.0 (0.5); 0.10	4.5 (0.5); 0.11	14.5 (1.5)
6	Use of imaging marker clip in patients for BCS undergoing primary systemic therapy		3.0	4.5 (0.5); 0.11	5.0 (0.5); 0.10	4.0 (1.0); 0.25	13.5 (2.0)
7	Documentation of staging:	a) preoperative (clinical)	4.0	4.5 (0.5); 0.11	5.0 (0.5); 0.10	5.0 (0.5); 0.10	14.5 (1.5)
		b) postoperative (pathologic) breast cancer staging	4.0	5.0 (0.5); 0.10	5.0 (0.5); 0.10	5.0 (0.5); 0.10	15.0 (1.5)
8	Documentation of outcomes:	a) locoregional recurrence rate	N/A	4.5 (1.0); 0.22	5.0 (0.5); 0.10	4.5 (0.5); 0.11	14.0 (2.0)
		b) progression-free survival	N/A	4.5 (1.0); 0.22	5.0 (0.5); 0.10	4.0 (1.0); 0.25	13.5 (2.5)
		c) overall survival	N/A	4.5 (1.0); 0.22	4.5 (0.5); 0.11	4.5 (1.0); 0.22	13.5 (2.5)

SD = standard deviation, CV = coefficient variation.

Table 4

Optional non-standard of care quality indicators with evidence levels and results from the Delphi rounds.

#	Quality Indicator Title		Level of Evidence	Round 1: Mean Score(SD); CV	Round 2: Mean Score(SD); CV	Round 3: Mean Score(SD); CV	Mean Sum Score (SD)
1	Timeliness of care:	a) first diagnostic examination to initial treatment	4.0	5.0 (1.0); 0.20	5.0 (0.5); 0.10	4.5 (1.0); 0.22	14.5 (2.5)
		b) needle biopsy to pathology report	4.0	4.5 (0.5); 0.11	5.0 (0.5); 0.10	5.0 (0.5); 0.10	14.5 (1.5)
		c) needle biopsy pathology report to treatment	4.0	4.5 (1.0); 0.22	5.0 (0.5); 0.10	4.5 (0.5); 0.11	14.0 (2.0)
		d) surgery to pathology report	4.0	4.0 (1.0); 0.25	5.0 (0.5); 0.10	5.0 (0.5); 0.10	14.0 (2.0)
		e) surgery to adjuvant chemotherapy	3.0	5.0 (0.5); 0.10	4.5 (1.0); 0.22	5.0 (0.5); 0.10	14.5 (2.0)
		f) last surgery or chemotherapy to adjuvant radiation	3.0	4.5 (0.5); 0.11	5.0 (0.5); 0.10	5.0 (0.5); 0.10	14.5 (1.5)
2	Postoperative complications:	a) Readmission rate	3.0	4.5 (1.0); 0.22	5.0 (0.5); 0.10	4.5 (1.0); 0.22	14.0 (2.5)
		b) Surgical site infection rate	3.0	4.5 (1.0); 0.22	4.5 (0.5); 0.11	4.5 (0.5); 0.11	13.5 (2.0)
3	Documentation of eligibility for BCS		4.0	4.5 (0.5); 0.11	5.0 (0.5); 0.10	4.5 (0.5); 0.11	14.0 (1.5)
4	Oncoplastic procedure rate		4.0	4.5 (0.5); 0.11	4.5 (0.5); 0.11	4.5 (0.5); 0.11	13.5 (1.5)
5	Discussion of surgical options		4.0	4.5 (0.5); 0.11	4.5 (0.5); 0.11	4.5 (0.5); 0.11	13.5 (1.5)
6	Failure of identification of the sentinel node		3.0	4.5 (0.5); 0.11	5.0 (0.5); 0.10	4.5 (0.5); 0.11	14.0 (1.5)
7	Node positivity rate in SLNB		4.0	4.5 (1.0); 0.22	5.0 (0.5); 0.10	4.5 (0.5); 0.11	14.0 (2.0)
8	Tracer use in sentinel lymph node biopsy		4.0	4.0 (1.0); 0.25	4.0 (1.0); 0.25	4.5 (1.0); 0.22	12.5 (3.0)
9	Concurrent SLNB with breast surgery		4.0	4.0 (1.0); 0.25	5.0 (0.5); 0.10	4.5 (0.5); 0.11	13.5 (2.0)
10	Efficient diagnosis (number of visits required to initiation of treatment)		4.0	4.0 (1.0); 0.25	4.5 (0.5); 0.11	4.5 (1.0); 0.22	13.0 (2.5)
11	Lymphoedema rate		4.0	4.5 (1.0); 0.22	4.5 (0.5); 0.11	4.5 (0.5); 0.11	13.5 (2.0)
12	Quality audit rate		4.0	4.0 (1.0); 0.25	4.5 (0.5); 0.11	4.5 (1.0); 0.22	13.0 (2.5)

SD = standard deviation, CV = coefficient variation.