Collection of Bilateral Cleft Lip and Palate Standard Set Variables: Establishing a Baseline.

BACKGROUND: The International Consortium for Healthcare Outcomes Measurement recently published a consensus Standard Set of clinical and patient-centered metrics to measure outcomes for patients with cleft lip and/or palate (CLP). This study aims to evaluate how the Standard Set compares to existing data collected to anticipate the impact that the Standard Set may have on quality and quantity of outcome data.
METHODS: Extraction of the Standard Set data points was attempted retrospectively for all nonsyndromic patients with bilateral cleft lip and/or palate who underwent primary lip and/or palate repair by a single surgeon (JGM) between June 2007 and June 2014.
RESULTS: Bilateral cleft lip repair was performed on 32 patients of which 29 also underwent palate repair. All but one of the baseline demographic and phenotypic variables were available. All perioperative variables were collected, but data quality was heterogeneous. There were no early complications. At 5 years, 29.6% of patients were lost to follow-up; however, a degree of data was available on 11 of the 12 clinical metrics for those remaining. Of patients with Veau IV cleft palate and follow-up at age 5, 1 patient (6.7%) had an oronasal fistula and 1 had velopharyngeal incompetence requiring Furlow palatoplasty (6.7%). No patient-reported data were collected for any time point.
CONCLUSION: Prospective collection of the International Consortium for Healthcare Outcomes Measurement Standard Set will improve consistency of clinical data and add the patient perspective currently lacking in outcome measures collected for patients with bilateral cleft.

INTRODUCTION

The ability to improve quality of care is contingent on the ability to measure it. Patients with cleft lip and palate (CLP) are cared for from infancy to adulthood by many specialties to ameliorate the sequelae of clefting on speech, eating/drinking, appearance, facial growth, hearing, and psychosocial well-being. A number of tools have been developed to measure diverse aspects of cleft care; however, the majority measure outcome exclusively from the clinician perspective. Inadequate emphasis has historically been placed on patient perception of outcome.[1-13] To unify patient- and clinician-reported metrics into a single tool suitable for everyday practice, the International Consortium for Healthcare Outcomes Measurement (ICHOM) assembled and published a CLP “Standard Set” that measures objective clinical and patient-reported outcomes at 7 key time points along the treatment pathway.[14] The Standard Set was developed by consensus between multidisciplinary experts and CLP patient advocates from 8 countries and combines many validated tools and questionnaires already utilized in cleft care.[6,15,16]

Implementing comprehensive outcomes measurement such as that defined in the CLP Standard Set can be daunting because of the volume of data collected and the infrastructure needed to do so. Individuals with bilateral CLP are especially complex; they have the greatest burden of care and thus greatest burden of outcomes data collection.[17-20] As such, bilateral CLP was specifically selected for this study to provide a comprehensive snapshot of data available for all specialties relevant to cleft care. The aim of this study is to evaluate how the CLP Standard Set aligns with data routinely collected at a large cleft center to anticipate the Standard Set’s impact on quality and quantity of outcome data on patients with bilateral cleft. Additionally, this retrospective cohort will provide a benchmark against which to compare results following implementation of prospective Standard Set data collection.

METHODS

A retrospective review was performed of all nonsyndromic patients with bilateral CLP who underwent primary lip and/or palate repair by a single surgeon (JGM) at a high-volume cleft center in the United States between June, 2007, and June, 2014, to ensure an adequate length of follow-up. International patients and those without follow-up were included to capture the true difficulty in both retrospective and prospective benchmarking. Patients who had their initial procedures by another clinician or at another center were excluded. One medically trained researcher systematically searched the electronic medical record to extract each of the clinical outcomes of the Standard Set. Plastic surgery, oral surgery, dental, otolaryngology, audiology, genetics, and social work documentation were reviewed, including clinic notes, operative records, and demographic data. No patient- or parent-reported metrics were collected during this time frame, and thus these elements of the Standard Set were excluded from review.

The study was approved by the institutional review board, protocol number IRB-P00022609.

RESULTS

Between June, 2007, and June, 2014, 32 nonsyndromic patients underwent Mulliken-style synchronous repair of bilateral cleft lip. Of those patients, 29 also underwent 2-flap palatoplasty with vomerine flaps and a modified Sommerlad intravelar veloplasty. Outcome data are presented according to the time points defined in the CLP Standard Set. A summary comparing the variables for collection in the ICHOM Standard Set and those available retrospectively is presented in Table 1.

Table 1.

Table Summarizing the Proportion of ICHOM Standard Set Variables Available from Retrospective Chart Review at Each Time Point

Initial Visit and 3 Months Old

All but one of the baseline and 3 month variables were collected as part of routine documentation, including age at first encounter, birth weight and change in weight percentile at 3 months old, sex, cleft phenotype, syndrome and type, medical comorbidities, ethnicity, language spoken at home, adoption status, insurance status, and distance to treatment facility. Key variables are reported in Table 2. The variable not collected was highest level of parental education, an indicator of socioeconomic status, and burden of care. Retrospectively stratifying phenotype with the required level of granularity proved challenging. Of the 32 patients, 29 (91%) had a CLP and the majority, 25 (78%), had bilateral complete cleft lip and Veau IV palate. Three patients had medical comorbidities. Of these, 2 had cardiac anomalies, 1 had genitourinary anomalies, and 1 had neurological dysfunction (Table 3). No patients were lost to follow-up during this early time frame, and 2 patients transferred care in from other institutions but had not previously undergone surgical repair.

Table 2.

Patient Demographics

Table 3.

Patient Phenotypes

Perioperative (30–45 d)

Following lip (n = 32) and palate (n = 29) repair, all treatment variables including type of operation, date of operation, length of stay, and all postoperative complication variables were documented on all patients but documentation quality was heterogeneous. There were no early postoperative complications, including no unplanned oronasal fistulae, death, unplanned reintubation, wound dehiscence, or readmissions. Two patients who presented at 6 and 15 months of age had an uncomplicated premaxillary osteotomy because of late presentation.

Five Years Old

Of 27 patients who had reached the age of 5 by the time of this study, 8 (29.6%) were lost to follow-up. Two were international patients who transferred care to their country of origin, and 1 moved within the United States and requested local follow-up. Eleven of the 12 clinical metrics were collected on the 19 patients who presented for follow-up. The uniformly missing variable was an objective measurement of the percentage consonants from a standardized speech assessment because the Pittsburgh Weighted Speech Scale was the only objective speech score used at that time. There was heterogeneity in the way data were recorded, and there were data gaps, with all 11 variables only available in 11 of the 19 patients (57.9%). Information on oronasal fistula, velopharyngeal competence, repeat speech surgery, and overjet-assessment was available and presumed accurate on all patients. Data for the 6 dental hygiene variables were available on 18 of the 19 patients (94.7%); however these data showed some surprising findings (few missing teeth recorded in this bilateral cleft cohort) calling into question the accuracy of this data category. Pure-tone audiologic assessment availability was inconsistent. Only 9 patients (47.3%) had the 5-view photographs as recommended.

Of the 15 patients with Veau IV cleft palate who presented for follow-up, 1 (6.7%) demonstrated a small fistula at the junction of the primary and secondary palate. Twelve (80.0%) had velopharyngeal competency as detected by a structured clinical grading system, 2 (13.3%) were marginally competent, and 1 (6.7%) was incompetent (Table 4). The single patient (6.7%) with velopharyngeal incompetence required conversion Furlow palatoplasty.

Table 4.

Results from Objective Assessment of Velopharyngeal Competence and Overjet

Eight Years Old

At age 8, all Standard Set outcome variables are patient or parent reported. Because this was not part of our clinical routine before implementing prospective data collection, no data meeting the Standard Set requirements were available for the 16 patients who reached age 8 by the time of the study.

DISCUSSION

Care of children and adults with CLP is complex. Over the last several decades, multiple longitudinal, prospective registries have demonstrated the importance of regular assessment of outcomes and the need to have uniform measurement tools that all institutions use in the same manner. The most pivotal of these are Eurocleft, ScandCleft, AmeriCleft, and CSAG/CCUK, among others.[10,21-23] These registries have brought new knowledge and spurned improvements in cleft care delivery. An important example is how the CSAG/CCUK studies demonstrated variable outcomes between low- and high-volume cleft centers. This caused the British National Health Service to restructure cleft care to be delivered in a limited number of high-volume centers to optimize outcome.

As we have gained knowledge and made clinical advancements in cleft care, there has been a push toward evaluating outcomes not just from the clinician perspective but from the patient and parent perspective as well. The CLEFT-Q was developed in 2016 to capture patient perception of appearance, quality of life, and physical function and is designed to be culturally appropriate across a range of settings and languages.[6] A large number of additional validated tools exist to assess outcomes applicable but nonspecific to cleft patients such as oral health, nasal breathing, and speech intelligibility.[13,15,16]

Although most multidisciplinary cleft teams engage in internal utilization review for quality-improvement purposes, historically there has been no consensus regarding which clinical or psychosocial outcomes should be formally assessed or how they should be measured. The CLP Standard Set was developed in conjunction with ICHOM, an organization that has spearheaded outcomes assessment and a push toward value-based care for a number of common conditions. Under their guidance, the Standard Set was designed to address several limitations of previous work: (1) being broadly applicable across multiple cultures, languages, and geographical regions; (2) including the majority of CLP diagnoses rather than a single phenotype; (3) allowing for risk stratification based on type of cleft, syndrome/medical comorbidity, sociodemographic factors, and adoption or transfer of care; (4) incorporating patient/parent perspective; and (5) streamlining the multiple tools available toward a minimum set of variable that most institutions can collect as part of routine practice. The integration of a number of already widely accepted patient/parent-reported surveys spanning both clinician-reported and patient/parent-reported assessments into a single tool gives parity to different dimensions of care and defines a common language for communicating CLP outcomes across the multidisciplinary team. Additionally, specifically designed tools create an environment for more accurate data capture to improve on reporting inaccuracies and omissions detected in this study. As this was our institution’s first attempt at comprehensively and prospectively collecting CLP data, we first needed to understand how our current clinical data collection aligns with the Standard Set to best create a strategy for scaling up our internal processes.

By completing this review through the lens of the CLP Standard Set, we identified areas that have historically performed strongly and those needing improvement. Although there was considerable overlap in the data being routinely collected and the Standard Set for demographic, phenotypic, surgical, audiologic, and oral health, the data available retrospectively had heterogeneity and gaps. The data were time intensive to retrieve and lacked clear definition. Moreover, there were areas where we hypothesize that the banality of documenting in the electronic medical record may have led to incorrect data being propagated forward. For example, this phenomenon may explain the fact that only 31% of patients in this series were documented as having missing deciduous teeth, and no patient was reported as missing permanent teeth when the bilateral cleft lip population is expected to lack both lateral incisors. These findings highlight the objective improvements in the quality and quantity of data that would be collected if the Standard Set was systematically and prospectively adopted.

The need for standardized data collection in less common cleft diagnoses such as bilateral cleft is further highlighted by the lack of robust data against which to compare the clinical results presented here. To date, most large, multicenter trials have focused exclusively on unilateral cleft.[10,22,23] Comparison of our data to other institutions’ bilateral cleft outcomes is limited by low number of subjects in each study and heterogeneous reporting methods, particularly for velopharyngeal competence. Reported rates of velopharyngeal incompetence (VPI) in bilateral CLP patients vary from 13.3% to 54%.[19,24-27] The largest study, by Sullivan et al, reports on 84 consecutive patients with Veau IV cleft, of which 23.8% required secondary surgery to correct VPI. Results in our study are somewhat better with a single child (6.7%) requiring a procedure for VPI by age 5 and 2 children (13.3%) with marginal incompetence not requiring an operation. In a similar example of the lack of large-scale bilateral CLP data, systematic reviews of unintended palatal fistulae for all cleft phenotypes puts the average fistula rate at between 4.9% and 8.6%.[28,29] However, the small reviews available for bilateral palatal clefts (19–63 patients) suggest the fistula rate increases to 20–79%[20,30-32] in this phenotype, substantially higher than the 6.7% fistula rate found in this study. The variability in these results is likely influenced by the small patient numbers included in each of these series, including our own. This further supports the need for standardized metrics to allow aggregation of multi-institutional data, especially for less common phenotypes such as bilateral CLP where single center studies will not yield sufficient cohort sizes.

In 2016, after a 2-year collaboration with ICHOM and a number of other large cleft centers to develop the CLP Standard Set, our center began prospective collection of Standard Set data for all cleft patients. To optimize the challenges identified in this retrospective review, a number of unique processes were implemented. A departmental champion along with a multidisciplinary advisory committee advocated for dedicated financial and human resources for the project. Staff across the multiple departments were sensitized to the benefits of comprehensive data collection for this patient population. The necessary software to collect, collate, and report the data has been developed and integrated into our existing electronic medical record to minimize additional burden for clinical staff. Clinical “buy-in” is required to continually test and iterate the software and to complete clinical information. Patient/parent-reported data are being collected at all time points from age 5 onward using a child-friendly iPad program to enhance patient willingness to participate. Thus far, patient-reported surveys have been administered in clinic by our staff both to optimize response rate and to provide a talking point to identify issues to be addressed during the annual team visit. Governance mechanisms to review and act on concerning responses have been put in place. A larger pilot comparing the quality of prospectively collected Standard Set data to retrospective data is planned to evaluate how successfully we have mitigated the challenges identified in this study.

Our long-term goal is that aggregation of international, multi-institutional data allows cleft centers to improve patient outcomes. Benchmarking and comparison between facilities will allow identification of practices that lead to the best outcomes and thus the development of evidence-based guidelines that will push forward standards in cleft care. The enhanced understanding of the patient perspectives of their care and outcomes provided by the ICHOM Standard Set will allow the supply of services to be tailored to patient demands and improve patient-centered outcomes. Implementation of the Standard Set can also improve the quality of patient care by improving the efficiency of care. Directly linking the health care delivered to outcomes can allow integration of these measures into value-based purchasing schemes and better link facility performance with compensation leading to a more efficient system.

One major challenge identified in this retrospective review for which we have not currently implemented a solution is loss of follow-up. In this series, no data were available for almost 30% of patients by the age of 5 years because of transfer of care or other loss to follow-up. It is possible that this may have biased our results, if patients lost to follow-up were those unhappy with their care. Although we do not currently have a solution for this problem, in time, as multiple facilities collect data, patients can continue to be standardly assessed even when they move location. As more facilities begin ICHOM implementation, pooling of international data will allow a sufficiently powered data set to appropriately risk adjust outcome data and answer questions as to the comparative effectiveness of different cleft interventions, something previously precluded by study size and heterogeneity.[14]

CONCLUSIONS

Retrospective outcomes data collection is challenging and results in limited data quality. Prospective collection of the internationally developed CLP Standard Set will improve the quantity and quality of evidence available to drive quality improvement in individuals with bilateral clefts.