Caries Risk Documentation And Prevention: eMeasures For Dental Electronic Health Records.

BACKGROUND: Longitudinal patient level data available in the electronic health record (EHR) allows for the development, implementation, and validations of dental quality measures (eMeasures).
OBJECTIVE: We report the feasibility and validity of implementing two eMeasures. The eMeasures determined the proportion of patients receiving a caries risk assessment (eCRA) and corresponding appropriate risk-based preventative treatments for patients at elevated risk of caries (appropriateness of care [eAoC]) in two academic institutions and one accountable care organization, in the 2019 reporting year.
METHODS: Both eMeasures define the numerator and denominator beginning at the patient level, populations' specifications, and validated the automated queries. For eCRA, patients who completed a comprehensive or periodic oral evaluation formed the denominator, and patients of any age who received a CRA formed the numerator. The eAoC evaluated the proportion of patients at elevated caries risk who received the corresponding appropriate risk-based preventative treatments.
RESULTS: EHR automated queries identified in three sites 269,536 patients who met the inclusion criteria for receiving a CRA. The overall proportion of patients who received a CRA was 94.4% (eCRA). In eAoC, patients at elevated caries risk levels (moderate, high, or extreme) received fluoride preventive treatment ranging from 56 to 93.8%. For patients at high and extreme risk, antimicrobials were prescribed more frequently site 3 (80.6%) than sites 2 (16.7%) and 1 (2.9%).
CONCLUSION: Patient-level data available in the EHRs can be used to implement process-of-care dental eCRA and AoC, eAoC measures identify gaps in clinical practice. EHR-based measures can be useful in improving delivery of evidence-based preventative treatments to reduce risk, prevent tooth decay, and improve oral health. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/).

Background and Significance

Health care is undergoing a transformation to provide better health, better care, lower cost, and an engaged workforce. 1 2 Continuous quality improvement is advocated for an ideal health care system to monitor care. 3 Each year in the United States, over 195,000 active dental practitioners provide care to more than 127 million patients, expending billions annually. 4 Quality measures applied to dentistry provide an understanding of the return in terms of quality and value within the health care system. 5 6

Tooth decay is a disease affecting 60 to 90% of children and up to 100% of adults worldwide. 7 Tooth decay is a complex and multifactorial disease resulting from an imbalance of protective and risk factors in the oral cavity and it remains the most prevalent chronic disease in the United States, affecting 91% of adults ages 20 to 64, with 27% having untreated tooth decay. 8 Healthy People 2030 goals for adults include reducing untreated decay and tooth loss. 9 Caries management has evolved from a traditional restorative, “drill-and-fill,” surgical intervention to a multilevel approach with assessment and determination of caries risk levels divided into low, moderate, high, and extreme. 10 11 Caries risk assessment (CRA) is conducted during a clinical examination using a form inclusive of the information on disease indicators, biological, and protective factors to predict the likelihood of future caries development with a goal to determine a personalized caries management plan based on the risk status. 10 12

In the last 20 years, the dental community has embraced the philosophy of CRA and management, 13 but our understanding to which extent dental institutions, that is, academic, private, and group practices perform CRAs, and deliver appropriate risk-based caries treatments to patients is limited. With structured patient-level data in electronic health records (EHRs), rather than relying on administrative claims data, it is now possible to use clinical quality measures to evaluate our patients' oral health care. A definition of the measure domain process of care is “a health care–related activity performed for, on behalf of, or by a patient,” and “supported by evidence that the clinical process has led to improved outcomes.” 14 The literature reports on the development and implementation of process of care clinical quality measures seeking to improve fluoride applications, oral evaluations for patients with diabetes, 15 tobacco use and cessation, 16 and placement of dental sealants. 17 18 In 2020, the Dental Quality Alliance released technical specifications of a process of care measure to evaluate caries risk documentation using program/plan administrative enrollment and claims data in children aged 21 years and younger, 19 combining primary, mixed, and permanent dentitions. Our study builds on the Dental Quality Alliance measure, utilizing EHR derived data.

Objectives

In this research study, we employed a process of care quality measurement to evaluate caries risk documentation in children and adults in clinical practice, and evaluated if patients at elevated risk received evidence-based caries preventive management care. Our research objectives are mentioned below:

eCRA: develop and validate a CRA measure to determine the percentage of patients (any age) of record who received a CRA within the reporting year.

Appropriateness of care eMeasure: develop and validate a risk-based measure to determine the percentage of patients (any age) who received a CRA within the reporting year and corresponding appropriate risk-based treatment(s) on or after the date of the CRA during the reporting year.

Methods

The development, implementation, and validation of eMeasures eCRA and eMeasure appropriateness of care (eAoC) were performed by our team with expertise in caries risk documentation and management, general dentistry, public health, dental informatics, and dental diagnosis terminology. Using retrospective patient-level data, we implemented the research study in three dental schools and one large dental accountable care organization (ACO). All four institutions use the axiUm EHR platform (Exan Corp, Coquitlam, BC, Canada) and the Standardized Diagnostic Dental terminology (SNODDS). 20 Each participating institution obtained respective Institutional Review Board (IRB) approval. As a first step, a query was developed and validated as described earlier. 15 Query validation was conducted in four dental institutions. One institution (site 4) had less than 25 charts with documentation of caries risk, therefore the participation of site 4 in this research study was limited to quality measure validation.

Dental Quality Measures eMeasure Carries Risk Assessment and eMeasure Appropriateness of Care: Development, Implementation, and Validation

Fig. 1 includes a flowchart with the eMeasures description, the numerator, and denominator details.

Fig. 1

Flow chart with description of caries risk assessment and appropriateness of care eMeasures (eCRA and eAoC, respectively).

eCRA—numerator: we defined the numerator population based on the latest documented caries risk levels in the reporting calendar year of 2019 with three approaches following a hierarchy of events, A1, A2, and A3.

A1. Caries risk diagnosis SNODDS—treatment valid pairs: a caries risk level–related SNODDS 20 diagnosis term (extreme, high, moderate, or low) associated with a planned, in process, or completed valid procedure code, recorded in the EHR. For example, caries risk high SNODDS term (596479), paired with topical fluoride varnish treatment code (D1206), is a valid pair because of a topical fluoride varnish application is an evidence-based preventive treatment indicated for a high-risk caries patient. Supplementary Appendix A (available in the online version) details the complete list of caries risk diagnosis-treatment valid pairs.

A2. CRA procedure codes: we followed the Dental Quality Alliance released specifications for practice-level measures for dental clinics to evaluate caries risk documentation using procedure codes 21 ; D0601 (caries risk low), D0602 (caries risk moderate), or D0603 (caries risk high) recorded in the patient chart with a “completed” status.

A3. CRA forms: while the Caries Management by Risk Assessment (CAMBRA) form 10 is in use across all participating institutions, we identified the various CRA forms used at each institution, and included those in the study to capture all risk assessment evaluations.

Patient-level caries risk documentation can happen more than once within the reporting year and the CRA may be recorded with any or all of the methods. To address this scenario, we utilized a three-step hierarchy of events for the classification of caries risk levels: (A1) caries risk diagnosis SNODDS–treatment valid pairs; (A2) CRA procedure codes; and (A3) CRA forms. We calculated the measure rate as the ratio of the numerator divided by the denominator.

eAoC is a measure of the caries preventive management plan that is risk-based with patients at elevated risk (moderate, high, or extreme) receiving evidence-based therapies. The consensus recommendation is prescription fluoride for patients at elevated risk levels, and antimicrobials treatments for high and extreme caries risk patients. 10 All participating institutions follow the evidence-based CAMBRA protocol for caries management and prevention which includes prescription 1.1% fluoride toothpaste, professionally applied fluoride (D1206 or D1208), and antimicrobials, or prescription chlorhexidine gluconate 0.12%. We also included the application of interim caries arresting medication (D1354) as fluoride therapy. We excluded low-caries risk patients from the numerator because they do not have active disease and the recommendation is to continue oral hygiene with over the counter toothpaste and regular dental visits. 10

Development of Automated Electronic Health Record Query

A structured query language (SQL) script was developed to enable standardized extraction of EHR data across participating institutions. During this test, we confirmed that the information extracted by the query from patient charts was 100% accurate and developed the final script for implementation at each site.

Validating the Automated Query

After implementing the script, we validated the query performance with manual chart reviews following our previously described process. 22

Specifically the below-mentioned steps:

Step 1: sample size calculations: the queried sample was the population, the respective proportions (measure scores) by site, and we used standard two-tailed z -score of 1.96 and a margin for error of 0.05 ( d  = 0.05). 23 The resulting sample sizes for evaluating CRA (n 1 ) and risk-based treatments (n 2 ) for each site are: site 1, n 1  = 314 and n 2 = 362; site 2, n 1  = 326 and n 2  = 317; and site 3 n 1  = 347and n 2  = 385.

Step 2: manual chart reviews: at each site, two independent reviewers manually reviewed 50 charts; Cohen's kappa 24 values for interrater reliability were κ  = 1.0 for all sites, allowing for a single reviewer to complete the remaining chart reviews at each site.

Step 3: validity analysis: to evaluate the concordance between the automated query and manual chart reviews, we calculated sensitivity, specificity, and positive and negative predictive values. The overall performance was above 90% for eCRA and eAoC ( Table 1 ).

Table 1

Overall validity analysis of eCRA and eAoC measures

Query vs. manual	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
eCRA	90.2	100	100	90
eAoC
Fluorides	90.4	100	100	90.2
Antimicrobials	100	100	100	100
Sensitivity	When the manual review confirmed that the automated query correctly identified the patient in the numerator
Specificity	When the manual review confirmed that the patient, as identified by the automated query was not a part of the numerator
PPV	When the automated query confirmed that the patient was part of the numerator as identified by the manual reviewer
NPV	When the automated query confirmed that the patient was only a part of the denominator as identified by the manual reviewer

Abbreviations: eAoC, eMeasures appropriateness of care; eCRA, eMeasures carries risk assessment; NPV, negative predictive value; PPV, positive predictive value;

Step 4: measure scores: calculated as the ratio of the numerator divided by the denominator for automated and manual queries. We assessed differences between the automated and manual queries by using a parametric, one sample z -test for proportions. For analysis purposes, we combined high and extreme caries risk patients, because few patients classified as extreme caries risk: 1, 91, and 590 for sites 1, 2, and 3, respectively; and the dental procedure codes do not differentiate between high and extreme caries risk.

Results

eCRA: across all three sites, 269,536 patients met the inclusion criteria for the denominator. The proportion of patients who received a CRA (numerator) was 94.43% ( n  = 254,510) across all sites. The number of female patients in the numerator and denominator was higher compared with males, with 95% of 150,273 female patients receiving a CRA ( Table 2 ). Using a Chi-squared test for homogeneity of proportions, there were statistically significant variations in the measure scores by gender ( χ 2  = 210.01, p  < 0.0001). We stratified our data in four age groups as follows: 0 to 5 (primary dentition), 6 to 12 (mixed dentition), 13 to 18 (permanent dentition), and 18+ (permanent dentition including wisdom teeth, 3rd molars; Table 2 ). Across all sites, the majority of our denominator patient population were adults aged 18 years and older with 96.5% of 269,536 receiving a CRA and the lowest were children aged 0 to 5 years with 70.1% with a CRA. The measure scores were statistically significant by age group ( χ 2  = 19,838, p  < 0.0001). Across all sites, the patient population was racial and ethnically diverse. Whites corresponded to 57.2% of the total denominator population. The measure scores were statistically significant by race with the highest proportion of Native Americans (98.8%) received a CRA and 82.5% of patients in the “Other” race category. The race data needs to be interpreted with caution because of the number of unknowns with 17.6% of the total study population. The measure scores for ethnicity were significant with higher proportion of patients being non-Hispanics (97.7% of 220,785 patients; Table 2 ).

Table 2

Percentage of patients with eCRA and eAoC treatment interventions based on gender, age group, race, and ethnicity

	Population distribution		eCRA		eAoC
Variable	eCRA denominator	Perc. (%)	eCRA numerator	Measure score (%)	eAoC denominator	Total patients receiving fluoride treatment interventions (eAoC numerator)	Measure score (%)	Total patients receiving antimicrobials (eAoC numerator)	Measure score (%)
Total patients	269,536	100	254,510	94.4	128,898	109,836	85.2	47,982	37.2
Gender
Male	118,956	44.1	111,485	93.7	59,569	50,937	85.5	23,330	39.2
Female	150,273	55.8	142,773	95.0	69,153	58,753	85.0	24,576	35.5
Transgender	218	0.1	215	98.6	–	–	–	–	–
Unknown	67	0.0	17	25.4	28	15	53.6	6	21.4
Chi-squared (df = 1)				210.01			7.59		179.93
p -Value				<0.0001			0.0058		<0.0001
Age group (y)
0 to 5	16,042	6.0	11,250	70.1	5,371	4,283	79.7	412	7.7
6 to 12	35,248	13.1	32,767	93.0	20,075	18,346	91.4	4,294	21.4
13 to 18	29,436	10.9	28,284	96.1	15,978	14,393	90.1	4,720	29.5
18+	188,810	70.1	182,209	96.5	87,474	72,814	83.2	38,556	44.1
Chi-squared (df = 3)				19,838			1305.2		6323.1
p -Value				<0.0001			<0.0001		<0.0001
Race
White	154,853	57.2	151,225	97.7	71,095	61,658	86.7	28,656	40.3
African American	9,012	3.3	8,131	90.2	5,294	4,176	78.9	1,695	32.0
Asian	14,250	5.3	12,555	88.1	5,798	4,710	81.2	1,660	28.6
Native American	1,031	0.4	1,019	98.8	586	509	86.9	287	49.0
Pacific Islander	1,185	0.4	1,152	97.2	687	626	91.1	323	47.0
Other	3,508	1.3	2,895	82.5	1,779	1,478	83.1	613	34.5
Multiracial	13,510	5.0	13,067	96.7	7,011	6,306	89.9	2,581	36.8
Unknown (blanks)	47,635	17.6	41,856	87.9	22,613	18,833	83.3	7,966	35.2
Chi-squared (df = 7)				10,506			688		596.77
p -Value				<0.0001			<0.0001		<0.0001
Ethnicity
Hispanic	27,875	11	26,025	93.4	15,066	13,233	87.83	5,186	34.42
Non-Hispanic	220,785	89	215,646	97.7	102,060	89,645	87.84	41,766	40.92
Other	221	0	112	50.7	106	71	66.98	–	–
Chi-squared (df = 2, 3)				3,382.8			7011.4		230.75
p -Value				<0.0001			<0.0001		<0.0001

Abbreviations: eAoC, eMeasures appropriateness of care; eCRA, eMeasures carries risk assessment.

We determined caries risk levels with three strategies. Fig. 2 illustrates the combined use of approaches (A1) caries risk diagnosis SNODDS—treatment valid pairs; (A2) CRA Current Dental Terminology (CDT) codes; and (A3) CRA forms by site. In site 1, approach, A3 alone represented 23.0% of the caries risk documentation. A combination of approaches A2 and A3 accounted for 60.0% of the caries risk documentation, indicating the caries risk status was determined with a combination of CRA forms and CDT codes ( Fig. 3 ). In site 2, a combination of A1 and A3 corresponded to 45.0% of the caries risk estimation. Lastly, in site 3, 98.4% of the caries risk document was done with a combination of A1, A2, and A3 approaches. In summary, site 3 used a combination of the A1, A2, and A3 strategies, while site 1 used more often a combination of A2 and A3, and in site 2, A1 and A3 were more common.

Fig. 2

Three approaches to determine the caries risk level (eCRA measure). eCRA, eMeasures carries risk assessment; SNODDS, Standardized Diagnostic Dental terminology.

Fig. 3

Percentage of patients with fluoride therapies and antimicrobial interventions with extreme/high and moderate risk across three sites.

eAoC: Table 2 also displays the proportion of patients at elevated (extreme, high, or moderate) caries risk levels who received the appropriate caries risk management interventions, fluoride therapies and antimicrobial interventions. Overall, 85.2% of 128,898 patients with elevated risk received fluoride therapy. Measure scores for fluoride delivery were not statistically significant by gender ( χ 2  = 7.59, p  = 0.0058) as both females and males received the treatment intervention equally. Whereas, antimicrobial intervention was more frequent in males (39.2%) than females (35.5%), and the scores were significant. Also, among 91.4% of elevated risk patients, In the 6 to 12 years of age group received a fluoride intervention and it was the lowest in the 0 to 5 years of age group with 79.7% of patients. The measure scores were statistically significant with antimicrobial intervention in the age group category with 44.1% of adult patients ( χ 2  = 6323.1, p  < 0.0001). In terms of race and ethnicity, the highest proportion of Pacific Islanders received a fluoride intervention (91.1%) and only 28.6% of Asians received antimicrobials. Hispanics and non-Hispanics both equally received fluoride therapies ( χ 2  = 7,011.4, p  < 0.0001; Table 2 ). Fig. 3 shows the proportion of patients at extreme, high, and moderate levels who received fluoride therapies and antimicrobial interventions. Among high and extreme caries risk patients, the most common treatment intervention were fluorides in site 1 (56%), site 2 (71.1%), and site 3 (93.8%). Also, 80.6% of patients in site 3 in the extreme and high risk categories received antimicrobials, whereas only 2.9% of them received in site 1.

Discussion

Dentistry, as part of the health care system, is embracing quality measures and advocating increased use of EHR data, 25 including interoperability between dentistry and medicine 26 27 and the development of research and clinical networks to share data. 28 29 Using dental informatics, the aggregation and sharing of data allow for electronic mining of patient care data beyond claims and administrative data for research, teaching, and clinical quality of care purposes. Electronic dental patient care data are used for usability testing, 30 adherence to clinical practice guidelines 31 32 as reported in this paper, clinic decision support, 33 treatment planning, 34 education, 35 quality metrics, 22 adverse events, 36 and patient safety. Electronically derived dental quality of care measures includes caries risk and appropriate care, as discussed in this paper, also include a wide variety of dental applications such as sealants, 18 37 tobacco cessation, 16 dental patients with diabetes, and pregnancy. 38 A critical aspect of the EHR is a robust dental diagnostic interface terminology 39 for providers to select diagnosis and pair the diagnosis to the procedure performed. 40 41 A robust EHR with dental diagnosis, clinic decision support, and electronic quality measures will allow for patient care data to be used for research, patient care, and education purposes including the expansion of quality improvement in dentistry. 27

We developed, implemented, and validated two practice-level eMeasures, the eCRA a process of care measure for caries risk documentation and the eAoC, AoC measure for the delivery of evidence-based preventive treatments according to caries risk status in two academic institutions and one dental ACO.

The philosophy of CRA is established in dentistry. 42 There is a lack of standardization in determining caries risk status with different forms being utilized across the profession, with differing levels of accuracy in determining caries risk. 12 43 44 45 The electronic measures developed and validated in this study can be utilized, irrespective of which CRA tool is used to determine caries risk. The important aspect of the eCRA and eAoC measures is that caries risk for the patient is made by the provider. Recent review papers have summarized the current status of CRA and prevention recommendations. 46 47 There many CRA tools available and used in research and clinical practice to assess risk in children and adults and many preventive intervention care recommendations. 10 48 49 Despite the variety of forms available, validation studies and risk prediction models of these forms are limited in their ability to accurately predict future tooth decay in some versions of CRA tools. 13 42 Validation studies with retrospective study designs conducted in academic settings established that past caries experience is the most critical predictor of future caries experience, 11 12 with high caries risk patients more likely to develop new caries lesions during the follow-up period. A randomized prospective clinical trial using the CAMBRA demonstrated reduction in tooth decay when caries risk was used to determine and deliver prevention in a systematic way. 50 This method of care was enhanced and implemented on a large scale which demonstrated CRA and standardized preventive treatments lowered new decay well below the Healthy People 2020 baseline and target levels. 51 Evidence-based guidelines including the American Dental Association 52 and the CAMBRA protocol recommend that individuals at lower caries risk continue the traditional oral hygiene regimen, consisting of over the counter fluoride toothpaste and attending regular dental visits. For patients with moderate, high, or extreme caries risk, the personalized caries management plan based on risk includes a combination of oral hygiene and dietary instructions, prescription fluoride therapies, and antimicrobials. 10

The reimbursement model in dentistry has procedure codes for CRA and documentation; however, no documentation is required for payment which accounts for the lack of standardization. This study utilized the intent of the Dental Quality Alliance measure which is limited in using administrative enrollment and claims data. This type of data are all that is available for payors and plan administrators. Our measures built on the DQA measure to be more provider centric and informative from a quality improvement perspective. One of the significant challenges encountered while evaluating caries risk documentation in this study's participating institutions was the heterogeneity in the usage of CRA tools. In this study, we developed quality measures derived from EHRs that can be applied to any CRA tool utilized. Looking solely in the electronic health record for a completed CRA form can be problematic as data elements may be spread throughout the EHR, from different software systems and different implementations, compromising the effectiveness of dental quality measurement and may lead to underrepresentation of the relevant population (in the numerator). To overcome this challenge, and capture the most complete and accurate information, we used three distinct approaches described in the methodology, diagnosis-treatment pair (A1), CDT codes (A2), and CRA forms (A3) to classify caries risk status. This method expands the applicability of the quality measures as we were successful at the three institutions that participated. For all sites, we were able to use the novel risk assessment strategy “diagnosis and treatment valid pairs (A1),” where each pair is comprised of the risk-level assignment and its corresponding treatment choice. In earlier work, we have documented the development of diagnosis–treatment pair strategy, 53 and the importance of using diagnostic terms in assessing adherence to treatment protocols. 39 To measure quality effectively using EHR data and ultimately improve the care, our patients received complete, accurate, and high-quality data that were available for quality measurement evaluations and research. We anticipate that the approach will be successful at a wide range of dental clinics with differing software and implementations, although customization of the measures would be required.

There were site variations in the measure scores that merit some explanation. Site 3, a dental ACO, had the highest proportion of patients with an eCRA relative to the other sites ( Table 2 ). Also, site 3 had higher scores for eAoC compared with sites 1 and 2, with the majority of patients at moderate, high and elevated caries risk receiving appropriate caries management interventions. Variations across sites may be partially explained by the differences in the distribution of at-risk patients and the ACO providers are continuously trained and received feedback on their performance regarding the use of diagnostic terminology and corresponding procedure codes.

Our study employed a process of care measure that can identify best practices and opportunities for quality improvement. The participating dental ACO in this study has such processes in place and the results clearly demonstrate improved delivery of caries management strategies to their high-risk patient population. Moving from measuring quality with data to implementing policies for sustained quality improvement takes a culture shift and leadership support as we have described earlier. 54 We expect that well-defined process of care measures for caries risk, eCRA and good AoC measures for caries prevention, eAoC would result in good outcomes of care measures, eAoC for caries. Recently, the literature reported on the development and validation of automated tools to calculate caries indices from data available in the EHR, and can be adapted to implement eAoC measures. 55 The next steps in this research will be to investigate the relationship between eCRA, eAoC for prevention, and eAoC for incidence of new caries lesions.

Early childhood caries (ECC) is the single most common chronic childhood disease. 56 As noted by the CDC, “oral health disparities are profound in the United States.” 57 Although the prevalence of early childhood caries is 37% among all children aged 2 to 8 years, there are great disparities in prevalence between Hispanic (46%) and non-Hispanic Black (44%) children compared with non-Hispanic white children (31%). 58 The prevalence of severe early childhood caries among disadvantaged groups has been reported up to 85%. 59 Clearly, the time has come to implement preventive measures up front to prevent draconian measures later on, for the children entrusted in our care. The ability to evaluate these quality measures including age, gender, and race/ethnicity can be helpful in understanding other factors that might influence the CRA and preventive management of care. A recent quality improvement study showed that children with a documented CRA were more likely to receive preventive services, while those who did not have a CRA had higher number of invasive restorative procedures. 6 We have begun to include other factors, such as socioeconomic status, rural and urban location adding to these measures based on EHR data which will be beneficial in exploring health disparities. 51 60 Given the prevalence of tooth decay, these measures have tremendous importance in their ability to provide for practitioners, patients, payors, and public health with tools to evaluate the quality of care and appropriateness of treatment provided.

Evidence-based caries risk management strategies for moderate-to-high caries risk patients continues to include sealants, modification in diet, adequate plaque removal, and fluoride toothpaste as important steps for maintaining and/or reversing caries risk and improving the oral health of our patients. 10

In summary, this work presents the use of process of care dental quality measure using longitudinal patient data available in the EHR to provide feedback on the clinical care related to the documentation of caries risk and risk-appropriate treatment to identify clinical workflows and areas of concern for clinical improvement initiatives. As future directions, we will expand and implement these process-of-care measures in other dental institutions that are a part of the BigMouth Dental Data Repository. 28 The routine adoption of dental quality measures will optimize clinical performance and, ultimately, precise and patient-centered clinical care.

Limitations

Despite obvious advantages of using clinical EHR data to improve clinical outcomes, we encountered several limitations worth noting. First, as process of care measures were limited to evaluate a procedure performed for the patient, we could not assess the reasons why evidence-based preventive care was not prescribed to a larger number of adult patients at elevated caries risk in sites 1 and 2. Also, we could not evaluate if a lower proportion of high-risk individuals in site 3, a dental ACO, is a result of a well-established caries preventive protocol, leadership structure and clinical organization, or because of potential different populations at each institution. Based on caries risk status, we are currently developing outcomes of care measures derived from EHR data to evaluate new and untreated caries lesions. Second, the number of patients with a CRA might have been underestimated in sites 1 and 2, since most were identified from a variety of CRA forms. As quality measures become more common in clinical practice, detailed information connecting disease conditions and dental procedures as available in SNODDS standardized diagnostic terminology, 61 it will be necessary to appropriately capture the care provided and include patients in the numerator. Lastly, our study was conducted using the same EHR (axiUm) across all three sites and calls into question the generalizability level in non-axiUm user institutions. Nevertheless, the methodology and measure specifications we described may be adapted in other institutions using different EHRs, if they are set to the same standardized codes for treatment and diagnosis.

Conclusion

This study demonstrated the use of patient-level EHR data to validate and implement two dental quality eMeasures on CRA and risk-based treatment in a large and diverse cross-sectional sample. Data, available in the EHR allowed for the dental quality eMeasures, developed and implemented in this study utilizing a combination of methods to capture the caries risk, providing the best results in documenting that patients received a CRA and a risk-based caries preventive treatment.

Clinical Relevance Statement

This study's results indicate that quality measurement research using clinical data can be a valuable tool to evaluate the performance of CRA and risk-based treatments. Findings from such evaluations could inform clinicians, researchers, and administrators to design interventions to optimize clinical performance and provide our patients with evidence-based personalized care.

Which type of dental quality measure was selected to determine the proportion of patients receiving a caries risk assessment and corresponding appropriate risk-based preventative treatments for patients at elevated risk of caries?

Process of care

Outcomes of care

Structural measures

Correct Answer : The correct answer is option a. Process of care measures can identify gaps in clinical care, and inform the provider, and administrators on what is working well and what needs action.

What approaches were used to validate the eMeasures (mark all that apply)?

Expert opinion

Concordance between manual chart reviews and automated queries

Comparison with external dataset

Correct Answer : The correct answer is option b. The automated query was validated with manual chart reviews in all participating institutions to verify if the query measured what was intended to measure (see Methods).