Detecting Patients With Nonvalvular Atrial Fibrillation and Atrial Flutter in the Canadian Primary Care Sentinel Surveillance Network: First Steps.

BACKGROUND: A recent feasibility assessment of quality indicators for nonvalvular atrial fibrillation/atrial flutter (NVAF/AFL) identified the Canadian Primary Care Sentinel Surveillance Network, a national outpatient electronic medical record (EMR) system, as a data source for measurement. As a first step, we adapted and validated an existing EMR case definition.
METHODS: A diagnosis of NVAF/AFL was defined using International Classification of Disease, 9th Revision, Clinical Modification codes (427.3) in either the physician billing, encounter diagnosis, or health condition fields. We identified all presumed cases in a single clinical site with the algorithm and selected a random sample of those who were presumed NVAF/AFL negative with the same algorithm. A chart audit diagnosis of "definite" NVAF/AFL was confirmed by electrocardiogram and nonvalvular diagnosis confirmed after echocardiogram, attending physician, or specialist letter review. To demonstrate face validity, clinical characteristics were compared for patients with and without NVAF/AFL.
RESULTS: The case definition identified a possible 184 patients with and 184 without NVAF/AFL. The case validation resulted in a sensitivity of 100% (95% confidence interval [CI], 100-100), specificity of 84.3% (95% CI, 78.8-89.9), and positive and negative predictive value of 74.7% (95% CI, 66.4-83.2) and 100% (95% CI 100-100), respectively. Patients with NVAF/AFL were older (63 vs 42 years) and had a higher proportion of cardiovascular comorbidities and relevant medications.
CONCLUSIONS: We think it is possible that with further validation work, NVAF/AFL can be accurately identified using this large pan-Canadian EMR system and used as a future tool to measure quality of care in the outpatient setting.

Atrial fibrillation, the most prevalent heart rhythm disorder, has a significant impact on outcomes and health care costs. Therefore, understanding the quality of nonvalvular atrial fibrillation/atrial flutter (NVAF/AFL) care in Canada is critical not only to ensure the delivery of optimal and cost-effective care but also to properly inform future resource allocation. The Canadian Cardiovascular Society AF/AFL quality indicator (QI) working group recently identified 5 QIs in the areas of therapy and outcomes and then performed an environmental scan to determine the feasibility of measurement across all settings. Of the multiple setting where patients with NVAF/AFL may be managed, primary care is one of the most important and yet it is the least well examined. A prior publication from Alberta found that the proportion of cases diagnosed in the hospital declined 21% between 2000 and 2005, whereas the proportion of cases diagnosed in the outpatient setting rose by 50%. Electronic medical records (EMRs) are increasingly used in the primary care settings and may address the potential data gap.

Reporting of QIs based on administrative data alone has important limitations including a cohort restricted to hospitalized patients, imperfect accuracy of NVAF/AFL diagnosis, and stroke risk factors by administrative coders, and difficulties in accurately assessing the use of stroke prevention therapy. Recently, the Canadian Cardiovascular Society AF/AFL working group published an update on the development and feasibility assessment of Canadian QI for NVAF/AFL.

Most patients with AF are treated in outpatient settings, and few admitted to the hospital for AF. A potential data source identified for measuring and reporting QIs in the outpatient setting was the Canadian Primary Care Sentinel Surveillance Network (CPCSSN), a national primary care EMR surveillance system. As a first step, we used an existing case definition of NVAF/AFL that has been validated for use using electronic health record data, and conducted a validation study to determine its usefulness as a tool to identify patients with NVAF/AFL in CPCSSN data.

Methods

CPCSSN is Canada’s only multidisease surveillance system that extracts and collects deidentified personal health information from primary care EMR. It consists of comprehensive data (demographics, health conditions, encounter and billing diagnosis, labs, medications, vitals, and referrals) from 1,820,500 patients from 13 primary care practice–based research networks representing 1378 primary health care clinics across 7 provinces and 1 territory. The data are extracted quarterly, anonymized, cleaned and coded, and then mapped to a common database structure.

A case validation study of NVAF/AFL using a full EMR chart review as the “gold standard” was performed at a single clinical site, the Queen’s Family Health Team, Department of Family Medicine at Queen’s University, using CPCSSN data extracted in the fourth quarter of 2017. The case definition of NVAF/AFL was any patient ≥20 years who had an International Classification of Disease, 9th Revision, Clinical Modification code 427.3 and without valvular disease recorded in either the physician billing, encounter diagnosis, or health condition fields (Table 1)., We also selected a simple random sample, using a random number generator, of the same number of patients who were not identified as having NVAF/AFL by the same algorithm in the same clinic. A chart audit diagnosis of “definite” NVAF/AFL was confirmed by the presence of either an electrocardiogram and nonvalvular confirmed after echocardiogram, the attendant physician or specialist letter review, or any other indication of a physician-diagnosed case of NVAF/AFL contained in the patient’s entire medical chart. Thus, patients who had no evidence of NVAF/AF using the algorithm could be classified as having NVAF/AF based on the available unstructured data. The chart auditor was a Health Information Management student trained by the study investigators and blinded to the chart selection process. In cases where the chart auditor was unable to determine either a positive or negative diagnosis, a content expert provided a second review to determine whether an NVAF/AFL diagnosis was present or absent.

Table 1

Case definition of NVAF/AFL

Any of the following ICD-9 codes in the Billing, Encounter Diagnosis, or Health Condition table:
427.3: Atrial fibrillation and flutter
427.31: Atrial fibrillation
427.32: Atrial flutter
Excluding:
394, 394.0, 394.1, 394.2, 394.9: Diseases of the mitral valve
395, 395.0, 395.1, 395.2, 395.9: Diseases of the aortic valve
396, 396.0, 396.1, 396.2, 396.3, 396.9: Diseases of mitral and aortic valves
424, 424.0, 424.1, 424.2, 424.3, 424.9, 424.90, 424.91, 424.99: Other diseases of endocardium
V43.3: Heart valve replaced by other means

ICD-9, International Classification of Disease, 9th Revision; NVAF/AFL, nonvalvular atrial fibrillation/atrial flutter.

We calculated sensitivity, specificity, and positive and negative predictive values. Overall accuracy, defined as the proportion of times the definition correctly classified true positives and true negatives, and Cohen’s k, which is a measure that indicates to what degree 2 raters agree with each other were also calculated. To demonstrate face validity (the extent to which we identify a group of patients who have the characteristics expected of a group of patients who have NVAF/AF), we compared the demographics, clinical profiles, and prescribed medications for NVAF/AFL present and absent patients. All data analysis was conducted using SAS version 9.4 TS. Ethics approval was granted by health sciences and affiliated teaching hospitals research ethics board at Queen’s University, Kingston.

Results

Of the 1,820,500 patients in CPCSSN, 13,806 were identified from a single site located in the Eastern Ontario Network (Fig. 1). The case definition identified a possible 184 patients with and 184 without NVAF/AFL. After accounting for archived charts that were unavailable to us because the patient had moved or died and had been removed from the EMR, the final sample consisted of 103 patients with a presumed diagnosis of NVAF/AFL and 140 without.

Figure 1

Study flow.

The validation metrics of the EMR algorithm and chart audit diagnosis of NVAF/AFL is shown in Table 2. We found a sensitivity of 100% (95% confidence interval [CI], 100-100), a specificity of 84.3% (95% CI, 78.8-89.9), and a positive and negative predictive value of 74.7% (95% CI, 66.4-83.2) and 100% (95% CI, 100-100), respectively. The overall accuracy was 89.3% (95% CI, 85.4-93.2) and Cohen’s k = 77.3 (95% CI, 69.3-85.4).

Table 2

Validation metrics

	Observations from chart abstraction
Classified by case definition	Yes	No	Row total
Yes	77	26	103 of 184
No	0	140	140 of 184
Column total	77	166	243
Sensitivity with 95% CI	100% (100%, 100%)
Specificity with 95% CI	84.3% (78.8%, 89.9%)
PPV with 95% CI	74.7% (66.4%, 83.2%)
NPV with 95% CI	100% (100%, 100%)
Accuracy with 95% CI	89.3% (85.4%, 93.2%)
Cohen’s kappa with 95% CI	0·773 (0·693, 0·854)

CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value.

The baseline characteristics and clinical description of patients with and without NVAF are shown in Table 3. Compared with patients without NVAF/AFL, an older average age (62 vs 42 years) and a higher proportion of cardiovascular comorbidities and relevant medications were seen in patients with NVAF/AFL. The health care providers of the patients with NVAF/AFL tended to have a higher median practice size (420 patients) than that of providers of the patients who did not have NVAF/AFL (333 patients).

Table 3

Baseline characteristics of patients classified with and without nonvalvular atrial fibrillation and atrial flutter (NVAF/AFL) by the case definition

Patient characteristic	NVAF/AFL + (n = 103)	NVAF/AFL − (n = 140)
	n (%)	n (%)
Demographic
Age (y), mean (SD)	62.6 (12.8)	42.0 (15.7)
Age group (y)
< 65	48 (46.6)	125 (89.3)
65 to < 75	37 (35.9)	11 (7.9)
≥ 75	18 (17.5)	4 (2.9)
Sex
Females	50 (48.5)	75 (53.6)
Location
Urban	94 (91.3)	114 (81.4)
Clinical profile
Comorbidities
Heart failure	21 (20.4)	3 (2.4)
Hypertension	52 (50.5)	27 (19.3)
Diabetes mellitus	28 (27.2)	14 (10.0)
Vascular disease	7 (6.8)	0 (0)
Coronary artery disease	30 (29.1)	12 (9.7)
Stroke/transient ischemic attack	4 (3.9)	3 (2.4)
CHADS2 score
0	8 (7.8)	76 (61.3)
1	16 (15.5)	22 (17.7)
≥ 2	79 (76.8)	26 (21)
CHA2DS2VASc score
0	7 (6.8)	51 (36.4)
1	12 (11.7)	54 (38.6)
≥ 2	84 (81.6)	35 (25.0)
Estimated glomerular filtration rate (mL/min/1.73 m2)
Mean (SD)	72.9 (19.0)	86.3 (16.8)
Medications
Warfarin	56 (54.4)	2 (1.6)
Apixaban	23 (22.3)	0 (0)
Other direct oral anticoagulants∗	35 (34.0)	1 (0.7)
Aspirin	42 (40.8)	14 (11.3)
Non-ASA antiplatelet agents	41 (39.8)	12 (9.7)
NSAIDs	12 (11.7)	25 (20.2)
ACEI or ARB	29 (28.2)	17 (13.7)
β-Blocker agents	15 (14.6)	3 (2.4)
Diuretics	0 (0)	3 (2.4)
Antiarrhythmic and/or digoxin	27 (21.8)	3 (2.9)
Lipid-modifying agent	1 (1.0)	0 (0)

ACEI, angiotensin-converting enzyme inhibitors; ARB, angiotensin II receptor blocker; ASA, acetylsalicylic acid; CHADS2, Congestive Heart Failure, Hypertension, Age ≥ 75, Diabetes Mellitus, and Prior Stroke/Transient Ischemic Attack; CHA2DS2-VASc, Congestive Heart Failure, Hypertension, Age ≥ 75 years, Diabetes, Stroke/Transient Ischemic Attack, Vascular Diseases, Age 65 to 74 years, Sex Category; NSAID, nonsteroidal anti-inflammatory drug; SD, standard deviation.

Dabigatran or rivaroxaban. None of the patients were prescribed edoxaban.

Conclusions

In this study, we adapted an existing methodology using administrative codes widely used in the context of administrative data sources but has yet to be validated using data contained within Canadian primary care. A systematic review on AF using EMR found that previous AF validation studies performed relatively well, reporting a wide range of sensitivity (57%-95%) and positive predictive value (70%-96%). However, the generalizability of the case definition validity was limited by few recent data (before 2000), use of nonrepresentative populations, and the majority of data focused on the inpatient setting. We found that it is possible to detect patients in an outpatient setting with a high degree of sensitivity and specificity within the structured electronic medical data housed within CPCSSN. Furthermore, our comparison of the demographic, clinical, and medication histories of the patients classified with or without NVAF/AFL reflects what we would expect to observe in a sample of patients with NVAF/AFL. Thus, we think that the case definition has high face validity. These data provide evidence that applying this case definition across the entire CPCSSN database is feasible and that we can become capable of describing the epidemiology and measurement of relevant NVAF/AFL QI indicators (stroke risk strata and the proportion of patients at high risk for stroke receiving oral anticoagulant therapy can be measured in the CPCSSN data.

Validating a case definition for NVAF/AFL for using CPCSSN data is not without limitations. Our validation exercise is limited to a single EMR and a single academic primary health care medical clinic. Most importantly, our study sample was not randomly selected and consisted of a control group (presumed negative for NVAF) that may not be representative of a population of patients at risk of NVAF. As such our results, although promising, should be interpreted with caution. We also do not know to what extent the performance of the algorithm changes by age group and by sex. However, we suspect that the algorithm will perform better in patients 65 years of age and older because of the expected increasing prevalence. Funding permitting, necessary next steps will include a larger, more comprehensive, validation study that includes a cross-country sample of randomly selected CPCSSN patients to further assess the validation metrics of the algorithm by age group, by sex, and to what degree accuracy differs by CPCSSN Network and/or EMR product. Although CPCSSN has validated case definitions for comorbidities such as diabetes and hypertension, validated case definitions for the other comorbidities have yet to be created. It is also important to note that the research-based primary care networks that make up the CPCSSN may not be representative of all primary care networks. Last, algorithm accuracy is dependent on the completeness of the structured data held in the EMR; thus we do not know to what extent patients only have relevant information in the unstructured data. We suspect that the specificity may be limited by unknown coding and recording variations within the source EMR that warrant further investigation. Future work will further refine the algorithm to improve the specificity without adversely affecting sensitivity. A supplemental appendix of limitations is provided (Supplemental Appendix S1).

We think that NVAF/AFL can be identified with a high degree of accuracy and face validity in CPCSSN. With further validation work, this large pan-Canadian EMR system can be used as a future tool for research, surveillance, and ongoing clinical monitoring of quality of care of patients with NVAF/AFL.