No consensus on definition criteria for stroke registry common data elements.

BACKGROUND: Stroke registries contribute to the conduct of clinical research and to the assessment of health care quality control. Efforts to compare clinical outcomes and quality indicators between centers are dependent on standardized data elements, but it is unknown how stroke physicians define common data elements, such as hypertension or diabetes, when collecting data for registries at their centers.
METHODS: We conducted an internet-based survey of 91 centers affiliated with a university to assess their definitions of common data elements (CDEs) and compared their responses with standardized definitions, including those from the American College of Cardiology (ACC).
RESULTS: More than half (52%) of centers completed the survey. There was only modest agreement among respondents regarding definitions of CDEs in the survey and even less agreement on how the respondents' definitions compared to ACC standards.
CONCLUSIONS: Surveyed respondents do not agree on the definitions of CDEs, making comparisons between centers problematic. Standardized definitions of CDEs are needed to improve data collection for patient care and clinical research.

Introduction

Stroke registries are prospectively maintained collections of patient data that are used for clinical research and health care quality, federal- and state-mandated programs for hospital certification, and NIH-sponsored research networks. In research focused on cerebrovascular disease, registries soon will be used for verification of adherence to Centers for Medicare/Medicaid pay-for-performance indicators. Historically, registries in stroke have provided important data on the prevalence and severity of vascular risk factors in the stroke population, approaches to acute stroke treatment, complications, and outcomes. Observed associations among variables in stroke registries have proven useful in the design of quality improvement projects and prospective clinical trials.

In 2001, the American College of Cardiology's (ACC) Task Force on Clinical Data Standards published ‘Key Data Elements and Definitions for Measuring the Clinical Management and Outcomes of Patients with Acute Coronary Syndromes’ [1, 2]. These definitions were proposed to provide standardization for all centers involved in the collection of clinical data related to cardiovascular disease and allow for effective pooling of information from different centers. Unfortunately, no equivalent published guidelines exist in the literature on cerebrovascular disease. The NINDS is in the midst of proposing a library of CDEs or ‘common data elements’ (http://www.commondataelements.ninds.nih.gov/) for use in stroke studies; however, while these elements include factors such as hypertension and diabetes, the NINDS standards currently do not offer definitions for the elements but rather only specify classification for these factors as ‘yes’, ‘no’ or ‘unknown’ [3, 4]. Since cardiovascular and cerebrovascular diseases share risk factors, the ACC standards could provide specific definitions for use in stroke studies as well. These ACC standards can be used to assess the validity and reliability of registry data from stroke centers.

If data elements and definitions for research in cerebrovascular disease are agreed upon and standardized, registry level data have the potential to promote research by allowing for comparison and pooled analyses of both clinical trial data and multi-site observational data. Registries often provide important data and observations upon which clinical trials are designed and clinical trials often provide the basis for clinical care decisions. Furthermore, without registries, it may not be possible to readily understand the prevalence of specific characteristics and co-morbid diseases among an otherwise captive cohort of patients. Finally, with the impending quality performance measurement initiatives such as pay-for-performance, providers would benefit from registry data that are supported by a common taxonomy, enhancing both validity and reliability.

The purpose of this study was to compare the current use of definitions for data elements among US stroke centers that have stroke registries, and compare the use of existing stroke registry definitions with those already established by the ACC. We hypothesized that vascular neurologists using registry data were not making use of existing definitions published in the cardiology and internal medicine literature.

Materials and Methods

Study Design, Participants, and Administration

From September to December 2009, we conducted a cross-sectional, internet-based survey of academic vascular neurologists in the United States at university-affiliated hospitals using a modified Dillman's tailored design method [5, 6]. The survey was designed and conducted at www.surveymonkey.com [7] and used our previously published methods for obtaining a complete list of US academic vascular neurologists [8]. Vascular neurologists were targeted, as they are typically the primary investigators of registry-based clinical research, and as such are responsible for supervising the research process, assuring study personnel are appropriately trained, and that they adhere to the study protocol. After an exhaustive effort to identify all stroke centers affiliated with a university in the United States, surveys were distributed to a total of 144 vascular neurologists. Seven of the 144 individuals had e-mail addresses that were nonfunctional and the correct address could not be found. This yielded a total sample number of 137 initial e-mails that were successfully sent to the target population at 91 universities.

After β-testing by three of the authors (K.C.A., S.M.-S., and S.I.S.) and two additional volunteers, the survey was formally released. Participants received a pre-survey cover letter explaining the objectives of the study and its importance. A request to participate in the study, with detailed instructions, followed which provided a link to the electronic survey. Non-responders were sent four reminders. No incentives were provided. This study was coordinated by the Vascular Neurology Program at the University of Texas School of Medicine in Houston.

Survey Content

The survey consisted of 3 general questions and 10 multi-part definition questions. All survey questions as they were presented to respondents are shown in the online supplementary material (www.karger.com/doi/10.1159/000334146). Respondents were asked to indicate (1) if their institution had an active stroke registry, (2) who abstracted data at their site, and (3) if their site made use of a registry codebook or data dictionary. The survey focused on 10 variables (i.e., data elements) commonly collected in stroke registries, many of which are specifically defined in the ACC standards: hypertension, diabetes, hyperlipidemia/dyslipidemia, atrial fibrillation, tobacco use, myocardial infarction, coronary artery disease, congestive heart failure, carotid disease, and peripheral vascular disease [1, 2, 9, 10, 11]. After an exhaustive search of available guidelines failed to reveal a formal definition of ‘carotid disease’, potential definitions were generated. Participants were provided choices and instructed to indicate if their center currently uses each provided choice as part of their definition for each variable (yes or no). Respondents were then instructed to indicate if they found each of the provided choices ‘acceptable’ as part of their definition for each variable (yes or no). Risk factor data elements included (but did not explicitly identify) criteria listed in definitions previously published by the ACC and American Diabetes Association (ADA) [1, 2, 11, 12].

Data Collection and Analysis

All data was entered electronically by the participants. Analysis was performed using SPSS version 15.0 (SPSS Inc., Chicago, Ill., USA). The anticipated response rate was estimated at 50% (95% CI 37–63%, adjusted Wald method) based on prior studies [12, 13, 14].

Results

Sample

Participants completed 55 self-administered surveys, representing 47 universities. Vascular neurologists reported that registry data were abstracted by trained abstractors (29%), medical assistants (4%), nurses (29%), physician assistants (2%), fellows (2%), attending physicians (6%), or a combination of medical professionals (29%). The response rate for individuals was 40.1% (55/137) compared to 51.6% (47/91) for universities. There was no significant difference in response rates based on Census Bureau regions (Northeast 25.5%, Midwest 29.1%, South 27.3%, and West 18.2%, p = 0.700).

Data Elements

The majority of respondents (80%, 44/55) reported using a stroke registry; however, only 46% of respondents using a registry reported using a data dictionary to standardize definitions within their center. Table 1 illustrates reported current use of examined definition criteria for 10 data elements. Patient or family report of having a data element was the most commonly used definition criterion, ranging from 55.8% for hyperlipidemia to 74.4% for hypertension and carotid endarterectomy as a surrogate for carotid disease. Current use of prior documentation of a condition ranged from 53.5% for tobacco use within the last year to 72.1% for hypertension. Current use of definition criteria related to patient medication ranged from 7.0% for peripheral vascular disease to 65.1% for diabetes. The use of laboratory findings to define data elements ranged from 11.6% in ‘high-density lipoprotein <40 mg/dl’ to 44.2% in ‘glycosylated hemoglobin (HbA1c)’ for diabetes. Use of data elements derived from prior diagnostic studies ranged from 11.6% for ‘left ventricular hypertrophy on admission electrocardiogram (ECG)’ for hypertension to 76.7% for demonstration of atrial fibrillation on admission ECG. Also depicted in table 1 is the reported acceptability for each data element. There was poor concordance between current use and acceptability of most of the definition choices, especially for the many choices offered for the data elements hypertension, diabetes, and carotid disease. For the most commonly used definition criteria of each of the 10 data elements, at least 20% fewer respondents found definition criteria to be acceptable.

Table 1

Current use and acceptability of definition criteria for 10 stroke registry data elements

	Current use (n = 43)	Also acceptable (n = 46)
	% (n)	% (n)
Hypertension (HTN)
1 Patient or family states that s/he has a history of diagnosed HTN	74.4 (32)	47.8 (22)
2 Prior documentation of HTN	72.1 (31)	47.8 (22)
3 Admission ECG reads left ventricular hypertrophy (LVH)	11.6 (5)	43.5 (20)
4 Prior echo shows concentric LVH	14.0 (6)	37.0 (17)
5 Patient is currently on any antihypertensive medication (e.g., ACE inhibitors, angiotensin receptor blockers, β-blockers, thiazide diuretics, calcium channel blockers) for no other known reason (e.g., systolic heart failure, rate control of atrial fibrillation, migraine prevention)	55.8 (24)	30.4 (14)
6 Other	2.3 (1)	4.3 (2)
7 We do not have a formal definition	25.6 (11)	4.3 (2)
Diabetes mellitus (DM)
1 Patient or family states that s/he has a history of diagnosed diabetes	69.8 (30)	41.3 (19)
2 Prior admission notes, discharge summaries or clinic notes list DM	72.1 (31)	41.3 (19)
3 Existing medical records show a random plasma glucose concentration ≥200 mg/dl in association with polyuria, polydipsia, or unexplained weight loss1	34.9 (15)	47.8 (22)
4 Existing medical records show a fasting (i.e., no caloric intake for at least 8 h) plasma glucose value ≥126 mg/dl (in an asymptomatic patient) 1	27.9 (12)	47.8 (22)
5 Existing medical records show a 2-hour plasma glucose value ≥200 mg/dl during an oral glucose tolerance test (in an asymptomatic patient) 1	27.9 (12)	56.5 (26)
6 Admission or previous HbA1c ≥7.01	44.2 (19)	37.0 (17)
7 Patient is currently on any of the following medications: metformin, glyburide, pioglitazone, rosiglitazone, glipizide/Glucotrol, any form of insulin (e.g., Humalog, Humulin, Lantus, Novolin, NovoLog, 70/30)	65.1 (28)	43.5 (20)
8 Other	0 (0)	2.2 (1)
9 We do not have a formal definition	23.3 (10)	4.3 (2)
Hyperlipidemia (HLP) or dyslipidemia
1 Patient or family states that s/he has a history of diagnosed HLP	55.8 (24)	34.8 (16)
2 Patient or family states that s/he has a history of diagnosed dyslipidemia [i.e., elevated triglycerides, low high-density lipoprotein (HDL)]	58.1 (25)	34.8 (16)
3 Prior documentation of HLP	65.1 (28)	37.0 (17)
4 Prior documentation of dyslipidemia	55.8 (24)	32.6 (15)
5 Admission or previous total cholesterol >200 mg/dl (5.18 mmol/l)	25.6 (11)	32.6 (15)
6 Admission or previous low-density lipoprotein (LDL) ≥130 mg/dl (3.37 mmol/l)	34.9 (15)	47.8 (22)
7 Admission or previous LDL is >100 mg/dl (2.59 mmol/l) in patients with known coronary artery disease or stroke	30.2 (13)	30.4 (14)
8 Admission or previous HDL <40 mg/dl (1.04 mmol/l)	11.6 (5)	28.3 (13)
9 Patient is currently on any of the following medications: statin/HMG-CoA reductase inhibitors, fibrates, nicotinic acid, resin drugs (e.g., atorvastatin, simvastatin, pravastatin, fluvastatin, lovastatin, cholestyramine, colestipol, probucol, gemfibrozil, Niaspan)	51.2 (22)	37.0 (17)
10 Other	2.3 (1)	2.2 (1)
11 We do not have a formal definition	23.3 (10)	4.3 (2)
Atrial fibrillation (AF)
1 Patient or family states that s/he has history of AF	62.8 (27)	39.1 (18)
2 Prior documentation of AF	74.4 (32)	39.1 (18)
3 Admission ECG shows AF	76.7 (33)	39.1 (18)
4 Admission ECG shows atrial flutter	39.5 (17)	30.4 (14)
5 Prior echo shows AF/atrial flutter	25.6 (11)	23.9 (11)
6 Prior Holter monitoring or event recording revealed AF	62.8 (27)	43.5 (20)
7 Prior Holter monitoring or event recording revealed atrial flutter	39.5 (17)	32.6 (15)
8 Patient reports a history of ‘irregular heartbeats’ without knowledge of term AF	2.3 (1)	2.2 (1)
9 Other	0 (0)	2.2 (1)
10 We do not have a formal definition	16.3 (7)	2.2 (1)
Tobacco use
1 Patient or family states that s/he currently uses tobacco (cigars, cigarettes, smokeless tobacco)	72.1 (31)	41.3 (19)
2 Patient or family states that s/he has used tobacco in the last year (cigars, cigarettes, smokeless tobacco)	67.4 (29)	37.0 (17)
3 Prior documentation of tobacco use in the last year	53.5 (23)	37.0 (17)
4 Other	0 (0)	2.2 (1)
5 We do not have a formal definition	16.3 (7)	2.2 (1)
Myocardial infarction (MI)
1 Patient or family states that s/he has a history of MI	62.8 (27)	34.8 (16)
2 Prior documentation of MI	67.4 (29)	34.8 (16)
3 Admission ECG shows ST-segment elevation in a lead group	25.6 (11)	23.9 (11)
4 Admission ECG shows either ST-segment depression or T-wave abnormalities – in the absence of ST elevation in a lead group	20.9 (9)	21.7 (10)
5 Q waves present on admission ECG in a lead group	27.9 (12)	32.6 (15)
6 History of MI symptoms in the presence or absence of chest discomfort; ischemic symptoms may include: unexplained nausea and vomiting or diaphoresis, persistent shortness of breath secondary to left ventricular failure, unexplained weakness, dizziness, lightheadedness, or syncope	9.3 (4)	10.9 (5)
7 Prior echo shows evidence of regional wall motion abnormalities consistent with prior MI	27.9 (12)	28.3 (13)
8 Prior nuclear imaging studies, exercise radionuclide ventriculography or pharmacologic stress echocardiography consistent with prior MI	25.6 (11)	32.6 (15)
9 Other	0 (0)	2.2 (1)
10 We do not have a formal definition	23.3 (10)	2.2 (1)
Coronary artery disease (CAD)
1 Patient or family states that s/he has history of prior coronary artery bypass graft (CABG) or percutaneous transluminal coronary angioplasty (PTCA)/percutaneous coronary intervention (PCI)	69.8 (30)	39.1 (18)
2 Prior documentation of CABG or PTCA/PCI including balloon angioplasty, atherectomy, stent, or other	67.4 (29)	37.0 (17)
3 Wires seen on chest X-ray without alternate explanation	18.6 (8)	17.4 (8)
4 Scar on chest or scar on leg from vein harvest	11.6 (5)	17.4 (8)
5 Has current prescription for long-acting nitrates (isosorbide mononitrate or dinitrate)	23.3 (10)	15.2 (7)
6 Has current prescription for sublingual nitroglycerin	27.9 (12)	19.6 (9)
7 Other	0 (0)	2.2 (1)
8 We do not have a formal definition	23.3 (10)	4.3 (2)
Congestive heart failure (CHF)
1 Patient or family states that s/he has a history of CHF	58.1 (25)	30.4 (14)
2 Prior documentation of CHF	62.8 (27)	34.8 (16)
3 Prior echocardiogram demonstrated reduced left ventricular ejection fraction (<50%)	25.6 (11)	21.7 (10)
4 History or current symptoms of dyspnea, orthopnea, and fluid retention	30.2 (13)	13.0 (6)
5 Examination demonstrates crackles, rales, or jugular venous distension	20.9 (9)	19.6 (9)
6 Admission chest X-ray demonstrates pulmonary edema with pulmonary venous congestion and cephalization of vasculature	25.6 (11)	28.3 (13)
7 Markedly elevated brain natriuretic peptide on current admission or any prior assessment	23.3 (10)	23.9 (11)
8 On combination regimen(s) suggestive of heart failure (e.g., ACE inhibitors + loop diuretic, hydralazine + nitrate)	11.6 (5)	15.2 (7)
9 Patients instructed to weigh themselves daily	2.3 (1)	6.5 (3)
10 Other	2.3 (1)	4.3 (2)
11 We do not have a formal definition	23.3 (10)	4.3 (2)
Carotid disease
1 Patient or family states that s/he has history of carotid disease	58.1 (25)	23.8 (13)
2 Patient or family states that s/he has history of carotid stenting	72.1 (31)	32.6 (15)
3 Patient or family states that s/he has history of carotid endarterectomy (CEA)	74.4 (32)	39.1 (18)
4 Prior documentation of carotid disease	72.1 (31)	34.8 (16)
5 Prior documentation of carotid stenting	72.1 (31)	37.0 (17)
6 Prior documentation of CEA	74.4 (32)	37.0 (17)
7 Evidence of >50% stenosis of an extracranial carotid artery on imaging (e.g., ultrasound, MRA, CTA, angiogram)	60.5 (26)	34.8 (16)
8 CEA scar	32.6 (14)	30.4 (14)
9 Evidence of carotid stent or CEA on imaging	46.5 (20)	37.0 (17)
10 Other	0 (0)	2.2 (1)
11 We do not have a formal definition	18.6 (8)	2.2 (1)
Peripheral vascular disease (PVD)
1 Patient or family states that s/he has a history of PVD	58.1 (25)	28.3 (13)
2 Prior documentation of PVD	58.1 (25)	32.6 (15)
3 History of claudication, either with exertion or at rest	41.9 (18)	34.8 (16)
4 History of amputation for arterial vascular insufficiency	48.8 (21)	37.0 (17)
5 History of vascular reconstruction, bypass surgery, or percutaneous intervention to the extremities	48.8 (21)	39.1 (18)
6 Documented aortic aneurysm	16.3 (7)	21.7 (10)
7 Positive noninvasive test (e.g., ankle brachial index <0.8) or lower extremity US with arterial Doppler	37.2 (16)	43.5 (20)
8 Absence of peripheral pulses on examination	20.9 (9)	26.1 (12)
9 On medication, e.g., Pletal/cilostazol	7 (3)	13.0 (6)
10 PVD demonstrated on imaging (e.g., angiography)	39.5 (17)	37.0 (17)
11 Other	0 (0)	2.2 (1)
12 We do not have a formal definition	32.6 (14)	2.2 (1)

Data elements derived from ACC Guidelines are in bold [1, 2].

Data elements derived from ADA.

Discussion

The majority of centers surveyed are actively collecting stroke registry data on key elements such as hypertension or diabetes. However, few centers appear to have formal guidelines in place (e.g., codebook/data dictionary) to assist them with the within-site standardization of definitions for these variables. No more than three fourths of respondents were using or found any provided definition criteria to be acceptable for any of the data elements. For each data element, far more respondents were using a stated history of the data element to define its presence, than those that actually found a stated history to be an acceptable criterion. Researchers using and combining registry data may not agree on the definitions or even know the definitions used in the acquisition of the data they house in their registries.

The results of this survey suggest there is little consensus among US centers regarding the actual definition of CDEs in cardiovascular and cerebrovascular diseases. The prevalence of these data elements and association of these data elements with other patient characteristics or outcomes will be substantially affected by the definitions applied, making the interpretation of results from these registries problematic. There are various possible reasons for the lack of consensus. There may be a lack of agreement about the purpose of the registry, i.e., what question or questions it will answer, or a belief that a registry can be created that will be able to answer a wide variety of questions without compromising quality. Without a prospective and limited list of questions to answer, one can understand why it is hard to agree on how data fields are defined.

In addition, our study found that vascular neurologists are not currently using published ACC or ADA definition criteria for data elements. It is possible that stroke neurologists are not familiar with the ACC guidelines; however, our survey found that they often do not agree with the data element definitions proposed by the ACC.

The evolving acceptance of definitions by specialty organizations (e.g., ACC or ADA) is one barrier to the adoption of definitions for stroke registries. For example, in 2010 the ADA approved the use of HbA1c measurement of ≥6.5% to meet the criteria for diabetes mellitus. If stroke registries uniformly incorporate this criterion, the prevalence of diabetes mellitus may increase from rates previously demonstrated. If the criterion is not uniformly incorporated, data regarding diabetes mellitus cannot be compared nor combined from center to center. As other methods of diagnosing stroke risk factors are proposed and validated, centers using registries need to have a plan for how to adapt and incorporate new criteria. Similar to the standardized definitions that support evidence-based process measures endorsed by the National Quality Forum for adoption by the Centers for Medicare/Medicaid Services, definitions for medical diagnoses or the presence/absence of symptoms or disease characteristics are essential. Without this standardization, the validity of findings from the best process measures is ultimately threatened due to measurement error.

Our study has a number of limitations. First, the generalizability of this study is limited by its 40.1% response rate; however, our response rate is in keeping with a review of 31 e-mail surveys that reported a mean response rate of 36.8% and recent stroke surveys assessing opinions of university-affiliated stroke neurologists [8, 12, 13, 14]. While self-administered web-based surveys have been shown to result in lower response rates and more missing data, they do have the advantage of diminishing social desirability bias and response order effects. Secondly, despite our efforts to β-test survey questions prior to disseminating the survey, we cannot be certain that the responses regarding currently used and acceptable definitions of CDEs accurately represent the opinions of other neurologists within the same stroke center. For example, respondents at centers that use a prespecified protocol with standardized definitions to perform retrospective chart abstraction may have indicated that they did not currently use a registry codebook or a data dictionary, despite using standardized definitions. Further, we cannot disregard the potential influence of close-ended, leading, or loaded questions on respondents, despite our efforts to avoid double-barrel questions by separating the ‘current use’ and ‘also acceptable’ answer choices. It is possible that our target population did not represent all hospitals collecting registry level data, as many hospitals not affiliated with universities may also collect data to support their programs of research as well as stroke center certification. Lastly, we wish to point out that the results of this study should not be misconstrued to equate registries with well-designed epidemiological studies and controlled clinical trials.

To the best of our knowledge, this is the first study to provide quantitative evidence supporting the NINDS proposal for creating a library of CDEs. The implications of our findings are practical, as existing inconsistencies in definitions may substantially impair our ability to detect associations between variables, limit the ability to generalize the observational data of a single center, and confound multi-site collaboration. Given the concerted efforts by academic institutions to collect data for clinical research, standardization of data elements is of paramount importance. This study supports the argument that consensus guidelines need to be developed to outline and define key data elements, to optimize and standardize data collection for the purpose of clinical research.

Disclosure Statement

The authors have no conflicts of interest to report.

Click here for additional data file.