The Adverse Event Unit (AEU): A novel metric to measure the burden of treatment adverse events.

OBJECTIVE: To design a physician and patient derived tool, the Adverse Event Unit (AEU), akin to currency (e.g. U.S. Dollar), to improve AE burden measurement independent of any particular disease or medication class. PATIENTS/
METHODS: A Research Electronic Data Capture (REDCap) online survey was administered to United States physicians with board certification or board eligibility in general neurology, subspecialty neurology, primary care internal medicine or family medicine, subspecialty internal medicine, general pediatrics, and subspecialty pediatrics. Physicians assigned value to 73 AE categories chosen from the Common Terminology Criteria of Adverse Events (CTCAE) relevant to neurologic disorder treatments. An online forced choice survey was administered to non-physician, potential patients, through Amazon Mechanical Turk (MTurK) to weight the severity of the same AE categories. Physician and non-physician data was combined to assign value to the AEU. Surveys completed between 1/2017 and 3/2019.
RESULTS: 363 physicians rated the 73 AE categories derived from CTCAE. 660 non-physicians completed forced choice experiments comparing AEs. The AEU provides 0-10, weighted values for the AE categories studied that differ from the ordinal 1-4 CTCAE scale. For example, CTCAE severe diabetes (category 4) is assigned an AEU score of 9. Although non-physician input changed physician assigned AEU values, there was general agreement among physicians and non-physicians about severity of AEs.
CONCLUSION: The AEU has promise to be a useful, practical tool to add precision to AE burden measurement in the clinic and in comparative efficacy research with neurology patients. AEU utility will be assessed in planned comparative efficacy clinical trials.

Introduction

There is increasing emphasis on adverse event(AE) burden in neurology as new treatments are approved [1-3]. AEs cost more than $136 billion per year and add an average of 5 days to neurological hospitalizations [4-7]. AEs are important to patients and represent a barrier to treatment adherence. When structuring neurological treatment paradigms, among medications with equal efficacy, treatment decisions will be dictated by differences in AE burden, treatment burden, and cost. We remain without a practical metric to measure AEs that facilitates comparison of medications within and across different classes based on AEs alone.

The Adverse Event Unit (AEU) is a physician and patient weighted consensus unit, akin to currency (e.g. US dollar), designed to quantify and compare AE burden over time. Unlike previous measures, the AEU facilitates AE measurement independent of any disease or medication class, in terms of a number of AEUs that can be compared over time [8-13]. AEU scores can be combined with other outcome metrics and quality of life scores to better define the differences among treatments in comparative efficacy trials and in the clinic. Understanding AE tolerance in different neurological conditions and AEU validation against other disease metrics is planned for future studies. This manuscript describes the derivation of the AEU and potential applications for this new tool.

Methods

Development of the AEU was designed as a two-phase protocol to obtain input from physician experts and potential patients. In the first phase, US physicians assigned weight to the severity of AE associated with treatments for neurological illnesses. In the second phase, non-physician potential patients recruited through the Amazon Mechanical Turk (MTurk) service (https://www.mturk.com) rated the severity of the same group of AE. Data obtained from both phases was combined to generate value for the AEU. Surveys were completed between 1/2017 and 3/2019.

Standard protocol approvals, subject consent

The institutional review board (IRB) at the University of Vermont approved this protocol with a waiver of consent as all subjects were recruited anonymously through on-line surveys. Survey completion implied consent.

Physician subjects

United States physicians completed an on-line survey utilizing the secure Research Electronic Data Capture tool (REDCap) [14] hosted at University of Vermont. The target population was physicians with board certification or board eligibility in general neurology, subspecialty neurology, primary care internal medicine or family medicine, subspecialty internal medicine, general pediatrics, and subspecialty pediatrics. These specialties were chosen to capture the broad range of physicians who provide medical care for neurological patients.

Champions (MKH, TMB, DBA, KR, NK, AK, and ED) identified at US centers recruited colleagues in their communities and at other centers through a combination of targeted emails and in person meetings with groups of physicians. The American Academy of Neurology facilitated recruitment of current and previous physician recipients of the development award that supported the current study. All respondents were encouraged to forward the survey to colleagues in the aforementioned medical specialties.

Potential patient subjects

The online survey tool, MTurk, was used to recruit potential patients to represent a sample of the general population in the United States. MTurk is a viable and validated method to collect data about clinical and social science populations [15, 16]. MTurk participants produced similar results when compared to in person university recruited populations in psychological surveys, behavioral tests, matched comparison groups, economic experiments, clinical studies, and social science studies [17-20]. In general, the MTurk participants tend to be of younger age. To sample a broad age range reflective of a typical neurology patient population, we stratified the surveys into the following available age cohorts: 25–30 years, 30–35 years, 35–45 years, 45–55 years, and greater than 55 years. The MTurk tool did not permit additional age stratification in the greater than age 55 years category. Subjects were paid $5 for survey completion.

Survey design and administration (Fig 1)

Items for analysis

The investigators (TB and MH) chose 73 AE categories relevant to medications prescribed across the field of neurology from the Common Terminology Criteria of Adverse Events (CTCAE) version 4 for analysis (Appendix 1 in S1 File) [21]. The CTCAE is a physician expert derived, widely employed, ordinal [1-5], unweighted scale commissioned by the National Cancer Institute used to measure AE in many clinical trials [21]. Although AE severity increases along the CTCAE scale, items given the same value may not be of equal burden. For example, the AE of moderate hypertension (level 3), which carries the long-term risk of cardiovascular complications, is given the same score as a high fever of <24 hours duration (level 3). The CTCAE category 5 corresponding to death was not analyzed as we are interested in assigning value to AE that can be monitored over time while a patient undergoes treatment. The finite category of death can be measured independently without weighting because death from any cause is presumably of equal importance and consequence. Under the guidance of a board certified pediatrician (DA), items to measure congenital complications were adapted from the DSM-5 definitions of intellectual disability, an epilepsy research classification of congenital abnormalities, and neural tube defect classification systems [22-25].

Physician subjects

Each physician subject was asked to assign values (0 = no significance to 10 = most significant) to a random sample of 30 AEs within and across the chosen 73 CTCAE and congenital categories of varying severities. They were asked to consider each AE independent of any one disease or treatment. Subjects were also asked to factor scores they assigned both within and across AE categories as they rated AE in the survey (Appendix 2 in S1 File). A separate pediatrician survey included all the congenital malformation AE evaluated in addition to non-congenital AE. Adult physicians also rated congenital AE. Median scores with associated interquartile ranges were calculated to assign initial value to the AEU. This method of assigning weighted values to the AE categories identified from the CTCAE was adapted from method used by members of our research team (TB and MC) in the construction of the MG-Composite, a weighted, consensus, outcome measure validated for use in evaluating patients with myasthenia gravis [26].

Potential patient subjects

A subset of AE derived from the CTCAE and weighted by the physicians in phase 1 were converted into lay descriptions informed by Mayo Clinic descriptions of symptoms and medical conditions (https://www.mayoclinic.org/symptoms; https://www.mayoclinic.org/diseases-conditions). Potential patient friendly versions of the CTCAE have been employed in other studies [27].

Potential patient subjects reviewed pairs of AE descriptions (Table 1 and Appendix 3 in S1 File) assigned different AEU values by the physician subjects. AE pairs to review were randomly computer generated so that the compared AEs were from different AE categories and had been assigned different scores by the physician subjects. In the style of a discrete choice experiment, subjects were asked “After reviewing each pair of AE, please choose which of the two AE would be least tolerable (i.e. the most severe of the pair)” [28, 29]. They were also asked to consider: impact on quality of life (QOL), impact on life expectancy, future medical complication risk, likelihood for AE resolution following therapy change, and other factors considered important to the subject. Potential patients were not told how the physicians weighted the AE being evaluated.

Table 1

Example of potential patient discrete choice.

Instructions:

In this survey, you will review information about potential medication associated adverse events. You will be provided with a description of pairs of potential medication associated adverse events. Consider the side effects alone without thinking of any particular medication or disease.

After reviewing each pair of adverse events, please choose which of the two adverse events would be least tolerable (i.e. the most severe of the pair).

Consider the following in making your decisions:

1. Impact of the side effect on your quality of life

2. Impact on life expectancy

3. Risk to develop future medical complications because of this adverse event

4. How likely it might be the side effect will go away if medication is stopped

5. Other factors of importance to you

Deep Vein Thrombosis (DVT):

Deep vein thrombosis (DVT) occurs when a blood clot forms in one or more of the deep veins in your body, usually in your legs. Deep vein thrombosis can cause leg pain or swelling, but also can occur with no symptoms. Deep vein thrombosis can be very serious because blood clots in your veins can break loose and lodge in your lungs, blocking blood flow (pulmonary embolism). Treatment of DVT includes anticoagulant medications (blood thinners) and in severe circumstances, placement of a filter in your blood vessels or treatment with a clot busting medication. Blood thinner treatment increases the risk for bleeding.

Complication: (AEU 7) **

You have developed a DVT in your leg as a result of medical treatment. No complications have occurred with this DVT, such as pulmonary embolism. You require treatment with a blood thinner for at least a few months. You may have been admitted to the hospital for a short time due to this issue.

VS

Headache

Headaches may include syndromes that cause discomfort on the head including throbbing pain, stabbing pain, and numbness. Severe headaches may result in impaired physical and cognitive function. Severe headaches can impair daily function and may require treatment with medications. Drug induced headaches are likely to improve with stopping an offending medication.

Complication: (AEU 5) **

You have developed a severe headache that limits routine daily activities and self-care as a result of medical therapy. This headache lasts less than 1 week, may require a short course of pain medication (such as ibuprofen), and improves with discontinuing the offending medication. No ongoing medical therapy is required.

** Potential patient subjects were not shown the AEU value assigned by the physicians during the experiment. They are presented to illustrate that these AEs were given different weights by the physician subjects.

Combining physician and potential patient data

Bradley-Terry models were fit to the choices made by the potential patients using Firth’s method for penalized maximum likelihood logistic regression in SAS version 9.4 [30, 31]. The Bradley-Terry model uses the paired comparisons obtained through Mturk to estimate a set of ‘less preferred’ parameters for the AE. These parameters have the property that, if an AE with parameter A is compared to a second AE with parameter B, we would estimate that a proportion A/ (A+B) of potential patients would choose the first AE as less tolerable. Once ‘less preferred’ parameters from potential patient choices were estimated, we created integer scores by applying K-means clustering to the less preferred estimates, setting the number of clusters equal to 9 to match the range of integer scores provided by the physicians (2 to 10). Final adjustment to AEU scores was done by comparing the physician and the potential patient AEU scores. If the potential patient AEU score was greater than the physician assigned AEU, we increased the physician AEU across an entire AE category (e.g. hypertension) rating by 1. If the potential patient AEU score was less than the physician assigned AEU, we decreased the physician AEU across an entire AE category rating by 1. This method of combining physician and potential patients AEU scores was chosen to give weight to the expertise of physicians in understanding the overall short and long-term sequelae of the rated AEs. It is essential to arrive at single AEU scale to achieve the ultimate goal of developing a combined, easy to administer, best fit, weighted, consensus unit that would be feasible to administer in a clinical practice setting or clinical trial.

Results

The targeted medical specialties were well represented (Table 2). The group was experienced and covered a wide geographic region. Recruited physicians had male and academic practice predominance. Primary care physicians practicing through university associated medical centers largely self-identified as in academic practice.

Table 2

Baseline characteristics.

Variable	Potential Patients	Physicians
		(n = 363)
	(n = 660)
Sex, % (n) female	49% (325/660)	36% (129)
Age years, % (n)		NA
25–30 years	13% (79)
31–35 years	10% (63)
36–45 years	14% (87)
46–55 years	11% (67)
> 55 years	52% (314)
Median years practice (range)	NA	12 (1–50)
Academic Practice % (n)	NA	86% (312)
Education, % (n)
Grade 11 or below	1% (3)	NA
Completed Grade 12	36% (239)	NA
College/Above	63% (418)	NA
Ethnicity, % (n)
Caucasian American	85% (547)	Did not ask
African American	8% (49)
Hispanic American	4% (26)
Asian American	3% (17)
Native American	1% (3)
Geographic Region United States, % (n)
Northeast	22% (147)	31% (112)
Southeast	32% (210)	42% (152)
Midwest	18% (120)	12% (44)
Southwest	11% (73)	7% (26)
West	17% (109)	8% (30)
Medical Specialty, % (n)
General Neurology	NA	16% (58)
Subspecialty Neurology	NA	43% (156)
Adult Primary Care	NA	17% (62)
Subspecialty Internal Medicine	NA	7% (25)
Pediatrics	NA	13% (47)
Other	NA	4% (15)

Potential patient numbers may not add to 660 because of missing values.

The potential patient cohort represented the wide range of ages typical of neurology patients (Table 2). The variables of geographic regions in the U.S. and sex were equally represented. Potential patients with college level of education or above were overrepresented. In addition, African Americans, Hispanic Americans, and Asian Americans were slightly underrepresented when compared to most recent U.S. Census data [32].

Phase 1: Physician weighting

Three hundred sixty three physicians provided data from 397 surveys; 34 physicians completed two different surveys with different sets of AE. On a 0–10 scale (0 = no importance and 10 = maximal importance), physician responses ranged from 2–10 across the 73 AE categories evaluated. Median values with interquartile ranges are available in Appendix 1 in S1 File. In many circumstances, the weighted values provided by the physicians did not match the rigid 1–4 ordinal CTCAE scale. For example, the CTCAE category 1, corresponding to a mild AE for pulmonary fibrosis, received an AEU score of 6. The CTCAE category 4, corresponding to a severe AE for diabetes, received an AEU value of 9. In contrast, the severe CTCAE category 4 for headache received an AEU value of 6, similar to the AEU values assigned for CTCAE category 2 diabetes and CTCAE category 1 for pulmonary fibrosis.

Phase 2: Potential patient forced choice

Each of 660 MTurk potential patient raters made 20 random paired AE discrete choice comparisons. Two sets of comparisons, presented to 20 participants each, could not be used because the computer randomly assigned items with same initial physician derived AEU score. These two sets of comparisons did not allow the participants to distinguish the choices, leaving 11,463 comparisons for analysis. All 73 AE categories were used in at least one paired comparison. Appendix Table 4 in S1 File provides estimates and standard errors for the logistic regression parameters estimated using Firth’s method as well as a calibration plot. The model has excellent discrimination (c-index = 0.866) and calibration. Appendix 5 in S1 File shows the results of the K-means clustering used to assign integer scores to the preference parameters. Subsequent analyses adjusted the preference parameters for demographic characteristics, age, sex, race/ethnicity, education and region of the country and of the mTurk respondents, but none of the characteristics were statistically significant, and more importantly, did not change the final ratings. Adjustment for the demographic characteristics altered the final rating in only 3 of the 73 items, and never by more than 1 point. Given the additional complexity in interpreting the results with additional covariates, we present the final ratings based on the model without adjusting for demographic characteristics.

The AE evaluated by the potential patients ranged from 2–10 AEU points on the scale generated by the physicians and the Bradley Terry method (Table 3). Severity choice values ranged from 0.33 for a mild degree of diarrhea (physician AEU 3) to 8.5 for treatment related malignancy (physician AEU 9).

Table 3

Combined physician and potential patient AEU values.

	Original AEU from physicians	AEU from potential patients’ choices	Final AEU rating
Diabetes	6	8	7
Osteoporosis	4	5	5
Weight Gain 6	6	4	5
Cognitive Impairment	4	4	4
Seizure	6	7	7
Heart Attack	9	9	9
Deep Vein Thrombosis (DVT)	7	7	7
Headache	5	3	4
Hallucinations 6	6	5	5
Treatment Related Malignancy (Cancer)	9	9	9
Myalgia (Muscle Pain) 6	6	2	5
Itchy Skin (Pruritus)	6	3	5
Low Platelets	5	4	4
Low Blood Sodium (Hyponatremia)	9	6	8
Flu-Like Symptoms	2	2	2
Abnormal Movements (Dyskinesia) 4	4	5	5
Kidney Stones	7	7	7
Diarrhea	4	2	3
Pulmonary Fibrosis 9	9	9	9
Hypertension (High Blood Pressure)	4	5	5
Liver Disease	3	7	4
Kidney Failure 8	8	8	8
Infection	8	6	7
Weight Loss	8	5	7
Dizziness (Vertigo)	3	3	3
Hair Loss	5	4	4
Headache 2	2	2	2
Congestive Heart Failure	5	8	6
Stroke	10	10	10
Cataracts	5	4	4
Dry Eyes	4	3	3
Diarrhea 7	7	4	6
Suicidal Thoughts	5	7	6
Dizziness (Vertigo)	5	5	5
Pancreatitis	8	8	8
Anemia (Low Red Blood Cells)	4	4	4
INR Elevation	3	4	4
Low White Blood Cells (Neutropenia)	8	6	7
Leg and Arm Swelling (Edema)	3	4	4
Constipation	6	4	5
Headache 6	6	4	5
Glaucoma 9	9	7	8
Nausea	6	3	5
Trouble Walking	5	4	4
Allergic Reaction	5	4	4
Avascular Necrosis of a Joint	8	8	8
Glaucoma 8	8	7	7
Osteoporosis	7	6	6
Stroke 6	6	7	7
Stomach Ulcer	4	5	5
Pulmonary Fibrosis 6	6	8	7
Weight Gain 3	3	3	3
Diarrhea 3	3	2	2
Hallucinations 5	5	5	5
Abnormal Movements (Dyskinesia) 6	6	7	7
Heart Rhythm Disorder (Arrhythmia)	4	5	5
Hypertension (High Blood Pressure)	5	6	6
Kidney Failure 5	5	6	6
Myalgia (Muscle Pain) 3	3	2	2
Depression	6	5	5
Fever	6	2	5
Sexual Dysfunction	6	2	5
Anxiety	5	4	4
Mania	8	6	7
Insomnia	3	3	3
Fatigue	3	2	2
Vascular Access Complications	9	8	8
Teratogen 1	10	10	10
Teratogen 2	2	6	3
Reproductive Dysfunction	4	3	3
Urine Retention	4	3	3
Heart Failure	5	9	6
Neutropenia	8	6	7

Phase 3: Combining physician and potential patient values

Final physician and potential patient combined AEU scores are presented in Table 4. Fifty-five of the 73 items were adjusted from the originally assigned physician scores to reflect input from the potential patients (Table 3). In three categories (hallucinations, dyskinesia, and thrombosis), the physician assigned AEU value of a more severe adverse event in a category had a lower score than the immediately preceding AE. For example, moderate hallucinations had an AEU score of 6 and Severe Hallucinations/Medical Intervention Indicated (Hospitalization Not Indicated) had an AEU score of 5. This likely occurred as physicians were not shown the full range of side effects in each category when assigning scores. We did not show physicians all the AEs in a category to reduce survey burden and to prevent bias from being shown a group of side effects in a previously determined, ordinal fashion. In these three circumstances after discussion among the investigators, the decision was made to rate both categories with the higher AEU score prior to obtaining potential patient input; e.g. both hallucination categories in question have a final AEU score of 6. This decision is also supported by overlap in the interquartile range of these categories in the original physician subject weighted values.

Table 4

Final values Adverse Event Unit (AEU).

Adverse Event	CTCAE 1	CTCAE 2	CTCAE 3	CTCAE 4
Chronic Illnesses
Diabetes	Pre-Diabetes:Hemoglobin A1C: 5.7–6.4% orFasting Glucose: 100–125mg/dL or 2H Oral Gluc Tolerance Test: 140–199 mg/dL4	DiabetesHemoglobin A1C > 6.4% orFasting Glucose > 125mg/dL or 2H Oral Gluc Tolerance Test > 200mg/dL Medical Intervention May Be Indicated 6	DiabetesHemoglobin A1C > 6.4% orFasting Glucose > 125mg/dL or 2H Oral Gluc Tolerance Test > 200mg/dL ANDRequiring multiple medication and/or medication escalation 7	Acute Life Threatening DiabetesHospitalization (eg Ketoacidosis) 10
Pulmonary Fibrosis	Drug Related Pulmonary FibrosisMild Hypoxemia with Radiologic Pulmonary Fibrosis < 25% lung volume 6	Drug Related Pulmonary FibrosisModerate Hypoxemia with Evidence of Pulmonary Hypertension ORRadiographic Evidence of Pulmonary Fibrosis 25–50% Lung Volume 7	Drug Related Pulmonary FibrosisSevere HypoxemiaEvidence of Right Side Heart Failure OR Radiographic Pulmonary Fibrosis > 50–75% Lung Volume 9	Drug Related Pulmonary FibrosisLife Threatening Consequences (e.g. hemodynamic complications)ANDIntubation with Ventilatory Support 10
Hypertension	Pre-HypertensionSystolic Blood Pressure: 120-139 orDiastolic Blood Pressure: 80-893	HypertensionSystolic Blood Pressure: 140-159 orDiastolic Blood Pressure: 90-99 ORIncrease or Start Anti-hypertension medications 5	HypertensionSystolic Blood Pressure: > 160 orDiastolic Blood Pressure: > 100 6	Life Threatening HypertensionHospitalization for Hypertensive Urgency/Emergency 10
Respiratory and Thoracic	Respiratory, Thoracic, Mediastinal Not Otherwise SpecifiedAsymptomatic or Mild SymptomsANDClinical or Diagnostic Observations Only3	Respiratory, Thoracic, Mediastinal Not Otherwise SpecifiedModerate, Minimal, Local, or Non-invasive Intervention IndicatedANDLimiting Age Appropriate Activities Daily Living4	Respiratory, Thoracic, Mediastinal Not Otherwise SpecifiedSevere or Medically Significant but Not Life ThreateningANDHospitalization or Prolong Existing Hospitalization Disabling8	Respiratory, Thoracic, Mediastinal Not Otherwise SpecifiedLife Threatening ConsequencesANDUrgent Intervention Indicated9
Osteoporosis	OsteoporosisRadiologic evidence of osteoporosis orBone Mineral Density t score -1 to -2.5 (osteopenia)ANDNo intervention indicated/No loss of height5	OsteoporosisBone Mineral Density t score < -2.5 ANDAnti-osteoporotic treatment indicatedLimiting Activities Daily Living5	OsteoporosisRadiographic osteoporosis ANDComplication not requiring hospitalization (eg fracture)7	OsteoporosisRadiographic osteoporosis ANDHospitalization Indicated (eg hip fracture)8
GI Ulcer	Gastric or Duodenal UlcerAsymptomaticANDDiagnostic Observations OnlyIntervention Not Indicated5	Gastric or Duodenal UlcerSymptomatic ANDAltered GI FunctionMedical Intervention Indicated6	Gastric or Duodenal UlcerSeverely Altered GI FunctionANDTPN Indicated ORElective Operative or Endoscopic Intervention Indicated9	Gastric or Duodenal UlcerLife-threatening ComplicationsANDUrgent Operative Intervention Indicated10
Endocrine	Endocrine DisordersAsymptomatic or Mild Symptoms ANDClinical or Diagnostic Observations Only3	Endocrine DisordersModerate Symptoms ANDMinimal, Local, or Non-invasive Intervention Only3	Endocrine DisordersSevere or Medically Significant but Not Immediately Life-threateningANDHospitalization or Prolongation of Existing Hospitalization8	Endocrine DisordersLife-threatening Consequences ANDUrgent Intervention Indicated9
Secondary Malignancy		Treatment Related Secondary MalignancyNon-Life Threatening Secondary Malignancy7	Treatment Related Secondary MalignancyChronic Life Threatening Secondary MalignancyANDShortens Life Expectancy(e.g. Metastatic Disease)9	Treatment Related Secondary MalignancyAcute Life Threatening Secondary Malignancy(e.g. blast crisis)10
Renal Fail	Kidney InjuryCreatinine level increase of > .3mg/dL ORCreatinine 1.5–2 times above baseline5	Kidney InjuryCreatinine 2–3 times above baseline5	Kidney InjuryCreatinine 3 times baseline or > 4mg/dL ANDHospitalization Indicated8	Kidney InjuryLife-threatening consequences AND Hospitalization, dialysis, or transplant indicated10
Congestive Heart Fail	Heart FailureAsymptomatic with lab (e.g. BNP [B-natiuretic peptide]) ORCardiac Imaging Abnormalities6	Heart FailureSymptoms with mild to moderate activity or exertion6	Heart FailureSevere with symptoms at rest or with minimal activity or exertion ANDIntervention indicated10	Heart FailureLife-threatening consequences ANDUrgent intervention indicated(e.g. continuous IV medications, mechanical hemodynamic support10
Cardiac Arrhythmia	Atrial or Ventricular Arrhythmia Asymptomatic intervention not indicated5	Atrial or Ventricular Arrhythmia ANDNon-urgent medical intervention indicated7	Atrial or Ventricular ArrhythmiaAND Acute Medical Intervention Indicated9	Atrial or Ventricular Arrhythmia Life-threatening ConsequencesHemodynamic Compromise AND Hospitalization and Urgent Intervention Indicated10
Cognitive Dysfunction	Encephalopathy or Cognitive DysfunctionMild SymptomsANDNot Interfering with Work/School/Life Performance4	Encephalopathy or Cognitive DysfunctionModerate SymptomsANDInterfering with Work/School/Life Performance But Capable of Independent Living6	Encephalopathy or Cognitive DysfunctionSevere SymptomsANDImpairing Work/School/Life Performance9	Encephalopathy or Cognitive DysfunctionLife-Threatening ConsequencesANDUrgent Intervention or Hospitalization Indicated9
Hepatic Dysfunction	Functional Hepatic ImpairmentAsymptomatic or Mild SymptomsANDClinical or Diagnostic Observation Only4	Functional Hepatic ImpairmentModerate SymptomsANDMild, Local, or Non-Invasive Intervention6	Functional Hepatic ImpairmentSevere or Medically Significant but Not Immediately Life-Threatening(e.g. Mild Encephalopathy)ANDHospitalization or Prolongation of Existing Hospitalization Indicated9	Functional Hepatic ImpairmentLife-Threatening Consequences(e.g. Moderate to Severe Encephalopathy, Coma, Hemorrhage) ANDHepatic Transplant and Urgent Hospitalization Indicated10
Seizures	Drug Related SeizureBrief Partial SeizureNo Loss of Consciousness6	Drug Related SeizureBrief Generalized Seizure withLoss of Consciousness7	Drug Related SeizureMultiple Seizures Despite Medical Intervention10	Drug Related SeizureLife-Threatening Prolonged Repetitive Seizures (Status Epilepticus)ANDRequiring Hospitalization and Urgent Intervention10
Glaucoma	GlaucomaElevated Intraocular Pressure (EIOP) without Visual Field DeficitsANDSingle Topical Agent Indicated3	GlaucomaElevated Intraocular Pressure (EIOP) with Early Visual Field DeficitANDMultiple Topical Agents and/or Oral Agent Indicated5	GlaucomaElevated Intraocular Pressure (EIOP) with Marked Visual Field DeficitANDOperative Intervention Indicated7	GlaucomaBlindness in Affected Eye (20/200 or Worse)8
Cataracts	CataractsAsymptomatic ANDClinical or Diagnostic Observation Only2	CataractsSymptomatic ANDModerate Decrease in Visual Acuity (20/40 or better)4	CataractsSymptomatic with Marked Decrease Visual Acuity (worse than 20/40 but better than 20/200)ANDOperative Intervention Indicated (e.g. cataract surgery)6	CataractsBlindness (20/200 or worse) in affected eye7
Acute Coronary Syndrome	Acute Coronary SyndromeSymptomatic Progressive Angina ANDCardiac Enzymes NormalHemodynamically Stable7	Acute Coronary SyndromeSymptomatic Unstable Angina OR Acute Myocardial InfarctionANDCardiac Enzymes AbnormalHemodynamically Stable8	Acute Coronary SyndromeSymptomatic Unstable Angina OR Acute Myocardial InfarctionANDCardiac Enzymes AbnormalHemodynamically Unstable9	Acute Coronary SyndromeLife Threatening ConsequencesHemodynamically UnstableANDICU Level Care Indicated10
Stroke	StrokeAsymptomatic or Mild Neurologic DeficitRadiographic Findings Only7	StrokeModerate Neurologic Deficit9	StrokeSevere Neurologic DeficitProlonged Hospitalization AND/OR Requires Care in Long-term Facility10	StrokeLife-Threatening ConsequencesIf Survive Requires Prolonged Use of Tracheostomy AND/OR Percutaneous Gastrostomy TubeRequires Care in Long-Term Facility10
Anxiety	Anxiety Mild symptoms; intervention not indicated 1	Anxiety Moderate symptoms; limiting instrumental ADL 4	Anxiety Severe symptoms; limiting self-care ADL; hospitalization not indicated 6	Anxiety Life-threatening; hospitalization indicated 8
Depression	Depression Mild depressive symptoms 2	Depression Moderate depressive symptoms; limiting instrumental ADL 5	Depression Severe depressive symptoms; limiting self-care ADL; hospitalization not indicated 6	DepressionLife-threatening consequences, threats of harm to self or others; hospitalization indicated8
Mania	ManiaMild manic symptoms (e.g., elevated mood, rapid thoughts, rapid speech, decreased need for sleep)4	ManiaModerate manic symptoms (e.g., relationship and work difficulties; poor hygiene)6	Mania Severe manic symptoms (e.g., hypomania; major sexual or financial indiscretions); hospitalization not indicated9	ManiaLife-threatening consequences, threats of harm to self or others; hospitalization indicated9
Adverse Events Typically Shorter Duration
Fever	Drug FeverTemperature 38–39C (100.4–102.2F) < 24Hours2	Drug FeverTemperature 39–40C (102.2–104F) < 24 Hours 2	Drug FeverTemperature > 40C for < 24Hours 5	Drug FeverTemperature > 40C for > 24Hours 5
Headache	HeadacheMild Pain < 1 week 2	HeadacheModerate Pain ANDLimits Activities Daily Living< 1 week2	HeadacheSevere Pain ANDLimits Self Care and Activities Daily Living < 1 week 4	Chronic HeadacheModerate to Severe PainDuration > 1 week5
Infection	InfectionAsymptomatic or Mild SymptomsDiagnostic intervention onlyMedical intervention not indicated 1	InfectionSymptomatic ANDMinimal, local, or non-invasive intervention indicated 2	InfectionSevere of medically significant but not immediately life threatening ANDHospitalization or prolongation existing hospitalization 7	InfectionLife threatening consequences ANDUrgent hospitalization or ICU level care indicated9
Weight Gain	Weight Gain5-10% increase from baseline3	Weight Gain10-20% increase from baseline4	Weight Gain> 20% increase from baseline6
Weight Loss	Weight Loss5-10% Decrease from Baseline1	Weight Loss10-20% Decrease from Baseline ANDRequires Nutritional Support4	Weight Loss> 20% Decrease from Baseline ANDTube feed or TPN Indicated7
Thrombosis	Venous Thrombosis(e.g. superficial thrombosis)3	Venous ThrombosisUncomplicated deep vein thrombosisANDMedical Intervention Indicated7	Uncomplicated Pulmonary Embolism or Non-embolic Cardiac Mural ThrombusANDMedical Intervention Indicated.7	Life Threatening Thrombotic Event(e.g. Complicated Pulmonary Embolism, Arterial Insufficiency)Hemodynamic InstabilityANDUrgent Intervention Indicated10
Avascular Necrosis	Avascular NecrosisAsymptomatic Clinical or Diagnostic Interventions Only4	Avascular NecrosisSymptomatic Limiting Instrumental Activities Daily Living6	Avascular NecrosisSevere SymptomsLimiting Self Care Activities Daily LivingANDOperative Intervention Indicated8	Avascular NecrosisLife Threatening ConsequencesANDUrgent Intervention Indicated9
Hair Loss	AlopeciaHair Loss < 50% of Normal for IndividualOnly Noticeable from Close InspectionDifferent Hair Style but Doesn’t Require Wig to Camouflage3	AlopeciaHair Loss >/= 50% of Normal for IndividualMay be associated with Psychosocial Impact4
Diarrhea	DiarrheaIncrease < 4 Stools Per Day Over Baseline ORMild Increase in Ostomy Output Compared to Baseline2	DiarrheaIncrease 4–6 Stools Per Day Over BaselineModerate Increase in Ostomy Output Compared to BaselineANDLimiting Instrumental Activities Daily Living3	DiarrheaIncrease < 7 Stools Per Day Over BaselineIncontinence ORSevere Increase in Ostomy Output Compared to BaselineANDHospitalization Indicated6	DiarrheaLife-threatening Complications ANDHospitalization Indicated8
Nausea	NauseaLoss of Appetite Without Alteration in Eating Habits1	NauseaOral Intake Decreased ANDWithout Weight Loss, Dehydration, or Malnutrition2	NauseaVomiting or Anti-Emetics Required5	NauseaInadequate Oral Caloric or Fluid IntakeORRefractory Vomiting with Tube Feed, TPN7
Vertigo	VertigoMild symptoms3	VertigoModerate Symptoms ANDLimiting Instrumental Activities Daily Living5	VertigoSevere Symptoms ANDLimiting Self Care7
Vascular Access	Vascular AccessDevice Dislodgement, Blockage, Leak, MalpositionANDDevice Replacement Indicated3	Vascular AccessDeep Vein or Cardiac ThrombosisANDIntervention Indicated (e.g. anticoagulation, lysis, filter, invasive procedure)6	Vascular AccessEmbolic Event Related to Vascular Access(e.g. pulmonary embolism or life threatening thrombus)8
Suicidal Ideation	Suicidal IdeationIncreased Thoughts of Death But No Wish to Kill Oneself7	Suicidal IdeationSuicidal Ideation with No Specific Plan or Intent7	Suicidal IdeationSpecific Plan to Commit Suicide ORSuicide Attempt without Serious Intent to DieMay Not Require Hospitalization10	Suicidal IdeationSuicide Attempt with Intent to Die ORSpecific Plan to Commit Suicide with Serious Intent to Die Requires Hospitalization10
Hallucinations	HallucinationsMild Hallucinations (e.g. perceptual distortions)5	HallucinationsModerate Hallucinations7	HallucinationsSevere Hallucinations ANDMedical Intervention IndicatedHospitalization Not Indicated7	HallucinationsLife-threatening ComplicationsThreats of Harm to Self or OthersANDHospitalization Indicated9
SICCA	Dry MouthSymptomatic (e.g. dry thick saliva) ANDWithout Significant Dietary Alteration1	Dry MouthModerate SymptomsAND Oral Intake Alterations(e.g. copious water, other lubricants) ORDiet Limited to Purees3	Dry MouthInadequate Oral IntakeANDTube Feeds, TPN Indicated7
Cushingoid	CushingoidMild Symptoms ANDIntervention Not Indicated3	CushingoidModerate Symptoms ANDMedical Intervention Indicated6	CushingoidSevere Symptoms ANDMedical Intervention or Hospitalization Indicated7
Myalgia	Drug Related MyalgiaMild Pain1	Drug Related MyalgiaModerate Pain ANDLimiting Instrumental Activities Daily Living5	Drug Related MyalgiaSevere Pain ANDLimiting Self Care Activities Daily Living5
Pruritus	Pruritus (Itching)Mild or Localized ANDTopical Intervention Indicated1	Pruritus (Itching)Intense or Widespread Intermittent ORSkin Changes from ScratchingANDOral Intervention Indicated3	Pruritus (Itching)Intense or Widespread, ConstantANDOral Corticosteroid or Immunosuppressive Therapy Indicated5
Dermatologic	Skin DisordersAsymptomatic or Mild SymptomsClinical or Diagnostic Observations Only1	Skin DisordersModerate, Minimal, Local, or Non-Invasive Intervention Indicated 2	Skin DisordersSevere or Medically Significant but Not Immediately Life ThreateningANDHospitalization or Prolongation of Existing Hospitalization6	Skin DisordersLife-threatening Consequences ANDUrgent Intervention Indicated7
Dry Eye	Dry EyeMild symptoms relieved by lubricants1	Dry EyeMultiple agents indicated to relieve symptomsANDLimiting instrumental activities of daily living3	Dry EyeDecrease in visual acuity (worse than 20/40)ANDLimiting self-care activities of daily living5
Constipation	Drug Related ConstipationNew mild symptomsOccasional use of stool softeners, laxatives, dietary modification, or enema.1	Drug Related ConstipationPersistent symptoms with regular use of laxatives or enemasANDLimiting instrumental activities of daily living3	Drug Related ConstipationObstipation with manual evacuation indicatedANDLimiting self-care activities of daily living5	Drug Related ConstipationLife threatening consequences ANDHospitalization indicated8
Infusion Site Reaction	Injection Site ReactionTenderness with or without associated symptoms(e.g. warmth, erythema, itching)2	Injection Site ReactionPain, Lipodystrophy, Edema, Phlebitis4	Injection Site ReactionUlceration or Necrosis with Severe Tissue DamageANDOperative Intervention Indicated8	Injection Site ReactionLife-threatening consequencesANDUrgent intervention indicated9
Falls	Drug Related FallMinor with no resultant injuriesIntervention not indicated2	Drug Related FallSymptomatic ANDNon-invasive Intervention Indicated4	Drug Related FallHospitalization Indicated6
Allergic Reaction	Allergic ReactionTransient Flushing ANDNo Intervention Indicated1	Allergic ReactionIntervention or Infusion IndicatedANDResponds Quickly to MedicationsProphylaxis < 24 Hours4	Allergic ReactionProlonged (Not rapidly responsive to medical intervention)ANDRecurrence of Symptoms Following Medical TreatmentHospitalization Indicated5	Allergic ReactionLife Threatening ConsequencesANDUrgent ICU Level Care Indicated(e.g. Stevens Johnson Syndrome, Anaphylaxis, Angioedema)9
Sexual Dysfunction	Sexual DysfunctionMild Sexual Dysfunction Not Adversely Affecting Relationship2	Sexual DysfunctionModerate Sexual Dysfunction ANDAdversely Affecting Relationship5	Sexual DysfunctionSevere Increase in Sexual Interest Leading to Dangerous Behavior7
Edema	Edema LimbsAsymptomatic or Mild SymptomsClinical or Diagnostic Observations Only3	Edema LimbsModerate Symptoms ANDMinimal, Local, or Non-Invasive Intervention Indicated4	Edema LimbsSevere or Medically Significant but Not Life-ThreateningANDHospitalization or Prolongation Existing Hospitalization Indicated8
Fatigue	FatigueFatigue Relieved By Rest2	FatigueFatigue Not Relieved by Rest2	FatigueFatigue Not Relieved by RestANDLimiting Self Care Activities Daily Living5
Anemia	AnemiaHemoglobin (Hbg) < Lower Limit Normal - 10 g/dL ORHgb < Lower Limits Normal - 6.2mmol/L ORHgb < Lower Limits Normal - 100g/L3	AnemiaHemoglobin (Hbg) < 10- 8 g/dL ORHgb < 6.2–4.9 mmol/L ORHgb < 100–80g/L4	AnemiaHemoglobin (Hbg) < 8 g/dL ORHgb < 4.9 mmol/L ORHgb < 80 g/LANDTransfusion Indicated8	AnemiaLife-Threatening Consequences ANDUrgent Intervention Indicated9
DIC	Disseminated Intravascular CoagulationLab Findings with No Bleeding5	Disseminated Intravascular CoagulationLab Findings with Bleeding9	Disseminated Intravascular CoagulationLife-Threatening ConsequencesANDHospitalization and Urgent Intervention Indicated10
Dyskinesia	DyskinesiaMild Restlessness or Increased Motor Activity5	DyskinesiaModerate Restlessness or Increased Motor ActivityANDLimiting Instrumental Activities Daily Living8	DyskinesiaSevere Restlessness or Increased Motor ActivityANDLimiting Self Care Activities Daily Living8
Kidney Stones	Renal Calculi (Kidney Stones)Asymptomatic of Mild Symptoms ANDOccasional Use of Non-Prescription Agents3	Renal Calculi (Kidney Stones)SymptomaticANDOral Anti-emetics ORAround the Clock Non-Prescription Analgesics or Any Oral Narcotic6	Renal Calculi (Kidney Stones)Hospitalization IndicatedANDIV InterventionElective Endoscopic or Radiographic Intervention Indicated7	Renal Calculi (Kidney Stones)Life-Threatening ComplicationsANDUrgent Endoscopic or Operative Intervention and Hospitalization Indicated9
Insomnia	InsomniaMild Difficulty Falling Asleep, Staying Asleep, or Waking Up Early3	InsomniaModerate Difficulty Falling Asleep, Staying Asleep, or Waking Up Early3	InsomniaSevere Difficulty Falling Asleep, Staying Asleep, or Waking Up Early6
Pancreatitis	PancreatitisEnzyme Elevation or Radiologic Findings Only3	PancreatitisSevere Pain, VomitingANDMedical Intervention Indicated (e.g. analgesia, nutritional support)8	PancreatitisLife-Threatening ConsequencesANDHospitalization and Urgent Intervention Indicated9
Flu Reaction	Flu Like SymptomsMild Flu-Like Symptoms2	Flu Like SymptomsModerate Flu-Like Symptoms > 1 day2	Flu Like SymptomsSevere Flu-Like Symptoms > 1 Day ANDLimiting Self Care Activities Daily Living5
Gait Dysfunction	Gait DisturbanceMild Change in Gait(e.g. wide based, limping, or hobbling)4	Gait DisturbanceModerate Change in Gait(e.g. wide based, limping, or hobbling)ANDAssistive Device Indicated4	Gait DisturbanceSevere Change in GaitANDDisabling Requires Wheelchair8
Febrile Neutropenia	Febrile NeutropeniaANC < 1000/mm3 with Single Temp > 38.3C (101F) ORSustained Temp >/= 38C (100.4) for more than 1 Hour6	Febrile NeutropeniaLife-Threatening Consequences ANDHospitalization and Urgent Intervention Indicated8
Laboratory Abnormalities
INR Elevation	INR IncreaseINR > 1–1.5 x Upper Limit Normal ORINR > 1–1.5 x Above Baseline if on Anticoagulation4	INR IncreaseINR > 1.5–2.5 x Upper Limit Normal ORINR > 1.5–2.5 x Above Baseline if on Anticoagulation6	INR IncreaseINR > 2.5 x Upper Limit Normal ORINR > 2.5 x Above Baseline if on Anticoagulation8
ALT/AST Elevation	ALT or AST ElevationLab 2–3 x Upper Limit Normal3	ALT or AST ElevationLab 3–5 x Upper Limit Normal4	ALT or AST ElevationLab 5 -20 x Upper Limit Normal7	ALT or AST ElevationLab > 20 x Upper Limit Normal8
Neutropenia	Neutrophil Count ReducedANC < Lower Limit Normal - 1500/mm3 ORANC < Lower Limit Normal - 1.5 x 10e9/L3	Neutrophil Count ReducedANC < 1500–1000/mm3 ORANC < 1.5–1 x 10e9/L4	Neutrophil Count ReducedANC < 1000–500/mm3 ORANC < 1–0.5 x 10e9/L7	Neutrophil Count ReducedANC < 500/mm3 ORANC < 0.5 x 10e9/L7
Low Platelets	Platelet Count ReducedPlatelets < Lower Limit Normal - 75,000/mm3 ORPlatelets < Lower Limit Normal - 75 x 10e9/L4	Platelet Count ReducedPlatelets < 75,000–50,000/mm3 ORPlatelets < 75–50 x 10e9/L6	Platelet Count ReducedPlatelets < 50,000–25,000/mm3 ORPlatelets < 50–25 x 10e9/L6	Platelet Count ReducedPlatelets < 25,000/mm3 ORPlatelets < 25 x 10e9/L7
Hypernatremia	HypernatremiaNa > Upper Limit Normal - 150 mmol/L3	HypernatremiaNa > 150–155 mmol/L3	HypernatremiaNa > 155–160 mmol/L AND Hospitalization Indicated7	HypernatremiaNa > 160 mmol/L and Life Threatening ConsequencesHospitalization Indicated7
Hyponatremia	HyponatremiaNa < 130 mmol/L - Lower Limit Normal2	HyponatremiaNa < 120–130 mmol/L4	HyponatremiaNa < 120 mmol/LLife Threatening Complications8
Hypokalemia	HypokalemiaK < 3 mmol/L - Lower Limit Normal3	HypokalemiaK < 3 mmol/L - Lower Limit Normal ANDSymptomatic Intervention Indicated5	HypokalemiaK < 2.5–3 mmol/L AND Hospitalization Indicated5	HypokalemiaK < 2.5 mmol/L with Life Threatening ConsequencesANDUrgent Hospitalization8
Hyperkalemia	HyperkalemiaK > Upper Limits Normal - 5.5 mmol/L2	HyperkalemiaK > 5.5–6 mmol/L5	HyperkalemiaK > 6–7 mmol/L5	HyperkalemiaK > 7 mmol/L8

Discussion

In the age of precision medicine, well-designed, practical outcome measures and decision support tools expand the data we track about patients to better inform medical decisions [33, 34]. Unlike previous measures, the physician and potential patient derived AEU quantifies AE burden in a common currency independent of any disease or medication class, that can be compared among different medications over time. The AEU may facilitate movement from more gestalt AE burden measurement to more precise AE burden measurement, enriching treatment discussions between patients and physicians.

Individual patients and physicians may not value AE in the same way, e.g. patients with more severe conditions such as cancer, may tolerate a higher burden of AE. As a consensus metric, the AEU is not designed to be an absolute measure of burden and distress for any particular patient but rather a way to keep the AE burden score. The AEU is designed to best estimate the market price of specific AEs, similarly to how the price is set for a good or service, e.g. $10 for a basketball and $30,000 for a car. Consumers decide if they are willing to pay consensus prices for these goods. Similarly, patients can be given AEU scores corresponding to the number and type of AEs they develop on a given therapy. In combination with measures of disease improvement, financial burden, overall QOL, severity of a patient’s medical condition, patient age, and other factors unique to a particular patient, patients can decide whether they are willing to tolerate a specific AEU burden when making treatment decisions with their physician. Future validation projects, like one underway in a population of patients with myasthenia gravis, will attempt to understand clinically meaningful differences in AEU score over time for different patient populations.

Attempts have been made to develop disease and medication specific measures of AE [8-13]. Disease and medication specificity limit broad applicability. Quality Adjusted Life Year (QALY) is a useful measure of population cost effectiveness of varied treatments [35, 36]. Since the QALY encompasses all aspects of health, financial cost, and QOL, it cannot measure AE burden alone. As a population based tool, the QALY is a less practical way to measure treatment burden in a comparative efficacy trial or in the clinic.

The CTCAE is a medication independent, physician derived AE measurement tool [21]. Due to lack of weighting and patient input, it provides only granular AE burden measurement. We built the AEU based on the strengths of the CTCAE. The diverse physician group incorporated a wide range of opinions about AE impact on overall health accounting for both current effects (e.g. joint pain) as well as future secondary consequences (e.g. stroke due to new diabetes) to assign AEU values. Although all physicians surveyed could rate congenital complication AEs, all pediatricians surveyed weighted these items as they care for impacted children.

The AEU incorporates potential patient opinions in assigning AE burden values. The use of potential patients rather than patients with particular diagnoses allows AE burden to be scored independent of any particular disease or medication. While we were not able to stratify the sample by whether MTurk respondents were parents, many subjects who rated congenital AEs self-identified as parents in the comment section. Since MTurk doesn’t permit stratification by ethnicity, some groups were slightly underrepresented in our sample. We observed even representation of U.S. geographic regions. Utilizing MTurk, we obtained hundreds of opinions within days of survey release. Although MTurk introduced bias due to requirement of basic computing skills, it reduced other bias, including the selection bias of clinicians when choosing patients for participation in research. We found recruitment through this online tool to be a logistical and cost-effective strategy to easily obtain opinions from large samples. This method has the potential to be a powerful method for studies like this one and to obtain preliminary data for clinical study design while reducing the inherent bias of the small focus group method.

We believe the weighted consensus AEU values provides a more complete measurement of AE burden. A CTCAE category 1 is often classified as mild [37]. However, all CTCAE categories across different AEs are not of equal value and were not weighted the same among our cohort. A CTCAE grade 1 may not reflect a good outcome in all circumstances. For example, CTCAE Grade 1 pulmonary fibrosis, received an AEU score of 7 and was rated the same as CTCAE Grade 4 osteoporosis (Table 4). We also believe the AEU’s independence of any particular disease or medication class is essential to allow comparison of treatments across medication classes. For example, prednisone and IVIG, treatments with different AE profiles, could be compared by the AEU in patients with myasthenia gravis.

Although 75% of AE categories required final AEU value adjustment when physician and potential patient values were combined, only 12% of items had a rating difference of 3 or more points between physicians and potential patients (Table 4). This suggests that while there is difference in physician and potential patient opinions on AE severity, there appears to be general agreement among the groups. Use of the Bradley-Terry paired comparisons model was a useful way to put the physician ratings, collected as scores, and the MTurk ratings, collected as a sequence of paired comparisons, on the same scale. Although physician opinions anchored AEU values, potential patient opinions were incorporated via the discrete choice surveys. We believe adjusting the AEU score to incorporate opinions of both groups strengthens the future applicability of this tool. In practice, patients often rely on physician expert caregivers to guide medical decisions.

We believe the AEU has great promise to be a useful, practical tool to add precision to AE burden measurement in the clinic and in comparative efficacy research for neurology patients. Future studies may show the AEU to be useful in other medical specialties. In comparative efficacy research, we anticipate that AE burden of drugs from different classes can be compared by AEU burden. Assigning an AEU score over time will account for more transient AEs that drop out over time (e.g. single headache) and more persistent AEs (e.g. new hypertension). The AEU score can be combined with other disease specific outcome metrics and QOL metrics to measure differences among medications over time. Evaluation of the validity, utility, and value of the AEU in comparative efficacy trials in myasthenia gravis and other neurological disorders is under way. If the AEU is useful in these studies, translation of some or all of the other items in the CTCAE could be performed to generalize the AEU to other medical subspecialties.

(DOCX)

Click here for additional data file.

Physician subject raw data form 2.

(CSV)

Click here for additional data file.

All potential patient choices raw data.

(XLSX)

Click here for additional data file.

Physician subject raw data form 5.

(CSV)

Click here for additional data file.

AEU item codes raw data.

(XLSX)

Click here for additional data file.

Physician subject raw data form 1.

(CSV)

Click here for additional data file.

Physician subject raw data pediatric form.

(CSV)

Click here for additional data file.

Physician subject raw data form 6.

(CSV)

Click here for additional data file.

Physician subject raw data form 4.

(CSV)

Click here for additional data file.

Physician subject raw data form 3.

(CSV)

Click here for additional data file.