Big data analysis of ASM retention rates and expert ASM algorithm: A comparative study.

OBJECTIVE: Only 50% of patients with new-onset epilepsy achieve seizure freedom with their first antiseizure medication (ASM). A growing body of data illustrates the complexity of predicting ASM response and tolerability, which is influenced by age, sex, and comorbidities. Randomized data with sufficient resolution for personalized medicine are unlikely to emerge. Two potential facilitators of ASM selection are big data using real-world retention rates or algorithms based on expert opinion. We asked how these methods compare in adult-onset focal epilepsy.
METHODS: ASM retention rates were determined by cross-referencing data from comprehensive Swedish registers for 37 643 individuals, with identified comorbidities. Eight fictive cases were created and expert advice was collected from the algorithm Epipick. We compared Epipick suggestions in representative patient subgroups, and determined whether ranking based on retention rate reflected expert advice.
RESULTS: The Epipick algorithm suggested six ASM alternatives for younger patients and three ASM alternatives for older patients. In the real-world data, retention rates for the ASMs ranked as best options by Epipick were high; 65%-72% for young patients and 71%-84% for older patients. The lowest retention rate for Epipick suggestions was 42%-56% in younger cases, and 70%-80% in older cases. The ASM with the best retention rate was generally recommended by Epipick. SIGNIFICANCE: We found a large overlap between expert advice and real-world retention rates. Notably, Epipick did suggest some ASMs with more modest retention rates. Conversely, clearly inappropriate ASMs (not recommended by Epipick) had high retention rates in some cases, showing that decision systems should not rely indiscriminately on retention rates alone. In future clinical decision support systems, expert opinion and real-world retention rates could work synergistically.

National Patient Register and prescription data were cross‐referenced to study antiseizure medication (ASM) retention rates in all adults in Sweden >30 years with epilepsy onset after 2007.

Eight fictive cases were used to compare real‐world ASM retention rates for subgroups defined by age, sex, and comorbidities, and expert advice in the Epipick tool.

The ASM with the highest retention rate was among Epipick‐recommended treatments in all eight cases.

Epipick generally recommended drugs that patients are likely to retain, but some suggestions had more moderate retention rates.

Combinations of expert opinion and big data analytics could provide more complete information for prescribers.

INTRODUCTION

Selecting the right antiseizure medication (ASM) for each patient is a major challenge in epileptology. Only 50% of patents achieve seizure freedom with the first ASM, and the trial‐and‐error process of finding the right drug can extend into years. The proportion of persons with epilepsy who remain on a suboptimal ASM and have seizures or side effects as a consequence is unknown. A growing body of evidence hints at the complexity of selecting the optimal first ASM; multiple factors such as age, sex, comorbidities, and etiology seem to influence retention rates. , Several strategies exist to improve ASM success rates: algorithms providing expert opinion, studies providing randomized evidence, and big data. There are pros and cons of all these approaches.

Expert algorithms, guiding physicians toward suitable options, are interactive guidelines. Their advantage is that users get tailored recommendations from experts integrating existing knowledge and vast clinical experience. A publicly available option is Epipick. Drawbacks include unclear evidence supporting the recommendations, that several ASMs are usually recommended leaving practitioners to do the final tailoring, and that rare epilepsy conditions are often not included.

Randomized data provide a high level of evidence, but is not likely to provide personalized data. Registration‐purpose randomized‐controlled trials (RCTs) have heterogeneous study populations and evaluate antiseizure effect by measuring the proportions of participants with fewer seizures, but are less informative about whether a first monotherapy will provide seizure freedom. Investigator‐initiated trials are another option, and large studies like Standard and New Antiepileptic Drugs (SANAD) and SANAD II are informative, but require enormous efforts, and the participant numbers allow only relatively crude stratification in subgroup analyses. ,  Niche RCTs, focusing on particular subgroups of patients, have proven very difficult. Even for poststroke epilepsy, the most common acquired epilepsy, trials frequently struggle to reach recruitment targets. , RCTs for every patient group (30‐year‐old women with posttraumatic epilepsy, 65‐year‐old men with epilepsy after brain infection, and so on) are not feasible.

Big data may offer a complementary approach. We recently showed that tracking real‐world ASM retention rates in large register‐based data sets of patients with acquired epilepsy gave results very similar to SANAD and RCTs on poststroke epilepsy. In addition, we found that age, sex, and comorbidities influenced retention rates and that 14%–21% of patients did not start with the ASM most likely to succeed for their strata. Other investigators using machine learning on large data sets have come to similar conclusions regarding the potential for improvement.

If big data is to inform clinical practice, there must be at least some congruence with randomized evidence and expert opinion. We have already demonstrated that register‐based big data can replicate randomized studies, but how they compare to expert opinion is not known. In the present study, we, therefore, compared the results of big data analytics to expert opinion. Based on eight fictive cases, we tracked ASM use on a nationwide scale for relevant patient groups and compared the results to expert opinion, represented by the Epipick algorithm.

METHODS

Registers

Data from several comprehensive Swedish health registers were cross‐referenced: the National Patient Register (NPR), the Cause of Death Register (CDR), and the Drug Register (DR). The NPR was established in 1987, with expanded outpatient coverage from 2001, and includes information on all diagnoses registered in in‐ or outpatient care. The CDR contains the date of death for all Swedish inhabitants. The DR was established in 2005 and contains information on all prescriptions in Sweden. Reporting to the NPR and CDR is mandatory for all health care providers, and the DR contains all prescriptions in Sweden. All registers are managed by the National Board of Health and Welfare, who anonymized the data before we were given access to it.

Cohort

We ordered anonymized information on all individuals (n = 94 321) with a first International Classification of Diseases, Tenth Revision (ICD‐10) code of epilepsy (G40) after 2007. For the purpose of this study, we included patients with a first epilepsy diagnosis after the age of 30 (presumed focal epilepsy) and dispensation of an ASM (Anatomical Therapeutic Chemical (ATC) code N03) at or after their first seizure (Figure 1). Individuals with a rare first ASM (confidence intervals wider than 50%) were excluded, resulting in a total of 37 643 individuals. An age limit to define focal epilepsy was used in the main analysis because the code G40.9 (unspecified epilepsy) is often used for practical reasons and onset of generalized epilepsy is rare after age 30. A more specific diagnosis of focal epilepsy was used in a sensitivity analysis, including individuals with an ICD‐code for focal epilepsy G40.1, G40.2, G40.6, or unspecified epilepsy G40.9 with a previous diagnosis of stroke (I61, I62, I63, or I69) or traumatic brain injury (S00‐S06 or S20.9), resulting in a total of 23 254 patients.

FIGURE 1

Study concept. Of all individuals with new‐onset epilepsy (n = 94 321), at total of 37 643 were selected based on cross‐referenced registers: the National Patient Register (NPR) provided medical variables, the Cause of Death Register (CDR) provided information on survival, and the Drug Register (DR) provided information on antiseizure medication (ASM) prescriptions. The ASM tracking was based on (1) retention‐rate calculations using prescription (P) interval (double arrow) to detect treatment stop (S), and (2) selecting subgroups for estimates

Patient characteristics and ASM tracking

Based on the NPR, age at epilepsy onset, sex, and comorbidities were defined by register searchers. The following comorbidities were identified: stroke, traumatic brain injury, psychiatric conditions, dementia, multiple sclerosis, central nervous system (CNS) infection, and developmental disorders/intellectual disability based on relevant ICD‐10 codes (Table S1). ASM retention was estimated with Kaplan‐Meier calculations based on a prescription interval of 12 months, and each dispensation was sufficient for 3 months, as described previously. Briefly, treatment was assumed to continue until 1 year passed without renewal, and patients were censored at death or date of export (December, 31 2019). Confidence intervals were calculated using Greenwood's Exponential formula.

Cases, retention rates, and Epipick

We created eight fictive cases of focal epilepsy (Table 2), representing a broad range of ages and etiologies. For these eight cases, we assessed retention rates in the data set for individuals matching the case age, sex, and comorbidities. For cases of unknown cause, we included all individuals, but in a sensitivity analysis we excluded patients with any comorbidity at all (only including G40 and R56.8, and excluding patients with stroke, trauma, multiple sclerosis, dementia, brain infection, intellectual disability, psychosis, depression, stress, personality disorder, mental development disorder, or behavioral and emotional disorders) (ICD codes are available in Table S1). For the cases of poststroke epilepsy, posttraumatic epilepsy, or epilepsy in dementia, we restricted the analyses to patients with these comorbidities.

TABLE 2

Case vignettes, Epipick suggestions, and register data results

Case	Epipick best	Real world rank	All with epilepsy onset >30 years			Specific codes for focal epilepsy
			1‐year retention rate	95% CI	n	1‐year retention rate	95% CI	n
30‐year‐old female, epilepsy and traumatic brain injury (MRI lesion)	Cbz	ltg	71	60–78	101	72	60–79	83
	Esl	lev	57	42–70	50	56	40–68	48
	Lcm	cbz	51	38–60	76	47	33–57	66
	Ltg	pgb	50	18–66	14	NA
	Lev
	Oxc
30‐year‐old female, epilepsy of unknown cause	Cbz	ocx	67	35–79	18	NA
	esl	ltg	64	60–68	591	66	59–70	282
	lcm	lev	53	47–57	387	47	40–54	193
	ltg	cbz	52	47–56	433	54	47–60	261
	lev	tpm	45	21–60	22	NA
	oxc	vpa	43	33–51	111	48	33–59	54
		pgb	33	21–46	55	44	24–62	25
		pht	32	10–47	19	27	3–44	11
30‐year‐old male, epilepsy and traumatic brain injury (MRI lesion)	cbz	ltg	65	54–74	86	64	53–74	76
	esl	vpa	60	45–69	62	59	43–69	56
	lcm	lev	57	48–65	130	56	47–65	116
	ltg	cbz	51	43–58	172	48	39–55	147
	lev	pgb	43	25–60	29	41	18–56	23
	oxc
40‐year‐old male, epilepsy of unknown cause	cbz	ltg	68	64–71	577	67	62–72	319
	esl	oxc	62	47–71	66	65	45–76	37
	lcm	lev	60	57–63	1005	60	56–64	597
	ltg	vpa	57	52–61	442	56	50–63	221
	lev	cbz	56	53–59	1432	58	55–62	806
	oxc	clb	45	25–57	35	40	12–56	15
		lcm	42	10–59	12	NA
		pgb	37	28–45	123	34	21–44	61
		gbp	32	23–42	99	31	19–45	48
		tpm	31	8–47	16	NA
		pht	26	10–38	31	NA
45‐year‐old female, epilepsy of unknown cause	cbz	lcm	72	42–89	15	NA
	esl	ltg	65	62–68	1051	64	59–68	508
	lcm	lev	58	54–61	849	56	51–60	435
	ltg	cbz	52	49–55	976	53	49–57	573
	lev	vpa	49	43–55	277	50	41–58	136
	oxc	oxc	45	28–57	42	42	19–56	24
		tpm	38	21–50	40	33	8–49	17
		gbp	38	26–46	92	30	14–48	31
		pht	36	19–49	37	30	8–46	18
		pgb	36	26–44	115	39	23–54	39
60‐year‐old male, epilepsy and stroke (concomitant medications, MRI lesion)	lcm	ltg	77	71–81	270	78	72–82	242
	ltg	lev	72	69–74	1054	71	68–74	921
	lev	vpa	68	62–72	338	66	61–71	306
		cbz	66	63–68	1147	64	61–67	1004
		oxc	57	40–71	39	61	41–76	32
		pht	54	38–64	55	51	34–62	47
		pgb	49	25–63	26	54	25–67	22
		gbp	35	21–46	56	36	20–47	50
65‐year‐old female, epilepsy and stroke (concomitant medications, MRI lesion)	lcm	ltg	71	63–76	199	70	62–75	177
	ltg	lev	70	66–74	521	70	65–74	440
	lev	vpa	68	59–74	158	67	57–73	140
		pht	61	37–73	29	61	34–74	24
		cbz	60	55–64	531	59	54–63	484
		gbp	50	27–63	30	52	27–66	27
		oxc	46	23–46	26	42	19–56	24
70‐year‐old male, epilepsy and traumatic brain injury (concomitant medications, MRI lesion)	lcm	ltg	80	74–84	279	79	73–83	246
	ltg	lev	73	69–75	888	72	68–75	776
	lev	vpa	71	65–76	274	70	63–75	242
		cbz	69	66–72	872	67	63–70	746
		oxc	64	40–76	28	54	27–68	22
		gbp	62	44–75	40	63	44–77	35
		pht	61	41–73	41	64	41–75	34
		pgb	52	22–66	21	45	15–60	17
70‐year‐old female, epilepsy and traumatic brain injury (concomitant medications, MRI lesion)	lcm	ltg	81	75–85	228	79	72–84	191
	ltg	pht	78	53–87	29	78	50–87	24
	lev	lev	72	68–76	496	70	66–75	426
		cbz	66	60–70	386	64	58–69	329
		vpa	66	57–73	148	63	52–70	126
		gbp	59	39–70	40	55	32–68	30
80‐year‐old male, dementia (concomitant medications)	lcm	pht	84	65–93	35	82	59–93	25
	ltg	ltg	84	79–87	318	82	76–87	216
	lev	lev	80	77–83	756	80	76–83	506
		oxc	79	50–87	26	NA
		cbz	78	74–81	609	77	73–81	411
		vpa	75	69–80	294	75	67–81	185
		gbp	55	38–70	42	64	43–79	31
80‐year‐old female, dementia (concomitant medications)	lcm	pht	81	64–90	41	73	52–86	28
	ltg	lev	80	76–83	669	79	74–83	402
	lev	gbp	79	67–88	70	80	57–88	32
		ltg	77	72–81	313	76	69–82	188
		vpa	74	68–79	289	72	64–79	170
		cbz	73	69–77	548	73	67–78	344
		oxc	68	32–80	16	NA

Epipick settings used are indicated in parenthesis in the vignettes.

Abbreviations: cbz, carbamazepine; esl, eslicarbazepine acetate; lcm, lacosamide; oxc, oxcarbazepine; ltg, lamotrigine; lev, levetiracetam; tpm, topiramate; pgb, prebabalin; vpa, valproic acid; pht, phenytoin.

Expert opinion was obtained from the online ASM guide Epipick (www.epipick.org). Epipick provides ranking of ASMs in three levels of prioritization: best, second best, and least desirable though still acceptable. We used the ASMs categorized as “best” as comparators for the real‐life data.

Ethical permission

The study was approved by the Ethics Review Authority, decision number 2020‐04902. We confirm that we have read the Journal's position on issues involved in ethical publication and affirm that this report is consistent with those guidelines.

Data availability statement

The underlying register data are protected by Swedish confidentiality laws and cannot be shared by the authors. The Swedish registers are available to researchers upon request to the Swedish National Board of Health and Welfare.

RESULTS

Retention rates

We first analyzed retention rates in all individuals in the data set and in those with a more specific diagnosis of focal epilepsy (Table 1). Lamotrigine had the highest retention rate and levetiracetam had the second highest.

TABLE 1

Retention rates for all individuals in the data set and for individuals with codes specific for focal epilepsy

Antiseizure medication	All with epilepsy onset >30 years			Specific codes for focal epilepsy
	1‐year retention rate	95% CI	N	1‐year retention rate	95% CI	N
Lamotrigine	71	69–72	5641	71	69–72	3383
Levetiracetam	68	68–69	12 974	68	67–69	7998
Phenobarbital	66	49–75	58	62	34–70	32
Valproate	62	61–64	4272	62	60–64	2651
Lacosamide	61	51–68	134	57	46–67	91
Carbamazepine	58	58–59	11 844	59	58–60	7578
Oxcarbazepine	57	52–61	478	56	49–61	303
Phenytoin	53	49–57	619	52	47–57	394
Gabapentin	45	41–48	943	45	40–49	517
Pregabalin	40	36–45	528	40	33–45	252
Clobazam	39	30–46	152	31	20–41	75
Topiramate	38	28–46	115	40	27–52	55

Epipick suggestions

Next, we used our fictive cases to assess retention rates in particular patient subgroups. The Epipick algorithm suggested six ASM alternatives for patients younger than 50 years, and three ASM alternatives for older patients (Table 2).

In the real‐world data, we first assessed retention rates with a permissive approach, including all ASMs with a confidence interval <50%. The retention rates for the best Epipick options were high: 65%–72% for young patients and 71%–84% for older patients; the lowest retention rates among the best Epipick suggestions were 42%–56% in younger cases and 70%–80% in older cases (Figure 2A). If precision was increased by including only ASMs with more than 50 users (resulting in a smaller confidence interval) the results were almost identical (Figure 2B), the one difference being a reduced gap between the Epipick suggestion with the highest and the lowest demonstrated retention rate.

FIGURE 2

Retention rates of the Epipick suggestion with the highest (gray area) and lowest (dashed) retention rate (large areas better). If all antiseizure medications (ASMs) with any number of individuals were included (A) some Epipick suggestions had low retention rates, but this was not seen if precision was increased by requiring 50 users (B). PTE = posttraumatic epilepsy, PSE = poststroke epilepsy

Retention rate–‐based rank

We next evaluated the reverse association: whether the drugs with the best retention rates were among those recommended by Epipick. The ASMs with the highest retention rates were generally recommended by Epipick. If ASMs with any number of users were included, the highest‐ranking ASM was among the Epipick suggestions in six of eight cases (Figure 2A). If ASMs with more than 50 users were included, the highest‐ranking ASM was among the Epipick suggestions in all cases (Figure 2B).

Finally, we asked whether retention‐rate ranking could distill the same drugs as those suggested by Epipick. If ASMs with any number of users were included, ASMs not suggested by Epipick had the highest or second‐highest retention rate in several cases. In some cases, a clearly inappropriate ASM had the highest retention rate (phenytoin in elderly patients with dementia). If the analysis was restricted to ASMs with more than 50 users, the ASM with the best and the second‐best retention rates were suggested by Epipick for all cases (Figure 3).

FIGURE 3

Evaluation of whether retention rates identify expert choice. Top row (larger area better): If all antiseizure medications (ASMs) with any number of users were included, the ASM with the highest retention rate was recommended by Epipick in six of eight cases (A). Performance of retention rates was improved if 50 users were required; then the ASM with the highest retention rate was recommended by Epipick in all cases (B). Lower row (points closer to center better): With any number of users, some potentially inappropriate ASMs not recommended by Epipick had the highest retention rates (C). With >50 users, “inappropriate” ASMs ranked three or lower in all cases (D)

Sensitivity analysis

For the main analysis, the three patients with unknown causes were represented by all patients. In a sensitivity analysis, we included patients with no comorbidity at all (Table S2). This analysis showed results that were similar to the main analysis: both top retention rate ASMs were recommended by Epipick for all cases.

DISCUSSION

We evaluated the potential of national big data routinely collected in administrative health care registers to inform ASM selection by stratification on age, sex, and comorbidities by comparing the performance of big data analytics to that of expert opinion, as illustrated by Epipick. We aimed to evaluate the congruence between these approaches for facilitating ASM selection. The analysis resulted in several interesting conclusions, and it is hoped that they are helpful in furthering personalized medicine in epilepsy.

First of all, retention rate in prescription data on a national level seems to find relatively appropriate ASMs, although some restriction is needed to avoid inappropriate alternatives. Conversely, expert opinion illustrated by Epipick recommends drugs that patients in Swedish registers are likely to retain. Nonetheless, our investigation did demonstrate that some Epipick suggestions had higher retention rates than others. These results have several implications for the development of future clinical decision support systems, one being that combinations of expert opinion and big data analytics are likely to yield better results than either one. Expert opinion algorithms can be enhanced by providing retention rates for the possible alternatives to the user. Conversely, clinical decision systems should not rely indiscriminately on retention rates alone, since this can be high also for inappropriate ASMs.

There are other interesting findings in our material. To our knowledge, it is the first attempt to use administrative data to track ASM retention in patient subgroups based on data from an entire country. Our findings that lamotrigine has the highest retention rate in focal epilepsy is in excellent agreement with the SANAD and SANAD II studies. ,  The results in our study are also similar to previously reported comparisons of Epipick and real‐world retention rates in a smaller population.  The ASMs suggested by Epipick had a significantly higher retention rate than many lower ranked ASMs.

Some inappropriate ASMs had high retention rates in our real‐world data, indicating a potential problem with putting too much emphasis on retention rate in guiding treatment. High retention of inappropriate ASMs may arise for several reasons. Long‐term adverse effects might not result in withdrawal until time points that are later than those analyzed by us, and low‐quality epilepsy care may cause patients to continue taking less suitable or tolerated ASMs. For instance, sodium valproate was prescribed to women with epilepsy of childbearing age to a non‐negligible extent in the study period, despite the now well‐known risks and 2018 European Medicines Agency regulations. ,

There are drawbacks to our method. The register method relies on several assumptions. The epilepsy diagnosis has been validated and has a 90% positive predictive value. Similarly, the ASM tracking by Kaplan‐Meier ignores competing risks. The real‐world retention rates also represent clinical reality, meaning that the higher ASM retention rates in the older cases may well represent less‐rigorous epilepsy care with fewer attempts at ASM revision or that patients have not survived long enough to change their treatment. The results may also be confounded by co‐medication and seizure frequency, which is considered in Epipick but not in our data, potentially leading to a difference in retention rates among Epipick suggestions. For instance, the titration required for lamotrigine may lead to it being used in cases with less frequent seizures, which may also require a longer evaluation period and thereby delayed discontinuation of the drug.

Although our fictive cases represent many patients, our investigation does not cover the huge individual variability of epilepsy. With more data, the method can probably give more precise estimates for even more narrowly defined patient groups. Although we used prescription data from an entire country for over a decade, the selection of individuals with specific comorbidities, age, and sex resulted in relatively few patients using individual ASMs and relatively imprecise estimates of retention rates for these drugs. Multinational efforts could be one possible counter‐effort. It would also be interesting to study ASM dosage, but extracting the doses actually used by patients from prescription register data is not possible.

A near step in personalized medicine of epilepsy is the use of artificial intelligence, to make even better use of big data accumulating in various health registers. Our study provides some clues on how expert opinion and big data analytics can interact to create even better outcomes.

CONFLICT OF INTEREST

JZ reports speaker honoraria for unbranded educations from Eisai and UCB, and as employee of Sahlgrenska university (no personal compensation) being an investigator/subinvestigator in clinical trials sponsored by UCB, GW Pharma, Bial, and SK life science. SH reports no disclosures.

Table S1‐S2

Click here for additional data file.