Mapping Outcomes and Registries Used in Current Danish Pharmacoepidemiological Research.

Purpose: There is an increasing need for national and international pharmacoepidemiological studies based on high-quality real-world data of which the Danish registries are a valuable source. In lack of a complete overview of which data are used to assess real-world drug safety and effectiveness outcomes, we aimed to map the outcomes, data sources, and the reporting of outcome quality in recent pharmacoepidemiological studies.
Methods: We conducted a systematic mapping review of pharmacoepidemiological studies based on Danish registries investigating drug safety and/or effectiveness, published in the period 2018-2019, identified in PubMed and Scopus. Extraction included: Anatomical Therapeutic Chemical level 2 code for drug exposures, outcomes, outcome data sources, and quality of outcomes.
Results: Of the 210 included studies, 96% used outcomes categorized as Clinical, 4% utilized outcomes categorized as Society-related, 5% used outcomes categorized as Healthcare cost and utilization, and 3% of the studies applied outcomes categorized as Patient-reported in which the percentages are not mutually exclusive. Diagnosis (66%) and Mortality (38%) were the two most utilized subcategories among those categorized as Clinical outcomes. Danish Health Data Authority and Statistics Denmark registries were the most reported outcome data sources (90%). Ninety-six studies (46%) reported one or more quality parameters related to their outcomes of interest with accuracy/validity being the most reported parameter (22%).
Conclusion: The Danish registries support a wide range of outcomes. Across therapeutic areas, most studies investigate traditional clinical outcomes of disease and mortality based on data from a small number of available registries. In contrast, clinical and biochemical databases, despite potentially offering outcomes with high responsiveness, and the high-quality social and healthcare cost registries were rarely used as outcome data sources.

Introduction

Pharmacoepidemiological and other real-world evidence (RWE) studies of drug safety and effectiveness are increasingly demanding to support regulatory decision-making.1–3 As response to a demand for faster access to drugs targeting diseases with unmet medical needs, more drugs are approved under fast-track processes such as conditional approval with requirements for close monitoring and performance of post-authorization studies within a defined timeline to support reassessment of the benefit–risk balance.1,4,5 These novel drugs are often designated orphan products (ie drugs for rare diseases) and advanced medical therapies (eg gene and cell therapies) targeting smaller patient populations, which may challenge the traditional pathway of evidence generation by randomized clinical trials (RCTs).4,6 Thus, there is a growing interest in studies combining cohorts from several countries to gain larger study samples and potentially increase the generalizability of the results, as well as using high-quality real-world data (RWD). In this context, the comparable healthcare and social security systems as well as health data infrastructures across the Nordic countries7 provide the (pharmaco-) epidemiologists with favorable opportunities for conducting international RWE studies, for example, using common data models.8

Similar to the other Nordic countries, Denmark has a long tradition of capturing a wide range of RWD on the Danish inhabitants (population: 5.8 million) offering more than 200 different registries and databases, of which some have more than half a century of longitudinal data. Linkage across registries is most often straightforward via a unique personal identification number. However, sometimes linkage also requires dates to identify the order of events in different registries.9 The available data includes various types of administrative, clinical quality and research data, harvested in a setting with universal access to healthcare free of charge, thereby limiting social and economic bias in coverage.9,10 The three main data holders and permission authorities in Denmark are the Danish Health Data Authority, Statistics Denmark and the Danish Clinical Quality Program – National Clinical Registries (RKKP). The Danish Health Data Authority and Statistics Denmark mainly govern the national administrative registries, which are either total population registries capturing sociodemographic information, such as births, deaths, migration (the Danish Central Person Registry, since 1968)10 and highest completed educational level (the Population’s Education Registry, since 1973)11 on all inhabitants, or population-based registries capturing all residents with a specific characteristic or event such as hospital contacts recorded with discharge diagnoses in the Danish National Patient Registry (since 1977),12,13 or prescriptions redeemed from community pharmacies registered in the Danish National Prescription Registry (since 1995).14 While some of the registries are only accessible via one data holder, others are accessible through both. RKKP manage disease- or procedure-specific clinical quality databases that typically cover all persons with a specific disease or undergoing a specific procedure, eg, the Danish National Lymphoma Registry, which covers all patients diagnosed with malignant lymphoma and has been referred to hematology departments in Denmark since 2000.15 As the purpose of establishing the clinical quality databases has been surveillance and improvement of clinical quality, these databases contain more detailed clinical data than recorded in the administrative registries.16–18 For more details about the Danish registries, we refer to the overview of content, coverage and data holders of 43 Danish registries provided in .

In general, the Danish registries are considered to have high validity and completeness. However, the inter- and intra-registry validity may vary.9,13 In a review of the Danish National Patient Registry, Schmidt et al13 reported, “The positive predictive values of diseases and treatments vary widely (<15–100%)”. This variability calls for attention to address quality parameters when conducting observational studies as it may influence the accuracy of the results.

Even though many Danish registries have been used extensively for health-related research, the Danish data landscape might contain important untapped potential that is relevant for the scope of both national and international pharmacoepidemiological studies.9,18,19 However, there is a lack of an overview of which data are currently used to assess drug safety and effectiveness. To address this gap in knowledge, the objectives for this study, to our knowledge the first of its kind, were to 1) provide an overview of the different types of outcomes used in recent pharmacoepidemiological studies using Danish registry data to assess drug safety and effectiveness, and the extent of their use, 2) to map these outcomes according to therapeutic level of drug exposure and outcome data source, and 3) to investigate to what extent the studies report on the quality of the applied outcomes.20

Materials and Methods

Study Design and Literature Search

The present study was conducted as a systematic mapping review, which is a review method focusing on the research characteristics rather than on the findings of the included studies.21–23 A systematic search was conducted in PubMed and Scopus to identify pharmacoepidemiological studies based on Danish registries and published in the period of January 1st, 2018 to December 31st, 2019 (electronic or print). In PubMed and Scopus, the final search was performed on the November 18th, 2020, and December 6th, 2020, respectively. First, the search strategy for PubMed was developed, and based on this, the search strategy for Scopus was developed. Both strategies are available in and in the study protocol20 (), which also explains the conduct of this review in further detail.

Screening and Study Selection

Study eligibility was assessed based on the following criteria: Inclusion criteria: Original research publications investigating drug safety and/or effectiveness using data on outcomes from Danish registries/databases. Exclusion criteria: Studies with exposures not complying with the European Medicines Agency (EMA) definition of a medicinal product, ie,

A substance or combination of substances that is intended to treat, prevent or diagnose a disease, or to restore, correct or modify physiological functions by exerting a pharmacological, immunological or metabolic action24

Articles not published in English language; reviews and short publication types.20

A pilot screening of 35 article titles and abstracts were conducted independently by three authors (CTP, KJJ, JP) to ensure consistency and agreement on study selection as well as to discuss and resolve any differences. The remaining article titles and abstracts were screened by CTP. In case of doubt of inclusion or exclusion, the title and abstract of the articles were also screened by KJJ and JP. If uncertainty remained, these articles were included in the full-text screening process. Full-text screening was conducted in conjunction with data extraction.

Data Extraction and Validation

From each included article, the following data were extracted: Journal, publication year, study design, patient enrolment period, whether the study was imposed by EMA, the US Food and Drug Administration (FDA) or Other, Anatomical Therapeutic Chemical (ATC) code, level 2 of the drug exposures,25 outcomes, outcome registry sources, and reported quality of the outcomes (ie text pieces and references). Further specifications are available in the study protocol.20 Outcomes were herein defined as events or measures identified in registry-based data to evaluate a specified exposure. If the information could not be obtained from the original article text, the supplementary material was investigated. In case of multinational studies, only the Danish information was recorded.

Data extraction was initially tested and validated on 12 articles by CTP and at least one of the three authors (KJJ, MR, JP) to ensure consensus on data items and the process. The remaining articles were extracted by CTP, BIvO, or by student assistants (see Acknowledgements) under careful instruction by CTP. Of these, 20 articles were extracted independently by CTP and either BIvO or the student assistants and compared to confirm the same extraction patterns and resolve disagreements. Uncertainties in data extraction were discussed with KJJ and JP.

Data Analysis

All recorded outcomes were mapped in a hierarchical way: Each study-specific outcome was assigned to an outcome subtype to aid the categorization. This assignment was driven by the type and context in which the outcome was applied in the included study. As an example, redeemed prescriptions could be assigned to “Prescriptions” if the drug exposure was investigated in relation to prescription of another specific drug (type), or to “Healthcare utilization” if the investigation was related to change in general medication use. The possible outcome subtypes (referred to as “outcome measure subgroups” in the protocol)20 were defined a-priori to conduct the study based on extracting study-specific outcomes of seven pharmacoepidemiological studies using Danish data.26–32 The seven studies were semi-randomly selected, ie the studies were randomly selected from a group of 50 studies identified as seemingly relevant during the early development of the search strategy. Following assignment of study-specific outcomes into outcome subtypes, the subtypes were further categorized into subcategories, which each constituted a part of one of the four overall categories: Clinical outcomes, Society-related outcomes, Healthcare costs and utilization outcomes, and Patient-reported outcomes. Clinical outcomes included measures evaluated only by clinicians or other healthcare professionals. Society-related outcomes included measures related to work, education, or criminal records. Healthcare costs and utilization outcomes included measures associated with cost or use of healthcare services, whereas Patient-reported outcomes included measures with a patient-reported element (or reported by relatives). When referring to the outcome (sub-)categories in the text, the names are capitalized to emphasize that the (sub-)categories are defined in the study. An overview of the hierarchical categorization of outcomes was generated based on the categories, subcategories, and subtypes. Outcome categories or subcategories (depending on the size of the category) were also mapped according to therapeutic level of drug exposure and to outcome registry source. The number of studies using each outcome category, subcategory, subtype, exposure group, and registry source, as well as combination between outcome and exposure group or registry source was determined (counted once per study). In addition, a number of studies reporting the following quality parameters: reliability, accuracy/validity, positive predictive value, negative predictive value, sensitivity/completeness, specificity, responsiveness were determined.

Results

Study Inclusion

From the search in PubMed and Scopus, 1246 records of which 371 records were duplicates and 210 studies were found eligible. The study selection process is presented in Figure 1. An overview of the included studies is provided in and . A request for study conduction by EMA, FDA or Other was reported in 2% (n = 5) of the studies.

Figure 1

Flow diagram illustrating the study selection process. Articles could be excluded due to more than one reason. Thus, the number of articles that meet the specific reasons for exclusion is not mutually exclusive. Adapted from Liberati A, Altman DG, Tetzlaff J et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339(jul21 1):b2700-b2700.67

Categorization and Distribution of Outcomes

The outcomes recorded from the 210 included studies were categorized through a hierarchy of 1) Categories, 2) Subcategories, and 3) Subtypes. The hierarchy and distribution of outcomes in the included studies are presented in Table 1. Clinical outcomes were the most frequently used outcome category, present in 96% (n = 201) of the included studies. Twelve percent of the studies (n = 25) investigated at least one non-clinical outcome, which herein was defined as those categorized as Society-related, Healthcare costs and utilization, and Patient-reported outcomes that were used in 4% (n = 9), 5% (n = 11) and 3% (n = 6) of the studies, respectively. Diagnosis and Mortality were the two most reported Clinical outcome subcategories with 66% (n = 138) and 38% (n = 80) of the studies, respectively. In total, 80% (n = 168) of the studies investigated outcomes within at least one of these two subcategories. Across the studies applying Diagnosis as outcome subcategory, the most reported diagnosis code category was related to diseases in the circulatory system (n = 60, 43%). An overview of the investigated diagnoses across studies has been provided in . Within the category of Healthcare costs and utilization outcomes, only one study33 used Costs as an outcome.

Table 1

Overview of Outcome Categorization and Distribution Across the 210 Included Studies

Categories	n (%)	Subcategories	n (%)	Subtypes	n (%)	Type of Measures
Clinical outcomes	201(96)	Mortality	80(38)	All-cause mortality	60(29)	All-cause mortality; Death by any cause
				Cause-specific mortality	35(17)	Disease/cause-specific mortality; Death by other causes than a specific one
		Diagnosis	138(66)	Diagnosis	131(62)	Diseases/events/hospitalization/hospital admission defined by specific diagnosis (ICD-8/10 codes*)
				Diagnosis/Prescription combination	11(5)	Composite measures defined both by specific diagnosis and by redeemed prescription
				Other	1(<1)	Suicide attempt (ICD-8/10 or contact codes*)
		Surgery and procedures	17(8)	Surgery and procedures	16(8)	Surgeries; Amputations; Blood transfusions; Dialysis, Implantable cardioverter defibrillator therapy; Other procedures
				Other	1(<1)	Organ supportive treatments
		Drug treatment	22(10)	Prescription	17(8)	Redeemed prescriptions of specific medications
				Drug discontinuation/ switch	6(3)	Overall- and cause-specific drug discontinuations, withdrawal, or switch; Add-on treatment
		Disease severity or morbidity	13(6)	Hospitalization	3(1)	All-cause hospitalization
				Severity scale (Disease), Clinical	6(3)	Expanded Disability Status Scale score; Psoriasis Area and Severity Index response score; Minimal joint activity score; Cognitive functioning score
				Other	7(3)	Disease relapse/progression (single measure or composite measure); In-hospital postoperative event (composite of diagnosis, surgery and procedure, and diagnostic measures); Adverse clinical event (composite of new prescription, hospitalization, or surgery); Bradyarrhythmic events (composite of surgery and procedure, and diagnosis); Future atopic dermatitis activity (composite of prescription, contact with private dermatologist and hospital contact)
		Clinical investigations	7(3)	Diagnostic measures	7(3)	Mammographic density; Creatinine kinase-MB level; Coronary angiography; Body Mass Index and overweight; Ultrasound scans; Abnormal electrocardiogram, heart rate, blood pressure, etc. (clinical signs and symptoms of poisoning); Tumor biopsy; Radiological findings; Cerebrospinal fluid cytology and/or flow cytometric immunophenotyping
		Pregnancy and offspring	21(10)	All-cause mortality offspring	5(2)	Stillbirth; Fetal death/abortions (elective or spontaneous); Infant mortality
				Cause-specific mortality offspring	1(<1)	Death due to heart defect in offspring
				Diagnosis offspring	18(9)	Diagnosis in offspring following parental exposure: Birth defects/malformations and other diseases defined by a diagnosis (ICD-8/10 codes*)
				Apgar Score	2(1)	5-min Apgar score
				Preterm birth	5(2)	Preterm birth/delivery
				Small for Gestational Age	5(2)	Low birth weight in relation to gestational age; Fetal growth restriction
				Other	2(1)	Cesarean (elective or acute); Normal or induced birth delivery; Molar and extra uterine pregnancies; Other pregnancy outcomes
Society-related outcomes	9(4)	Work-related	2(1)	Work ability/Productivity/ Connection to labor market	2(1)	Return to work; Workforce detachment (paid sickness leave, early retirement, or any other economic support from the state due to reduced working capability)
		Educational	4(2)	Other	4(2)	Learning disabilities (special educational needs); School performance and cognitive abilities (national academic tests); Student academic performance and/or behavior (exam or teacher-evaluated grade point average); Delayed admission in elementary school
		Criminality	3(1)	Other	3(1)	Prison sentences/incarcerations, and convictions
Healthcare costs and utilization	11(5)	Healthcare costs and utilization	11(5)	Costs	1(<1)	Costs for inpatient and outpatient contacts, primary sector contacts, and medication
				Healthcare utilization	11(5)	All-cause somatic/psychiatric hospital admission; Somatic/psychiatric hospital days; Intensive care unit admissions; Outpatient visits; Activities in connection to hospital contacts; Any readmission/reoperation; Medication use
Patient-reported outcomes, PRO	6(3)	Generic PRO	3(1)	Quality of Life	1(<1)	Visual analogue scale (VAS) of pain, VAS global patient health assessment, Health Assessment Questionnaire-Disability Index
				Other	2(1)	ADHD/Hyperactivity-related behavior (Strength and Difficulties Questionnaire)
		Disease-specific PRO	4(2)	Severity scale (Disease), PRO	3(1)	Disease Activity Score in 28 joints with or without C-reactive protein measurement; Clinical Disease Activity Index score; American College of Rheumatology treatment response score; European League Against Rheumatism treatment response score
				Quality of Life	1(<1)	Dermatology Life Quality Index

Notes: The table present an overview of how outcomes have been categorized into subtypes, into subcategories and further into categories forming the overall hierarchy of outcome types. The number of unique studies using the subtype, subcategory, or category have been stated by frequency (n) and rounded percentage (%) relative to the total number of included studies (N=210). As studies can have investigated more than one outcome subtype/subcategory/category, the numbers and percentages are not mutually exclusive. The last column presents the types of outcomes included under each subtype (mostly stated in an unspecific manner). *In Denmark, diagnoses recorded in registries are coded according to the ICD system with the 8th revision (ICD-8) being used up to 1994, followed by the 10th revision (ICD-10) from 199513.

Abbreviations: ICD, International Classification of Diseases; PRO, patient-reported outcomes.

Mapping Outcomes According to Drug Exposure Group

The outcome (sub-)categories were mapped to 69 different exposure groups (68 out of 94 available ATC level 2 codes, and one not specified on ATC level (named “Other”)) identified in the included studies (Table 2). All exposure groups were paired to an outcome categorized as Clinical outcome, and mostly to the subcategories Diagnosis (n=63, 91%), Mortality (n = 45, 65%) or Pregnancy and offspring (n = 37, 54%). The pattern of the most investigated drug exposure group, ATC B01: Antithrombotic agents (n = 38, 18%) was similar. The antithrombotic agents were most often studied in relation to Diagnosis (n = 29), especially cardiovascular diagnoses, and Mortality (n = 21) with the subtypes All-cause mortality (n = 18) and Cause-specific (cardiovascular or cancer) mortality (n = 4). The number of studies investigating each exposure group or outcome are not mutually exclusive. The immunosuppressants (ATC: L04) were the only exposure group paired to all seven clinical subcategories. Mapping to non-clinical outcomes was seen in 17% (n = 12) of the 69 exposure groups. Of these, six ATC groups (ATC: A14, C07, C09, L04, N05, N06) were coupled to three out of four categories. No exposure group was paired to all four overall categories.

Table 2

Outcomes Mapped According to Drug Exposure Groups

Studies Included N = 210	Clinical (n = 201)							Society-Related (n = 9)	Healthcare Costs and Utilization (n = 11)	Patient-Reported Outcome (n = 6)
	Mortality (n = 80)	Diagnosis (n = 138)	Surgery and Procedures (n = 17)	Drug Treatment (n = 22)	Disease Severity or Morbidity (n = 13)	Clinical Investigations (n = 7)	Pregnancy and Offspring (n = 21)
Exposure Groups	n	n	n	n	n	n	n	n	n	n
A01: Stomatological preparations (n = 2)	1	2		1
A02: Drugs for acid-related disorders (n = 13)	7	9		1	1		1
A03: Drugs for functional gastrointestinal disorders (n = 2)		1					1
A04: Antiemetics and antinauseants (n = 1)							1
A06: Drugs for constipation (n = 4)		4
A07: Intestinal anti-infectives (n = 7)	1	6		1	1
A09: Digestives, incl. enzymes (n = 3)		3
A10: Drugs used in diabetes (n = 7)	3	6	2				1
A11: Vitamins (n = 1)				1			1			1
A14: Anabolic agents for systemic use (n = 2)	1	1	1	1				1	1
B01: Antithrombotic agents (n = 38)	21	29	3	2			1
B03: Anti-anemic preparations (n = 3)		1		2			2			1
B04: Statins (before 1997)63 (n = 1)		1
B05: Blood substitutes and perfusion solutions (n = 4)	3	4	1	1	1	1			1
C01: Cardiac therapy (n = 9)	2	6	1		1		1
C02: Antihypertensives (n = 2)		1					1
C03: Diuretics (n = 9)	4	5					1
C05: Vasoprotectives (n = 2)	1	2		1
C07: Beta blocking agents (n = 15)	5	11	2		2		1	1	1
C08: Calcium channel blockers (n = 12)	4	8	1		1		1		1
C09: Agents acting on the Renin-Angiotensin System (n = 12)	4	9						1	1
C10: Lipid modifying agents (n = 15)	7	11	2	1
D01: Antifungals for dermatological use (n = 2)	1	2		1
D05: Antipsoriatics (n = 1)					1
D06: Antibiotics and chemotherapeutics for dermatological use (n = 3)	1	2		1	1
D07: Corticosteroids, dermatological preparations (n = 3)	1	2		1	1
D08: Antiseptics and disinfectants (n = 2)	1	2		1
D10: Anti-acne preparations (n = 2)	1	2		1
D11: Other dermatological preparations (n = 3)		2			1
G01: Gynecological anti-infectives and antiseptics (n = 2)	1	2		1
G02: Other gynecologicals (n = 3)	2	1					1
G03: Sex hormones and modulators of the genital system (n = 10)	3	7				1	2
G04: Urologicals (n = 4)	1	4
H01: Pituitary and hypothalamic hormones and analogues (n = 4)		2					1			1
H02: Corticosteroids for systemic use (n = 7)	1	4	1		1		1
H03: Thyroid therapy (n = 3)	2	1					1
H05: Calcium homeostasis (n = 1)		1		1
J01: Antibacterials for systemic use (n = 16)	3	11		1	1	1	2
J02: Antimycotics for systemic use (n = 3)	1	3		1
J04: Antimycobacterials (n = 3)	1	3		1
J05: Antivirals for systemic use (n = 5)	1	3		1	1		1
J07: Vaccines (n = 12)	3	11		1			2
L01: Antineoplastic agents (n = 10)	4	6
L02: Endocrine therapy (n = 4)	1	2					1
L03: Immunostimulants (n = 3)	1	1			2
L04: Immunosuppressants (n = 23)	5	12	3	7	6	2	1		2	4
M01: Anti-inflammatory and antirheumatic products (n = 18)	6	14	1				1
M02: Topical products for joint and muscular pain (n = 2)		1					1
M03: Muscle relaxants (n = 1)							1
M04: Antigout preparations (n = 2)		2
M05: Drugs for treatment of bone diseases (n = 2)		2		1
N02: Analgesics (n = 11)	5	4	1				2
N03: Antiepileptics (n = 9)	1	4					3	2
N04: Anti-Parkinson drugs (n = 1)							1
N05: Psycholeptics (n = 19)	7	11	2	3		1	1	2	3
N06: Psychoanaleptics (n = 17)	5	9	3	1		1	2	5	3
N07: Other nervous system drugs (n =2)		1					2
P01: Antiprotozoals (n = 4)		3		1
P02: Anthelmintics (n = 3)		2		1
P03: Ectoparasiticides, incl. scabicides, insecticides and repellents (n = 2)		1		1
R01: Nasal preparations (n = 2)					1		1
R02: Throat preparations (n = 2)	1	2		1
R03: Drugs for obstructive airway diseases (n = 9)		3		2	1		3
R05: Cough and cold preparations (n = 4)	1	1			1		1
R06: Antihistamines for systemic use (n = 2)		1					1
S01: Ophthalmologicals (n = 4)	1	3		2			1
S02: Otologicals (n = 2)	1	2		1
S03: Ophthalmological and ontological preparations (n = 2)	1	2		1
Other* (n = 6)	4	4					1

Notes: The table present the number (n) of studies using each outcome-exposure group combination as well as the general number of studies using an exposure group (ATC level 2), outcome category (bold font) and subcategory. The table cells are color-coded according to the number size: dark grey (≥10), medium grey (5–9), and light grey (<5). White cells represent no outcome-exposure group combination identified. As one study can have investigated more than one outcome and more than one exposure group, the numbers are not mutually exclusive. *Drug–drug interaction exposures (ATC level 2: B01 + J01, N05 + C03 or C09, L01 + M01, L04 + M01), Phthalate-containing drug exposures,64 or “Torsade de pointes risk drugs”65.

Abbreviation: ATC, Anatomical Therapeutic Chemical Classification.

Mapping Outcomes According to Data Source

Across the 210 included studies, 43 different registries/databases were identified as the data sources of outcomes. Outcome (sub-)categories were mapped to registries as presented in Table 3. The registries have been divided into three categories based on the data holder/permission authority, each stated with the number of studies using data sources within each category: Danish Health Data Authority or Statistics Denmark (n = 188, 90%), RKKP (n = 23, 11%) and Other data holders (n = 17, 8%). In 84% (n = 176) of the studies, at least one outcome was retrieved from one of the following five data sources: Danish Central Person Registry, Danish Causes of Death Registry, Danish National Patient Registry, Danish National Prescription Registry, and the Danish Cancer Registry. Across all the registries, the Danish National Patient Registry was the most common data source of outcomes used in 54% (n = 114) of the 210 studies being the source of Diagnosis in 44% (n = 93), Surgery and procedures in 5% (n = 10), Drug treatment in 1% (n = 2), Disease severity or morbidity in 3% (n = 6), Clinical investigations in <1% (n = 1), Pregnancy and offspring in 6% (n = 13), and Healthcare costs and utilization in 3% (n = 7) of the studies in which the numbers and percentages are not mutually exclusive. Most registries (n = 30, 70%) were only sources of outcomes categorized as Clinical. However, 19% (n = 8) of the registries were coupled to more than one outcome category of which the Danish Rheumatology Database DANBIO was the only data source paired with three out of four outcome categories.

Table 3

Outcomes Mapped According to Data Sources

Studies Included N = 210	Clinical (n = 201)							Society-Related (n = 9)	Healthcare Costs and Utilization (n = 11)	Patient-Reported Outcome (n = 6)
	Mortality (n = 80)	Diagnosis (n = 138)	Surgery and Procedures (n = 17)	Drug Treatment (n = 22)	Disease Severity or Morbidity (n = 13)	Clinical Investigations (n = 7)	Pregnancy and Offspring (n = 21)
Reported Data Source	n	n	n	n	n	n	n	n	n	n
Danish Health Data Authority/Statistics Denmark (n=188)
Danish Central Person Registry (n = 39)	39
Danish Causes of Death Registry (n = 43)	41						2
Danish National Patient Registry (n = 114)		93	10	2	6	1	13		7
Danish National Prescription Registry/ Registry of Medicinal Products Statistics (n = 29)		11		17	2				1
Danish National Health Insurance Service Registry (n=2)					1				1
Danish Psychiatric Central Research Registry (n=20)		9		1		1	9		2
Danish Medical Birth Registry (n = 2)		1					2
Danish in Vitro Fertilization Registry (n = 1)		1
Registry of Legally Induced Abortions (n = 1)							1
Danish Pathology Registry (n = 6)		5				1
Danish Cancer Registry (n = 29)		27					2
Danish Education Registries (n = 3)								3
DREAM Database (n = 2)								2
Danish Central Crime Registry (n = 3)								3
Danish Clinical Quality Program - National Clinical Registries, RKKP (n = 23)
Danish Rheumatology Database, DANBIO (n = 4)				3	1	1			1	3
Danish Dermatology Database, DERMBIO (n = 1)		1		1	1					1
Danish Breast Cancer Group (DBCG) Database (n = 2)		2
Danish Colorectal Cancer Group Database (n = 2)		2
Danish Pancreatic Cancer Database (n = 1)	1
Danish National Acute Leukemia Registry (n = 2)	1	1
Danish National Lymphoma Registry (n = 2)	1				1	1
Danish Registry of Childhood Cancer (n = 1)							1
Danish Heart Registry (n = 1)		1
Danish Vascular Registry (n = 1)		1
Danish Cardiac Arrest Registry (n = 1)		1
Danish Pacemaker and ICD Registry (n = 1)			1
Danish Hip Arthroplasty Registry (n = 1)		1	1
Danish Knee Arthroplasty Registry (n = 1)		1	1
Danish Multiple Sclerosis Treatment Registry (n = 1)					1
Danish Intensive Care Database (n = 1)			1						1
Danish Transfusion Database (n = 2)			2
Other data holders (n = 17)
Danish Twin Registry (n = 1)					1
Danish National Birth Cohort Database (n = 2)										2
Danish Microbiology Database (n = 1)		1					1
Danish Multiple Sclerosis Registry (n = 3)	1			1	2
Western Denmark Heart Registry (n = 2)	2	1	2		1	1			1
Danish Familial Relational Database (n = 1)							1
Danish National Cerebral Palsy Registry (n = 2)							2
Danish Cytogenetic Central Registry (n = 1)							1
Danish Pathology Data Bank (n = 2)		2
Copenhagen Mammography Registry (n = 1)						1
Danish Poison Information Centre (DPIC) Database (n = 1)						1
National Academic Test* (n = 1)								1

Notes: The table present the number of studies using each outcome–registry combination as well as the general number of studies using a registry, outcome category (bold font) and subcategory. The table cells are color-coded according to the number size: dark grey (≥10), medium grey (5–9), and light grey (<5). White cells represent no outcome–registry combination identified. The number of studies is not mutually exclusive as one study can have investigated more than one outcome and used more than one outcome data source. The registries have been categorized according to data holder/permission authority (italic font). *Access to data through the Danish Agency for Information Technology and Learning66.

Reporting of Outcome Quality

One or more quality parameters related to the outcomes were reported in 46% (n = 96) of the included studies. Nine studies (4%) reported reliability of their outcomes (either related to mortality, diagnosis, healthcare utilization or patient-reported outcomes). Positive predictive values were reported in 20% (n = 42) of the included studies and were almost exclusively reported in relation to Diagnosis except for a single study reporting the parameter in relation to (cause-specific) Mortality. Completeness/sensitivity of outcomes were reported in 11% (n = 23) of the studies. The parameter was reported in relation to Diagnosis in 16/23 studies. Other outcomes included Mortality, Surgery and procedures, and Healthcare utilization. In 1% (n = 3) of the studies that reported specificity, all in relation to Diagnosis. The most reported parameter was accuracy/validity (n = 46, 22%) hereof 37 studies reported the parameter in relation to Diagnosis. No studies reported the parameters negative predictive value and responsiveness. As the studies could report more than one parameter, the percentages are not mutually exclusive. In total, 116 unique references associated with the quality of the outcomes and/or the registry source were identified, which are available in the last column of .

Discussion

In this systematic mapping review covering two years of original research publications, we identified outcomes from 210 pharmacoepidemiological studies based on Danish registries and divided them hierarchically into categories (Clinical, Society-related, Healthcare cost and utilization, and Patient-reported outcomes), subcategories and subtypes. It was evident from the hierarchical-structured overview of outcome categorization (Table 1) that the Danish registries support a great variety of outcomes, making a lot of different outcome-analyses possible. Although many different outcomes were identified, Clinical outcomes constituted by far the most utilized category (n = 201, 96%). Diagnosis and Mortality were the top-two subcategories accounting for the outcomes in 80% (n = 168) of the studies. The majority (n = 63, 91%) of exposure groups were also paired to one or more of these two Clinical subcategories. While mainly two registries were sources of Mortality outcomes; the Danish Central Person Registry and Danish Causes of Death Registry, 19 different registries were identified as sources of Diagnosis with the Danish National Patient Registry being the most frequently used (n = 93). The Danish National Health Insurance Service Registry captures information on primary care contacts but no diagnoses linked to the registered contacts. In Denmark, diagnoses are usually registered only when patients have a hospital contact. For some conditions/diagnoses, prescription data can be used as a proxy.34,35 Therefore, the Danish National Prescription Registry was also identified as a data source of Diagnosis. Non-clinical outcomes were only used in 12% (n = 25) of the studies with coupling to 17% (n = 12) of the exposure groups and 30% (n = 13) of the registries. We also found that 46% of the included studies reported one or more outcome quality parameters. Despite an increasing regulatory demand for pharmacoepidemiological studies,5,36,37 only 2% of the studies were reported as imposed by EMA or FDA.

Outcomes

For all exposure groups (n = 69) across therapeutic areas, the Clinical outcomes were dominating. The more frequent use of these outcomes was expected as disease and mortality have traditionally been the outcomes of interest to clinicians.38 These outcomes may also be more similar to traditional endpoints used in RCTs than those categorized as non-clinical,39,40 thereby facilitating bridging between RCT-generated evidence and RWE. Finally, a simple explanation is that the pharmacoepidemiological researcher likely have a background in healthcare-related educations, and therefore thinks of healthcare-related outcomes as the most relevant. Despite most exposure groups (n = 57, 83%) were only coupled to outcomes categorized as Clinical, a minor group of exposures (ATC codes: A14, C07, C09, L04, N05, N06) were paired to three out of four categories. The more diverse use of outcomes of these exposures may be explained by different factors, such as burden of disease, eg psychiatric disorders (treated with antipsychotics and antidepressants covered in ATC codes N05/N06) may be associated with a large burden for the patient and the society by being one of the leading causes of long-term sick leave;41 advances within a therapeutic field, eg treatment of heart failure (covered in ATC codes C07/C09), may allow a shift in focus from Clinical outcomes to other types; availability of disease-specific registries, eg the Danish rheumatology database DANBIO, a disease treated with immunosuppressants (ATC: L04), allow access to a broad range of outcomes,42,43 or potentially due to high treatment expenses.

Data Sources

Despite several reported data sources, most studies (n = 188, 90%) obtained outcomes from the registries governed by Danish Health Data Authority or Statistics Denmark. However, most often the studies only used a small portion of the registries available as the top-five registries; Danish National Patient Registry, Danish Causes of Death Registry, Danish Central Person Registry, Danish National Prescription Registry, and the Danish Cancer Registry accounted for the outcome sources in 84% (n = 176) of the studies. In comparison, the clinical quality databases managed by RKKP were, regardless of a larger number of available databases (85 databases per January 2022),44 less frequently used (n = 23, 11%) data source. The same was evident for the registries owned by Other data holders (n = 17, 8%). The clinical quality databases are often disease- or treatment-specific which may provide more depth in the available clinical information such as diagnostic measurements or scores of disease severity,9,16,19 as compared to the broader registries of the Danish Health Data Authority or Statistics Denmark. However, this specificity may also naturally limit the use of the RKKP databases for studies investigating these specific disease areas or procedures. Generally, the registries held by the Danish Health Data Authority or Statistics Denmark have a long history with national coverage, eg the Danish National Patient Registry was established in 1977,13 well-described variables, and are often considered reliable and valid data sources.9 The clinical quality databases have been established at various points in time from 1960 to 2021, most established after 2005.9,45 Hence, the completeness and history may vary greatly,9 which may also affect the relevance of the additional variables in specific research projects. Finally, the fact that the different data providers/permission authorities use different application platforms or processes for data access, may be a considerable contributor to the current difference in use of registries/databases. In general, access to data for research purposes can be granted if the following requirements can be met: Demonstrated social relevance of the project with justification for the requested data; and demonstrated secure, and safe handling and usage of data with respect for the individual’s right to privacy, which usually takes the form of an authorization of the research institution as well as data processing agreements.9,46 Authorization can only be granted to Danish institutions. Data access for foreign researchers therefore usually requires research collaborations.47 Moreover, as opposed to the data permission process of Danish Health Data Authority or Statistics Denmark’s registries, access to data from RKKP databases require approval through the steering groups of the specific databases.48 This may deter some researchers from applying for the clinical quality data or other research data due to the administrative burden.

Outcome Quality

The reporting of data quality is important as it is a basis for assessing the value of published results. We found that 46% of the studies reported one or more outcome quality parameters. The reported quality parameters were mostly related to diagnosis codes in the Danish National Patient Registry. However, most often the studies referred to the quality of the data source rather than the quality of the specific outcome. Although the validity and completeness of the Danish registries are generally considered to be high, the varying intra-registry validity can have a great influence on the certainty in the findings.9,13 Therefore, we believe that researchers should be extra vigilant in reporting outcome quality either by referring to estimates in validity studies49,50 or by pointing out if relevant documentation of outcome quality is lacking. Such a reporting practice ensures transparency, which gives the reader a better opportunity to assess the quality and reliability of the study results.38

Strengths and Limitations

The present work is, to our knowledge, the first study to systematically map the reported outcomes in pharmacoepidemiological studies across therapeutic areas.20 This study only builds on studies based on data from Denmark, as we believe that knowledge of the data sources used was important to make the mapping review. However, the rest of the Nordic countries have very comparable registries and access to these are to some extent similar. We therefore believe that mapping of their pharmacoepidemiologic outcome studies would yield similar results. However, a detailed and broad comparison of the health data structure has not been performed. The comparison of study outcomes used and the registries available across countries may serve as inspiration for future study designs and may help drive the evolvement and enrichment of research databases and make investigation of rare events possible. The review included studies covering drugs within 68 out of 94 available ATC level 2 codes where the remaining exposure groups included drugs mostly dispensed at hospitals or purchased over-the-counter and therefore either not recorded or only to a small extent in the Danish National Prescription Registry, which is the primary data source of drug exposure.14 The variety of studied exposures across the included studies emphasize that the findings of this review are representative of the research practices in the field of (Danish) pharmacoepidemiology and not only in a few therapeutic areas. Only studies published in English were included in the review. We find it unlikely that this should have caused any bias, as most scientific articles based on Danish registries are published in English.20 We experienced some difficulties in identifying which registries were the data sources of the specific outcomes used. In cases where the registry source of the specific outcome was ambiguous, more than one data source was paired to the specific outcome, which may have led to overreporting of some outcome-registry combinations. In addition, reporting of quality parameters varied from study to study, making it difficult to capture in the text and to determine whether these were related to the outcome. In general, these limitations are not expected to have had a major influence on the results.

Developments in the Coming Years

In the light of conditional approvals of drugs, it might be reasonable to assume that we will observe more studies with clinical investigation outcomes, such as laboratory measurements in the future. These faster reacting outcomes compared to those slower responding, such as mortality, may become important to monitor and assess treatment response. In this context, the Danish nationwide Registry of Laboratory Results for Research made available for research in 201851 could become very important. Moreover, the new National Hospital Medication Registry will provide unique access to information regarding medication administered under the auspices of hospitals52 and may become an important registry for future pharmacoepidemiological studies of especially drugs targeting small populations and rare diseases. In the last five years (2016–2020), the Danish public expenditures on medicine have increased annually.53 Novel orphan products and advanced medical therapies may not only challenge the traditional pathway of evidence generation but also represent a potential economic burden on society due to high expenses of such novel therapies.4,6 This may call for more research on drug effects in a more holistic picture of disease/treatment burden on individuals and on society54–56 using other types of outcomes, such as Society-related or Healthcare costs and utilization measures. Moreover, we might observe more studies investigating drugs in relation to Patient-reported outcomes (n = 6, 3%) as these outcomes are increasingly accepted as valuable supporting information in drug evaluations related to regulatory decisions57 or Health Technology Assessments.58,59 However, this might require increased data availability and confidence in the use of Patient-reported outcomes among researchers. In the context of achieving a more holistic picture of disease and treatment burden, it is also important to highlight some of the most important gaps in the Danish data landscape; despite the availability of administrative registries capturing services provided in primary care60 and home nursing/care,61 these registries capture no diagnosis or reason for the provided service and almost no clinical information. In addition, most registries do not capture information on lifestyle.9 As a consequence, most pharmacoepidemiological studies based on Danish data investigate more severe outcomes (eg, resulting in hospitalization) or use proxy outcomes (eg, new prescriptions) to capture milder cases or diseases treated in the primary sector. Facilitation of future research within disease areas mostly treated in the primary sector would probably require either alteration of what is mandatory to report into these registries or establishment of new registries capturing clinical information from the primary sector. Finally, a common gateway for the application of data from multiple databases including the registries owned by the Danish Health Data Authority and RKKP is under development.46,62 These changes in the data infrastructure may contribute to better exploitation of the Danish health data potential.

Conclusion

The Danish registries support a broad range of outcomes, comprising numerous Clinical-, Society-related, Healthcare cost and utilization, and Patient-reported outcomes, to investigate drug safety and effectiveness in pharmacoepidemiological studies. Across therapeutic areas, drug exposures are mostly investigated against Clinical outcomes, while non-clinical outcomes are more rarely used. A few registries are used much more as outcome data sources than others, such as the Danish National Patient Registry, which is characterized by easy accessibility, good data coverage and clinically relevant outcomes. Specialized clinical databases are more rarely used, which might be a consequence of a potential administrative burden associated with the data access process. In contrast, clinical and biochemical databases, despite potentially offering outcomes with high responsiveness, and the high-quality social and healthcare cost registries were rarely used as outcome data sources. Outcome-related data quality is less frequently described in pharmacoepidemiological studies, which emphasizes a need for improving the reporting practice.