CNODES: the Canadian Network for Observational Drug Effect Studies.

Although administrative health care databases have long been used to evaluate adverse drug effects, responses to drug safety signals have been slow and uncoordinated. We describe the establishment of the Canadian Network for Observational Drug Effect Studies (CNODES), a collaborating centre of the Drug Safety and Effectiveness Network (DSEN). CNODES is a distributed network of investigators and linked databases in British Columbia, Alberta, Saskatchewan, Manitoba, Ontario, Quebec and Nova Scotia. Principles of operation are as follows: (1) research questions are prioritized by the coordinating office of DSEN; (2) the linked data stay within the provinces; (3) for each question, a study team formulates a detailed protocol enabling consistent analyses in each province; (4) analyses are "blind" to results obtained elsewhere; (5) protocol deviations are permitted for technical reasons only; (6) analyses using multivariable methods are lodged centrally with a methods team, which is responsible for combining the results to provide a summary estimate of effect. These procedures are designed to achieve high internal validity of risk estimates and to eliminate the possibility of selective reporting of analyses or outcomes. The value of a coordinated multi-provincial approach is illustrated by projects studying acute renal injury with high-potency statins, community-acquired pneumonia with proton pump inhibitors, and hyperglycemic emergencies with antipsychotic drugs. CNODES is an academically based distributed network of Canadian researchers and data centres with a commitment to rapid and sophisticated analysis of emerging drug safety signals in study populations totalling over 40 million.

The need for drug safety research using an epidemiological approach has been clearly understood for decades.1,2 Prescription medications remain one of the most common causes of severe adverse reactions in clinical medicine, accounting for an estimated 1800 to 10 000 deaths annually in Canada.3,4

Canadian population health databases have been used to assess the risks and benefits of non-steroidal anti-inflammatory drugs (NSAIDs), beta-agonist inhalers for the treatment of asthma, anti-psychotic drugs, gastric-acid suppressants and many other pharmaceutical therapies.5-8 A population-based approach is particularly important for less frequent, severe or long-term adverse effects that cannot be detected by the randomized controlled trials required for initial drug approval. Such trials are not powered for rare outcomes, exclude vulnerable populations and do not provide sufficient follow-up for the quantification of long-term effects.9

Recent experience concerning the cardiovascular effects of cyclo-oxygenase-2 inhibitors and thiazoli-dinediones demonstrates the need to rapidly detect and confirm low relative risks, in the order of 1.2–1.5, to be able to distinguish between individual members of drug classes with respect to their associated risks and to identify clinical factors that increase the risk of adverse drug effects.10,11 This requires very large sample sizes, which can be achieved only through the use of population databases.

To date, such research has suffered from a lack of coordination. For example, investigations of the adverse cardiovascular effects of rofecoxib were conducted by separate teams of researchers using databases in Ontario, Quebec and Saskatchewan.12-14 The time taken to respond to the first report on safety concerns, published in November 2000,15 ranged from 3 to 9 years—an excessive period, considering the potential threat to public health posed by a widely used drug. Investigators used different approaches in designing their studies and analyzing their results, discrepant results were obtained, and individual risk estimates were imprecise. These studies were performed by small academic groups working within a system of competitive funding that rewards individual rather than collective effort.

The challenges are to organize sufficient financial and human resources, to coordinate responses to safety signals, to standardize methodological approaches and to obtain rapid access to data sets that are large enough to give precise estimates of risk. The Canadian Network for Observational Drug Effect Studies (CNODES), an investigator-led, multi-provincial distributed network of data repositories and researchers, has been established to do this.

The development of CNODES

CNODES is part of the Drug Safety and Effectiveness Network (DSEN), a joint initiative of Health Canada and the Canadian Institutes of Health Research (CIHR). The principal aim of CNODES is to use collaborative, population-based approaches to obtain rapid answers to questions about drug safety and effectiveness. Funding for the CNODES infrastructure was granted in January 2011 on the basis of a single, directed, internationally refereed application to CIHR, with representation from 7 provinces (British Columbia, Alberta, Saskatchewan, Manitoba, Ontario, Quebec and Nova Scotia). The application added a mechanism for accessing data from the United Kingdom Clinical Practice Research Datalink (CPRD)—previously known as the General Practice Research Database—in view of its size and the direct and rapid access it provides to comprehensive data, including on drugs marketed in the United Kingdom before they are licensed in Canada.16 Because CPRD is a compilation of electronic health records, it enables adjustment for potential confounders that are not routinely captured in administrative records (e.g., smoking). Work within CNODES began in March 2011.

Database structure

Legal and privacy concerns made it unfeasible to pool data from multiple provinces in a single, central repository. There was a strong preference for a distributed network, as this would enable the data to remain and be analyzed “in situ,” meaning that local regulations concerning data access and approval processes for research would be respected, and local knowledge, expertise and analytical capacity would be valued and supported. The main disadvantages are that those same features can lead to delays in accessing data or obtaining approvals for individual studies. Some barriers to timely data access persist in British Columbia and Quebec.

CNODES data sources are summarized in Table 1. The provincial databases vary in their capacity to answer different questions. For example, if a question concerns a drug used primarily in patients under the age of 65, this can be studied comprehensively in only 3 provinces (British Columbia, Saskatchewan and Manitoba) plus the CPRD.

Table 1

Overview of CNODES Canadian provincial databases and the CPRD

Governance

The CNODES governance structure is depicted in Figure 1. Prospective research questions regarding safety and effectiveness of drugs marketed in Canada come from Health Canada and are directed to the DSEN Coordinating Office. Federal and provincial drug plans may also refer questions to Health Canada. Before consideration by CNODES, the questions are assessed and prioritized by a DSEN Scientific Advisory Committee, which includes staff from Health Canada and provincial drug plans, as well as experts in pharmacy and pharmaco-epidemiology). The CNODES Database Team, comprising the provincial leads, is responsible for ensuring access to the linked provincial data sets and maintaining analytical capacity at each site. If a study is feasible within CNODES, the Database Team selects a Project Team composed of nominees from each province with relevant methodological and content expertise and a liaison from the Methods Team. Data analysts are included at all stages of project planning and discussion. The 15 to 20 members of the Project Team develop the common scientific and analytical protocols, enable the sharing of algorithms and statistical analysis software, organize local approvals for the research and arrange for the analyses to be conducted in each database. The 7 members of the Methods Team are statisticians, biostatisticians or epidemiologists. They provide methodological leadership, assist with the development of the scientific and analytical protocols for each project and conduct meta-analyses of the results across databases. The Methods Team also reports on bias, precision and validity of results, and is charged with developing new methods for the design and analysis of multiple observational studies.

Figure 1

How questions are prioritized and answered by the Canadian Network for Observational Drug Effect Studies (CNODES)

Methods

Linked prescription drug and outcome data are analyzed separately in each province using a range of multivariable techniques; the results from the provinces and the CPRD are combined to give a summary estimate. In the studies conducted to date (see Table 2), a range of methods and tools have been used, including high-dimensional propensity scores to generate and compare exposed and reference cohorts and nested case-control analyses with propensity score adjustment. A study of the association between proton pump inhibitors (PPIs) and pneumonia employed a highly restricted cohort, selected from new users of NSAIDs, to minimize confounding by indication (esophagitis) and reverse causality (PPIs being prescribed for chest pain from pneumonia) (Table 2). This was possible only because of the large source populations.

Table 2

Summary of CNODES projects completed between March 2011 and May 2012

Measures to reduce bias

The complete coverage of provincial populations and subpopulations minimizes selection biases. The initial screening of study questions ensures their appropriateness for evaluation using observational methods and administrative data. There is strong clinical and methodological input into the design of scientific and analytical protocols. The standardization of these reduces variation in results arising from study design and analysis. We use common exposure, covariable and outcome definitions and make maximum use of shared common, tested, stattistical analysis (SAS) code. Sophisticated analytical techniques reduce confounding (see examples in the previous section on methods). All research protocols are logged at the CNODES and DSEN coordinating offices, and will be registered with Clinicaltrials.gov. All study outcomes and analyses are pre-specified, and site-specific protocol deviations are permitted for technical reasons only. Each participating site lodges results in a secure drop box without knowing the results obtained at other sites. The Project Team and Steering Committee jointly review and interpret results. All reports are reviewed and approved by a Publications Committee, which currently consists of the CNODES executive but will include two additional external members in future. These procedures should reduce publication and reporting biases.17,18 All CNODES researchers are required to make a full disclosure of their relationships with life sciences companies. Open-ended relationships, such as stock holding and membership in speakers’ bureaus are discouraged; researchers with such relationships cannot take primary responsibility for leading specific CNODES research projects, including developing study protocols or drafting study reports.

Studies undertaken to date

CNODES has begun 5 studies since March 2011. Analyses were completed within 12 months for 3 of these studies (Table 2). Initial study questions were designed to use data from all member provinces. The details of the studies illustrate the capacity of the network to assemble very large cohorts, to agree on standardized analytical protocols and to carry out suitable adjustments simultaneously in multiple data sets. Early experience indicates that important questions can be addressed in most provinces within 4 months. Full results of studies are being published separately and are not provided here. Briefly, the results documented a small increase in the relative risk of acute kidney injury with high- versus low-potency statins and no increase in the risk of community-acquired pneumonia with proton pump inhibitors.19,20

Limitations

The main limitations are those of the data sets themselves. Eight of the 9 repositories used by CNODES comprise linked health administrative data, which generally lack information on risk factors such as obesity, smoking, non-prescription drugs and complementary therapies.21 The CPRD database in the United Kingdom comprises electronic health records that include some of this information,16 and we found no difference between the CPRD and the provincial administrative data in the results obtained to date. Databases do not include hospital drug use, and they capture information on drug prescribing (CPRD) or dispensing, rather than consumption. Controlled observational studies are susceptible to a range of biases, most significantly information bias and confounding by indication and disease severity.22 These are limitations of all non-randomized pharmaco-epidemiological studies, but reviews suggest that there is good agreement between properly designed non-randomized and randomized studies in the investigation of the adverse effects of drugs.23

Capacity building and knowledge translation

Training and knowledge translation teams have been established to support the work of CNODES. Key objectives of the network are to train graduate students and analysts, and to inform clinical practice and health care programs and policies.

Discussion

The main advantages of this multi-provincial distributed network are the generalizability, statistical power and timeliness of the study results. The standardization of design and analyses minimizes variation. The large group of researchers in 7 provinces is building substantial capacity in pharmaco-epidemiology across Canada.

In establishing CNODES we considered that a distributed network in which data are analyzed in each province and outputs are combined by meta-analysis would be the only feasible model. This overcame the substantial legal and privacy obstacles to pooling the raw data. However, the distributed network approach has not guaranteed timely access to data in every province. There were, and still are, substantial delays in British Columbia and Quebec.

CNODES exploits the scientific principle of replication. A single study is rarely sufficient to provide definitive information on a serious adverse drug effect unless it finds an extremely high risk for a drug. In CNODES the drug-disease association is examined in up to 9 databases simultaneously, and early experience has shown that heterogeneity in the estimates is low. This contrasts with the typical situation, in which studies are conducted in a haphazard, sequential manner over several years before a meta-analysis is performed to assess the totality of the evidence.10 In CNODES, meta-analyses are planned prospectively as part of the study protocol, providing immediate summary evidence. The value of such a network is illustrated by the study of high-dose statins and acute kidney injury, which explored an uncommon, serious outcome and a small excess risk with a widely used medication.19

CNODES exploits the enormous statistical power that results from combining provincial health databases. This discriminatory power is necessary to detect small but important increases in risk and to make comparisons between drugs, so that benefit–harm relationships can be quantified and contrasted. Drug regulation is moving from a traditional “all or nothing” attitude to safety to a “risk management” approach throughout the life cycle of the drug.24 This necessitates more rapid access to larger and more comprehensive data sets and people with the skills to analyze them.

The strategy used to establish CNODES differs from that applied in the creation of Mini-Sentinel, a prominent drug safety network in the United States. Mini-Sentinel is a distributed network of 17 data partners representing a total population of 99 million.25 Much development work went into the creation of data extraction tools that generate identical core data sets within each node. These data nodes can be queried directly and simultaneously from the coordinating centre using common code with rapid response times (days rather than weeks).25 Although CNODES is not capable of functioning on this time scale, it has the advantages of complete population coverage and the academic and methodological capacity that is being built at each provincial site. This new expert network will serve the long-term interests of Canadians with respect to drug safety.

Currently, CNODES has access to data from a total population of over 40 million, and many pharmaco-epidemiologists in Canada either already have or will have some connection with the enterprise. Studies that are independent of CNODES will continue and hopefully increase as a result of the larger capacity engendered by the network. The first real test will be the ability of CNODES to quantify the next major drug safety signal. Experience suggests that we will not have to wait very long for that event.