Electronic Alerts for Acute Kidney Injury Amelioration (ELAIA-1): a completely electronic, multicentre, randomised controlled trial: design and rationale.

INTRODUCTION: Acute kidney injury (AKI) is common among hospitalised patients and under-recognised by providers and yet carries a significant risk of morbidity and mortality. Electronic alerts for AKI have become more common despite a lack of strong evidence of their benefits. We designed a multicentre, randomised, controlled trial to evaluate the effectiveness of AKI alerts. Our aim is to highlight several challenges faced in the design of this trial, which uses electronic screening, enrolment, randomisation, intervention and data collection. METHODS AND ANALYSIS: The design and implementation of an electronic alert system for AKI was a reiterative process involving several challenges and limitations set by the confines of the electronic medical record system. The trial will electronically identify and randomise 6030 adults with AKI at six hospitals over a 1.5-2 year period to usual care versus an electronic alert containing an AKI-specific order set. Our primary outcome will be a composite of AKI progression, inpatient dialysis and inpatient death within 14 days of randomisation. During a 1-month pilot in the medical intensive care unit of Yale New Haven Hospital, we have demonstrated feasibility of automating enrolment and data collection. Feedback from providers exposed to the alerts was used to continually improve alert clarity, user friendliness and alert specificity through refined inclusion and exclusion criteria. ETHICS AND DISSEMINATION: This study has been approved by the appropriate ethics committees for each of our study sites. Our study qualified for a waiver of informed consent as it presents no more than minimal risk and cannot be feasibly conducted in the absence of a waiver. We are committed to open dissemination of our data through clinicaltrials.gov and submission of results to the NIH data sharing repository. Results of our trial will be submitted for publication in a peer-reviewed journal. TRIAL REGISTRATION NUMBER: NCT02753751; Pre-results. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

RCT Entities:   Population Interventions Outcomes

This manuscript discusses the design, ethical framework and plan of execution of a multicentre randomised controlled trial of electronic acute kidney injury alerts.

A multicentre, randomised controlled trial which is the first of its kind to rigorously test the efficacy of an electronic alert at multiple hospitals before broad implementation.

Carefully designed selection criteria to reduce the rate of false positives.

Complete reliance on the electronic medical record for subject screening, enrolment and randomisation and delivery of the intervention, which reduces cost and increases scope of data collection, efficiency and generalisability.

Reliance on the electronic medical record presents limitations in alert design and randomisation methods.

Introduction

Acute kidney injury (AKI) is defined as an abrupt decrease in kidney function, which results in the accumulation of metabolic waste products, as well as dysregulation of volume status, electrolyte and acid-base balance. This condition is common, estimated to occur in about 5%–20% of all hospitalised patients, and carries a significant, independent risk of mortality of up to 20% in some studies.1–3

International guidelines for the treatment of AKI focus on appropriate management of drug dosing, avoiding nephrotoxic exposures and careful attention to fluid and electrolyte balance.2 Early nephrologist involvement may also improve outcomes in the care of AKI.3 Without appropriate provider recognition of AKI, however, none of these measures can be taken, and patient outcomes may suffer. Unfortunately, AKI (which is asymptomatic) is frequently overlooked by clinicians and carries a substantial cost, morbidity and mortality burden.1 In a prior study at a tertiary care academic hospital, we found that only 43% of patients had documentation of AKI in the medical record and that AKI documentation was associated with decreased mortality, adjusted for admission type and severity of illness.4 Further, patients with AKI often continue to be exposed to kidney-toxic medications.5 6

Automated alert systems have emerged as a strategy to influence clinician detection of specific clinical states and subsequent behaviour. Several randomised trials have demonstrated the efficacy of using alerts, particularly in minimising drug interactions in hospital settings.7–11 In 2014, our group was the first to conduct a pilot, randomised trial of electronic alerts for AKI.12 The trial, which randomised 2373 patients with AKI, found that alerting a single physician to the presence of AKI did not improve the course of AKI or reduce dialysis or death rates. Our pilot study demonstrated that there is clinical equipoise regarding the effectiveness of alerting and that alerting to the presence of this condition should not be considered standard of care.

However, our pilot study had several limitations which we address in a new randomised trial. The prior study was conducted in a single hospital, and the alert itself did not describe specific actions that a provider could take in response to the alert. The alert was delivered only once, to a single provider (and a unit pharmacist) without contextualisation (ie, it occurred outside of the relevant electronic health record). In the present trial, we expand on our prior study to determine the efficacy of an electronic alert system to modify provider behaviour and reduce patient outcomes. We hypothesise that an electronic AKI alert with an attached AKI-specific order set will improve best practices in regards to care of hospitalised patients with AKI and improve rates of progression of AKI, dialysis or death in hospitalised patients. This trial is notable for its reliance on the electronic health record (EHR) to screen, enrol, randomise and deliver the intervention to patients. It further differs from our pilot study in that alerts are integrated at the point of care, that they are delivered to multiple providers and in its use of a multicentre design which allows for assessment of heterogeneity of alert effect across different hospital types with diverse patient populations.

Methods and analysis

The study design was approved by the Yale Institutional Review Board (Yale IRB# 1604017596) and is registered under clinicaltrials.gov NCT02753751. It operates under a waiver of informed consent (see ethics section). The protocol conforms to the principles of the Declaration of Helsinki and the full study protocol is accessible at www.akistudy.org. This manuscript was submitted using the SPIRIT reporting guidelines.13

This is a multicentre, parallel-group, randomised, controlled trial to evaluate the efficacy of an AKI alert system for hospitalised patients with AKI. The six participating centres are described in table 1 and were selected on the basis of their shared use of an electronic health record (Epic Systems, Verona, Wisconsin, USA). AKI alerts were not previously present at any of the sites, with the exception of Yale New Haven Hospital, where we piloted the alert for a month prior to beginning the trial (discussed below). This piloting phase was followed by a month-long washout period where no alerts were firing to reduce contamination of the study. The trial began on 26 March 2018 and is expected to enrol patients for 1.5–2 years.

Table 1

Participating centres in the ELAIA-1 trial

Institution	Location	Type	Teaching	Beds
Bridgeport Hospital	Bridgeport, Connecticut, USA	Community	Yes	383
Greenwich Hospital	Greenwich, Connecticut, USA	Community	Yes	206
The Hospital of St. Raphael	New Haven, Connecticut, USA	Community	Yes	511
Lawrence and Memorial Hospital	New London, Connecticut, USA	General/acute care	No	280
Yale New Haven Hospital	New Haven, Connecticut, USA	Acute/Tertiary	Yes	1030
Westerly Hospital	Westerly, Rhode Island, USA	Community	No	60

Patient and public involvement

Patients or the public were not involved in the development of the research question, design of the study or outcome assessment.

Participants

All inpatients 18 years of age or older at the six participating centres who develop AKI will be automatically enrolled into the trial. The Kidney Disease: Improving Global Outcomes (KDIGO) guidelines define AKI as an increase in serum creatinine concentration of 0.3 mg/L above baseline within 48 hours or a relative increase of 50% above baseline within 7 days.2 Because of limitations within Epic’s best practice alert framework, we define AKI in our study as a 0.3 mg/L increase above the lowest proceeding creatinine value within a 48 hours period or a 50% increase above the lowest proceeding creatinine value within a 7-day period. This slight variation on the KDIGO definition avoids the need for imputation of a baseline creatinine value and potentially artificially prolonged AKI duration. AKI may also be defined by urine output criteria; however, because it is difficult to collect the necessary output data in most (non-intensive care unit) patients, this component was not used in our definition.

Exclusion criteria

Our exclusion criteria are designed to reduce the rate of ‘false-positive’ alerting (alerts sent for individuals without true AKI). Patients with initiation of dialysis prior to AKI onset and with an end-stage kidney disease (ESKD) diagnosis code will be excluded. Patients with an initial serum creatinine ≥4.0 mg/L will also be excluded due to a lack of consensus definitions of AKI in this population. Patients admitted to hospice services or who are made ‘comfort measures only’ will also be excluded, as the use of an electronic alert is not expected to impact their care. We will also exclude patients within 6 months of kidney transplant, as these individuals are monitored closely for changes in kidney function when hospitalised.

Intervention

Patients will be randomised to either usual care or to the AKI electronic alert system (Alert). The alert consists of a ‘pop-up’ generated within the EHR when the provider accesses a patient record (figure 1).

Figure 1

The ‘pop-up’ electronic alert. The alert gives relevant information regarding recent creatinine values and provides access to an AKI order set as well as relevant trial information. AKI, acute kidney injury.

The electronic alert text was designed to inform providers of the presence of AKI in their patient as well as provide a minimum and maximum creatinine value within the prior 7 days. Therefore, the language was kept broad and simple. The alert reads ‘Your patient has been identified as having AKI. Relevant creatinine values over the last 7 days are listed below’. The most recent creatinine value as well as the lowest and highest values in the past 7 days, will be listed. This is followed by the following statement “THIS ALERT DOES NOT FIRE FOR ALL PATIENTS. This patient is part of a randomized trial. For more information, click here: www.akistudy.org’. The electronic alert also includes a link to an AKI order set, which includes labs and imaging to further work-up AKI. The order set was designed so as not to promote or increase the use of any one particular therapeutic strategy, as this could vary from patient to patient. While no patient-specific guidance or recommendations are made, our trial website does include a list of KDIGO clinical practice guidelines for AKI care, which can be referenced by clicking the link in the alert. Finally, at the bottom of the alert, an option to either ‘agree’ or ‘disagree’ with the alert is provided.

In our previous study, only one primary in-house provider received an alert per patient; however, we hypothesised that creating a more comprehensive alerting system may improve alert efficacy. Providers who will receive an alert include physicians, physician assistants, nurse practitioners and advanced practice registered nurses. Any of the above type of providers, regardless of relationship to the patient, will receive the alert when the patient’s chart is opened. This population of providers was chosen as (unlike nurses, pharmacists or medical students) they are able to enter and discontinue diagnostic and treatment orders that may impact the course of AKI.

Mode and invasiveness of alerting were important considerations of the present study. In our previous study, we used a text-based paging system for alerting providers to the presence of AKI. This was a minimally invasive approach that was disconnected with provider activity in the EHR. In this trial, the alert occurs at the point of care and is linked to both an AKI order set containing generic options for further workup as well as a link to our study website that contains evidence-based practice guidelines.

Alert frequency

The AKI alert will be displayed to the relevant provider whenever the patient’s chart is opened while they have AKI. If the provider ‘dismisses’ the alert, it will continue to ‘pop-up’ on each subsequent opening of the patient’s medical record by that provider. The alert will stop firing for the provider under the following conditions:

The provider acknowledges the alert by ‘agreeing’ that AKI is present or by ‘disagreeing’ that AKI is present with an accompanying explanation (alert will then be suppressed for 48 hours).

The patient’s most recent creatinine no longer meets criteria for AKI.

The patient receives an order for haemodialysis, continuous renal replacement therapy or peritoneal dialysis.

The patient is transferred to the hospice service, is made comfort measures only or dies.

The patient is discharged from the hospital.

Though we recognise that repeated alerts may become onerous and lead to alert fatigue,14 we felt that physicians may not recognise the presence of AKI or fully read the alert if only provided with one alert. In addition, we wanted to study the utility and usefulness of the AKI order set and/or use of the link for KDIGO clinical practice guidelines, which would be more likely to occur if providers view and read the alert multiple times. In order to counteract potential alert fatigue, we do give providers the option to suppress the alert, as stated above. Further, because our definition of AKI is based on changes in creatinine compared with a lowest previous creatinine value within either 48 hours or 7 days, alerting will stop if a patient’s creatinine remains unchanged or undergoes little change, for an extended period of time that would take the patient outside of this window. This can help reduce alert fatigue by stopping alerts on patients for which AKI is presumably already well-known.

We are also aware that repeated alerting may lead to variable ‘dosing’ of the intervention. Because our prior study involved one alert per patient, a uniform intervention was guaranteed for all patients in the alert arm. Here, it is feasible for patients to experience different ‘doses’ of the alert, dependent on the duration of AKI, frequency and timing of EHR access and provider response to the alerts. As such, this trial is best conceptualised as an attempt to measure the effectiveness of an alert protocol rather than of an individual alert per se.

Randomisation

Simple randomisation is achieved within the Epic EHR system using an internal random number rule. Randomisation occurs the first time the patient’s chart is opened by an eligible provider after an AKI-defining creatinine value has been reported into the EHR. If the patient has AKI according to KDIGO creatinine criteria, and if the patient meets all other inclusion and exclusion criteria, the patient is automatically enrolled and randomised. Once randomised into either arm, the patient remains in this arm for the duration of their hospital stay. Beyond the primary intervention, no further tests or procedures will be performed on subjects in this trial.

Though commonly used in clinical trials, a permuted block randomisation method was not used as this EHR has neither the functionality to generate permuted block lists or to import external randomisation lists. Allocation concealment is maintained as the alert process is completely automated and performed within the EHR. In addition, we chose not to randomise at the provider level because this was deemed infeasible; patients at the participating hospitals are cared for by multiple providers, who may change during the course of a patient’s hospital stay. While cluster randomisation is a commonly used strategy that could reduce the risk of contamination across study arms, this method was deemed infeasible in this study given the limited number of clusterable entities (ie, six hospitals).15 Ward-based clustering would not be feasible given the fact that physicians (especially consultants) see patients throughout a given hospital. Additionally, because our six study sites range from small community hospitals to larger tertiary care centres, it would be difficult to assess differences between study arms containing confounders arising from potentially vastly different patient and provider populations. Performing simple randomisation at the patient level will allow for subanalyses of alert efficacy independently at each hospital and in individual wards. Stepped-wedge clustering has also been increasingly used in the evaluation of interventions related to service delivery. This method allows for both intercluster and intracluster comparisons. However, the effect across study arms is likely to be confounded by unanticipated temporal and seasonal trends. Further, it would be difficult for study investigators to remain blinded as the time of crossover would be known. Both study designs have inherently greater statistical complexity to account for intracluster correlation and reduced statistical efficiency that does not outweigh their advantages. Given these considerations, we believe that a simple randomisation scheme would be best for our study design; however, we do recognise that this does have important implications with regards to contamination (see below).

Blinding

Participants and the study team will be blinded to the intervention, though obviously care providers will be aware of treatment assignment.

Clinician outreach

While the unit of randomisation is the patient, clinicians may also be considered subjects of this research. We will engage in pretrial and periodic outreach to all clinicians who may be exposed to this study, informing them of the nature of the study, the fact that it is a randomised trial and that alerts do not fire for all patients with AKI. We will additionally inform them that limited data are being collected regarding provider behaviour. The pretrial education will occur in the form of short education presentations at group and departmental meetings given by either the study’s principle investigator (a clinical nephrologist) or a study coordinator. The study coordinator will also be at each site when the alerts become active in order to provide further provider education and answer any questions. Periodic site visits by the study coordinator or check-ins with local site investigators will occur on a monthly basis at each site to ensure that the alerts are functioning correctly and to reeducate any new providers on the floors. While we believe clinician education is important, we feel it is best that this process remain relatively simple to allow for broader adoption in the future should alerting prove beneficial. Further, the alert pop-up contains methods to contact the study team. Most notable, if ‘disagree’ is clicked, a free-text box is opened that allows providers to communicate their concern directly to the team. While piloting the popup in pretrial activities, we used these responses to further tailor the language of the alert. We will also make it clear that data subject to clinician behaviour (such as AKI documentation) will NOT be linked to individual clinicians.

Primary outcome

The primary outcome will be a composite of progression to a higher stage of AKI, inpatient dialysis and inpatient death within 14 days of randomisation, chosen such that we can objectively measure hard clinical outcomes of AKI that can be easily extracted from the EHR. Severity of AKI is strongly associated to longer-term outcomes, such as chronic kidney disease and ESKD, while dialysis and death allow us to capture events that limit the rise in creatinine and that, if not accounted for, would potentially lead to missed cases of severe AKI. We chose this time frame for outcome assessment as the effect of an AKI alert might be diluted over time as more issues arise during the hospitalisation. However, we also recognise that this time frame will potentially capture outcomes that occur in a period of time representing acute kidney disease, a period of continued kidney dysfunction after AKI.16

Secondary outcomes of interest

Secondary outcomes of interest are listed in table 2. Many of these are process measures, as we are particularly interested in measures that may change as a result of AKI alerts. In determining secondary outcomes of interest, we needed to balance outcomes that would be of interest to clinicians and other care providers with the feasibility of accurately determining those specific outcomes. This was particularly relevant for ‘best practices’ such as dose-adjustment for medications, which can be somewhat subjective and thus requires direct chart review for determination.

Table 2

Secondary outcomes of interest

Endpoint and definitions	Data source
Mortality outcomes
Inpatient mortality	Hospital record
Dialysis outcomes
Inpatient dialysis	Order entry system
Discharged on dialysis	Social work records
Renal failure outcomes
Percent who progress to Stage 2 AKI	Laboratory values
Percent who progress to Stage 3 AKI	Laboratory values
Duration of AKI	Laboratory values
Readmission rate and costs
30-day readmission rate	Hospital record
Cost of index hospitalisation	Billing records
Individual ‘Best Practice’ outcomes (proportion achieved per patient in study arm during index hospitalisation)
Contrast administration	Order entry system
Fluid administration	Order entry system
Aminoglycoside administration	Order entry system
NSAID administration/cessation	Order entry system
ACE inhibitor administration/cessation	Order entry system
Urinalysis order	Order entry system
Documentation of AKI	Postdischarge ICD-10 codes
Monitoring of creatinine	Order entry system
Monitoring of urine output	Hospital Record
Renal consult	Direct chart review
Provider awareness outcomes
Chart documentation of AKI (by post discharge ICD-10 codes)	Billing records
Chart documentation of AKI (adjudicated)	Direct chart review

ACE, angiotensin converting enzyme; AKI, acute kidney injury; NSAID, non-steroidal anti-inflammatory drug.

To operationalise our ‘duration of AKI’ endpoint, we define ‘C1’ as the AKI-defining creatinine and ‘C0’ as the lowest preceding creatinine in the past 48 hours or past 7 days depending on which KDIGO AKI criteria was met. For those defined by both, C0 will be the lower of the two creatinine values. Cessation of AKI will occur when a subsequent creatinine measure is within 0.3 mg/L or 50% of C0, again depending on the KDIGO AKI criteria initially met (and within both if both criteria were met). While C0 may not represent true ‘normal’ kidney function for a patient, selecting this time point avoids imputation of a baseline while avoiding potentially artificially prolonged AKI duration. Further, as it is conceivable that patients can be discharged prior to recovery and receive no follow-up creatinine measurements, we will evaluate differences in duration of AKI through the use of Kaplan-Meier estimators, allowing for censoring at death or discharge.

Subgroup analysis

The effect of the alert may differ based on several patient characteristics. We have therefore prespecified several subgroups of interest that are outlined in table 3 and that will be considered hypothesis-generating for the future design of targeted alerts towards populations that are more likely to benefit from an alert system. Additionally, because we are enrolling patients across six hospitals that vary in size, type and patient population, we will perform an exploratory analysis to determine the efficacy of alerts independently at each site. This analysis will employ logistic regression with a site-by-randomisation interaction term to allow for simultaneous assessment of site-by-site baseline event rates and the site-by-site effect of AKI alerts.

Table 3

Planned subgroup analyses and justification

Subgroup of primary interest	Justification
Surgical patients (defined by admission to a surgical team)	Risk of underdocumentation (reference)
Subjects with baseline creatinine<1.0 mg/L	AKI occurs when creatinine in ‘normal range’
Subjects with baseline creatinine<0.5 mg/L	AKI occurs when creatinine in ‘normal range’
Females	Lower rate of creatinine increase after AKI (reference)
African Americans	Higher rate of creatinine increase after AKI (reference)
Elderly (age>65, age>70 and age>75)	Lower rate of creatinine increase after AKI (reference)
Subjects in an intensive care unit at the time of the alert	AKI may be overlooked in the setting of multiple clinical problems
Subjects who enter the study based on a 50% increase in creatinine vs. a 0.3 mg/L increase in creatinine vs both	Clinicians may be less likely to recognise a 0.3 mg/L change vs a 50% change

Contamination

As providers are not randomised and will be aware of patients who are randomised to the experimental arm, there is a risk of contamination of the intervention. Providers may use the information provided in the AKI alert (ie, definition of AKI, best practices with respect to AKI care and so on) to improve their ability to detect AKI in patients not randomised to the experimental arm. In addition, improved knowledge with respect to the definition of an AKI and its appropriate management may improve the ability to detect AKI over time. In order to address this issue, we will examine the outcome rate in the control arm over time; if the outcome rate in the control arm improves over time, this may suggest contamination.

Beyond that, we will establish a pretrial baseline cohort of patients that would be enrolled were the trial actively recruiting by retrospectively collecting a year’s worth of pretrial patient data from each study site. While temporal shifts in treatment may change outcomes over time independent of alerting, a significant improvement in AKI outcomes in the control arm of the trial versus the pretrial cohort would further suggest contamination.

Finally, it is possible that providers exposed to alerts may actually be at risk of increased inattention to AKI in patients of the control arm, as they may become accustomed or dependent on receiving an alert as recognition of AKI. We will attempt to mitigate this through periodic outreach to clinicians and explicitly stating on the alert that not all patients with AKI trigger an alert.

Statistical analysis

The primary outcome will be analysed as a simple combination of progression of AKI, dialysis, and death at 14 days after randomisation or at discharge (whichever comes first). If any one of these three elements is positive, the composite outcome will be considered positive. The primary analysis will use the intention to treat principle. The proportion of patients who experience the primary outcome in the intervention and control groups will be compared by the χ² test with Mantel-Haenszel correction for the six study strata (by hospital). Statistical significance will be based on a two-sided p<0.05. As all prespecified secondary outcomes are categorical in nature, these will be similarly analysed, using the χ² test with Mantel-Haenszel correction. We will not be correcting for multiple testing, especially because many of our secondary outcomes are likely correlated, making a true Bonferoni correction overly conservative. Therefore, we consider these outcomes as hypothesis-generating only, and any significant findings should be further explored.

Power and sample size considerations

To estimate the sample size, we conducted a retrospective analysis of patients with AKI at three of the six study hospitals. The composite outcome of progression of AKI, dialysis or death occurred in 24.5% of 29 027 individuals with AKI in this analysis. A 20% reduction in this proportion (to 19.6%) would be clinically meaningful. To that end, a sample size of 2512 in each arm achieves 90% power to detect a difference this large at a two-sided alpha of 0.05. This was calculated with the PASS software package V.13.0,17 using the continuity-corrected form of the Cochran-Mantel-Haenszel test to account for the six hospital strata.18 19 This gives a total population of 5025 individuals with AKI. We have elected to increase this number by 20% to account for potential contamination of the effect across study arms, leading to a final sample size of 6030 individuals. In addition to adequately powering for the primary clinical outcome, this sample size will allow us to detect at least a 16% increase in the odds of more best practices being completed in the intervention group.

Interim analysis

We plan to have one interim analysis at the mid-point of the trial when 50% of patients have been enrolled. The interim analysis will allow us to alter the sample size or stop the trial earlier for ethical considerations, unexpected adverse events or high efficacy. The trial will stop for declaring efficacy if the effect size is large. We will use the O’Brien and Fleming stopping rule to stop the trial at a p-value of 0.001 for efficacy.20 Alerting harm will be also be assessed using the primary outcome, but the threshold for stopping the study will be greater, at a p value of 0.005. The DSMB will be unblinded to the study outcomes for these assessments, but the study team will remain blinded throughout.

Preintervention data

Pilot in the Medical Intensive Care Unit (MICU)

Prior to the implementation of the electronic alert across all six hospital systems, we piloted the alert from 01/08/2018 to 02/08/2018 in the medical intensive care unit at Yale-Haven Hospital. The purpose of this pilot phase was to evaluate the appropriateness of alerting, to solicit feedback from providers and to ensure that the electronic methods of data capture were valid. There were 77 patients randomised (37 to alert and 40 to control). The alert fired a total of 2355 times, a median (IQR) of 48 (23–89) alerts per patient. Of 509 providers eligible to receive alerts, 323 providers received at least one, with a median of 1 (0–9) alerts per eligible provider (figure 2). The median number of providers per patient was 17 (9–24), which may explain the low number of alerts seen per provider despite a high total number of alerts fired. The maximum number of alerts received by a single provider over the 30 days of the pilot was 78 alerts from 12 different patients. That provider was an acute care nurse practitioner assigned to the MICU for the duration of the pilot. Median alert duration was 0.84 (0.47–1.64 days (figure 3).

Figure 2

AKI order set. This order set can be opened directly from the electronic alert and contains generic options for further work-up. AKI, acute kidney injury.

Figure 3

Histogram demonstrating the number of alerts received by providers during the pilot phase.

Outcomes for all randomised patients were as expected for a medical intensive care unit population. Inpatient death occurred in 29 (37%) of patients, while 4 (5.1%) were discharged home. The remainder of patients were discharged to a nursing facility, hospice or transferred to another medical facility. In terms of AKI outcomes, the majority of these patients (52%) never had progression of AKI, 29% progressed to stage 2 and 18% progressed to stage 3.

Several iterations to the alert were made over the course of the pilot in response to provider feedback and internal testing. These are summarised in table 4:

Table 4

Summary of changes made to alerting during the piloting of the alert system

Change	Motivation
Excluded patients with ‘deceased’ status	Occasional alerting on patients who had recently died was ghoulish and unactionable
Modified language to make it clear that ‘agreeing’ with alert suppresses future alerts for themselves only	Providers are eager to find ways to suppress alerts once they have been alerted
Extended the ‘prior dialysis’ exclusion criterion to 1 year	Some chronic dialysis patients would initially lead to an alert
Excluded providers who cannot enter orders	AKI Order set is not useful for individuals (such as nurses, medical students) who are unauthorised to enter orders.

AKI, acute kidney injury.

Ethics and dissemination

Ethical issues

This study posed several ethical issues that are worthy of discussion. First, in order to efficiently proceed with the study, we obtained a waiver of informed consent. United States federal guidelines require that in order to obtain a waiver of consent, (1) the research pose no more than minimal risk to the subject, (2) the waiver not adversely affect the rights and welfare of the subject, (3) the research could not be practicably carried out without a waiver and (4) whenever appropriate, the subjects be provided with additional pertinent information after participation. We felt that our study met all of the criteria noted above to qualify for a waiver of informed consent.

Subjects will not be informed of their randomisation status or participation in this trial as the trial could not be feasibly performed if subjects were told they were enrolled. We do not feel that postfacto informing of patients randomised in this trial is appropriate for several reasons. First, there is no guideline-based specific follow-up or intervention for AKI. Second, many patients may incorrectly assume that AKI is an iatrogenic condition, caused by poor medical care, when this is not always the case and, rather, their AKI is indicative of the severity of the underlying medical condition. Finally, most patients will not be familiar with ‘acute kidney injury’ and informing them of the presence of the condition may engender significant stress or anxiety without offering a tangible benefit. Because the intervention (alert) is a tool to make a provider aware of information already obtainable from the EHR, it is at the discretion of the provider to inform the patient of any relevant information regarding their AKI diagnosis, severity and prognosis. We believe that the determination of the clinical impact and significance of AKI for a given patient rests with the primary providers and trust that they will act ethically with regard to the disclosure of the relevant medical information.

Prior to pursuing a waiver of informed consent, however, we weighed the issues of patient autonomy with the feasibility of actually obtaining consent from each patient. In order to obtain consent, we would need to either rapidly enrol all patients with an AKI at the moment of their AKI occurrence or prospectively inform all patients about the possibility of developing an AKI so that they would have already been consented at the moment of their AKI occurrence. The former method would be inefficient and use significant time and effort by study personnel, as about 10% of all hospitalised patients experience AKI. The latter method would risk loss of confidentiality for a significant number of hospitalised patients, ~90% of which will never go on to develop AKI. In addition, informing patients about the presence of an AKI will act as a separate alert of sorts, as patients in the control arm may inadvertently relay this information to providers or be placed in the position of withholding information to providers (in both the control and experimental arms), which may undermine the physician-patient relationship.

We also felt that the harm to patients with a waiver of informed consent will be minimal, as informing providers of the presence of an AKI is a low-risk intervention relying on a novel presentation of data that is theoretically already available.

Data dissemination

As we recognise the novel strategies and potential impact of our trial, we are committed to the open and timely dissemination of our data. Our trial has been registered with clinical trials.gov (NCT02753751) and will be continually reviewed and updated. We intend to submit the results of our trial no later than 1 year following the completion date and will include aggregate-level primary and secondary outcomes, participant demographics, statistical analyses and any adverse events. We also intend to disseminate information through publications and through the submission of our results the NIH data sharing repository.

Discussion

AKI significantly increases the risk of morbidity and mortality in hospitalised patients. We designed a multicentre, randomised, controlled trial to determine whether the use of an AKI alert system will improve outcomes with regards to patients with this condition. The design of this trial was challenging for several reasons, presented above and summarised here. First, we needed to create a novel electronic alert system specific to this clinical trial; to do this, we needed to work within the limitations of the Epic electronic medical record system. Second, the choice of a composite outcome of progression to AKI, inpatient dialysis and inpatient death within 14 days of randomisation was carefully chosen. Our process outcomes were carefully chosen as well, with a balance between utility of the best practice outcome and feasibility of measurement. Finally, the ethical issues associated with a lack of informed consent were carefully considered.

Randomised trials are of utmost importance to prevent implementation of alert systems that lack any demonstrable benefits on clinician behaviour or patient outcomes and may precipitate unforeseen consequences or burdens on the healthcare system.21 The potential utility of an alert system is complicated by a variety of patient-specific and provider-specific factors that must be considered before its implementation. Positive outcomes on clinical efficacy should be weighed against potential risks. As an example, one frequently documented phenomenon, alert fatigue, is a decreased attention to alerts due to frequent or overabundant alerting.22–24 This may lead to lack of efficacy in the studied alert and it can negatively impact pre-existing alerts once deemed successful. Further, as alert override from physicians is a common problem of current alerting systems, careful thought must be put into design and implementation of the alert so as to create elements that are likely to increase provider adherence and thus improve alert success.25–29 User feedback and positive user perception of the benefits of alerting are critical in creating successful alert systems that are well-received by providers.30 31

In conclusion, through a reiterative process of design, implementation and testing, we have developed an autonomous AKI alert coupled to an automated trial screening, enrolment and randomisation engine. This approach decreases the costs of such a trial dramatically, while simultaneously increasing generalisability (as virtually all eligible patients are enrolled). While this approach is not feasible for all clinical trials, especially those using novel therapeutics, it is an ideal system to rigorously study systems-based interventions.