Validation study of German inpatient administrative health data for epidemiological surveillance and measurement of quality of care for sepsis: the OPTIMISE study protocol.

INTRODUCTION: Sepsis is a major cause of preventable deaths in hospitals. This study aims to investigate if sepsis incidence and quality of care can be assessed using inpatient administrative health data (IAHD). METHODS AND ANALYSIS: Design: Retrospective observational validation study using routine data to assess the diagnostic accuracy of sepsis coding in IAHD regarding sepsis diagnosis based on medical record review. PROCEDURE: A stratified sample of 10 000 patients with an age ≥15 years treated in between 2015 and 2017 in 10 German hospitals is investigated. All available information of medical records is screened by trained physicians to identify true sepsis cases ('gold standard') both according to current ('sepsis-1') definitions and new ('sepsis-3') definitions. Data from medical records are linked to IAHD on patient level using a pseudonym. ANALYSES: Proportions of cases with sepsis according to sepsis-1 and sepsis-3 definitions are calculated and compared with estimates from coding of sepsis in IAHD. Predictive accuracy (sensitivity, specificity) of different coding abstraction strategies regarding the gold standard is estimated. Predictive accuracy of mortality risk factors obtained from IAHD regarding the respective risk factors obtained from medical records is calculated. An IAHD-based risk model for hospital mortality is compared with a record-based risk model regarding model-fit and predicted risk of death. Analyses adjust for sampling weights. The obtained estimates of sensitivity and specificity for sepsis coding in IAHD are used to estimate adjusted incidence proportions of sepsis based on German national IAHD. ETHICS AND DISSEMINATION: The study has been approved by the ethics commission of the Jena University Hospital (No. 2018-1065-Daten). The results of the study will be discussed in an expert panel to write a memorandum on improving the utility of IAHD for epidemiological surveillance and quality management of sepsis care. TRIAL REGISTRATION NUMBER: DRKS00017775; Pre-results. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The results of the study will allow to calculate adjusted estimates of true sepsis incidence for Germany.

The study uses a multicentre validation sample of inpatient cases that allows assessment of both sensitivity and specificity of several coding abstraction strategies for sepsis.

Based on its multicentre sample, the study can investigate the variability in sepsis coding across hospitals.

Based on the study personnel necessary to conduct the study only tertiary care and university hospitals participate in the study, restricting generalisability of results.

True sepsis cases are identified by a retrospective review of medical chart and not by prospectively gathered data, which could cause a limitation of internal validity.

Introduction

Sepsis is final pathway to death from infectious diseases,1 causing more than 6 million deaths worldwide.2 It is among the most expensive conditions treated in hospital,3 and was regarded a major cause of preventable deaths in hospitals.4 5 Recognising shortcomings in prevention, diagnosis and treatment of sepsis, a resolution was released by WHO in May 2017. It urges member states—among other points—to improve epidemiological surveillance as well as quality of care.6 7

Administrative health data are widely used for disease surveillance, outcome research and health services research, since they allow an easy investigation of whole populations.8 Epidemiological studies on sepsis have been conducted relying on administrative health data both in the USA9 10 and Germany.11 12 Administrative data are also commonly used for the purpose of measuring quality of care.13 Several models for assessing risk-adjusted sepsis-related mortality based on administrative data have been proposed in the USA14 15 and Germany.16

When using administrative data for these purposes, the validity of the method to identify patients with the disease of interest is of high importance, since a misclassification bias might threaten the validity of study conclusions.8 Additionally, knowing sensitivity and specificity of an identification method allows for an adjusted estimation of the true number of sepsis cases in a population.17 Several methods have been reported to identify sepsis in inpatient administrative health data (IAHD). These are based on abstraction of International Classification of Diseases (ICD) codes. Some define sepsis by the presence of specific codes for sepsis (eg, A41.0 for Sepsis due to Staphylococcus aureus, or R65.1 for systemic inflammatory response syndrome of infectious origin with organ failure (severe sepsis)). Others use algorithms that define sepsis by the concurrent presence of a code for sepsis or other infections (eg, I30—for acute pericarditis) with codes for organ dysfunctions (eg, N17—for Acute kidney failure).9 10 18 The validity of different methods has been investigated in comparison to a gold-standard diagnosis mostly derived by review of medical charts. A systematic review revealed an overall low validity and high heterogeneity across different coding abstraction strategies and data sources.18 Beside our own single-centre pilot study, the validity of sepsis coding in German IAHD has yet not been studied.19 Additionally, previous studies did not investigate how validly sepsis-related risk-adjusted mortality can be derived from IAHD.

In 2016, new clinical definitions of sepsis (‘sepsis-3’) were introduced.1 These sepsis definitions exclude any sepsis without organ dysfunction and changed the criteria for the presence of organ dysfunction. Guidelines for ICD coding of sepsis in IAHD continued to rely on the old sepsis definitions (‘sepsis-1’) for several years (eg, in Germany, coding guidelines were changed in 2020). Thus, no estimates on the incidence of sepsis-3 in comparison to sepsis-1 definitions are yet available for Germany.

Based on these considerations, two interlinked studies are conducted: a validation study and a national incidence study. The validation study has the aims (1) to investigate the validity (sensitivity, specificity) of different coding abstraction strategies to identify sepsis in IAHD, (2) to identify possible causes of undercoding and overcoding of sepsis in IAHD and (3) to investigate the validity of models to calculate risk-adjusted mortality based on IAHD. Using sensitivity and specificity of coding abstraction strategies obtained by the validation study, the national incidence study has the aim (4) to obtain adjusted estimates of the incidence of hospital treated sepsis in Germany both according to sepsis-1 and sepsis-3 definitions.

Methods and analysis

The investigator initiated study ‘Validation and optimisation of the utility of routine data for improving the quality of sepsis management in hospitals’ (Validierung und Optimierung der Nutzbarkeit von Routinedaten zur Qualitätsverbesserung des Sepsis-Managements im Krankenhaus—OPTIMISE) is funded by governmental public funds between September 2018 and October 2020. This study protocol takes into account the recommendations of the statement on the Reporting of studies Conducted using Observational Routinely collected health Data (RECORD)20 and of a version of the Standards for Reporting of Diagnostic accuracy (STARD) adapted to validation studies of health administrative data.8 The checklists are given as online supplementary materials 1 and 2. Two separate but interlinked studies will be conducted, a validation study, and a national incidence study for Germany.

Validation study

Design

A multicentre, retrospective, observational validation study using IAHD and a medical chart review is conducted. Based on IAHD of ten German hospitals, a sample of 1000 medical records per hospital of inpatient cases of the years 2015–2017 is investigated. For these 10 000 hospital episodes all available information of medical records is screened to identify true sepsis cases (‘gold standard’) both according to current (‘sepsis-1’) definitions21 and newly proposed (‘sepsis-3’) definitions.1 The validity of sepsis case identification based on coding in IAHD is then assessed by comparing it to the information based on clinical judgement of medical records.

Setting

Ten German hospitals participate in the study. Eight of the hospitals are university hospitals. The two non-university hospitals are teaching hospitals providing tertiary level care, one is in public and one in private ownership. The mean number of beds for inpatient care is 1388 (minimum: 755, maximum: 3000).

Sample

This study includes hospital episodes of patients, which received inpatient care, were billed according to the German diagnosis-related groups (DRG) system, and were at least 15 years of age at time of admission. Based on the IAHD provided by the ten participating hospitals, a sample of 1200 hospital episodes per hospital is drawn. Since the rate of sepsis among hospital cases is only about 2%,11 12 19 a disproportional stratified sampling is conducted to increase the proportion of ‘true’ sepsis cases in the sample. The strata are defined by the cross-tabulation of the following criteria: (1) presence of a procedure code (Operationen- und Prozedurenschlüssel; OPS) for complex intensive care treatment (yes vs no, OPS-code: 8–890); (2) hospital length of stay (≤6 days vs >6 days), (3) year of discharge (2015–2017). The same number of cases is sampled from each of the resulting 12 strata. The strata were chosen based on the experiences from a single-centre pilot study, where the rate of cases with sepsis in a sample obtained by this method was 16%.19 The aim of the study is to analyse 1000 episodes per hospital, 200 additional episodes are sampled, since some medical records might be unavailable for analysis by the local study teams.

Procedure

After applying the inclusion criteria, the ten participating hospitals transfer pseudonymised IAHD of the respective hospital to the study’s epidemiologist at the University Hospital Jena. Sampling of cases is done and pseudonyms of sampled cases are reported back to the study centres.

To identify cases with sepsis, a combination of criteria needs to be fulfilled. To judge these criteria, study physicians need to review (1) information on the presence of infection (clinical signs or results of microbiology), as well as (2) information on specific symptoms, vital signs and laboratory values, which might indicate a criterion of a systematic inflammatory response syndrome (SIRS) or of an organ dysfunction (see table 1). Finally, a causal link between (1) and (2) needs to be highly probable based on the available information. Study physicians are not informed about which ICD-10 codes are present in the IAHD, but they cannot be blinded to ICD-10 codes that might be present within the medical records. For purpose of training of study physicians to identify cases with sepsis from medical records, 40 cases are sampled per study centre including 20 cases with coded sepsis with organ dysfunction or septic shock according to sepsis-1 (ICD-10-German Modification (GM) codes R65.1 or R57.2), 10 cases with coding of any other infection and 10 cases without any infection code. Based on the structured electronic case report form (eCRF), a written working instruction, and a training session, all local study physicians (at least two necessary) of the centres screen and discuss every of the 40 cases, of which five are monitored by the study physician of the core study team. After the training, a second sample of 40 cases is provided to assess the objectivity of the identification method. These cases are screened independently by two trained study physicians and information on sepsis criteria is documented in an eCRF. To assess objectivity of the process of identifying cases with sepsis, the inter-rater agreement is calculated. If the objectivity for identifying cases with sepsis with organ dysfunction (sepsis-1) in a study centre is below the cut-off of 0.6, a second monitoring of five cases is conducted.

Table 1

Defining criteria for status according sepsis-1 and sepsis-3 definitions for review of medical records

Criteria*	Status according to sepsis-1			SIRS-negative sepsis	Status according to sepsis-3
	Sepsis without organ dysfunction	Sepsis with organ dysfunction without shock	Septic shock		Sepsis	Septic shock
Presence of infection	Yes	Yes	Yes	Yes	Yes	Yes
Co-occurrence of at least two of four SIRS criteria	Yes	Yes	Yes	No
At least one newly occurring organ dysfunction caused by infection	No	Yes		Yes†
Arterial hypotension caused by infection	No	No	Yes
Increase of SOFA score by at least two points caused by infection					Yes
Co-occurrence of arterial hypotension and increase of lactate >2 mmol/L						Yes

Table presents the criteria that have to be present to define the status of a case according to current sepsis-1 definitions21 and newly proposed sepsis-3 definitions.1 The status of SIRS-negative sepsis was suggested by Kaukonen et al 26 Criteria with a ‘yes’ in the same column need to be present simultaneously while at the same time criteria with a no need to be not present (logical ‘and’), cells left blank are irrelevant for the respective status (can or cannot be present).

*Detailed definitions of each criteria of the definitions are given in the respective literature.1 21

†Either arterial hypotension or other organ dysfunctions caused by infection are present.

SIRS, systematic inflammatory response syndrome; SOFA, sepsis-related organ failure assessment.

After the training and validation phase is finished, the stratified sampling of 1200 cases per study centre is conducted (flow diagram presented in figure 1). The records of these cases are screened in random order to avoid bias by learning effects. Each study centre needs to document at least 1000 cases. Study physicians document sepsis criteria according to sepsis-1 and sepsis-3 definitions via the eCRF. For cases with sepsis additional clinical information is documented. Finally, the data managers of the core study team at the University Hospital Jena document how the sepsis-1 status of cases was according to the coding in IAHD in the eCRF of every case. This is done after the eCRF parts on sepsis criteria according to medical records is completed by the local study physicians. The eCRF automatically does a comparison between sepsis-1 according to records and sepsis-1 according to coding. If false positive coding of sepsis has occurred, local study nurses are demanded to document possible reasons for that. After documentation of cases is finished, data from the eCRF are exported and then linked with the IAHD via the pseudonym.

Figure 1

Flow diagram of the validation study.

Data sources

Inpatient administrative health data

The IAHD contain: patient demographics, reasons and type of admission, ICD-10-GM coded primary and secondary diagnoses, conducted surgeries and procedures using procedure codes, treating hospital departments, discharge destinations (including hospital death) and billing information. The data are limited, since they provide no dates for diagnoses coded in ICD-10-GM. Therefore, it cannot be identified, if a diagnosis was present at admission. Additionally, they do not contain follow-up information after hospital discharge. DRG data that can be accessed via the Federal Bureau of Statistics (‘national IAHD’) do not contain information on which hospital episodes belong to the same patient and also concatenate some successive episodes to one case based on specific billing rules (eg, readmission for the same primary cause). Since the results of the study shall be generalisable to German national IAHD, the IAHD of the participating hospitals are provided without considering which episodes belong to the same patient and with concatenated episodes.

Medical record data

All clinical information contained in the medical records of the respective cases is screened to identify patients with sepsis; this involves information form the patient data management system, electronic charts as well as paper charts, laboratory data, results of medical imaging and other tests, and discharge letters. Study physicians and nurses are demanded not to inspect the IAHD coding for identification of cases for sepsis. Data from medical records are documented in an eCRF using the open source study management software OpenClinica (V.3.1).

Data linkage

IAHD are pseudonymised within the participating hospitals. The pseudonym is used to identify the cases within the eCRF and thereby to link the data from medical records with the information obtained from the IAHD. To assure correct linkage patients’ age, gender, exact time of admission and discharge obtained from IAHD is provided to the study centres additional to the pseudonym. By comparing this information (age, sex, admission and discharge date) with the information provided via the eCRF the evaluation of the correctness of the linkage is done by the data managers and an epidemiologist at the University Hospital Jena.

Measures

Measures based on medical records

The eCRF of the study was developed based on previous research18 22–25 as well as an own pilot study.19 The complete CRF is presented in online supplementary material 3. It contains three parts. Part A is documented by trained study physicians and aims to identify patients with sepsis. If an infection was present, characteristics of the infection as well as sepsis criteria are documented. Table 1 presents the defining criteria for sepsis according to current sepsis-121 and newly proposed sepsis-31 definitions. Additional to the sepsis-1 definition this study also defines the status of infection with infection-related organ dysfunction or infection-related shock without at least two SIRS criteria (‘SIRS-negative sepsis’). This is done because the validity of the necessity of presence of at least two SIRS criteria has been challenged,26 and we want to estimate the prevalence of such cases. All criteria are documented separately. It is also documented if there is not enough information in the medical records to judge if a criterion is fulfilled. Based on logical combination of the criteria, the eCRF automatically reports to the study physician the sepsis-1 and sepsis-3 status or the case. To finalise the identification of cases with sepsis, the study physician needs to confirm this status. Further data are only documented for patients with sepsis (either according to sepsis-1, SIRS-negative sepsis or sepsis-3) by local study nurses in part B of the eCRF and has several purposes.

First, the patient and the course of treatment are described (basic demographics, reasons for admission, organ replacement therapies). Second, risk factors that can also be identified based on IAHD16 are documented to serve as a gold standard of comparison: comorbidities according to the categories of Charlson and Elixhauser indices,27–29 pre-existing illnesses or conditions that are associated with immunosuppression or increased mortality risk (asplenia, transplanted organ status, tracheostomy status, dependence on renal replacement therapy), treatment in the current hospital episode associated with increased mortality risk (chemotherapy, stroke treatment, palliative care). Additional risk factors and factors influencing mortality, which are not available from IAHD are documented: cardiopulmonary resuscitation occurring before onset of sepsis, and presence, type and timing of therapy limitations. Results of microbiological diagnostics are documented to identify sepsis-related pathogens and as a standard of comparison to evaluate the validity of pathogen specific ICD-10-GM codes for sepsis present in the IAHD. Because knowledge about the importance of sepsis as cause of death has only been systematically investigated in one study in the USA,25 study physicians also judge if sepsis was the immediate cause of death, and what primary underlying cause of death was present. Finally, to identify the influence of physician documentation in records for false negative coding of sepsis in IAHD, study physicians document if a diagnosis of sepsis was present anywhere in the medical record.

The final part C of the eCRF has the purpose to document reasons for false positive coding of sepsis. False positive coding occurred, if an ICD-10-GM code for sepsis was present, but no sepsis was judged to be present based on records, or if an ICD-10-GM code for organ dysfunction or shock was present, but only a sepsis without organ dysfunction was diagnosed according to records. In case of discrepancy, the eCRF demands the study nurses to document if SIRS criteria, criteria for organ dysfunctions or criteria for a shock were present without being related to an infection, since this might have caused the false negative coding of sepsis.

Measures based on IAHD

IAHD are used to identify the sepsis status of patients based on several coding abstraction strategies. The primary interest is on the identification of patients with sepsis with organ dysfunction including septic shock by the ICD-10-GM codes R65.1 and R57.2. Table 2 presents all coding abstraction strategies. Parts of the strategies use explicit sepsis codes, and parts of the strategies combine codes for infection or sepsis with codes for organ dysfunction, which is known as the implicit coding strategy.9 The strategies have been partly adapted from the ICD-922 to ICD-10-GM and investigated in a pilot study.19 Surpassing previous literature, it will also be tested if the inclusion of procedure codes for organ supporting therapies to define organ dysfunction will enhance predictive validity. Respective procedure codes will be defined and all codes relevant for the abstraction strategies will be checked and if necessary revised before the primary analyses of the study will be conducted.

Table 2

Coding abstraction strategies for identification of sepsis status according to sepsis-1 in administrative health data

Coding abstraction strategy	Sepsis status	Definition
Explicit sepsis coding	Sepsis without organ dysfunction	Presence of any explicit ICD-10-GM sepsis codes with R57.2 and R65.1 not being present
	Sepsis with organ dysfunction without shock	Presence of ICD-10-GM R65.1 but not R57.2
	Septic shock	Presence of ICD-10-GM R57.2
	Sepsis with organ dysfunction including shock	Presence of ICD-10-GM R65.1 or R57.2
	Sepsis	Presence of any explicit ICD-10-GM sepsis code
Explicit sepsis coding and organ dysfunction codes	Sepsis with organ dysfunction including shock	Presence of any explicit ICD-10-GM sepsis code and any ICD-10-GM code for organ dysfunction
Implicit approach	Sepsis with organ dysfunction including shock	Presence of any ICD-10-GM code for infection and any ICD-10-GM code for organ dysfunction
Explicit sepsis coding and organ dysfunction codes including procedure codes	Sepsis with organ dysfunction including shock	Presence of any ICD-10-GM sepsis code and (any ICD-10-GM code for organ dysfunction or any OPS code for organ dysfunction*)
Implicit approach including procedure codes	Sepsis with organ dysfunction including shock	Presence of any ICD-10-GM code for infection and (any ICD-10-GM code for organ dysfunction or any OPS code for organ dysfunction*)

ICD-10-GM explicit sepsis codes, infection codes and codes for organ dysfunction are presented in online supplementary material 4.

*OPS codes for identification of organ dysfunctions will be identified before conduction of the main analyses and documented in the analyses plan.

ICD-10-GM, International Classification of Diseases, 10 Revision, German Modification; OPS, Operationen- und Prozedurenschlüssel (German classification of surgeries and procedures).

Statistical analysis

As a measure of objectivity inter-rater agreement is calculated between two independent study physicians per study centre based on 40 medical records of the validation phase. Inter-rater agreement is calculated by Gwet’s AC1, which is a more robust alternative to Cohen’s Kappa.30 Primary aim of the validation phase is to achieve at least a substantial inter-rater agreement above 0.631 regarding the judgement of the presence of sepsis with organ dysfunction including septic shock (sepsis-1). Inter-rater agreement is also calculated regarding the full classification of sepsis status (sepsis-1: no sepsis, sepsis without organ dysfunction, sepsis with organ dysfunction but no shock, septic shock; sepsis-3: no sepsis, sepsis without shock, septic shock).

Based on the main study involving 1000 medical records per study centre proportions are calculated for sepsis and sepsis subgroups according to sepsis-1 and sepsis-3 definition (sepsis with and without organ dysfunction and septic shock), as well as for sepsis according to the different coding abstraction strategies. Differences between the different definitions and coding strategies will be investigated by cross-tabulation. Severity of illness might differ between gold-standard sepsis cases and coded sepsis cases.32 33 Therefore, indicators of illness severity and hospital mortality will be compared between the different groups.

The performance of the different coding abstraction strategies for identification of sepsis and different subgroups of sepsis-1 and sepsis-3 in IAHD will be assessed. Dichotomous classification problems like presence of sepsis with organ dysfunction (including septic shock, sepsis-1) or presence of sepsis (including septic shock, sepsis-3) will be assessed by sensitivity, specificity, positive and negative predictive value. Classification with more than two groups (sepsis-1: no sepsis, sepsis without organ dysfunction, sepsis with organ dysfunction but no shock, septic shock; sepsis-3: no sepsis, sepsis without shock, septic shock) will be assessed by Cohen’s κ (or similar measures) and performance measures for multiclass classification like microaveraged and macroaveraged F-scores.34 Primary outcome is the sensitivity of the explicit coding of sepsis with organ dysfunction including septic shock (ICD-10-GM codes R65.1 and R57.2).

The presence of different pathogens that are identified to be sepsis-related by review of medical records is cross-tabulated with the ICD-10-GM coding of pathogen-specific sepsis in IAHD to assess the validity of the coding.

Finally, the validity of risk-adjusted mortality calculation based on IAHD is assessed. Risk factors that can be identified both based on review of records as well as on IAHD are cross-tabulated and the diagnostic accuracy of the coding is assessed. Risk models relying solely on information from IAHD are compared with risk models that rely on information from review of medical records regarding discrimination, calibration and net reclassification index. To assess the validity of mortality risk predicted from a model based on IAHD, the correlation to the mortality risk predicted from a model based on medical records is calculated.

Planned subgroup analyses will be conducted to assess the stability of results. Since previous research has shown that sepsis is poorly reported outside the intensive care unit (ICU),18 the validity of the coding abstraction strategies will be examined among patients that were treated on an ICU and among patients not treated on an ICU. Subgroup analyses will also be conducted for patients discharged in the years 2015, 2016 and 2017 to assess the stability of predictive accuracy across time. To assess the variability of sensitivity, specificity and predictive value across the participating study centres a logistic regression model with random effects that is an extension of the model presented by Coughlin et al 35 will be calculated. The amount of missing data will be reported, especially regarding the information that is necessary to judge the presence of sepsis based on medical records. Analyses will be adjusted for sampling weights where appropriate. A detailed analysis plan will be written before the analysis of the 10 000 study cases will be conducted.

Sample size calculation

The sample size was calculated regarding the primary endpoint of the study—the sensitivity of the coding of sepsis with organ dysfunction (ICD-10-GM codes R65.1 and R57.2) in IAHD. In the pilot study, the sensitivity was estimated to be 0.39. To estimate sensitivity with a 95% CI of width ±0.03 a sample of number of about 850 true sepsis cases with organ dysfunction according to judgement of record data are necessary. In our pilot study, the rate of such cases in the sample was 8.6% resulting in necessary total sample of number of about 10 000 hospital episodes. This sample size also allows for an exact estimate of the proportion of true sepsis with organ dysfunction in the validation study sample (95% CI of width 0.005). Finally given this sample size, also risk models for hospital mortality among these cases can be calculated involving more than 30 predictors, given a mortality of 45%, what results in 380 deceased cases.36

National incidence study

Using estimates of sensitivity and specificity of sepsis coding in IAHD, adjusted estimates of the yearly incidence of hospital-treated sepsis are calculated by analysing German national IAHD data of the years 2015–2017.17

In Germany, hospitals are reimbursed based on a DRG system. Every year a standardised data set is transferred to the federal Institute of Hospital Reimbursement (Institut für das Entgeltsystem im Krankenhaus; InEK) by every hospital providing acute care (legal base: §21 KHEntgG). These data are passed to the Federal Bureau of Statistics and can be analysed for research purposes.

The same inclusion criteria as described for the validation study (inpatient cases, DRG-billing, age of at least 15 years, discharge in years 2015–2017) are applied to German national IAHD, which are used to calculate an adjusted national incidence of hospital-treated sepsis.

National IAHD are hosted by the Federal Bureau of statistics and can be accessed via a form of remote data processing. Based on completely anonymised sample data files, statistical syntaxes are written and sent to the Federal Bureau where they are applied to the original data files. Output files are then transferred back to the researcher.

German national IAHD have been used previously to calculate yearly incidence proportions of sepsis.11 12 These were calculated by obtaining the number of hospital episodes with ICD-10-GM coded sepsis per year and dividing them by the size of the German population within the same year. Rogan and Gladen17 presented a method to calculate the prevalence of a disease, if the disease is identified by an imperfect screening test. The observed prevalence is adjusted by a formula involving the sensitivity and the specificity of the screening test. Therefore, sensitivity and specificity of a coding abstraction strategy can be used to calculate an adjusted prevalence of hospital episodes with sepsis and from this an adjusted number of hospital episodes with sepsis. Methods to calculate CIs for the adjusted prevalence also are available.37 38 Adapting these methods, an adjusted incidence proportion of sepsis with a CI can be calculated using the sensitivity and specificity estimates of coding abstraction strategies from the validation study.

Since the national IAHD of Germany involve more than 18 Million inpatient cases per year and more than 130 000 cases with codes for sepsis with organ dysfunction or shock, test power will not be an issue in the analysis of these data.

Project organisation

The project is coordinated at the University Hospital Jena led by a senior research physician (KR) and involving an epidemiologist (DS), a study physician (HD) and two data managers. Local study teams led by a senior research physician and involving study physicians and study nurses run the project at the study centres. Each participating study centre receives funding for a 50% full-time equivalent for study physicians over 6 months and 50% full-time equivalent for study nurses over 8 months.

Project status

In March 2020, the training and validation phase have been completed and the documentation of data from medical records for the main study has been begun in all 10 study centres. It is planned to begin analyses in September 2020 and to finalise the results report in October 2020. Based on this report, the expert panel discussions are planned to be held in October 2020 leading to a draft for a memorandum to be available in December 2020.

Patient and public involvement

An expert panel is involved in interpretation and dissemination of results (see the section ‘Dissemination’).

Limitations

This study has a limitation of generalisability. It is only conducted in Germany and results could not be applied to other countries where IAHD have different structure and different rules apply for coding of diagnoses. The sample of the validation study consists of university and tertiary care hospitals and it is unclear if primary or secondary care hospitals would show a different validity of coding of sepsis in IAHD. Our calculation of an adjusted incidence estimate in the national incidence study is based on the assumption that sensitivity and specificity of sepsis coding in our sample of ten hospitals can be applied to the full population of German hospitals. Given the selective nature of the study sample, this might not be the case. A general disadvantage of IAHD is the scarce clinical information they contain. Electronic health record data have been shown to have much greater validity for sepsis surveillance,39 40 but no multicentre data bases with electronic health record data are available in Germany yet—like in many other countries. Therefore, our study seeks the validation and improvement of the usefulness of the available data sources.

Ethics and dissemination

Ethics

This study was approved by the ethics commission of the Jena University Hospital (No. 2018–1065-Daten). The processing of data is conducted based on art. 9 par 2 lit. i, j EU-GDPR, which allow processing of health data for ensuring quality and safety of healthcare and for scientific research. The need for informed consent was waived by the ethics commission and the data protection officer of the Jena University Hospital. To protect the rights and interests of the patients, data are only processed in pseudonymised form. Anonymisation of data is conducted after completion of the project.

Dissemination

The results will be published in peer-reviewed journals. To also influence decision-makers, a memorandum will be written, based on discussions of study results by an expert panel. The aim of the discussions is to formulate recommendations to increase the utility of IAHD for surveillance and quality management of sepsis. Strategies for improving the utility could involve linkage of additional data—like laboratory results or vital signs—improvement of physician documentation in charts, development of specialised procedures for coding of sepsis in IAHD and training of coding staff,18 or changing of legislation and development of financial incentives for correct coding. The planned panel will involve experts on quality management, health services research, health administrative data, intensive care, and medical controlling and billing.