The PROgnostic Value of unrequested Information in Diagnostic Imaging (PROVIDI) Study: rationale and design.

We describe the rationale for a new study examining the prognostic value of unrequested findings in diagnostic imaging. The deployment of more advanced imaging modalities in routine care means that such findings are being detected with increasing frequency. However, as the prognostic significance of many types of unrequested findings is unknown, the optimal response to such findings remains uncertain and in many cases an overly defensive approach is adopted, to the detriment of patient-care. Additionally, novel and promising image findings that are newly available on many routine scans cannot be used to improve patient care until their prognostic value is properly determined. The PROVIDI study seeks to address these issues using an innovative multi-center case-cohort study design. PROVIDI is to consist of a series of studies investigating specific, selected disease entities and clusters. Computed Tomography images from the participating hospitals are reviewed for unrequested findings. Subsequently, this data is pooled with outcome data from a central population registry. Study populations consist of patients with endpoints relevant to the (group of) disease(s) under study along with a random control sample from the cohort. This innovative design allows PROVIDI to evaluate selected unrequested image findings for their true prognostic value in a series of manageable studies. By incorporating unrequested image findings and outcomes data relevant to patients, truly meaningful conclusions about the prognostic value of unrequested and emerging image findings can be reached and used to improve patient-care.

Introduction

In this article the rationale and methods of the PROgnostic Value of unrequested Information in Diagnostic Imaging (PROVIDI) study are presented. It was designed to investigate the growing amount of unrequested information, identified on diagnostic radiological examinations. PROVIDI’s relatively large study sample and its innovative use of a case-cohort design place it in an ideal position to address some of the more challenging issues pertaining to unrequested radiological findings.

Rationale

Advances in radiological imaging techniques have led to scans of increasingly high resolution and contrast being deployed ever more widely, with the Computed Tomography (CT) [1] and Magnetic Resonance Imaging (MRI) [2] modalities leading the way. CT in particular is being deployed for a growing number of clinical indications and over the last decade alone CT image quality has evolved tremendously from single-section helical CT to the greater spatial and temporal resolution offered by multi-detector row CT (MDCT). 4-, 8- and 16-slice scanners are now widely in use, 64- and 256-slice scanners are being introduced into routine clinical practice whilst 320-slice scanners are in late phases of testing [3].

The increasingly widespread use of higher quality CT in routine diagnostic clinical care is causing a corresponding increase in the numbers of unrequested findings being detected, which previously would have gone unnoticed; characterized as unrequested information which is unrelated to the initial scanning indication [4, 5]. As the referrals for routine scanning are typically highly targeted at investigating specific pathologies in specific organs, there is plenty of scope for unrequested findings that are not linked to these initial referrals. These referrals are typically the only additional clinical information that radiologists possess, adding to the uncertainty over how to handle unrequested findings. Evidence from the literature indicates that, on the whole, a large number of unrequested findings are not addressed in routine clinical settings [6], whilst others may be subject to aggressively defensive follow-up [7, 8]. Due to their uncertain clinical significance, such findings pose a novel challenge to radiologists and referring clinicians alike.

Contributing to this trend of increasingly detected unrequested information is the growing culture of medical litigation wherein radiological lawsuits pertaining to missed diagnoses and perceived failure to initiate follow-up investigations form a growing majority of the total [9]. This follow-up exposes patients to potential harm in the form of unnecessary radiation exposure, invasive procedures, the anxiety and stress associated with an uncertain disease-state, as well as financial cost [10].

Additionally, to date the impact of emerging diagnostic findings, such as arterial calcium scores and volumetric analysis of lung nodules [11], are only partially understood and go largely unutilized in routine care. The fact that many potentially predictive findings, as unrequested detected findings, may be available in some form for free on routine scans makes the investigation of their implementation necessary. The potential for risk stratification and preventative treatment for a range of disorders spanning from cardiovascular disease to osteoporosis using unrequested imaging characteritics extracted from the hodgepodge of routine scanning equipment and protocols could be unlocked by demonstrating the prognostic utility of this approach.

The question of how best to adapt to the growing number of unrequested findings has engendered lively debate amongst radiologists, with opinions ranging from advocacy of maximal pursuit for and follow-up of all available findings, to deliberately ignoring anatomical regions and findings beyond the mandates of the scan indication [12-15]. This debate takes place in the context of the limited clinical information typically available to the radiologist and such discussions are a consequence of rapidly advancing techniques with attendant lack of knowledge of their implications. Debate will continue until follow-up studies are performed to investigate which unrequested scan findings are significant and which findings have no clinical impact. For these reasons, there is the urgent need of follow-up studies in routine radiologic settings; there have been but a few admirable attempts [10] at collecting follow-up data on patients with unrequested findings but virtually none investigating the prognostic endpoints. The work carried out in screening settings may not be representative for routine care settings due to for example different hazards for experiencing outcome events and concurrently, there is no basis for comparison between routine-care settings and the radiological prognostic research carried out so far in research settings using screening populations.

The PROVIDI study is the first study that aims to address these issues. It is designed as a longitudinal study, linking unrequested information detected on routine diagnostic Chest CT scans to major health outcomes via national health registries. In doing so we are able to identify those readily accessible unrequested findings that are prognostically relevant and sort them from findings that have little or no value to the patient (Fig. 1). This may allow clinical radiologists to contribute more generally than before to patient care by more effectively utilizing the increasing amount of diagnostic information available, thereby increasing the efficiency of health care.

Fig. 1

Objective of the PROVIDI-study

Methods

Study design

The PROVIDI study is designed as a multi-center retrospective case-cohort study. The cohort consists of subjects with routinely made chest CT scans obtained from eight participating hospitals in the Netherlands (Elkerliek Hospital (Elkerliek), Gelre Hospital (Gelre), St. Antonius Hospital (Antonius), Academic Medical Center Amsterdam (AMC), VU University Medical Center (VUMC), University Medical Center Groningen (UMCG), Academic Hospital Maastricht (AZM), and University Medical Center Utrecht (UMCU)), of which the latter five are tertiary referral centers. A case-cohort design offers significant gains in efficiency over a standard cohort study [16], at the cost of increased complexity in data analysis and the addition of a theoretical layer of assumptions about the uniformity of the cohort. The crucial difference from a normal cohort study is that a random control sample of the cohort is selected at baseline to represent the cohort, making it unnecessary to analyse the whole cohort. Each (cluster of) unrequested finding of interest can then be studied in such an individual sub study that combines the cases that suffered outcome(s) relevant to that finding during follow-up with a new random sample of controls that did not. The control group is to be randomly sampled so that the group is twice as large as the case group for the individual sub study. This permits the use of the PROVIDI cohort in the investigation of the prognostic value of a range of unrelated unrequested findings, as well giving researchers the freedom to pursue unforeseen candidate findings as they emerge. Valid absolute risks, which are indispensable to prognostic research, can be readily calculated.

It was not possible to evaluate the extent of clinical follow-up undergone by patients as a result of unrequested findings with this study design. We did not consider this to be a serious limitation as this is rare in practice [6] and at any rate, it would cause an underestimation of the prognostic effect (as those receiving preventative therapies are probably less likely to experience the outcome of interest).

The PROVIDI study was approved by the Medical Ethics Committee of the University Medical Center Utrecht. The need for written informed consent was waived for all patients due to the retrospective design of this study. A privacy protocol was implemented to ensure that no patient information would be visible whilst reading CT scans and no additional information could be obtained from patient’s medical records and no patient would be contacted as a result of this study.

Study subjects

All patients aged 40 years and older who were referred to one of the participating hospitals with an indication for a chest MDCT between January 2002 and the end of December 2005 were evaluated for inclusion in the PROVIDI study (N = 23.443). Patients with suspected primary lung cancer (including mesothelioma) or distant metastatic disease from other types of cancer (excluding haematological malignancies), 9.077 cases in all, were excluded, on the basis that it is highly unlikely that detection of unrequested imaging findings will alter clinical decision making in patients with such a poor prognosis. This selection was first fully evaluated in a pilot study (Table 1, textbox, pilot study). Consequently, the PROVIDI cohort consists of 14.366 patients, and an equal number of chest CT’s. In cases where patients underwent more than one chest CT examination, only the first CT scan of the series was used for analysis. The chest CT’s were obtained with 2-, 4-, 8-, 16-, 32- or 64-slice scanners of different vendors. All types of chest CT protocols, including contrast and non-contrast scans, were eligible for the PROVIDI study. The CT scans were initially assessed by local hospital radiologists, consistent with routine practice. Subsequently, anonymous copies of all images were stored on disk and transferred to the University Medical Center Utrecht. Patient characteristics and information on type of CT protocol used, including section thickness, tube voltage (kVp), tube load (mAs), and the use of a contrast agent, was abstracted from CT reports by a research physician, who also assessed the CT indication from the CT reports.

Table 1

Textbox, explanation of pilot study

Pilot study
A pilot study was conducted to investigate if the indication categories could be used to identify patients with a very bad prognosis a priori
For this pilot study, all patients, older than 40 years and undergoing thoracic CT at the University Medical Center Utrecht and Academic Hospital Maastricht were included. Information about mortality was gathered through linkage with the National Death Registry after complete follow-up
A comparison was made between patients who deceased within 6 months of follow–up and patients who deceased after 6 months of follow-up
Patients with indication categories ‘Suspected primary lung cancer (including mesothelioma)’ and ‘distant metastatic disease from other types of cancer (excluding haematological malignancies)’ had an average mortality rate within 6 months of 33.9%. This was much higher compared to the mortality rate for the other indication categories (mean 10.4%, P < 0.001)
We concluded that the indication categories can discriminate well between patients’ prognoses and could be used to exclude patients with a bad prognosis a priori

Image findings

The specific candidate image findings will differ for each individual study investigating different specific groupings of disease entities and will be selected for their potential prognostic value, as suggested by the available literature and expert opinions from experienced radiologists as well as other clinical specialists and epidemiologists. CT scans will be reviewed, blinded for general scan parameters, CT indication and the outcome status, at a computer workstation, by trained research physicians and supervised by an experienced chest radiologist. Images are to be viewed at standard lung, soft tissue, and bone settings, which are also readily available to radiologists in clinical practice. Reproducibility of image findings will be evaluated and must be sufficient. Some examples of the types of candidate unrequested scan findings are listed in Fig. 2.

Fig. 2

Examples of unrequested scan findings. Upper left non-contrast CT image, lower left contrast CT image, upper right CT image in lung setting, lower right contrast CT image in mediastinum setting. a calcifications in Left Main coronary artery and Left Anterior Descending artery, b calcification in descending thoracic Aorta, c: Irregular descending thoracic Aorta with calcification, d diameter of left ventricle, e diameter of heart, f Lung emphysema, g bronchiectasia, h Calcificated plaque in ascending thoracic Aorta, i enlarged lymph node, j Pleural effusion

Endpoints

Main endpoints of the PROVIDI study—established after a first initial linkage—are listed in Table 2. Ideal endpoints would be prevalent diseases in with a major clinical impact and can be treated preventively when diagnosed at an early stage. The endpoints were ascertained through linkage with the National Death Registry and the National Registry of Hospital Discharge Diagnoses for the period January 2002-December 2006. Research showed that the quality of these databases was acceptable [17, 18]. Patients were identified through a combination of a patient’s date of birth, sex, and zip code, using a validated probabilistic method [17, 19, 20]. In these databases, cause of death and the occasion of hospitalization are coded according to the International Classification of Disease, 9th [21] and 10th revision [22]. In the initial linkage, performed at a mean followup of 17 months, a total 5.225 out of the 14.366 patient cohort experienced a valid endpoint (Table 2). Note that this initial overview gives only a global indication of the numbers (here death prevailed over admission) and will be updated for each sub study to be performed.

Table 2

Outcome events and their codings according the ninth international classification of disease

Outcome events	ICD [1] Codes	Initial numbers
Mortality
All cause mortality	All codes	3677
Hospital admissions
Cardiovascular disease		659
Ischemic heart disease	ICD9 410–414
Cerebrovascular diseases	ICD9 431, 434–438
Thoracic aortic aneurysm	ICD9 441
Peripheral arterial disease	ICD9 443,444
Cardiac valve disease	ICD9 424
Sudden cardiac death	ICD9 427
Pericarditis	ICD9 420,423
Interventions	ICD9-CvV 5360–3, 5369, 5350–5354, 5380–3, 5385, 5399
Pulmonary embolism	ICD9 4151	69
Neoplasmata		375
Bronchus, lung, thymus, heart, oesophagus	ICD9 162–164, 150
Thyroid	ICD9 193, 226
Hodgkin	ICD9 201,
Non-Hodgkin	ICD9 2020
Other mediastinal structures	ICD9 1642–9, 2125
(Struma)	ICD9 240–242 2126,
Pulmonary disease		392
Chronic obstructive lung disease	ICD9 490, 4912, 4919, 4939, 496
Bronchiectasis	ICD9 494
Pulmonary emphysema	ICD9 492
Pulmonary fibrosis	ICD9 515
Muskuloskeletal disease		53
Fracture of hip	ICD9 820
Fracture of spine	ICD9 805
Osteoporosis and osteoporotic fractures	ICD9 733
Intervertebral disc disease	ICD9 722

[1] ICD International classification of disease, CvV Intervention classification score

Evaluation of data

PROVIDI will consist of a series of studies investigating potentially predictive scan findings for specific groupings of disease entities (e.g. cardiovascular diseases). The study populations of these studies will consist of the patients that suffered the relevant disease(s) under study during the follow-up period, as well as a random sample of patients from the PROVIDI cohort (the sub-cohort). In each of these studies, patients with a reported CT indication directly related to the disease under study will be excluded, to ensure that the image findings under study can legitimately be defined as unrequested information, in keeping with the stated goals of PROVIDI. (Table 3, textbox, example study population) Fig. 3 shows the flowchart of the PROVIDI design.

Table 3

Textbox, example study population

Example study population

The study population for a study investigating the prognostic value of unrequested detected image findings that may be predictive for cardiovascular disease will consist of all patients that experienced a cardiovascular event during follow-up plus a random sample from the PROVIDI cohort. Patients with a cardiovascular indication for obtaining the chest CT are excluded, making sure that the image findings under study are truly unrequested

Fig. 3

Flowchart of PROVIDI study. Question mark indicates that these numbers differ per conducted study within PROVIDI

We intend to analyse the scan findings both univariately and multivariately under study using Cox proportional hazard models, the most widely used statistical model for survival outcome in medical research. In multivariate analysis infromation readily available to radiologists in routine care (age, gender, CT indication, scanning parameters and quality) will be incorporated. The hazard ratios and standard errors will be modified based on robust variance estimates. These adaptations are to be carried out using the method according to Prentice, in which all sub cohort members are equally weighted [16]. Cases outside the sub cohort are not to be weighted before failure and at failure receive the same weight as members of the sub cohort. This method has been shown to resemble most closely estimates from a full-cohort analysis [23], without requiring a full analysis.

Discussion

Investigating which image findings have prognostic value and—perhaps more importantly—which findings do not, is the first step in a process that has great potential to improve clinical care. The findings of the PROVIDI study will be of great interest to radiologists, but also to referring physicians. The unique longitudinal study design, that places outcomes that matter to patients at its heart, gives this study the potential to meaningfully quantify the uncertainties facing diagnosticians and protect patients from potential over-diagnosis and over treatment. To illustrate: in the face of an absence of research investigating patient outcomes in routine-care, non-screening settings, radiologists are regularly responding defensively with exhaustive follow-up. An illustrative example is that of lung nodules, which are routinely detected on thoracic CT scans of all types, and which are frequently followed-up, despite evidence pointing to a low yield of significant pathology, at least in research/screening settings [7, 8].

By drawing on non-screening, routine clinical data it may both alleviate the dearth in radiological routine-care outcome data and serve as a basis for comparison with previous evidence from screening populations. A consequence of this study design is that patients at risk for a certain outcome will be identified using these models by radiologists while they may be already identified as such by other specialists, resulting in an overestimation of the clinical impact of certain incidental findings. Such double risk stratification would not have any negative effect for patients: the high risk indication from a radiologist is obtained freely (without additional radioation exposure) and can serve as a stimulus to verify whether optimal treatment has been initiated. In some cases this treatment is already optimal and a referring clinician can then ignore the ‘red flag’ if not, the referring specialist can consider additional investigations or start preventative treatment.

Translating the insights that might be gained from this study into clinical practice remains beyond the scope and means of PROVIDI itself but constitutes a crucial next step. Future clinical studies prospectively evaluating the efficacy and feasibility of implementing changes to clinical practice based upon PROVIDI’s findings could demonstrate the benefit of fully utilizing the potential of modern scanning technology.