Reproducibility of thrombus volume quantification in multicenter computed tomography pulmonary angiography studies.

AIM: To evaluate reproducibility of pulmonary embolism (PE) clot volume quantification using computed tomography pulmonary angiogram (CTPA) in a multicenter setting.
METHODS: This study was performed using anonymized data in conformance with HIPAA and IRB Regulations (March 2015-November 2016). Anonymized CTPA data was acquired from 23 scanners from 18 imaging centers using each site's standard PE protocol. Two independent analysts measured PE volumes using a semi-automated region-growing algorithm on an FDA-approved image analysis platform. Total thrombus volume (TTV) was calculated per patient as the primary endpoint. Secondary endpoints were individual thrombus volume (ITV), Qanadli score and modified Qanadli score per patient. Inter- and intra-observer reproducibility were assessed using intra-class correlation coefficient (ICC) and Bland-Altman analysis.
RESULTS: Analyst 1 found 72 emboli in the 23 patients with a mean number of emboli of 3.13 per patient with a range of 0-11 emboli per patient. The clot volumes ranged from 0.0041 - 47.34 cm3 (mean +/- SD, 5.93 +/- 10.15cm3). On the second read, analyst 1 found the same number and distribution of emboli with a range of volumes for read 2 from 0.0041 - 45.52 cm3 (mean +/- SD, 5.42 +/- 9.53cm3). Analyst 2 found 73 emboli in the 23 patients with a mean number of emboli of 3.17 per patient with a range of 0-11 emboli per patient. The clot volumes ranged from 0.00459-46.29 cm3 (mean +/- SD, 5.91 +/- 10.06 cm3). Inter- and intra-observer variability measurements indicated excellent reproducibility of the semi-automated method for quantifying PE volume burden. ICC for all endpoints was greater than 0.95 for inter- and intra-observer analysis. Bland-Altman analysis indicated no significant biases.
CONCLUSION: Semi-automated region growing algorithm for quantifying PE is reproducible using data from multiple scanners and is a suitable method for image analysis in multicenter clinical trials.

Core tip: Blood clots that occur in deep leg veins can break away and cause the serious complication of clots (then termed emboli) lodged in the lungs. Measuring the volume of the emboli with a semi-automated region growing software program using computed tomography pulmonary angiogram data can be helpful to evaluate treatment efficacy in clinical drug trials. This study demonstrates the technique to be reproducible both between image analysts and when repeated by the same image analyst when the data is obtained in a multicenter setting.

INTRODUCTION

Venous thromboembolism (VTE) represents the combined disease states of deep venous thrombosis (DVT) and pulmonary embolism (PE). PE is the most significant sequela of DVT, occurring in more than a third of DVT patients[1]. In the United States, there is a combined average annual incidence of over 275000 new cases of VTE among whites of predominantly European origin and African-Americans[2]. A retrospective study demonstrated an incidence of 117 VTE cases per 100000 in a demographically white population[3]. The same study showed VTE to be predominantly a disease of older age, with males slightly more affected than females, whereas in the younger population the incidence is higher in females during child-bearing age[3].

PE has a high mortality rate[4,5]. The two week and three month post diagnosis all-cause mortality rate was found to be 11.4% and 17.4% respectively[6]. Risk of mortality in PE rests upon multiple factors including clinical findings of shock and hypotension and elevated markers of right ventricular dysfunction and myocardial injury[7,8]. Clot burden alone is not a principal marker in clinical risk stratification, however computed tomography pulmonary angiogram (CTPA), is a commonly used diagnostic tool that has been shown to help with risk stratification of PE[9-11]. Objectively measuring thrombus volume from CTPA data can be particularly useful to evaluate the efficacy of treatments for PE[12]. Indeed, clinical pharmaceutical trials of drugs such as thrombolytics rely upon objective measures including total thrombus volume (TTV), the sum of volumes of all PE present in an individual, to assess drug effectiveness and potency and to help determine the optimal duration of therapy[13]. This study is not performed to assess CTPA measured clot volumes as a clinical prognostic indicator. Rather, the goal of this study is to evaluate the reproducibility of a quantifiable metric; the TTV in PE, especially as new drugs are being developed that aim to eliminate and reduce clot size. Thrombus volume measured by contrast enhanced CTPA could potentially serve as an imaging biomarker for evaluating burden and/or severity of PE in clinical trials. One such current study is the DS-1040b, a Randomized Study to Assess the Safety, Pharmacokinetics/Dynamics of DS-1040b in Subjects With Acute Submassive Pulmonary Embolism, NCT02923115 (www.clinicaltrials.gov).

Quantifying clot burden with CTPA requires segmentation of emboli from non-thrombotic contrast-enhanced blood within the pulmonary vasculature and computation of clot volume. Semi-automated region growing algorithms can be used for this purpose[12]. The inter and intra-observer reliability of this technique has been tested in data obtained from a single center using one fixed CTPA imaging protocol[12]. Accuracy of this approach has also been established by the relative volume measurement error[12]. However, these types of studies have not been implemented in a multicenter setting. This is a retrospective study performed on sample data obtained from subjects undergoing CT pulmonary angiography for suspected PE. Images were obtained from multiple centers as part of a study qualification visit for a multicenter drug trial evaluating a new thrombolytic agent and were from cases of suspected PE at the site obtained in the week prior to the qualification visit. Therefore, in addition to the stated study goal of evaluating the quantifiable metric of TTV in PE, the broader purpose of this study is to evaluate the hypothesis that repeatable volume quantification can be made using a semi-automated region growing algorithm on in vivo PE data obtained in a multicenter setting with inherent variability of CT scanners and acquisition and reconstruction parameters using TTV as a primary endpoint and individual thrombus volume (ITV), Qanadli score and modified Qanadli score as secondary endpoints.

MATERIALS AND METHODS

This study was performed using anonymized data in conformance with HIPAA and IRB Regulations (March 2015-November 2016).

CTPA data was acquired from 23 scanners from 18 different centers using the standard image acquisition for PE at the site. Table 1 describes the image acquisition and reconstruction parameters and contrast agent use protocol at each center. The data was completely anonymized. Patient demographic data was not included in the study as the cases obtained from the multicenter sites were requested as part of our core lab qualification assessment for participation in a clinical pharmaceutical trial, and as such, a deidentified random sample case(s) from each site was requested. From image metadata, limited information on gender and/or age on less than half of the cases were determined. No other demographic information was available to the authors. Of the 23 patients studied, seven were known to be male. Where the male’s age was identified the following ages were known: 72, 60, 82, 30 and 73. Two patients were known to be female where their ages were known to be 21 and 77. There were also two patients of unknown gender that were known to be 72 and 81 years old.

Table 1

Scanner Information and computed tomography pulmonary angiogram acquisition protocol from each scanner

Site	Scan	Manufacturer	Model	Number of slices	Recon thickness (mm)	kVp	Pitch	Contrast agent
Academic Medical Center, Amsterdam, Netherlands	1	Siemens	Definition AS+	128	1	120	0.45	Ultravist 300
	2	Siemens	Force	384	1.5	90	0.55	Ultravist 300
Azienda Ospedaliero Universitaria Ospedali Riuniti Di Ancona, Ancona, Italy	3	Philips	Brilliance 16	16	2	120	0.9	Isovue 350
Fondazione Poliambulanza Istituto Ospedaliero, Brescia, Italy	4	GE	Optima CT660	128	0.625	120	1.375	Ultravist 370
Hôpital de la Cavale Blanche, Brest, France	5	Siemens	Definition AS +	128	1	100	1.2	Iomeron 400
Cliniques Universitaires Saint-Luc, Brussels, Belgium	6	Philips	ICT Brilliance 64	256	1	120	0.797	Iomeron 400
CHU de Clermont-Ferrand, Clermont-Ferrand, France	7	GE	Revolution GSI	64	0.625	120	1.375	Ultravist 370
Hospital Universitario Dr. Josep Trueta, Girona, Spain	8	Philips	Ingenuity CT	64	1	120	0.952	Omnipaque 350
Medical University Graz, Graz, Austria	9	Toshiba	Aquilion64	64	0.5	120	1	Optiray 350
	10	Toshiba	Aquilion64	64	0.5	120	1	Iomeron 400
Universitätsmedizin Greifswald, Greifswald, Germany	11	Siemens	Sensation 16	16	1.5	120	1.2	Iomeron 350
Hopital Michallon - CHUGA, Grenoble, France	12	GE	Optima CT 660	128	0.625	100	0.984375	Iomeron 350
Leiden University Medical Center, Leiden, Netherlands	13	Toshiba	Aquilion ONE	320	1	80	0.8129	Ultravist 370
UZ Gasthuisberg, Leuven, Belgium	14	Siemens	Definition Flash	128	1	100	0.55	Visipaque 320
	15	Siemens	Definition Flash	128	1	100	0.55	Visipaque 320
	16	Siemens	Definition Flash	128	1	100	0.55	Visipaque 320
Cedars Sinai Medical Center, Los Angeles, United States	17	GE	LightSpeed VCT	64	0.625	120	0.984375	Omnipaque 350
Hospital Universitario Ramon y Cajal, Madrid, Spain	18	Toshiba	Aquilion ONE	320	0.5	100	1.375	Iomeron 350
Intercoastal Medical Group, Sarasota, United States	19	Siemens	Definition AS+	128	0.6	120	1.1	Isovue 370
CHU de St. Etienne, St. Etienne, France	20	Siemens	Definition DS	64	1	120	1.2	Xenetix 350
Strasbourg University Hospital, Strasbourg, France	21	GE	Discovery CT750 HD	64	0.625	100	1.375	Iomeron 400
	22	Toshiba	Aquilion One	320	1	100	0.8129	Iomeron 400
Ospedale di Circolo, University of Insubria, Varese, Italy	23	Siemens	Sensation 40	40	0.75	100	1	Iomeron 370

All images were obtained with dose modulation of tube current. Filter and reconstruction kernels varied.

Two experienced image analysts performed the in vivo study. AEK (a board-certified diagnostic radiologist with six years clinical experience) and ANP (Bachelor of Science with two years vascular imaging experience under the supervision of board certified radiologists (AJ, AEK) assessed each patient for the presence of emboli within the pulmonary arterial tree. The extent of the analysis was from the main pulmonary artery up to and including emboli within segmental arteries bilaterally. See Figure 1 for a schematic of the segmental distribution of the pulmonary arteries. An embolus appeared as a filling defect of soft tissue density within the otherwise contrast enhanced pulmonary arterial tree. An endoluminal filling defect was deemed a single embolus if it was contiguous and demonstrated no intervening contrast material fully separating it from adjacent clot. Embolus volume was the measured parameter and the endpoints were TTV and ITV in each patient and both the Qanadli and modified Qanadli score in all patients. Each separate embolus was analyzed using a semi-automated region growing algorithm implemented in the FDA approved Siemens syngo.via image analysis platform. This region growing algorithm was similar to what has been used in previous studies for quantification of PE clot volume[12] and was also based on the methods shown in the following papers[14-16]. Briefly, the region growing algorithm was a pixel-based image segmentation approach involving the initial selection of seed points by the user. This segmentation method examined the nearest neighboring pixels of initial seed points and determined if the pixel neighbors should be added to the region. The process was then iterated on, using data clustering algorithms based on image pixel data intensity and texture.

Figure 1

Schematic of segmental distribution of pulmonary arteries. MPA: Main pulmonary artery; RPA: Right pulmonary artery; TA: Truncus anterior; RILA: Right interlobar artery; RBT: Right basal trunk; RA1: Right upper lobe, apical; RA2: Right upper lobe, posterior; RA3: Right upper lobe, anterior; RA4: Right middle lobe, lateral; RA5: Right middle lobe, medial; RA6: Right lower lobe, superior; RA7: Right lower lobe, medial basal; RA8: Right lower lobe, anterior basal; RA9: Right lower lobe, lateral basal; RA10: Right lower lobe, posterior basal; LPA: Left pulmonary artery; LILA: Left interlobar artery; LBT: Left basal trunk; LA1: Left upper lobe, apical; LA2: Left upper lobe, posterior; LA3: Left upper lobe, anterior; LA4: Lingula, superior; LA5: Lingula, inferior; LA6: Left lower lobe, superior; LA7: Left lower lobe, medial basal; LA8: Left lower lobe, anterior basal; LA9: Left lower lobe, lateral basal; LA10: Left lower lobe, posterior basal.

At the viewing workstation, the image analyst performed segmentation and volumetric quantification of emboli. To acquire these readings, the cursor was placed on the clot and the left mouse button was pressed and held in order to grow into adjacent regions with similar density readings. The analyst then assessed the need to include and/or exclude portions of the segmented mask by visually assessing the anatomic extent of the embolus and the surrounding anatomic structures and then made appropriate corrections and adjustments to the segmented mask. Figure 2 shows an example of in vivo region growing. In order to determine the inter observer reproducibility of the data analysis, the two independent observers were blinded to each other’s interpretations when computing the clot volumes for each of the datasets using the same semi-automated algorithm and image analysis package[12,14-16]. One reader (AEK) repeated the analysis a second time to establish the intra-reader reproducibility.

Figure 2

Computed tomography pulmonary angiogram images demonstrating segmentation of a saddle embolus in three orthogonal views (arrows).

A Qanadli score was calculated by assessing the presence and degree of obstruction in the pulmonary arterial tree bilaterally[17]. More specifically, the Qanadli score was calculated by assessing the CTPA in two ways, first, by evaluating the distribution of emboli in order to arrive at a weighting factor, and second, by determining a level of occlusion score. There are 10 segmental arteries on each side with three segmental arteries feeding the upper lobes, two feeding the right middle lobe and lingula (left) and five feeding the lower lobes. The presence of an embolus in a segmental vessel was given a weighting score of one. Central arteries were given heavier weighting than more distal vessels using the following method: the weighting factor for a vessel with an embolus was based on the number of segmental vessels it feeds. The more distal vessels were then excluded from the score. The scale for the level of obstruction of a given vessel was as follows: 0 represents no embolus detected; 1 represents a partially occlusive embolus; and 2 represent a fully occlusive embolus. When the weighting factor and obstruction factors were multiplied then added together, there was a maximal possible score of 20 per side for a total possible raw score of 40. With this information the Qanadli score was then calculated and reported as a percentage by dividing the raw score by 40, the maximal score[17].

A Modified Qanadli score was also calculated by solely assessing the presence and degree of obstruction in the ten segmental vessels bilaterally[18]. We performed this additional obstruction index to pull out any difference that may be seen by using the Qanadli with its focus on weighted central vessels vs simply assessing obstruction in the individual segmental vessels alone as we have done in our modified version. The same Qanadli scale for the level of obstruction was used in our modified version with 0 representing no embolus, 1 representing a partially occlusive embolus, and 2 representing a fully occlusive embolus. As with Qanadli scoring, a maximum raw score of 40 and a final score reported as a percentage relative to 40 was used.

Statistical analysis

All statistical analysis presented here was performed or guided by a biomedical statistician. Intra-class correlation coefficients (ICCs) and Bland-Altman analyses were performed using GraphPad Prism 7 for Mac, GraphPad Software, San Diego California, United States, www.graphpad.com. The ICCs were used to compare the results obtained between the two readers as well as to compare the results obtained from the two analyses of the same reader. The agreement between the measurements by each observer in the two reading sessions and the agreement between the two observers were also assessed using Bland-Altman analyses. According to this method, the mean difference between measurements is defined as “bias” and represents the systemic error in measurements. We calculated 95%CI for bias and for the limits of agreement. ICC provides the overall inter- or intra-observer agreement whereas the Bland-Altman method gives the discrepancy in measurements at the individual level. A one sample t-test was performed to determine if any bias was observed on the Bland-Altman analysis. For TTV, in addition to evaluating absolute differences by Bland Altman analysis, we also evaluated the % difference between the two readers or two reads.

RESULTS

Analyst 1 found 72 emboli in the 23 patients with a mean number of emboli of 3.13 per patient. With the three negative cases excluded the mean number of emboli per patient was 3.6. Considering all cases there was a range of 0-11 emboli per patient. (0 in 3 patients (pts); 1 in 3 pts; 2 in 7 pts; 3 in 2 pts; 4 in 1 pt; 5 in 3 pts; 6 in 2 pts; 7 in 1 pt; and 11 in 1 pt). The clot volumes ranged from 0.0041–47.34 cm3 (mean +/- SD, 5.93 +/- 10.15 cm3). On the second read, analyst 1 found the same number and distribution of emboli. The clot volume range varied on the upper extent as compared to the initial read. The clot volumes for read 2 ranged from 0.0041-45.52 cm3 (mean +/- SD, 5.42 +/- 9.53 cm3).

Analyst 2 found 73 emboli in the 23 patients with a mean number of emboli of 3.17 per patient. With the three negative cases excluded the mean number of emboli per patient was 3.65. There was a range of 0-11 emboli per patient. (0 in 3 pts; 1 in 3 pts; 2 in 7 pts; 3 in 2 pts; 4 in 1 pt; 5 in 2 pts; 6 in 3 pts; 7 in 1 pt; and 11 in 1 pt). The clot volumes ranged from 0.00459-46.29 cm3 (mean +/- SD, 5.91 +/- 10.06cm3).

The ICC calculated for the TTV measurements per patient for inter-observer analysis was 0.998 and for intra-observer analysis was 0.997, while the associated Bland-Altman analyses for inter-and intra-observer agreement for TTV demonstrated no inter- or intra-observer biases (P = 0.23 for inter-observer results, P = 0.26 for intra-observer results for TTV using a one-sample t-test with 0, two tailed). The ICC and Bland-Altman analyses are seen in Figure 3. The ICC calculated for the ITV measurements per patient for inter-observer analysis was 0.996 and for intra-observer analysis was 0.997, while the associated Bland-Altman analyses for inter-and intra-observer agreement for ITV demonstrated no inter- or intra-observer biases. (P = 0.55 for inter-observer results, P = 0.24 for intra-observer results for TTV using a one-sample t-test with 0, two tailed). The ICC and Bland-Altman analyses for the ITV are seen in Figure 4.

Figure 3

Total thrombus volume inter- and intra-observer reproducibility (A) Intra-class correlation coefficient (ICC) plot and (B) Bland Altman plot comparing the total PE thrombus volume results of the primary and secondary image analyst for the inter-observer reproducibility analysis (C) ICC plot and (D) Bland Altman plot comparing the total pulmonary embolism thrombus volume results of the first and second read of the primary image analyst for the intra-observer reproducibility analysis.

Figure 4

Individual thrombus volume inter- and intra-observer reproducibility (A) Intra-class correlation coefficient (ICC) plot and (B) Bland Altman plot comparing the individual PE thrombus volumes results of the primary and secondary image analyst for the inter-observer reproducibility analysis (C) ICC plot and (D) Bland Altman plot comparing the individual pulmonary embolism thrombus volumes results of the first and second read of the primary image analyst for the intra-observer reproducibility analysis.

A Qanadli score was calculated for each patient. The range of for Analyst 1 was 0%-52.5% for read one and 0%-52.5% for read two. The range for Analyst 2 was 0%-72.5%. The 20 positive cases had a mean Qanadli score with associated standard deviations of 37.25% +/- 16.93 and 39.125% +/- 19.20 for Analysts 1 and 2 respectively. The mean and standard deviation for Analyst 1’s second read was 37.25% +/- 16.93. The ICC calculated for the Qanadli score per patient for inter- and intra-observer was 0.944 and 1 respectively. ICC plots and Bland Altman plots for inter- and intra-observer reproducibility for Qanadli assessment are shown in Figure 5.

Figure 5

Qanadli score inter- and intra-observer reproducibility (A) Intra-class correlation coefficient (ICC) plot and (B) Bland Altman plot comparing the Qanadli score results of the primary and secondary image analyst for the inter-observer reproducibility analysis (C) ICC plot and (D) Bland Altman plot comparing the results of the first and second read of the primary image analyst for the intra-observer reproducibility analysis of the PE obstruction index (Qanadli score).

The Modified Qanadli score was calculated for each patient as a secondary study endpoint. The range for Analyst 1 was 0%-70% for read one and 0%-72.5% for read two. The range of scores for Analyst 2 is 0%-72.5%. The 20 positive cases had a mean and associated standard deviations of 38.125% +/- 20.87 and 36.625% +/- 21.11 for Analysts 1 and 2 respectively. The mean and standard deviation for Analyst 1’s second read was 38.5% +/- 21.34. The ICC calculated for the Modified Qanadli score per patient for inter- and intra-observer was 0.996 and 0.999 respectively. ICC plots and Bland Altman plots for inter- and intra-observer reproducibility for the Modified Qanadli assessment are shown in Figure 6.

Figure 6

Modified Qanadli score inter- and intra-observer reproducibility (A) Intra-class correlation coefficient (ICC) plot and (B) Bland Altman plot comparing the modified Qanadli results of the primary and secondary image analyst for the inter-observer reproducibility analysis (C) ICC plot and (D) Bland Altman plot comparing the results of the first and second read of the primary image analyst for the intra-observer reproducibility analysis.

Finally, we also evaluated the % difference in TTV between the two analysts (P = 0.074, one sample t-test with zero, two tailed) as well as the two reads by Analyst 1 (P = 0.063, one sample t-test with 0, two tailed). These Bland-Altman plots are shown in Figure 7.

Figure 7

Bland Altman analysis of total thrombus volume normalized to clot size (Mean vs Difference/Mean) for (A) intra and (B) inter observer analysis.

DISCUSSION

Emboli to the lungs cause physical obstruction of the pulmonary arteries yielding a pathophysiologic cascade resulting in varying degrees of cardiovascular distress[19]. Although the embolic load and distribution are considered less pertinent than the patient’s hemodynamic status in evaluating risk stratification[7] clot burden has been shown to have predictive value of mortality in patients with acute PE[20]. For example, Collomb et al[21] have shown that the hemodynamic severity of acute PE can be determined by assessing clot burden through the use of a vascular obstruction index as well as other vascular load measurements such as right ventricular (RV): left ventricular (LV) ratio, minimum LV diameter and diameter of the central PA[21]. Furthermore, Furlan et al[12] have shown that quantification of clot burden can be accurate and reproducible at a single institution. The challenges faced in multicenter studies include the variety of imaging systems and models used and the resultant variability in data acquisition. Additional variables could include differences in the energy level (kVp), pitch, reconstruction methods and contrast agents utilized.

Overall, inter- and intra-observer variability measurements indicated excellent reproducibility of the semi-automated analysis method for quantifying PE thrombus volume/burden. The ICC coefficient for all endpoints TTV (primary), ITV (secondary), Qanadli score (secondary) and modified Qanadli score (secondary) all had ICC values greater than 0.95 for both the inter- and intra-observer analysis. The Bland-Altman analysis also indicated no significant biases in any of the metrics evaluated. Our data showed that even with the variability inherent in multicenter data acquisition and reconstruction, the image analysis methodology employed here is reproducible and therefore suitable for use in a multicenter setting. These findings have implications on future studies of PE by allowing for both the option of multisite data acquisition with evaluation at an imaging core laboratory, as well as inclusion of direct PE volume measurements and assessment of temporal changes in embolic load. For example, in the SEATTLE II study, a multicenter investigation to assess the safety and efficacy of ultrasound-facilitated catheter-directed, low dose fibrinolysis therapy in patients with acute submassive PE, the RV/LV diameter ratio was used as a primary efficacy outcome[22]. Subsequent research in SEATTLE II and similar studies can add direct clot burden volumetrics in furtherance of the body of knowledge in this area.

Our results compare well with those obtained by Nakada et al[23], who evaluated inter- and intra-reader reproducibility studies on manually measured PE volumes and showed no statistical differences in either inter- or intra-reader analyses. The intra-reader analysis in that study was nine months, much greater than in our study.

Implications of our findings on multicenter studies: In a recent study, we have shown that quantification of clot volumes by CT is unaffected by most imaging acquisition and reconsturction parameters (except large differences in pitch used during acquisition)[24]. Results of the current study show that the image analysis of data acquired from a multicenter setting is robust with the size of the clot not affecting the reproducbility of quantification significantly. Absolute quantification differences for TTV and ITV for both inter and inra-observer results were in the range of 2-4 cm3 (95%CI on Bland Altman plots, Figures 3 and 4). As a percent difference, these were of the order of 5%-10% (95%CI, Figure 7). This indicates that with our methodology we should be able to robustly pick up changes in absolute thrombus volume greater approximately 2 cm3 or 5% regardless of the starting size of the clots. this data can be used to estimate sample size requirements for clinical trials using clot burden quantification as an endpoint in PE treatment studies.

The study was limited by several factors. First, a small number of subjects were evaluated at a small number of sites. Images of subjects were chosen for this study during a site qualification visit prior to the start of a multicenter clinical trial evaluating a new thrombolytic agent for PE. Cases were also chosen to provide a spread of scanners and variations in imaging protocols across the sites. There could therefore be a selection bias in how these individuals were included as part of this current study. Second, the intra-reader variability analysis may have been affected by recall bias as there was a short interval of two weeks between the reads. This may have been insufficient to eliminate recall bias which could have resulted in the perfect agreement between the scores in the two reads. However, to reduce recall bias, images were presented to the image analyst in a randomized fashion for the two reads. Third, we were unable to assess radiation dose as a variable because the various scanners employ dose modulation and this setting is dependent upon patient size. Fourthly, we do not have access to demographic information of the subjects as the images analyzed were read anonymously and devoid of any Protected Health Information to comply with HIPAA requirements and our IRB approval. Fifthly, the inter reader assessments were only performed by 2 analysts and the intra reader analysis was performed on two reads by only a single analyst. Analysis by more individuals will improve the robustness of the results. Lastly, Qanadli scoring may also be inherently skewed leading to high ICC because it relies on the weighting of affected proximal vascular branches thereby yielding specific discreet scores. This bias could be emphasized in our patient population that demonstrated large and often proximal PEs.

In conclusion, the data showed that our image analysis methodology is reproducible and therefore suitable for future use in a multicenter setting.

ARTICLE HIGHLIGHTS

Research background

In regard to clinical evaluation of pulmonary embolism (PE), clot burden is not a principal marker for clinical risk stratification, however clot burden is being used to assess for pharmaceutical characteristics in clinical drug trials in multicenter settings. To this point the technique has been studied with data obtained from a single imaging center using one fixed computed tomography pulmonary angiogram (CTPA) imaging protocol.

Research motivation

Data obtained from multicenter sites has not previously been studied. Thus, in order to validate the methods employed in multicenter clinical pharmaceutical trials of drugs such as thrombolytics, this study was commenced to assess for repeatability and consistency of clot volume measurements being obtained using semi-automated region growing techniques. Confirming the reliability of these measures has value in furthering the assessment of drug effectiveness, drug potency and in determination of optimal duration of therapy.

Research objectives

The key objective is to evaluate reproducibility of PE clot volume quantification using a semi-automated region growing algorithm on CTPA data in a multicenter setting.

Research methods

Anonymized CTPA data was acquired from 23 scanners from 18 imaging centers using each site’s standard PE protocol. Two independent analysts measured PE volumes using a semi-automated region-growing algorithm on an FDA-approved image analysis platform. Total thrombus volume was calculated per patient as the primary endpoint. Secondary endpoints were individual thrombus volume, Qanadli score and modified Qanadli score per patient. Inter- and intra-observer reproducibility were assessed using intra-class correlation coefficient (ICC) and Bland-Altman analysis. The methods employed in this study were novel in that they previously have not been used in a multicenter setting.

Research results

The results showed excellent reproducibility of inter- and intra-observer variability measurements using the semi-automated region-growing method for quantifying PE volume burden. ICC for all endpoints was greater than 0.95 for inter- and intra-observer analysis. Bland-Altman analysis indicated no significant biases. The results confirm the validity of the methods used in multicenter pharmaceutical trials thereby allowing for advancement in this field.

Research conclusions

Semi-automated region growing algorithm for quantifying PE is reproducible using data from multiple scanners and is a suitable method for image analysis in multicenter clinical trials. The utility of validating this method could affect the advancement of thrombolytic therapy and other interventions that may be used to treat PE.

Research perspectives

Computer-assisted image analysis has a growing role in both diagnostic and investigative imaging. This study solidifies the foundation of semi-automated region growing for volume quantification by proving the repeatability of the technique when used in a multicenter setting.

ACKNOWLEDGEMENTS

This study is supported by Daiichi Sankyo Pharma Development. In addition, we wish to acknowledge the following hospitals, site names, principal investigators, and radiologists. Dr. Saskia Middeldorp and Dr. Ludo Beenen - Academic Medical Center, Amsterdam, Netherlands; Dr. Aldo Salvi, Dr. Cinzia Nitti and Dr. Arianna Lorenzoni - Azienda Ospedaliero Universitaria Ospedali Riuniti Di Ancona, Ancona, Italy; Dr. Claudio Cuccia and Dr. Silvia Magnaldi - Fondazione Poliambulanza Istituto Ospedaliero, Brescia, Italy; Dr. Francis Couturaud and Dr. Michel Nonent - Hôpital de la Cavale Blanche, Brest, France; Dr. Franck Verschuren and Dr. Benoit Ghaye - Cliniques Universitaires Saint-Luc, Brussels, Belgium; Dr. Jeannot Schmidt and Dr. Lucie Cassagnes - CHU de Clermont-Ferrand, Clermont-Ferrand, France; Dr. Fernando Garcia-Bragado and Dr. Pedro Ortuno Muro - Hospital Universitario Dr. Josep Trueta, Girona, Spain; Dr. Marianne Brodmann and Dr. Reinhard Raggam - Medical University Graz, Graz, Austria; Dr. Klaus Empen and Dr. Birger Mensel - Universitätsmedizin Greifswald, Greifswald, Germany; Dr. Helen Bouvaist and Dr. Adrien Jankowski - Hopital Michallon - CHUGA, Grenoble, France; Dr. Menno Huisman and Dr. Lucia Kroft - Leiden University Medical Center, Leiden, Netherlands; Dr. Peter Verhammo, Dr. Johny Verschakelen and Dr. Walter Coudyzer - UZ Gasthuisberg, Leuven, Belgium; Dr. Victor Tapson and Dr. Peter Julien - Cedars Sinai Medical Center, Los Angeles, United States; Dr. David Jimenez and Dr. Agustina Vicente Bartulos - Hospital Universitario Ramon y Cajal, Madrid, Spain; Dr. Maurizio Concha - Intercoastal Medical Group, Sarasota, United States; Dr. Laurent Bertuletti and Dr. Pierre Croisille - CHU de St. Etienne, St. Etienne, France; Dr. Dominique Stephan and Dr. Michael Ohana - Strasbourg University Hospital, Strasbourg, France and Dr. Walter Ageno and Dr. Chiara Floridi - Ospedale di Circolo, University of Insubria, Varese, Italy.