| Literature DB >> 33783960 |
Alex Kubli1, Jason Pukala2, Amish P Shah2, Patrick Kelly2, Katja M Langen3, Frank J Bova4, Rafael R Mañon2, Sanford L Meeks2.
Abstract
PURPOSE: The purpose of this study was to evaluate the performance of three common deformable image registration (DIR) packages across algorithms and institutions. METHODS AND MATERIALS: The Deformable Image Registration Evaluation Project (DIREP) provides ten virtual phantoms derived from computed tomography (CT) datasets of head-and-neck cancer patients over a single treatment course. Using the DIREP phantoms, DIR results from 35 institutions were submitted using either Velocity, MIM, or Eclipse. Submitted deformation vector fields (DVFs) were compared to ground-truth DVFs to calculate target registration error (TRE) for six regions of interest (ROIs). Statistical analysis was performed to determine the variability between each DIR software package and the variability of users within each algorithm.Entities:
Keywords: deformable; error; image; registration; virtual phantoms
Year: 2021 PMID: 33783960 PMCID: PMC8130225 DOI: 10.1002/acm2.13242
Source DB: PubMed Journal: J Appl Clin Med Phys ISSN: 1526-9914 Impact factor: 2.102
Fig. 1Virtual phantom example. (a) Start‐of‐treatment image. (b) End‐of‐treatment image. (c) Start‐of‐treatment image with the ground‐truth DVF overlaid.
Details of the DIR algorithms evaluated in this study.
| Software | Versions | DIR algorithm | Algorithm type | Number of institutions |
|---|---|---|---|---|
| Velocity | 3.1 to 3.2.1 | Deformable multi‐pass or extended deformable multi‐pass | Multiresolution B‐spline | 16 |
| MIM | 6.2.2 to 6.7.7 | VoxAlign | Constrained, intensity‐based, free‐form | 13 |
| Eclipse | 11 to 13.7 | SmartAdapt | Accelerated demons | 6 |
Summary statistics for all institutions that submitted data using Velocity, MIM, and Eclipse. Errors shown are ±standard error of the mean.
| Region | Velocity (N = 16) | MIM (N = 13) | Eclipse (N = 6) | |||
|---|---|---|---|---|---|---|
| Mean (mm) | Max (mm) | Mean (mm) | Max (mm) | Mean (mm) | Max (mm) | |
| Brainstem | 1.23 ± 0.23 | 4.8 | 0.42 ± 0.10 | 2.4 | 1.15 ± 0.10 | 7.3 |
| Cord | 1.40 ± 0.24 | 12.2 | 0.38 ± 0.15 | 2.7 | 1.12 ± 0.04 | 4.6 |
| Mandible | 1.34 ± 0.23 | 9.0 | 0.68 ± 0.24 | 8.2 | 1.97 ± 0.08 | 9.4 |
| Left parotid | 1.92 ± 0.21 | 10.8 | 1.12 ± 0.11 | 13.1 | 1.92 ± 0.12 | 11.0 |
| Right parotid | 1.59 ± 0.26 | 13.1 | 1.23 ± 0.28 | 26.9 | 1.75 ± 0.10 | 8.9 |
| External | 2.04 ± 0.35 | 40.7 | 1.10 ± 0.29 | 34.7 | 2.35 ± 0.15 | 25.0 |
Fig. 2Mean TRE for each institution in this study. The error bars represent one standard deviation of the mean registration error for each of the ten phantoms. Note that the Y‐axis scale is 4 mm and that the mean TRE difference between institutions is typically <1 mm.
Results of Welch’s t‐test to compare mean TREs across algorithms. Tests marked with an asterisk (*) indicate a significant difference in mean TRE.
| Algorithm 1 | Algorithm 2 | Region |
|
|---|---|---|---|
| Velocity | MIM | Brainstem | <0.001* |
| Velocity | Eclipse | Brainstem | 0.312 |
| MIM | Eclipse | Brainstem | <0.001* |
| Velocity | MIM | Cord | <0.001* |
| Velocity | Eclipse | Cord | <0.001* |
| MIM | Eclipse | Cord | <0.001* |
| Velocity | MIM | Mandible | <0.001* |
| Velocity | Eclipse | Mandible | <0.001* |
| MIM | Eclipse | Mandible | <0.001* |
| Velocity | MIM | Parotid L | <0.001* |
| Velocity | Eclipse | Parotid L | 0.977 |
| MIM | Eclipse | Parotid L | <0.001* |
| Velocity | MIM | Parotid R | 0.001* |
| Velocity | Eclipse | Parotid R | 0.058 |
| MIM | Eclipse | Parotid R | <0.001* |
| Velocity | MIM | External | <0.001* |
| Velocity | Eclipse | External | 0.009 |
| MIM | Eclipse | External | <0.001* |
Results of one‐way ANOVAs performed on mean TRE for each region. Regions that show a significant institutional dependence are indicated by an asterisk (*).
| Region | Magnitude (mm) | F‐value |
|
|---|---|---|---|
| Velocity | |||
| Brainstem | 1.23 | 1.62 | 0.074 |
| Cord | 1.40 | 0.873 | 0.596 |
| Mandible | 1.34 | 2.92 | < 0.001* |
| Left parotid | 1.92 | 0.928 | 0.535 |
| Right parotid | 1.59 | 1.87 | 0.031 |
| External | 2.04 | 2.93 | < 0.001* |
| MIM | |||
| Brainstem | 0.42 | 5.56 | < 0.001* |
| Cord | 0.38 | 31.7 | < 0.001* |
| Mandible | 0.68 | 12.4 | < 0.001* |
| Left parotid | 1.12 | 0.526 | 0.894 |
| Right parotid | 1.23 | 0.650 | 0.795 |
| External | 1.10 | 13.8 | < 0.001* |
| Eclipse | |||
| Brainstem | 1.15 | 0.426 | 0.829 |
| Cord | 1.12 | 0.119 | 0.988 |
| Mandible | 1.97 | 0.096 | 0.993 |
| Left parotid | 1.92 | 0.466 | 0.800 |
| Right parotid | 1.75 | 0.168 | 0.974 |
| External | 2.35 | 2.46 | 0.044 |
Registration error and Welch’s t‐test results for MIM before and after version 6.6. The t‐test compared mean TRE across version grouping. Errors shown are ±standard error of the mean. Asterisk (*) indicate significant values.
| Pre‐6.6 (N = 8) | Post‐6.6 (N = 5) |
| |||
|---|---|---|---|---|---|
| Mean (mm) | Max (mm) | Mean (mm) | Max (mm) | ||
| Brainstem | 0.45 ± 0.01 | 2.4 | 0.31 ± 0.00 | 1.4 | <0.001* |
| Cord | 0.47 ± 0.03 | 2.7 | 0.23 ± 0.00 | 2.1 | <0.001* |
| Mandible | 0.87 ± 0.08 | 8.2 | 0.41 ± 0.00 | 5.8 | <0.001* |
| Left parotid | 1.20 ± 0.05 | 13.1 | 0.97 ± 0.01 | 10.7 | <0.001* |
| Right parotid | 1.46 ± 0.02 | 26.9 | 0.92 ± 0.01 | 10.9 | <0.001* |
| External | 1.36 ± 0.04 | 34.7 | 0.79 ± 0.01 | 26.3 | <0.001* |
One‐way ANOVA results for MIM mean TRE before and after version 6.6.
| Region | Magnitude (mm) | F‐value |
|
|---|---|---|---|
| MIM pre‐6.6 | |||
| Brainstem | 0.50 | 0.002 | 1.00 |
| Cord | 0.49 | 0.033 | 0.999 |
| Mandible | 0.85 | 0.087 | 0.999 |
| Left parotid | 1.19 | 0.015 | 0.999 |
| Right parotid | 1.44 | 0.003 | 1.00 |
| External | 1.34 | 0.008 | 1.00 |
| MIM post‐6.6 | |||
| Brainstem | 0.30 | 0.002 | 0.999 |
| Cord | 0.20 | 0.000 | 1.00 |
| Mandible | 0.40 | 0.001 | 0.999 |
| Left parotid | 1.00 | 0.001 | 0.999 |
| Right parotid | 0.90 | 0.001 | 0.999 |
| External | 0.82 | 0.004 | 0.999 |