Comparison of quantitative measurements of four manufacturer's metal artifact reduction techniques for CT imaging with a self-made acrylic phantom.

BACKGROUND: Metal artifact reduction (MAR) techniques can improve metal artifacts of computed tomography (CT) images.
OBJECTIVE: This work focused on conducting a quantitative analysis to compare the effectiveness of four commercial MAR techniques on three types of metal implants (hip implant, spinal implant, and dental filling) with a self-made acrylic phantom.
METHODS: A cylindrical phantom was made from acrylic with a groove in the middle, and then three types of metal implants were placed in the groove. The phantom was scanned by four CT scanners and four commercialized MAR techniques were used to analyze the images. The techniques used were single-energy metal artifact reduction (SEMAR, Canon), smart metal artifact reduction software (Smart-MAR, GE), iterative metal artifact reduction (IMAR, Siemens), and metal artifact reduction for orthopedic implants (OMAR, Philips). Quantitative analysis methods included objective and subjective analysis.
RESULTS: The expected value of SEMAR, Smart-MAR, IMAR, and OMAR were 36.6, 37.8, 5.0, and 2.3, respectively. SEMAR and Smart-MAR achieved optimal results.
CONCLUSION: This study successfully evaluated the effects of four commercial MAR techniques on three types of metal implants in a phantom. All MAR techniques effectively reduced metal artifacts, but the effect was not significant with dental fillings due to high-density material.

Introduction

In a computed tomography (CT) scan, a severe artifact is produced when metal implants are present. The degree of the artifact depends on the size, shape, and density of the metal objects. A metal implant may produce a beam-hardening, partial volume artifact, and photon starvation. The main expression of metal objects is high-density or low-density artifacts in a strip shape diverging from the metal objects [1]. This phenomenon may reduce image quality and even affect the interpretation and diagnosis of images [2, 3]. A metal artifact reduction (MAR) technique can effectively reduce artifacts caused by metal implants [4]. Gjesteby et al. divided MAR techniques into six categories, namely metal implants optimization, acquisition improvement, physics-based preprocessing, projection completion, iterative reconstruction, and image post-processing [5]. Projection completion and iterative reconstruction are the most common techniques of the six.

Several commercialized MAR techniques have been developed to overcome various types of metal artifacts. The techniques include single-energy metal artifact reduction (SEMAR) by Canon Medical Systems [6, 7], metal artifact reduction software (MARS) and smart metal artifact reduction software (Smart-MAR) by GE Medical Systems [8, 9, 10], iterative metal artifact reduction (IMAR) by Siemens Healthcare [11, 12, 13], and metal artifact reduction for orthopedic implants (OMAR) by Philips [14, 15]. It has been clinically verified that these techniques can effectively reduce metal artifacts in a CT scan, improve image quality and diagnostic value, and even improve treatment planning of radiation therapy [16].

Previous studies have made simple comparisons between the effects of two of the commercialized MAR techniques [17, 18, 19]. Andersson et al. subjectively and objectively compared the application of three commercialized MAR techniques (OMAR, SEMAR, and MARS) on hip implants [20, 21]. They then compared the application of OMAR and IMAR in radiation therapy planning of head and neck [22]. Wagenaar et al. quantified and compared the application of MAR techniques with metal deletion techniques (MDT) of three companies (Philips, GE, & Siemens) on dental implants [23]. The results further indicated that the improvement in artifact reduction can be achieved by the MDT technique. Bolstad et al. compared the effects of four commercialized MAR techniques on metal implants made of different materials on a phantom leg [24].

In the literature mentioned above, the comparison of other metal implants (hip implants, spinal implants, and dental fillings) with commercialized MAR techniques of four major CT vendors is still lacking. The aim of this work is therefore to objectively and subjectively evaluate the image quality and effectiveness at reducing metal artifacts of four vendors’ MAR techniques (SEMAR, Smart-MAR, IMAR, and OMAR). We applied these four techniques on the metal implants mentioned above, the results of which would be beneficial for future clinical applications.

Material and methods

Phantom and metal implants

A cylindrical phantom was made from acrylic with a diameter of 20 cm and a length of 24 cm. A rectangular groove with a length of 20 cm, the width of 10 cm, and a depth of 16 cm was on the side of the phantom (Fig. 1). Three types of metal implants were (1) a one-sided artificial hip joint (vitallium, 8.4 g/cm), (2) two titanium bone rods (radii of 4.5 mm, 4.5 g/cm), and (3) an artificial dental stand with filler (calcium sulfate dihydrate as the material, 2.3 g/cm) that contained a mercury amalgam ( 13.9 g/cm). These three types of implants are commonly used in clinical settings. We fixed the metal implants on an ultrathin acrylic board with a height of 20 cm, the width of 10 cm, and a depth of 0.1 mm. The size of this acrylic board was equivalent to the groove in the middle of the cylindrical phantom. Therefore, each time the acrylic board was inserted into the groove, the metal implants on the board would be in a fixed position. This reduced experimental error caused by displacement.

Figure 1.

Customized acrylic phantom and scanned images. a. A cylindrical phantom with a rectangular groove in the side of the phantom. b. The phantom with metal implant is placed on the platform of a CT scanner. c. Scout image of hip implant. d. Scout image of spinal implant and dental filling.

The groove of the phantom was filled with water after metal implants were placed into it. Therefore, the metal implants were surrounded by water, which could be used for analyzing the degree to which metal artifact affected the CT number of the water (Fig. 1).

Image acquisition

The cylindrical phantom containing metal implants was scanned by CT of four manufacturers (Fig. 1): Canon Aquilion One Vision Edition (Canon Medical Systems, Otawara, Japan); GE Revolution CT (GE Healthcare, Milwaukee, WI, USA); Siemens Somatom Definition AS (Siemens Healthcare, Erlangen, Germany); Philips iCT 256 (Philips Healthcare, Best, Netherlands). The scanning protocols of each scanner are shown in Table 1. The parameters of all CT scans are frequently used in clinical settings.

Table 1

CT scan parameter

Parameter	CT scan parameters
	Canon Medical Systems (Otawara, Japan)	GE Healthcare (Milwaukee, WI)	Siemens Healthcare (Erlangen, Germany)	Philips Healthcare (Best, Netherlands)
Scanner type	Canon Aquilion ONE ViSION Edition (SE)	Revolution CT (DE by fast kV switching)	Siemens SOMATOM® Definition AS 384 (DE with one source)	Philips iCT 256 (SE)
CT protocol	Volume (SEMAR not compatible with helical scanning)	Helical (pitch 1)	Helical (pitch 1)	Helical (pitch 1)
Rotation time	0.5	0.5	0.5	0.5
Tube voltage (kVp)	120	DE: 80/140	DE: 100/140	120
CTDI_vol (mGy)	9.2	10.1	10.3	10.4
Collimation (mm)	160 × 0.5	128 × 0.625	128 × 0.6	128 × 0.625
Slice thickness (mm)	0.5	0.625	0.6	0.8
Reconstruction FOV (mm)	300	300	300	300
MAR technique	SEMAR	Smart-MAR	IMAR	OMAR
IR technique	AIDR 3D Level standard 50%	ASIR Level 50%	SAFIRE Level 3	IDOSE Level 3
Kernel	FC08	Standard	Q30	Standard B

SE, single energy; DE, dual energy; CTDIvol, volume CT dose index; MAR, metal artifact reduction; IR, iterative reconstruction.

0.5 seconds was chosen for the rotation time of the gantry. The pitch was set at 1.0 to prevent overlapping scans and the distortion of CT images caused by interpolation of the data. The volume CT dose index (CTDI) was set at approximately 10 mGy for each scanner. Collimation and slice thickness were smallest for all CT scanners. We used soft tissue kernel when reconstructing images. All four CT image constructions employed an iterative reconstruction technique at a level of 50%.

After obtaining images from four CT scanners, MAR technique of each scanner was applied to images. The Canon CT scanner of this study was operated with single energy of 120 kVp and in axial scan mode only because the SEMAR technique cannot be used in the helical scan mode. The dual energy mode (80/140 kV) and 120 keV monoenergetic images were chosen when using the GE CT. The helical mode was chosen for GE scanner because the Smart-MAR technique in axial mode has a limited scan range (40 mm). IMAR can be used both in single and dual energy modes when using the Siemens CT, but only dual energy images were presented because the quality of dual energy images was considerably higher than the quality of single energy images. The dual energy technique used 140/100 kV and energy modules of 120 keV for Siemens scanner. When using IMAR, different types of IMAR mode could be selected according to the type of metal implant. Hip implants, dental mode, and spinal implant mode of IMAR were applied to images of Siemens scanner in this work. The Philips iCT scanner in this study only provides a single energy mode, and the thickness of its smallest slice was 0.8 mm with a pitch of 1; the slice was slightly thicker than those of the other CT scanners. OMAR of Philips can be applied to images before or after the scan.

Quantitative assessment

Metal volume

ImageJ (Java 1.8, National Institutes of Health, Bethesda, USA) software was used to measure the area of metal implants () in each CT image, and the threshold value was set to half of the highest CT number of metal, which was 1548 HU for all CT images in this study. The area of metal implants higher than the threshold value was (Fig. 2). The method and the threshold to measure the metal area was in reference to the study of Huang et al. [25]. The metal area may change according to a different threshold, but the differences between four scanners will remain the same if we set a single threshold for all images.

Figure 2.

Measurement of metal volume and fraction of bad pixel area (FBPA). a. Volume of hip implant in red region. b. Measurement of fraction of bad pixel area (FBPA) of hip implant in a rectangular area of 10000 mm.

The volume of a metal object (mm) can be obtained by multiplying the sum of (mm) on slices with metal implants of each group of CT images by the slice thickness of each scanner, as per Eq. (1):

The slice thickness of each scanner is listed in Table 1. The length of hip implant and titanium rods are 195 and 75 mm respectively, and the total numbers of slices of each group of CT images will be different due to different slice thicknesses. The volume of dental fillings was not measured because the metal was close to denture which CT number was also relatively high, and it is difficult to distinguish the metal, denture, and artifact from each other with a fixed threshold.

We subtracted the volume of metal implants in the original images when MAR was not used from the volume of metal implants after MAR was used. Dividing the volume after subtraction by provided us with the improvement rate of metal volume as shown in Eq. (2). It was expected that a more advanced MAR technique would have a smaller metal area and larger metal volume improvement rate.

Fraction of bad pixel area (FBPA)

In the study by Bolstad et al. [24], a threshold above 500 HU was chosen for evaluating the degree of artifacts. However, metal artifacts also consist of dark bands or low-intensity zone, and a high-level threshold could not evaluate the dark zone. In this study, we circled a rectangular zone with an area of 10,000 mm from the water zone with metal implants in the groove of the phantom (Fig. 2), which zone is close to the maximum area of the water in order to maximize assess the area of metal artifacts. The method for measuring the area of metal objects (mm) in this zone was identical to the description in the previous section. The pixels with a CT number ranged between 40 and 40 HU in the rectangular zone were defined as water area (mm). The area beside and in the rectangular zone were defined as artifact area. FBPA was calculated by dividing the artifact area by 10,000 mm, as shown in Eq. (3):

The rectangular zone contained only water and metal objects, and thus CT numbers other than these two materials were noises and CT number errors caused by metal artifacts. The FBPA might also change according to a different threshold of water, but the differences remain between four CT scanners if the same threshold was chosen for measuring all images. 40 HU was chosen in this work to better distinguish the differences between scanners and was used in previous research [25]. An advanced MAR technique was expected low FBPA. For measuring of all images with dental fillings, the threshold was set at 1000 HU in order to include denture for calculating FBPA. The FBPA of two images without metal implants (with and without MAR) was also measured for reference.

We calculated the FBPA of all CT images of each type of metal implant and obtained the average FBPA. The FBPA of hip implant was divided into two parts (head and body) because the difference in the metal area of image cross-sections of the hip implant is too large; the spinal implant was in a cylindrical shape, and calculating the average of FBPA for all metallic images directly would suffice. Differences in the metal area of each cross-section of dental filling were overly large; the average value of FBPA of it was calculated directly. The FBPA improvement value was calculated by subtracting the average FBPA value of uncorrected images () from the average FBPA value of images after the MAR technique () was applied (Eq. (4)).

Visual ranking analysis

The image containing the most severe artifacts was selected for evaluation. A total number of 24 images were chosen (Figs 3–5); three metal implants, four scanners, and two images per scanner (with and without MAR). Eight images (with and without MAR from four vendors of each metal implant) were anonymized and randomized compared as one group for three times by each reviewer. Two radiologists (15 and 18 years of experience) and one radiographer (17 years of experience) were invited to review the image quality. The reviewers evaluated the CT images and ranked the images that optimally visualized the water area surrounding metal implants from the best to worst (first to eighth). After the ranking has been done, the best to the fourth images were scored 4, 3, 2, and 1, respectively, and the rest were scored 0.

Figure 3.

CT images of hip implants. a. Image without SEMAR; b. with SEMAR. c. Image without Smart-MAR; d. with Smart-MAR. e. Image without IMAR; f. with IMAR. g. Image without OMAR; h. with OMAR.

Figure 5.

CT images of dental filling. a. Image without SEMAR; b. with SEMAR. c. Image without Smart-MAR; d. with Smart-MAR. e. Image without IMAR; f. with IMAR. g. Image without OMAR; h. with OMAR.

Statistical method and figure of merit (FOM)

One-way analysis of variance and the Bonferroni post hoc test were used to examine the differences of FBPA improvements between four MAR techniques. Kendall coefficient of concordance was used to evaluate the consistency between observers of visual ranking analysis.

The total quantitative results that combined subjective and objective evaluation for each MAR technique were defined by the figure of merit. The average and standard deviation of visual ranking scores for each metal implant were used for the calculation of the expected value (), combined with the FBPA and FBPA improvement. The performance of each MAR technique on three metal implants ( 1 to 3) was calculated as one expected value (Eq. (5)). The better MAR technique was desired greater expected value. The FOM is basically revised from the definition of signal to noise ratio (S/N) in the Taguchi’s optimal analysis [26].

The key kernel of FOM is trying to quantify the digitized performance from various factors in the study. The definition of FOM herein is defined to fulfill “larger-the-better” tendency, therefore a high FBPA improvement or visual ranking score is always preferable whereas a low FBPA or standard deviation is preferable. Furthermore, the FOM tries to integrate all the performances of various CT scanners in one rule of thumb, thus, all the implanted materials have to be considered thoroughly to obtain a compiled digitized index for comparison among CT scanners.

Results

Objective analysis

The results of metal volume measurement are shown in Table 2. In the original images, the hip implant and spinal implant in GE’s images measured the smallest volumes (55,596.1 mm and 13,561.9 mm). After using the MAR technique, the smallest volume of two metal implants were all Canon images, and their metal volumes were 54,898.2 mm and 13,340.2 mm. The second lowest measured volume with MAR was in GE’s images (55,139.9 mm and 13,699.7 mm).

Table 2

Metal implant volume results

Image sequence	Hip implant volume (mm^3)	Spinal implant volume (mm^3)
Canon
Original	59274.8	13845.5
SEMAR	54898.2	13340.2
Improvement (%)	7.4	3.6
GE
Original	55596.1	13561.9
Smart-MAR	55139.9	13699.7
Improvement (%)	0.8	-1.0
Siemens
Original	57813.8	14150.9
IMAR	57304.1	13904.4
Improvement (%)	0.9	1.7
Philips
Original	59798.3	15197.7
OMAR	57630.4	14322.6
Improvement (%)	3.6	5.8

With hip implants, the optimal metal volume improvement rate was Canon’s 7.4%, followed by 3.6% for Philips. No obvious improvement was found in the images of GE and Siemens, which achieved rates of 0.8% and 0.9%. With spinal implants, the optimal metal volume improvement rate was 5.8% for Philips, followed by 3.6% of Canon. It is noteworthy that the metal volume of GE showed no improvement (less than 0%).

The results of FBPA are shown in Fig. 6. The FBPA of reference images without metal implants (with and without MAR) both are 0.1%. The FBPA of the head of hip implants found using an MAR technique with Canon, GE, Siemens, and Philips machines were 11.6 4.1%, 6.2 2.1%, 25.8 4.5%, and 24.4 6.2%, respectively; GE was the smallest. The MAR images of the body of hip implants were 9.1 1.6%, 4.1 0.9%, 8.0 2.4%, and 16.7 2.6%, respectively; GE was once again the smallest. In MAR images of spinal implants, the smallest FBPA was 4.9 0.3% for Canon, followed by 5.5 0.5% for GE. In MAR images of dental fillings, the smallest FBPA was 19.0 2.6% for GE, followed by 22.9 6.6% for Siemens.

Figure 6.

Measurement results of fraction of bad pixel area (FBPA) of three metal implants. Hip implant is divided into two parts; head and body.

FBPA improvement rate results of groups of images after using a MAR technique are shown in Table 3. In the head of hip implant images, the optimal improvement rate was 33.7 6.6% for Canon, followed by 32.3 9.0% for GE ( 0.32). In the body of hip implant images, the optimal improvement rate was again Canon with 7.0 6.6%, followed by Philips with 5.4 3.6% ( 0.001); it is noteworthy that GE only achieved a 0.8% improvement rate because the FBPA of its original images were already low at first (4.9 1.7%). In the images of spinal implants, the optimal FBPA improvement was Canon with 14.0 1.0%, followed by 12.6 1.0% for Philips ( 0.001). In MAR groups of dental fillings, the optimal improvement rate was 10.6 5.5% for Canon, followed by 9.1 5.3% for GE ( 0.182).

Table 3

Results of FBPA improvement, visual ranking score, and expected value

MAR	SEMAR		Smart-MAR		p value	IMAR		p value	OMAR		p value
FBPA improvement (%)
Hip implant (Head)	33.7	± 6.6	32.3	± 9.0	0.320	20.0	± 6.2	< 0.001	18.2	± 8.3	< 0.001
Hip implant (Body)	7.0	± 6.6	0.8	± 1.1	< 0.001	2.4	± 6.2	< 0.001	5.4	± 3.6	< 0.001
Spinal implant	14.0	± 1.0	8.9	± 1.1	< 0.001	3.8	± 0.8	< 0.001	12.6	± 1.0	< 0.001
Dental filling	10.6	± 5.5	9.1	± 5.3	0.182	6.1	± 5.0	< 0.001	5.7	± 3.7	< 0.001
Visual ranking score
Hip implant	3.6	± 0.7	3.3	± 0.5	–	2.1	± 0.3	–	0.9	± 0.3	< 0.001
Spinal implant	3.9	± 0.3	3.1	± 0.3	–	1.4	± 0.5	–	0.9	± 0.9	0.001
Dental filling	3.1	± 0.6	3.8	± 0.4	–	1.3	± 0.5	–	0.3	± 0.5	0.002
Expected value	36.6		37.8			5.0			2.3

For FBPA improvement, all value are compared to SEMAR. For Visual ranking score, only one value from Kendall’s coefficient of concordance. FBPA, fraction of bad pixel area.

Subjective analysis and FOM

The results of average ranking scores by the three experienced experts are shown in Table 3. The SEMAR technique of Canon was optimal for hip and spinal implants; and the Smart-MAR technique of GE for dental filling. The OMAR technique of Toshiba achieved the least desirable results in all types of metal implants.

The expected values of four MAR techniques are also shown in Table 3. The greatest value was 37.8 of Smart-MAR and second value was 36.6 of SEMAR. The performance of IMAR and OMAR were only 5.0 and 2.3.

Discussion

Quantitative comparison

Of the four MAR techniques, Smart-MAR and SEMAR are superior to the other two, which is in line with the results of Bolstad et al. [24]. The SEMAR technique has better FBPA improvement for all the metal implants and better visual ranking scores for hip and spinal implants then Smart-MAR, but still Smart-MAR reaches the greatest expected value. The reason for this is Smart-MAR has the smallest FBPA value in all images except for the images of spinal implants. The metal volume and FBPA of GE’s original image attained the optimal performance mainly because of the overall performance of its CT scanner (dual energy and lesser noise). The improvement rate of metal volume was not included in the figure of merit because of the lack of metal volume of dental filling.

The images of metal implants after correction of the Smart-MAR produced new artifacts in the acrylic part of the phantom (Fig. 7). This showed that the Smart-MAR may exhibit errors for materials with a CT number higher than water when reducing streak artifacts produced by the metal. The images of dental filling after using IMAR also produced new artifacts (Fig. 7). With the spinal implant, an obvious artifact remained in the central zone of the two titanium rods for all four types of MAR techniques (Fig. 4).

Figure 7.

Images of artifacts caused by MAR. a. Image of hip implant with artifact caused by Smart-MAR (red arrow). b. Image of spinal implant with artifact caused by Smart-MAR (red arrow). c. Image of dental filling with artifact caused by Smart-MAR (red arrow). d. Image of dental filling with artifact caused by IMAR (red arrow).

Figure 4.

CT images of spinal implants. a. Image without SEMAR; b. with SEMAR. c. Image without Smart-MAR; d. with Smart-MAR. e. Image without IMAR; f. with IMAR. g. Image without OMAR; h. with OMAR.

The performance of the four MAR techniques in improving artifacts produced by the dental filling was less satisfactory than the other two metal implants (Fig. 5). This was because the density of the amalgam alloy in the dental filling ( 13.9 g/cm) was considerably higher than the other metals. This phenomenon is matched to the study of Huang et al. [25].

On the basis of the results from analyzing the three aforementioned metal implants, this study suggests that the FBPA improvement and objective visual ranking were nearly the same. The measurement and improvement of FBPA are similar to the study of Huang et al. [25]. Huang et al. calculated the HU error (bad pixel) map by subtracting the baseline image from the metal image (excluding regions of air and the metal implant). FBPA only calculated bad pixel area within the same image, which is easier to perform and has less experimental errors. This is an effective and objective quantitative method for evaluating the optimal adjustment for various MAR techniques.

Comparison with previous studies

Bolstad et al. compared surgical sheet metal made of three types of materials on leg phantoms using subjective and objective quantitative analysis of four MAR techniques [24]. The results of the subjective quantitative analysis showed that SEMAR had superior performance in two types of metal with lower densities compared with the other three MAR techniques. Smart-MAR achieved the optimal performance in metal with relatively high density, which is highly consistent with the spine implant analysis results in this study. The objective analysis of Bolstad et al. focused on the CT number of fractions of pixels higher than 500 HU in the area surrounding metal implants; the metal volume they found was consistent with that of this study. However, their objective measurement was unable to analyze the effect of metal artifacts on the surrounding soft tissue. The FBPA proposed in this study can evaluate the degree to which a metal artifact influences the water and adjacent soft tissue CT number.

Wagenaar et al. used an objective method to analyze the effect of three techniques (Smart-MAR, OMAR, and IMAR) on a head phantom with a dental filling [23]. The errors of CT number of Smart-MAR and OMAR were extremely close and were higher than IMAR, but the study did not analyze SEMAR. In the dental filling analysis of this study, SEMAR attained the optimal results of subjective and objective analyses, followed by Smart-MAR, OMAR (third place), and IMAR. Our study results are highly similar to those of Wagenaar et al., but we added an analysis of SEMAR.

Anderson et al. used subjective and objective quantitative methods to analyze the effect of OMAR and SEMAR in hip implants [20, 21]. When the soft kernel was forming images, VGA ratings of the two techniques were the same, which is inconsistent with our finding that SEMAR was superior to OMAR. However, the study by Anderson et al. used an artificial phantom with a double-sided hip implant, which is different from the single hip implant in this study. Another difference is that this study simultaneously analyzed the hip implant using four MAR techniques (SEMAR, Smart-MAR, IMAR, and OMAR).

Limitations

This work evaluated four commercial MAR techniques on three common types of metal implants in a clinical setting for the first time. However, there are limitations to the study. Dual-energy technique has been available with MAR for Philips scanners in a recent update, and CT images with dual-energy cause lesser noise [8]. The helical mode with the MAR for Canon scanner has also been made available recently and can provide more insight into comparison.

The scan protocols used in the study were all common conditions used in clinical settings and were not optimally adjusted with the MAR technique for each scanner. Therefore, the performance was not optimal for four vendors. The best MAR conditions for each vendor should be studied in future work.

Conclusion

This study successfully used subjective and objective quantitative methods to evaluate the effect of using four commercial MAR techniques on three types of metal implants (hip implant, spinal implant, and dental filling) in a phantom for the first time in literature. All four MAR techniques effectively reduced metal artifacts, but the effect deteriorated when applied to dental fillings with higher amalgam alloy density. SEMAR of Canon and Smart-MAR of GE achieved optimal subjective and objective analysis results for the three types of metal implants among four MAR techniques.