Evaluation of Imaging Parameters of Ultrasound Scanners: Baseline for Future Testing.

BACKGROUND: Regular quality control is required in Poland only for those methods of medical imaging which involve the use of ionizing radiation but not for ultrasonography. It is known that the quality of ultrasound images may be affected by the wearing down or malfunctioning of equipment. MATERIAL/
METHODS: An evaluation of image quality was carried out for 22 ultrasound scanners equipped with 46 transducers. The CIRS Phantom model 040GSE was used. A set of tests was established which could be carried out with the phantom, including: depth of penetration, dead zone, distance measurement accuracy, resolution, uniformity, and visibility of structures.
RESULTS: While the dead zone was 0 mm for 89% of transducers, it was 3 mm for the oldest transducer. The distances measured agreed with the actual distances by 1 mm or less in most cases, with the largest difference of 2.6 mm. The resolution in the axial direction for linear transducers did not exceed 1 mm, but it reached even 5 mm for some of the convex and sector transducers, especially at higher depths and in the lateral direction. For 29% of transducers, some distortions of anechoic structures were observed. Artifacts were detected for several transducers.
CONCLUSIONS: The results will serve as a baseline for future testing. Several cases of suboptimal image quality were identified along with differences in performance between similar transducers. The results could be used to decide on the applicability of a given scanner or transducer for a particular kind of examination.

Background

Ultrasonography is currently one of the most widely used methods of medical imaging of soft tissues, the circulatory system, and the nervous system, with new applications being constantly developed [1-6]. One of the main advantages of ultrasonography is the lack of ionizing radiation and the risks associated with it. With spatial resolution reaching 0.5 mm, ultrasonography can provide a good quality of images. However, as shown in many publications [7-16], the quality should be systematically controlled. For example, Mårtensson et al. have shown that 40% of 676 ultrasound transducers used in 32 hospitals in southern Sweden were defective [7]. In the results published by Sipilä et al. [8], 25% of the transducers evaluated had physical flaws, and for 15% of the transducers image quality was defective.

Regular quality control is required by law in Poland for those methods of medical imaging which involve the use of ionizing radiation [17,18]. Despite a rising number of ultrasound scanners and examinations, systematic quality control of ultrasound equipment is not required.

In the present study, imaging parameters of ultrasound equipment were evaluated for ultrasound scanners and transducers used in the Maria Skłodowska-Curie Memorial Cancer Center and Institute of Oncology in Warsaw, Poland (COI). Test methods and acceptance criteria were based on APPM and IPEM reports and on the Gammex/RMI manual [9,10,16].

Material and Methods

Ultrasonography equipment

Evaluation of image quality was carried out between November 2015 and March 2016 for 22 ultrasound scanners equipped with 46 transducers (22 linear, 19 convex, 5 sector transducers), comprising 92% of all the scanners in the COI. All tests were performed for B-mode presentation images. A list of all the tested ultrasound scanners is presented in Table 1.

Table 1

List of evaluated ultrasound scanners and transducers.

Manufacturer	Type	Year of manufacture
Siemens	Acuson S2000	2013
	ACUSON Aspen	2002
Philips	HDI5000SonoCT	2005
	IU22	2010
	Sparq	2013
	IU22	2007
	HD15	2014
	HDI 4000	2003
	HD 15 Pure Wave	2014
Hitachi	HI Vision Preirus	2014
	Prosound Alpha 7	2011
	HI Vision Preirus	2014
	HI Vision Preirus	2015
	HI Vision 900	2007
	HI Vision 900	2007
	EUB 7500	2008
ATL	Ultra Mark 4 Plus	1995
	HDI 3000	1997
GE Healthcare	Voluson 730 EXP	2002
OLYMPUS	EU-ME2	2015
Supersonic Imagine	AIXPLORE	2012
BK Medical	Ultra View 800 Pro Focus	2012

Ultrasound frequency range

All transducers were broad-band, with frequencies in the range of 4–18 MHz for linear transducers, 1–6 MHz for convex transducers, and 1-5 MHz for sector transducers. For some ultrasound scanners, it was possible to choose one of three frequency ranges, namely „Res”, „Gen”, and „Pen”. „Res” stands for „resolution” and refers to the highest frequency range; this mode is meant for imaging superficial tissues (optimization of resolution is done at the cost of the range). „Pen” stands for „penetration” and refers to the lowest frequencies; it is meant for deeply located tissues or patients with obesity. „Gen”, which is usually the default setting, stands for „general” and designates an intermediate range of frequencies [19].

Equipment

A Multi-Purpose, Multi-Tissue Ultrasound Phantom model 040GSE manufactured by CIRS (Computerized Imaging Reference Systems Inc.) [20] was used to test the ultrasound scanners. The phantom is filled with a gel material called Zerdine® (a trade name). The speed of ultrasound in the material is 1540±10 m/s. The phantom is divided into two parts with different attenuation coefficients to mimic different tissues; namely, 0.5 dB/(cm·MHz) and 0.7 dB/(cm·MHz). The phantom contains several sets of objects for the evaluation of the dead zone, the axial and lateral resolution, the visibility of structures with various contrast (grayscale), and the visibility of anechoic structures. Additionally, several small targets are placed at known distances in horizontal and vertical directions for verification of distance measurement accuracy. A scheme of the phantom is presented in Figure 1.

Figure 1

Scheme of the phantom used for evaluation of ultrasound scanners (source: [20]). During measurements, the transducer is in contact with the top surface of the phantom.

Conditions for the assessment of image quality

Results of the evaluation of image quality strongly depend on scanner settings, such as the anatomical program, gain, harmonic mode etc. [21]. The following conditions were met during the measurements:

The same anatomical program (preset) as used clinically (based on user information);

TCG (time gain compensation) optimized for good uniformity of the image;

Focal zone on the evaluated structure;

Depth optimized for the best visibility of the evaluated structure.

Advanced functions, such as harmonic mode, Sono CT, XRes and similar, were turned off. All the settings were noted down for reference during future tests. Displayed values of MI and TI or TIB indexes were also recorded for comparison in future tests.

Scope of tests and criteria

For each scanner and each transducer, the following parameters were evaluated visually:

Depth of penetration;

Dead zone;

Distance measurement accuracy in horizontal and vertical directions;

Axial and lateral resolution;

Uniformity;

Visibility of anechoic structures;

Visibility of structures with various contrast (grayscale).

The scope of the tests was based on APPM and IPEM reports and on the Gammex/RMI manual [9,10,16]. We chose such tests that could be carried out with the CIRS 040GSE phantom in the same way for as many scanners and transducers as possible. Apart from the distance measurement accuracy, all the tests were based purely on visual assessment of images. Visual evaluation of image quality was carried out by the same observers for all ultrasound scanners in order to eliminate inter-observer variability. All of the tests were done separately for two areas with different attenuation coefficients (0.5 dB/(cm·MHz) and 0.7 dB/(cm·MHz)).

Depth of penetration was defined as the maximum depth at which the background texture of the image was still distinguishable from electronic noise. The depth of penetration depends on ultrasound frequency, output power of the transducer, gain and TGC settings, focal depth, the display format (number of lines), and electronic noise. It can change in such cases as output intensity alternations or due to a physical damage of the transducer or cable.

The dead zone is the distance between the front surface of the transducer and the echo of the first visible “near field” structure (Figure 1). The existence of the dead zone is caused, among other reasons, by the finite length of each emitted pulse. Echoes from low depths may be not detected if they coincide in time with the excitation pulse. If the frequency of pulses is higher, the pulses are shorter and the dead zone is smaller. The dead zone can get larger if the pulses are longer, which can be caused, for example, by a crack in the piezoelectric crystal or a malfunction of electronics.

The accuracy of distance measurement was evaluated by comparing known distances between the phantom structures with the distances measured with the software of the ultrasound scanners. The distances were measured between the structures identified in Figure 1 as the “horizontal distance” and the “vertical distance” (in two perpendicular directions). Ultrasound scanners measure the time between pulse emission and echo registration, and convert the measured time into a distance with the assumption that the speed of ultrasound in tissues is constant. Distance errors can be caused by a failure of the internal clock of the scanner. An error of the horizontal distance can be also caused by a failure of the scan mechanism. This is possible especially in mechanical transducers in which the wear of the motor can affect the timing of the acquisition of each line of the B-mode presentation image. Obviously, the distances measured will be erroneous if the speed of ultrasound in the phantom material is different than the speed assumed in the tissue. It is worth noting that the speed of ultrasound in phantoms usually depends on temperature (a change of approximately 1.5 m/s per 1°C).

The resolution in axial and lateral directions was evaluated visually by identifying the nearest objects which could be clearly identified as separate and by recording the distance between the objects. The resolution in the axial direction (along the axis of the ultrasound beam) is limited by the wavelength and by the length of the ultrasound pulse. The resolution in the lateral direction (along the line perpendicular to the ultrasound beam) is approximately equal to the width of the ultrasound beam, and changes when the depth, focal zone, gain, or sensitivity settings are changed. When possible, resolution in both directions was evaluated at three different depths (3.0 cm, 6.5 cm and 10.0 cm). For systems with multiple focal zones or with dynamic focus, the resolution in the lateral direction is uniform over a larger range of depths. The resolution in the lateral direction can be affected by malfunction of elements of transducers or of the beam-forming system.

Image uniformity, which is an essential parameter of image quality, was evaluated visually by the observation of uniform regions of the phantom. The presence of non-uniformities and artifacts may mask real structures or may lead to diagnosis of non-existent pathologies.

Visibility of anechoic structures depends on spatial resolution, contrast resolution, uniformity, and the presence of electronic noise and side lobes of the beam. Three parameters describing visibility of anechoic structures were used: their shape (it should be round), the edges (sharpness), and the interior (it should be anechoic). Each of those three parameters was scored on a three-step scale (details in Table 2). Overall, a total score of 3 referred to the best image, and 9 was the worst possible score. The visibility of structures with various contrast (–9 dB, –6 dB, –3 dB, +3 dB, +6 dB, >15 dB) was scored with one point for each visible structure and half a point for each partly visible structure.

Table 2

Score scale for visibility of anechoic structures.

Score	Shape	Edge	Interior
1	Round	Sharp	Black (anechoic)
2	Elliptical	Blurred	Single bright pixels
3	>20% difference between width and height	Hard to define	Filled with bright pixels

The results were compared with the criteria outlined in Table 3, based on APPM and IPEM reports and on the Gammex/RMI manual [9,10,16]. Two levels of criteria are defined. If the results exceed action levels, a corrective action should be taken to ensure that the results will not exceed defect levels. For some tests, including the depth of penetration and the visibility of structures, the results are compared with baseline values (results obtained during the first tests, usually after the installation of the scanner).

Table 3

Criteria for evaluation of test results.

Test	Suggested action level	Suggested defect level
Depth of penetration	Change from baseline > 0,6 cm	Change from baseline >1.0 cm
Dead zone	7 mm for f <3 MHz 5 mm for 3 MHz< f <7 MHz 3 mm for f >7 MHz	10 mm for f <3 MHz 7 mm for 3 MHz< f <7 MHz 4 mm for f >7 MHz
Distance accuracy - vertical	>1.5 mm or 1.5%	>2 mm or 2%
Distance accuracy - horizontal	>2% or 2 mm	>3% or 3 mm
Axial resolution	Change from baseline	Change from baseline
Lateral resolution	Change from baseline	Change from baseline
Uniformity	Change from baseline	Change from baseline
Visibility of anechoic structures	Change from baseline	Change from baseline
Visibility of structures with various contrast (grayscale)	Change from baseline	Change from baseline

Results

Table 4 and Figure 2 present the depth of penetration for the transducers evaluated. As there were no baseline values, it was not possible to decide on the acceptance of the results. During the analysis of the results, it was noted that for one convex transducer the results were obviously different than for the rest of them. The depth of penetration for that particular transducer was 5.4 cm for a 0.5 dB/(cm·MHz) attenuation, which would be a rather typical result for a linear transducer.

Table 4

Results of the depth of penetration test.

		Attenuation 0.5 dB/(cm·MHz)			Attenuation 0.7 dB/(cm·MHz)
		Minimum [cm]	Maximum [cm]	Median [cm]	Minimum [cm]	Maximum [cm]	Median [cm]
Transducer type	Linear	3.0	8.0	5.0	3.1	5.5	4.0
	Convex	5.4	17.0	12.2	5.5	12.0	9.6
	Sector	12.3	16.0	13.0	9.0	13.0	10.6

Figure 2

Results of the depth of penetration test.

The dead zone test results are presented in Figure 3. The results were acceptable, none of the values exceeded the levels presented in Table 3. For 89% of the transducers controlled, the dead zone was 0 mm. The largest and worst result (3 mm) was obtained for one sector transducer. Interestingly, it was the oldest one, in use since 1995.

Figure 3

Results of the dead zone test (zero is the best result).

Table 5 and Figures 4 and 5 present results of the distance accuracy test. None of the results exceeded the levels presented in Table 3. Maximum errors of the measured distance were observed for one of the convex transducers and were 2 mm in the vertical direction and 2.6 mm in the horizontal direction.

Table 5

Results of the test of distance accuracy in two directions.

		Maximum difference between nominal and measure distance [mm]
		Attenuation 0.5 dB/(cm·MHz)	Attenuation 0.7 dB/(cm·MHz)
Horizontal direction
Transducer type	Linear	1.7	1.1
	Convex	2.6	2.0
	Sector	1.6	1.5
Vertical Direction
Transducer type	Linear	1.2	1.7
	Convex	2.0	2.0
	Sector	1.0	1.4

Figure 4

Difference between nominal and measured distance in the horizontal direction.

Figure 5

Difference between nominal and measured distance in the vertical direction.

Table 6 presents the results of the axial resolution test. The resolution for linear transducers was at least 1 mm. It was worse for convex transducers, especially at higher depths, but did not exceed 5 mm.

Table 6

Resolution in the axial direction.

		Resolution in axial direction [mm]
		Attenuation 0.5 dB/(cm·MHz)		Attenuation 0.7 dB/(cm·MHz)
		Minimum	Maximum	Minimum	Maximum
Location of resolution pattern (depth): 3.0 cm
Transducer type	Linear	0.25	1.0	0.25	1.0
	Convex	1.0	2.0	0.25	1.0
	Sector	0.5	4.0	0.5	2.0
Location of resolution pattern (depth): 6.5 cm
Transducer type	Linear	0.5	1.0	1.0	x
	Convex	0.5	4.0	1.0	2.0
	Sector	0.5	4.0	1.0	4.0
Location of resolution pattern (depth): 10.5 cm
Transducer type	Linear	x	x	x	x
	Convex	1.0	4.0	1.0	5.0
	Sector	2.0	4.0	2.0	x

Table 7 presents the results of the lateral resolution test. The resolution did not exceed 5 mm, and the best results were obtained for linear transducers (between 0.5 mm and 3 mm). Similar to the axial direction, the results were worse at higher depths. The results of the resolution test will serve as a baseline for future testing.

Table 7

Resolution in the lateral direction.

		Resolution in lateral direction [mm]
		Attenuation 0.5 dB/(cm·MHz)		Attenuation 0.7 dB/(cm·MHz)
		Minimum	Maximum	Minimum	Maximum
Location of resolution pattern (depth): 3.0 cm
Transducer type	Linear	0.5	3.0	0.5	3.0
	Convex	0.5	4.0	2.0	3.0
	Sector	1.0	4.0	2.0	3.0
Location of resolution pattern (depth): 6.5 cm
Transducer type	Linear	1.0	2.0	2.0	x
	Convex	1.0	4.0	2.0	4.0
	Sector	2.0	4.0	3.0	4.0
Location of resolution pattern (depth): 10.5 cm
Transducer type	Linear	x	x	x	x
	Convex	2.0	5.0	2.0	5.0
	Sector	4.0	5.0	4.0	5.0

As a result of the uniformity test, line artifacts were detected for three linear and one convex transducer. The images were very noisy for three linear and one convex transducer. For two linear transducers, it was not possible to regulate the gain.

The visibility of anechoic structures is presented in Figure 6. For 71% of transducers, there were no distortions of shape, edges, or of the interior of structures, and the total score was 3. However, for the rest of the transducers, suboptimal image quality was observed, and the score was between 4 and 8 points. The largest distortion of anechoic structures was observed for one convex transducer (8 points).

Figure 6

Visibility of anechoic structures. A score of 3 points means no distortions, a higher score represents suboptimal image quality.

Figure 7 presents the number of distinguishable structures with different contrasts (grayscale). For 89% of transducers, 5 or more structures were visible. For four transducers, only 4 structures were visible, and for one linear transducer only 2 structures were visible in the higher attenuation zone.

Figure 7

Number of visible structures with various contrasts (largest result is best).

Discussion

The AAPM report on quality control procedures in ultrasonography [9] and other similar documents outline the methods for testing of quality. For some tests, such as distance measurement, the documents give clear acceptance criteria. It is then possible to evaluate the results, even if the test is carried out for the first time. For some tests, such as the depth of visualization, absolute criteria are not available. The results of initial tests become a baseline against which the results of subsequent tests are compared. It is easy to say if the results are constant but not if they are optimal. At the same time, the depth of visualization is very important clinically. Serious differences were observed in the present study between the results for similar transducers. The depth of visualization for linear transducers ranged from 3 cm to 8 cm. It seems that not all of these transducers can be used in examinations of the same anatomical regions, despite that they are all linear. The same applies to convex transducers, with the depth of penetration ranging from 5.4 cm to 17 cm.

The tests described in the present study may be used to evaluate imaging parameters of ultrasound scanners. If the tests are repeated periodically, a decrease in parameters may be detected due to the wearing down or damage of the equipment. The test results could be used to decide on the applicability of the scanner and transducer for a given kind of examination. Polish standards of ultrasound examinations [1] outline technical requirements for equipment used in various types of examinations. For instance, examinations of the thyroid should be performed with a linear transducer which should be broad-band (5–10 MHz), have a minimum of 128 channels, etc. The standards do not specify direct requirements for the depth of visualization or the resolution. The decision on the applicability of the transducer has then to be based on the knowledge and experience of physicians and ultrasonographers.

A few other transducers were also tested, e.g. endoscopic ones. However, the phantom was not designed for such transducers, and usually it was not possible to carry out a full set of tests. Therefore, only the results for linear, convex, and sector transducers are presented.

The tests described herein were based on visual assessment of the images. A maximum of care has been taken to perform the tests for all scanners in a reproducible manner. Nevertheless, objectivity and reproducibility of the results could have been be improved, if numerical methods of image analysis had been used. It is generally possible to evaluate several parameters in an objective way [14,22,23], but the possibility of export of captured images is required. Unfortunately, for most of the ultrasound scanners evaluated in the present study, DICOM export was not possible, and the possibility of export in other formats was varied.

Conclusions

Tests of imaging parameters of ultrasound scanners have been carried out for the first time in the Maria Skłodowska-Curie Memorial Cancer Center and Institute of Oncology. The results will serve as baseline for future testing. Several cases of suboptimal image quality were identified along with differences in performances between similar transducers. The results could be used to decide on the applicability of a given scanner and transducer for a particular kind of examination.