Factors influencing daily quality assurance measurements of magnetic resonance imaging scanners.

Magnetic resonance imaging is commonly used in hospitals and clinics to aid medical diagnoses. Scanner performance should be assessed regularly, including daily, weekly, and yearly evaluations to ensure high-quality and artifact-free images. Of these assessments, the daily quality assurance monitors the image quality of the scanner using a manufacturer-provided protocol. In this study, we sought to determine the factors that introduced variability in daily quality assurance data. A phantom was scanned using a head coil in two schemes: with varied phantom placement daily, and with a single phantom placement, and evaluated over approximately 1 month. Minor placement and localization changes accounted for approximately 50% of the variability in the signal-to-noise ratios observed in these measures, driven by changes in the measured signal, while the noise remained constant. The changes in the signal-to-noise ratios were small over the 2-month study period.

Background

Magnetic resonance imaging (MRI) is a commonly used imaging modality for medical diagnosis in hospitals and clinics. A recent clinical review reported an average of 82 MRI scans per 1000 people among the most affluent nations, which was, second only to computed tomography at 151 scans [1]. These scans were interpreted by a radiologist and subsequently used to guide healthcare decisions. For the scans to be used for diagnosis, they must be of high quality and free of image artifacts [2]. Therefore, MRI scanners are routinely assessed for proper functioning and image quality. Quality assurance (QA) assessments typically consist of a multi-pronged approach that includes daily, weekly, quarterly, and yearly evaluations.

To assess the image quality, the daily QA of MRI scanners typically involves using a standard phantom scanned in a head coil using a manufacturer-provided imaging protocol. The collected data are used to measure the signal-to-noise ratio (SNR) of the protocol to identify any degradation of system performance that may require service of the scanner. The SNR depends on the acquired signal and background noise, as provided by the following equation: [3]

where μsignal and σnoise are the mean signal within a given region of interest (ROI) and standard deviation, respectively.

Significant changes in the scanner SNR alert the technologist or imaging center that the equipment needs hardware correction. Efforts by vendors and researchers have led to automation of the QA process [4-7]. In addition, standards have been developed nationally and internationally to specify acceptable values of quality measures [8, 9]. All major scanner manufacturers have now implemented automated acquisition and analysis of daily QA measures to ensure unbiased and reliable collection of QA assessments. Often, these routines have a priori information about the phantom size and assume that the phantom is placed at the isocenter of the scanner using provided positioning devices. The SNR metric is strongly affected by issues in the receive chain, such as a bad pre-amplifier that directly impacts the strength of the acquired signal. The acquired signal can also be affected by changes in the performance of the electromagnetic shims, causing changes in the center frequency of the acquired images or gradient performance, impacting the object shape [10]. The noise estimate in μsignal can be impacted by metals on the bore of the scanner introducing radiofrequency (RF) noise into the images or changes in the integrity of the RF cabin, thereby allowing RF noise into the scan room.

The daily QA, which is a single measure of the scanner SNR in a single RF coil, is part of a comprehensive scanner evaluation program that should also include weekly, quarterly, and annual evaluations of the system performance. A typical program would also include weekly evaluations of the American College of Radiology phantom using a standardized imaging protocol to evaluate geometric distortion [10], image intensity uniformity [11], slice thickness [12], image geometry, slice position accuracy, image ghosting, and low-contrast object detectability [13]. Quarterly preventative maintenance is typically performed to ensure that the system meets the manufacturer’s standards. Finally, annual testing ensures proper functioning of the system and all the RF coils.

At the University of Iowa Magnetic Resonance Research Facility, we performed daily QA assessments of the scanners. With the outbreak of coronavirus disease 2019 and the shutdown of human subject research-related activities on the university campus, we had the unique opportunity to study factors that may influence the daily QA measurements and variations they may have on the resulting measurements.

Methods

Daily QA data were acquired on a SIGNA Premier 3.0 T scanner (GE Healthcare, Chicago IL) using a 48-channel head coil with a 17.78-cm-diameter silicon oil phantom. This scanner is located in a room with an HVAC unit dedicated to the scan room with a set point of 70° F and a standard operating range of ±2° F. We recently upgraded the SIGNA Premier from a 750 W scanner. Daily QA data were obtained from Monday to Friday from 3 February to 17 March 2019, resulting in 37 measurements. During this time, the silicon oil phantom was placed in the head coil with a manufacturer-designed holder (Fig. 1C), landmarked, and positioned at the scanner isocenter at the end of the day. A QA scan was then performed as the first scan of the next business day. We refer to the daily placement of the phantom within the scanner as “QA scheme 1.”

Fig. 1

Example of signal (A) and noise (B) images acquired from the spin-echo based quality assurance data acquisition scan. The QA phantom, holder, and placement inside the head coil is shown in C

When human subject research was suspended at the University of Iowa, the phantom was placed in the head coil as described above and positioned at the center of the magnet on 18 March 2019. Daily QA measurements were then obtained over 23 business days without phantom repositioning to ensure proper scanner functioning. No other scans were performed during this time; thus, the table was not moved. We will refer to the assessment of quality assurance without phantom repositioning as “QA scheme 2”.

An additional experiment was conducted once human subject research resumed at the University of Iowa to determine the extent to which the assigned landmark location had on the SNR measurements. During this experiment, the phantom used for the daily QA measurements was placed in the head coil once and different landmark locations (0, ± 5, ±10, ± 20 mm) relative to the target landmark location marked on the coil with a “ + .” A negative distance represented an inferior shift of the landmark location, whereas a positive value represented a superior shift in the landmark location. After assigning the landmark location, the marked location was placed at the isocenter of the scanner, and the QA procedure was performed after waiting for 5 min. Linear regression was performed on the SNR values with respect to the distance from the coil reference location.

Daily QA measurements

Daily QA measurements were obtained from two separate acquisitions. The first acquisition was conducted to measure the signal in the phantom and was acquired with a 2D axial spin-echo sequence using the parameters described in Table 1. The second scan was used to measure the background noise level in the system and was acquired with the RF transmitter disabled, allowing for assessment of the electronic noise measured by the receiver chain.

Table 1

Protocol parameters of the spin-echo sequence used to acquire signal and noise images

Protocol parameters	Signal scan	Noise scan
TR (ms)	750	50
TE (ms)	30	30
Flip angle (°)	90	90
BW (Hz/px)	122.1	122.1
Matrix	256 ×  256	256 ×  256
Slice thickness (mm)	5.0	5.0
FOV (mm)	300 ×  300	300 ×  300

An automated processing algorithm designed by the manufacturer was then used to estimate the SNR of the head coil. This algorithm uses an automated circular ROI placed in the phantom, which assumes that the phantom is positioned at the center of the head coil using the phantom holder. A square which tightly bound the phantom was first estimated from the image. A circular ROI was then defined with a diameter of 85% of the area of the square, and was used to estimate the average signal intensity in the image, and the standard deviation from the noise image collected with the RF transmitter turned off. The SNR value was then estimated as the mean phantom signal divided by the standard deviation measured from the noise image. In addition to the above measurements, the scanner shim currents, center frequency, and transmitter gains were recorded. The coefficient of variation (CV) was computed for these system values for each QA scheme. A hypothesis test was performed using a two-sampled Welsh’s t test with unequal variance to determine the likelihood of the samples having different SNRs. Comparisons were then made to assess variations between the QA phantom being repositioned daily (QA scheme 1) and a constant phantom position within the scanner (QA scheme 2).

Results

All images acquired in this study had no large visible artifacts and were similar to those shown in Fig. 1A. The CV values for both QA schemes were below 0.01%, as shown in Table 2. As expected, the SNR measurements showed greater variability in QA scheme 1 than in QA scheme 2 (Fig. 2A). As seen in Table 2, a lower ratio (QA scheme 2/QA scheme 1) value of < 1.0, indicating a lower CV for QA scheme 2, while a value > 1.0 indicates a lower CV for QA scheme 1. The CV for QA scheme 2, in which the phantom was not repositioned, was approximately half of that observed in QA scheme 1 (p = 0.04), due to increased variation in the measured signal from the phantom in QA scheme 1, with only a small difference in the noise estimates between the schemes. Figure 3 shows the variation in the signal (3A), noise (3B), transmitter gain (3C), and center frequency (3D) measurements over the course of the study. As expected, the center frequency exhibited the smallest variation across all measurements.

Table 2

Coefficient of variation (CV) in QA Schemes 1 and 2 for each measure recorded

	Coefficient of variation (%)
	Signal	Noise	SNR	TG	Center frequency
Scheme 1	0.559	0.518	0.768	0.128	7.68e-6
Scheme 2	0.267	0.499	0.285	0.102	1.23e-6
Ratio	0.476	0.963	0.371	0.794	0.161

*Ratio = (QA scheme 2)/(QA scheme 1)

Fig. 2

A Plot of signal-to-noise ratio (SNR) data across the time of the study. Data before and after the vertical dotted line in represent QA Schemes 1 and 2, respectively. B SNR dependence based on landmark location. Displacement is from the specified landmark location denoted by a “ + ” on the head coil. Landmarks were assigned at 0,  ± 5,  ± 10, and  ± 20 mm from this location. Negative distance was the inferior direction and positive distance was the superior direction

Fig. 3

Plots of signal, noise, transmitter gain (TG), and center frequency acquired the entire duration of the study. Data before and after the dotted line represents QA Scheme 1 and 2, respectively

The SNR measurements were linearly related to the landmark position (Fig. 2B) within the range evaluated (± 20 mm). A linear regression showed an excellent fit to the data, with an R2 of 0.92. In addition, we observed a change of approximately 1.2 SNR units (0.5%) for every millimeter shift in landmark location.

Discussion

We undertook this study to determine the factors that affected and introduced variations in daily QA measurements. It was found that human factors, such as positioning and landmarking of the phantom in the scanner, accounted for approximately half of the daily variation in QA measurements, which in turn are affected by variations in the signal measurements obtained from the phantom. The background noise, which indicates the electronic noise in the receiver chain, was constant and unaffected by these human factors.

The calculated signal variation is consistent with that reported in previous studies. In a QA study, Firbank et al. observed a CV of 1.7% for their SNR measurements based on 15 measurements at 1.0 T [14] obtained on a single day. It was not clear from their study whether the phantom was repositioned in the scanner over the course of the experiment. The CV increased to approximately 2.1% when the measurements were performed over 60 days [14]. The CVs of the SNR measurements obtained in the present study were well below 1.0% for both QA schemes. The time elapsed for our study was approximately the same as that reported by Firbank et al. [14]. Accounting for the difference in field strength between the two studies, QA scheme 1 produced a similar CV as Firbank et al. [14], but it should be noted that they did not consider the differences in the phantoms, such as their T1 relaxation times, which will also significantly impact the SNR measurements obtained. In another study, Peltonen et al. assessed daily QA performances at 1.5 T over the course of a year and measured a CV of 2.45% for SNR measurements obtained using a similar spin-echo sequence [6]. While our assessment of daily QA measurements was conducted over a period of only 2 months, the CV was approximately one-fourth of that observed by Peltonen et al. [6] It should be noted that our measurements were obtained at 3 T, where the signal strength would be expected to be two times that at 1.5 T.

The greater signal variation observed in QA Scheme 1 is likely due to positioning and landmarking variations of the phantom within the head coil. The follow-up experiment evaluating the effect of landmark location on the SNR measurements observed a variation of 0.5% for every millimeter shift in landmark location. During a typical QA scan, two factors contribute to this positioning: (1) the location of the phantom and (2) the landmark location on the head coil. This study only evaluated the second source of variation, because it was easier to quantify. Given that both factors contribute to the SNR variation, it is likely that we evaluated the best case scenario and that typical SNR variations due to positioning are likely twice as large. Among the factors affecting SNR was the estimated transmitter gain by the scanner used to define a reference 90° flip angle during imaging [15]. The transmitter gain variation for QA scheme 1 was 24% greater than that for QA scheme 2. However, the standard deviation of the transmitter gain for QA scheme 1 is less than 1. We then estimated the impact of the variance in the transmitter gain (TG) on the resulting flip angle. A TG difference of one unit would result in an approximately 1% change in the flip angle [16]. Thus, the impact on the resulting flip angle was considered negligible, even though it was significantly greater in QA scheme 1. Additional outside factors, such as temperature, convection, and motion‐induced fluid currents, can also affect the resulting SNR measurements. However, it was unlikely that convection and motion‐induced fluid currents had any impact on the resulting measurements because the phantom was placed at the isocenter at least 8 h before the QA scan, allowing it to equilibrate to the scan room temperature and dissipate any fluid currents. While the scanner room is climate-controlled with its own air handling system to maintain the temperature within a fairly narrow range, the temperature was not assessed during the study.

The drift in the center frequency was not sufficient to affect the SNR for the QA scheme 1 data. The permissible drift rate in the center frequency for any scanner proposed by the American Association of Physicians in Medicine is 0.25 ppm/day, [17] which translates to approximately 32 Hz/day at 3 T. During QA scheme 1, the daily difference in the center frequency ranged from 0 to 21 Hz. Outliers that occurred on day 20 were not included in this range. One possible explanation for the large difference observed at a single measurement obtained during scheme 1 was the poor magnetic field shim that increased the frequency by 353 Hz. Overall, we observed the expected downward drift in the center frequency, which drifted downward by 78 Hz over the course of the study.

The likely source of the variation in the SNR with respect to the landmark position was the layout of the coil elements within the coil. The inferior aspect of the coil is open, while the superior aspect of the coil has elements designed to wrap around the head. One would expect that a shift in landmark position towards the superior aspect of the coil, where a greater density of coil elements exists, would increase the SNR, which is what was observed. The influence of landmark position was only evaluated for a limited range (± 20 mm), which is considered as within the standard positioning error. The linear effect observed within this range may become non-linear as a larger positioning variation occurred; however, this was not studied.

Conclusion

Daily QA measurements are a good way to identify issues with the system hardware and calibration. Since the CV of variation was twice as large when the phantom was placed and landmarked daily as compared to no position change, approximately half of the variance typically observed in daily QA measurements was likely due to human setup of the phantom, while the other half was true system noise. Given that the human step of the QA process can account for a significant portion of the variation, it may be worthwhile to repeat a QA scan in the case where a system error is reported due to large variations in the measurements to verify that the error is likely due to degradation in hardware performance and not human error.