Technical and radiological image quality comparison of different liquid crystal displays for radiology.

BACKGROUND: To inform cost-effective decisions in purchasing new medical liquid crystal displays, we compared the image quality in displays made by three manufacturers.
METHODS: We recruited 19 radiologists and residents to compare the image quality of four liquid crystal displays, including 3-megapixel Barco(®), Eizo(®), and NEC(®) displays and a 6-megapixel Barco display. The evaluators were blinded to the manufacturers' names. Technical assessments were based on acceptance criteria and test patterns proposed by the American Association of Physicists in Medicine. Radiological assessments were performed on images from the American Association of Physicists in Medicine Task Group 18. They included X-ray images of the thorax, knee, and breast, a computed tomographic image of the thorax, and a magnetic resonance image of the brain. Image quality was scored on an analog scale (range 0-10). Statistical analysis was performed with repeated-measures analysis of variance.
RESULTS: The Barco 3-megapixel display passed all acceptance criteria. The Eizo and NEC displays passed the acceptance criteria, except for the darkest pixel value in the grayscale display function. The Barco 6-megapixel display failed criteria for the maximum luminance response and the veiling glare. Mean radiological assessment scores were 7.8±1.1 (Barco 3-megapixel), 7.8±1.2 (Eizo), 8.1±1.0 (NEC), and 8.1±1.0 (Barco 6-megapixel). No significant differences were found between displays.
CONCLUSION: According to the tested criteria, all the displays had comparable image quality; however, there was a three-fold difference in price between the most and least expensive displays.

Introduction

Due to the fact that the diagnostic 3-megapixel (MP) liquid crystal displays (LCDs) currently used in our radiology department have degraded in quality, they are scheduled to be replaced. The intention is to replace them with either a pair of 3-MP LCDs or a 6-MP single LCD. Also, due to ongoing developments in radiology, color images are becoming more important;1 therefore, we opted to upgrade to color LCDs.

Rapid developments in the field of LCDs have led to more stable, consistent image quality. This has resulted in a broad choice of medical grade monitors with similar technical specifications, but widely variable prices. For example, in the present study, we found a three-fold difference between the retail prices of the most and least expensive LCDs.

Several studies have compared imaging monitors with different specifications, like cathode ray tubes (CRTs) versus LCDs,2–5 diagnostic versus office grade or clinical review grade monitors,6,7 and monochrome versus color monitors.6,8–10 Also, comparisons have been made between monitors that varied in number of pixels and pixel size.11–16 Often, the monitors compared have differed in more than one variable (eg, monochrome CRT versus color LCD).

A number of studies have found that, under the conditions considered, medical grade LCDs were comparable with CRT monitors and hard-copy radiograph films for diagnostic readings in radiology.2,3,11,12,14,17 However, the use of office-grade monitors for diagnosis has been discouraged, despite some studies that demonstrated acceptable quality.6,7 In most studies, no significant differences were found between color and monochrome monitors.8,9,10,18 Finally, the number of pixels and pixel size should match the size of the image to minimize the time spent on zooming and magnification.1,15,16 Overall, no diagnostic differences have been found between monitors with different specifications; thus, it is not clear whether equally calibrated, diagnostic LCDs made by different manufacturers vary in image quality. To our knowledge, no studies have compared equally calibrated 3-MP or 6-MP color LCDs made by different manufacturers, based on the technical and clinical guidelines of the American Association of Physicists in Medicine (AAPM).19

Due to the current global economic situation, we were motivated to perform a study on differently priced LCDs. This study aimed to perform a technical and radiological comparison of 3-MP and 6-MP color LCDs made by three major manufacturers, and establish whether they differed significantly in image quality. This information provided the basis for cost-effective decisions in purchasing new diagnostic LCDs.

Materials and methods

This prospective study was conducted in June and July, 2011. We investigated four different models of color LCDs made by three different manufacturers. We compared an NEC® MD213MC 3-MP LCD (NEC, München, Germany), an Eizo® RX320 3-MP LCD (Eizo Nanao Corporation, Hakusan, Japan), a Barco® MDNC-3121 3-MP LCD (Barco, Kortrijk, Belgium), and a Barco MDCC-6130 6-MP LCD. The NEC, Eizo, and Barco 3-MP LCDs had a diagonal dimension of 21 inches and a resolution of 1,536×2,048 pixels, with a pixel size of 0.21 mm. The Barco 6-MP LCD had a diagonal dimension of 30 inches and a resolution of 3,280×2,048 pixels with a pixel size of 0.20 mm. Both types of Barco LCDs had an extra protective glass shield. The manufacturers were asked to supply optimally calibrated LCDs. All LCDs were calibrated according to the Digital Imaging and Communications in Medicine grayscale standard display function to obtain optimal black and white images at a brightness of 400 cd/m2. In a darkened, film-reading room, the displays were placed next to each other. Ambient light was 0.5 lux for technical measurements and 7.4 lux for radiological assessments. The names of the manufacturers were shielded to minimize bias. The 3-MP LCDs were set up in pairs in portrait mode, as commonly used in radiology. The 6-MP single LCD was assessed as if it were two 3-MP displays.

Technical assessment

Technical assessment of LCD performance was performed according to standard guidelines provided by the AAPM.19 Digital test patterns from the AAPM Task Group 18 (TG-18) were evaluated with a calibrated spot luminance meter (Konica Minolta type LS-100, Osaka, Japan) and a calibrated colorimeter (Philips type PM 5639, Best, The Netherlands). The following parameters were measured: geometric distortion, reflection, luminance response, luminance dependencies, resolution, noise, veiling glare, and chromaticity (Table 1).

Table 1

Short descriptions of technical parameters assessed for each liquid crystal display

Parameter	Section in reference19	Test pattern TG-18	Short description
Geometric distortion	III.A	QC, LPH, LPV	Visual evaluation of whether lines appear straight and parallel and whether squares appear square
Reflection	III.B	AD	Visual evaluation of low-luminance, low-contrast patterns in near total darkness and in ambient lighting
Luminance response	III.C	LN12	Quantitative measurement of luminance at 18 grayscales to determine conformity with grayscale display function
Luminance: nonuniformity	III.D	UN-L10, UN-L80	Quantitative measurement of luminance at four corners and in the center of the display
Luminance: angular dependence	III.D	LN12-01, LN12-18	Quantitative determination of view angle at which 70% luminance of frontal position is maintained
Resolution	III.E	QC	Visual score of CX patterns in the center and four corners of the display
Noise	III.F	AFC	Visual evaluation of number of visible low-contrast squares in the center and all corners of the display
Veiling glare	III.G	GV, GVN	Visual evaluation of number of visible globes to determine light spread within the display
		GQ	Quantitative measurement of glare
Chromaticity	III.H	UNL80	Quantitative measurement of color of pattern in the center and four corners

Note: Section numbers and test patterns are as designated by the American Association of Physicists in Medicine in an Executive Summary report.19 Copyright © 2005. American Association of Physicists in Medicine. Adapted from Samei E, Badano A, Chakraborty D, et al., Assessment of display performance for medical imaging systems: executive summary of AAPM TG18 report. Med Phys. 2005;32(4):1205–1225.19

Abbreviation: TG-18, Task Group 18.

Radiological assessment

For radiological assessments, 12 radiologists (7–42 years of experience) and seven residents (3 months to 3 years of experience) independently performed a side-by-side comparison of the four different types of LCDs. The observers were blinded to the brand of the LCD because we shielded the names of the manufacturers. The order of evaluating the different LCDs was randomized. The AAPM TG-18 anatomical test patterns included a chest X-ray image and a knee X-ray image.19 For a more complete evaluation, we included two AAPM TG-18 mammography images, although these were not commonly examined on 3-MP LCDs. Furthermore, we included anonymized thorax computed tomographic and brain magnetic resonance images from the hospital database. For all images, the evaluators responded to AAPM questions concerning the difficulty in interpretation, overall contrast, and overall sharpness of the images (Table 2). The AAPM TG-18 images also included questions about specific anatomical and structural details, according to the AAPM TG-18 criteria.19 All images were scored on a visual analog scale (range 0–10).20 The visual analog scale consisted of a horizontal line, with anchors at 0 and 10. The evaluators marked their scores on the line.

Table 2

American Association of Physicists in Medicine Task Group-18 criteria used for radiological assessments

Test pattern	Evaluation criteria
TG-18-CH (chest X-ray)	Difficulty of interpretation
	Overall contrast
	Overall sharpness
	Symmetrical reproduction of the thorax, as shown by the central position of the process between the medial ends of the clavicles
	Medial border of the scapulae
	Reproduction of the whole rib cage above the diaphragm
	Visually sharp reproduction of the vascular pattern in the lungs
	Sharp reproduction of the trachea and proximal bronchi
	Sharp reproduction of the borders of the heart and the aorta
	Sharp reproduction of the diaphragm
	Visibility of the retrocardial lung and the mediastinum
	Visibility of the subdiaphragmatic features
	Visibility of the spine through the heart shadow
	Visibility of small details in the whole lung, including retrocardial areas
	Visibility of linear and reticular details of the lung periphery
TG-18-KN (knee X-ray)	Difficulty of interpretation
	Overall contrast
	Overall sharpness
	Reproduction of trabecular detail
	Reproduction of bony and soft tissues
TG-18-MM1 and TG-18-MM2 (mammograms)	Difficulty of interpretation
	Overall contrast and brightness
	Overall sharpness (no blur)
	Sharp appearance of cooper’s ligaments
	Clip structure and the presence of the apical gap (TG18-MM1 only)
	Appearance and visibility of subtle microcalcifications (TG18-MM1 only)
	Visibility of structures at the margins of the breast (TG18-MM1 only)
Chest CT	Difficulty of interpretation
	Overall contrast
	Overall sharpness
Brain MR	Difficulty of interpretation
	Overall contrast
	Overall sharpness

Notes: Each question was scored on a visual analog scale of 0–10 for each liquid crystal display. Criteria are as designated by the American Association of Physicists in Medicine in an Executive Summary report.19 Copyright © 2005. American Association of Physicists in Medicine. Adapted from Samei E, Badano A, Chakraborty D, et al., Assessment of display performance for medical imaging systems: executive summary of AAPM TG18 report. Med Phys. 2005;32(4):1205–1225.19

Abbreviations: CT, computed tomography; MR, magnetic resonance; TG-18, Task Group 18.

Statistical analysis

Cronbach’s alpha was used to determine the internal consistency (validation) of the questions used to evaluate the images. LCD quality was compared with a global score of image quality for each LCD; the global score was the average score for all questions on image quality, based on the mean scores (from 19 evaluators) for each question, and each question was weighted identically.

Because the questions might not be equally important, we also performed statistical testing on individual questions to compare individual qualities of the LCDs. Some images were associated with additional questions; therefore, we examined a total of 18 scores. All five images were evaluated for difficulty in interpretation, overall contrast, and overall sharpness (five images × three scores =15 scores). Only three images were evaluated with the detailed anatomical questions. The scores for all the anatomical questions were averaged before performing statistical analyses; this resulted in three anatomical scores (three images × one anatomical score). Thus, we analyzed 18 scores (15+3 individual scores), and determined differences among LCDs with the repeated-measures analysis of variance method. We determined the sphericity of the data with Mauchly’s test. When no sphericity was found, the Huynh-Feldt statistic was reported. After applying the Bonferroni adjustment, P-values ≤0.0028 (0.05/18) were considered to be statistically significant. All statistical analyses were performed with Statistical Package for the Social Sciences for Windows version 18 (PASW Statistics, Armonk, NY, USA).

Results

Table 3 shows the results of the technical evaluations. The NEC and Eizo LCDs passed the AAPM acceptance criteria, except for the darkest pixel value in the grayscale standard display function. The Barco 3-MP LCD passed all AAPM acceptance criteria, but the Barco 6-MP LCD only passed the tests for geometric distortion, reflection, resolution, noise, and chromaticity.

Table 3

Technical assessments of the different LCD models

	Description	Criterion	NEC®	Eizo®	Barco® 3-MP	Barco® 6-MP
Geometric distortion			Absent	Absent	Absent	Absent
Reflection	Number of visible patterns in total darkness		40	34	34	34
	Number of visible patterns in ambient lighting		39	27	6	34
Luminance response	Maximum, cd/m2	≥170	410.9	394.0	399.7	328.1
	Deviation from 400 cd/m2 (calibrated maximum luminance), %	≤5	2.7	1.5	0.1	18.0
	Deviation from GSDF over 18 grayscales, % (mean±SD)		3.7±2.8	2.4±2.5	2.1±1.9	3.2±2.2
	Maximum deviation from GSDF, %*	≤10	11.9	11.1	7.3	8.3
Luminance: nonuniformity	Nonuniformity for UNL10 (low luminance pattern), %	≤30	0.4	0.4	0.2	0.2
	Nonuniformity for UNL80 (high luminance pattern), %	≤30	8.9	8.4	3.3	10.5
Luminance: angle dependencies	View angle for LN12-01 (low luminance pattern), degrees		35	35	40	55
	View angle for LN12-18 (high luminance pattern), degrees		25	25	25	45
Resolution	Similarity between CX patterns**	≤4	1	1	1	1
Noise	Number of visible squares	≥15	15	16	16	16
Veiling glare	Number of visible globes	≥3	5	5	4	4
	Glare ratio	>400	1,711	1,406	485	364
Chromaticity	Difference in color coordinates	≤0.01	0.0017	0.0042	0.0023	0.0024

Notes:

For all four displays, this was measured at the lowest gray level (LN12-01 pattern);

lower scores indicate better resolution.

Abbreviations: GSDF, grayscale standard display function; LCD, liquid crystal display; MP, megapixels; SD, standard deviation.

For all images evaluated, Cronbach’s alpha indicated that the questions used for the assessment had high internal consistency (Table 4).

Table 4

Reliability of image assessments for predicting liquid crystal display quality, determined with Cronbach’s alpha

	Cronbach’s alpha*
Chest X-ray	0.990
Knee X-ray	0.989
Mammogram	0.963
Chest CT	0.974
Brain MR	0.975

Note:

Alpha values >0.70 indicate acceptable reliability.

Abbreviations: CT, computed tomography; MR, magnetic resonance.

Overall scores of 8.1±1.0 (NEC), 7.8±1.2 (Eizo), 7.8±1.1 (Barco 3 MP), and 8.1±1.0 (Barco 6 MP) were obtained by averaging the scores of all questions. Figure 1 shows the scores for individual questions for each image evaluated on each type of LCD. The corresponding P-values are shown in Table 5. Although the NEC LCD and the Barco 6-MP LCD showed a tendency to higher scores than the other LCDs on most questions, there were no statistically significant differences found between the four different LCDs. Also, scores for the five different test patterns were similar.

Figure 1

Radiological assessments for each type of LCD, according to AAPM TG-18 criteria. Symbols indicate different LCD types; scores are represented as the mean and standard deviation on a scale from 0–10 (n=19 evaluators).

Abbreviations: AAPM, American Association of Physicists in Medicine; TG-18, Task Group 18; LCD, liquid crystal display; MP, megapixels; sd, standard deviation; CT, computed tomography; MR, magnetic resonance.

Table 5

P-values for comparisons of radiological assessments made with different liquid crystal displays

	Degree of difficulty	Overall contrast	Overall sharpness	Anatomy
Chest X-ray	0.109*	0.236*	0.008**	0.067*
Knee X-ray	0.346*	0.140*	0.088*	0.095*
Mammography	0.023**	0.031*	0.059*	0.072*
Chest CT	0.416*	0.637**	0.089*
Brain MR	0.153*	0.166*	0.024**

Notes: After Bonferroni adjustments for multiple comparisons, P≤0.0028 was considered statistically significant.

P-value according to the Huynh-Feldt statistic

P-value according to Mauchly’s test (spherical data).

Abbreviations: CT, computed tomography; MR, magnetic resonance.

On average, the radiologists’ scores were higher than the residents’ scores. The radiologists reported overall scores of 8.5±0.9 (NEC), 8.2±1.1 (Eizo), 7.9±1.2 (Barco 3 MP), and 8.4±0.9 (Barco 6 MP). The residents reported scores of 7.4±1.0 (NEC), 7.2±1.0 (Eizo), 7.4±0.9 (Barco 3 MP), and 7.5±0.9 (Barco 6 MP).

Discussion

To our knowledge, this study represents the first technical and radiological assessment of medical digital LCDs with similar specifications, made by different manufacturers, based on international criteria. We established some technical differences, but like Samei et al, we could not find any statistically significant differences in radiological assessments made with different LCDs.5 This was consistent with most reported studies, where no diagnostic differences were found between LCDs with different specifications.5,6,8,9–12,14–16

Overall, on technical assessments, all the included LCDs scored well on the AAPM acceptance criteria. However, we measured some technical differences. In contrast with the two Barco displays, the NEC and Eizo LCDs did not pass the criterion for the darkest grayscale in the grayscale standard display function. This may be due to the larger variations in measurements at lower grayscales, as reported by Fetterly et al.21 Nonetheless, this may be cause for concern, because it may lead to suboptimal contrast for dark structures.

The Barco 3-MP LCD passed all the AAPM acceptance criteria, but scored less well on the evaluations of reflections in ambient lighting and veiling glare. This might be due to its glass protective cover. This can be removed optionally by the manufacturers when ordering new LCDs. The American College of Radiology recommends against the use of protective shields, because they may contribute to reflections.1 On the other hand, the protective cover may prolong the lifespan of the LCD.

The maximum luminance response of the Barco 6-MP LCD deviated substantially from the calibrated value. However, when we evaluated the 6-MP LCD, we considered it equivalent to two 3-MP LCDs; therefore, we performed the measurements in the center of one half of the 6-MP LCD, instead of in the center of the whole LCD. Possibly, the LCD was not optimally calibrated to ensure the uniformity desired for a 2±3-MP set up. This problem only became apparent when we analyzed the technical results. This emphasizes the importance of carefully calibrating luminance uniformity and verifying the calibration.

Radiological image assessments showed no significant differences among LCDs, although there was a tendency for the Barco 6-MP and the NEC 3-MP LCDs to receive slightly higher scores than the other LCDs. Thus, the Barco 6-MP exhibited radiological results that were slightly in contrast with the technical results. Despite the precautions taken to avoid bias by obscuring the names of the manufacturers, the observers might have recognized the Barco 6-MP LCD, because it was the only wide screen LCD currently available. Therefore, we could not fully exclude observer bias.

The results of this study are consistent with a report by Crespi et al, who stated that calibrated medical monitors possessed good quality.17 To maintain this quality, they strongly recommended periodic quality assurance tests.

There are several limitations to our study. First, we did not measure stability over time. Deterioration of the color filters in color LCDs might lead to rapid decreases in luminance.8 Kuroki et al suggested that the luminance of LCDs affected observer evaluations.6 Second, our radiological assessments were based on AAPM TG-18 criteria, but we did not investigate specific diagnostic criteria. Further study is required to rule out possible diagnostic differences. Third, the number of readers who participated in the radiological assessment is limited, which explains the width of the uncertainty around the means. However, it is possible that with a larger sample size the standard deviations would be smaller. Last, in considering the price differences between the LCDs, we did not consider potential expenses associated with endurance, support, and maintenance in the overall cost.

Within the experimental context of the study we wanted to reflect daily practice. This is why both radiologists and residents participated in the radiological assessment. Since radiologists are more experienced than residents in radiology, they could be more certain about image quality in comparison with residents. This might explain the small differences between the average scores reported by the radiologists and the average scores reported by the residents. However, the standard deviations are completely overlying. Several studies have found that the technical specifications and performance can vary among different monitor types (LCD, CRT) and also among hard-copy radiograph films; nevertheless, very few differences have been reported in their accuracy for clinical evaluation.2,3,6,8,11–14 In the present study, although slight technical and radiological differences were noted, the evaluated medical grade diagnostic 3-MP and 6-MP LCDs showed comparable image quality when calibrated accurately. The lack of significant differences in overall scores in this study may have been due to the apparently high (medical grade) quality of all the tested LCDs. Care should be taken to calibrate the LCDs to suit the task at hand and to verify the calibration with technical testing.1,22

Conclusion

We showed that, despite a large price difference, there were no significant differences in image quality between the investigated LCDs. In deciding which LCD to purchase, in addition to the price, maintenance costs and stability should be taken into consideration.