Display Characteristics and Their Impact on Digital Pathology: A Current Review of Pathologists' Future "Microscope".

Digital displays (monitors) are an indispensable component of a pathologists' daily workflow, from writing reports, viewing whole-slide images, or browsing the Internet. Due to a paucity of literature and experience surrounding display use and standardization in pathology, the Food and Drug Administration's (FDA) has currently restricted FDA-cleared whole-slide imaging systems to a specific model of display for each system, which at this time consists of only medical-grade (MG) displays. Further, given that a pathologists' display will essentially become their new surrogate "microscope," it becomes exceedingly important that all pathologists have a basic understanding of fundamental display properties and their functional consequences. This review seeks to: (a) define and summarize the current and emerging display technology, terminology, features, and regulation as they pertain to pathologists and review the current literature on the impact of different display types (e.g. MG vs. consumer off the shelf vs. professional grade) on pathologists' diagnostic performance and (b) discuss the impact of the recent digital pathology device componentization and the coronavirus disease 2019 public emergency on the pixel pathway and display use for remote digital pathology. Display technology has changed dramatically over the past 20 years and continues to change at a rapid rate. There is a paucity of published studies to date that investigate how display type affects pathologist performance, with more research necessary in order to develop standards and minimum specifications for displays in digital pathology. Given the complexity of modern displays, pathologists must become better informed regarding display technology if they wish to have more choice over their future "microscopes." Copyright:

INTRODUCTION

Digital displays (monitors) are an integral part of the clinical practice of pathology, especially given the Food and Drug Administration's (FDA) current recommendation for digital pathology vendors to include a specific display model within their whole-slide imaging (WSI) system's device submissions.[1] Whether for digital pathology or routine daily use, choosing the best display for a pathologist's work should be considered just as important as selecting the most appropriate microscopy setup for an individual pathologist rendering formal diagnoses.

Unfortunately, selecting the “right” display for one's clinical and daily use cases can be fraught with challenges. Current displays vary greatly in size, shape, esthetic design, resolution, brightness, contrast ratio, refresh rate, screen reflection, viewing angle, etc., In fact, most display specification sheets have a considerable number of criteria that, to the average consumer (and pathologist), may include terms whose meanings are not well understood [Figure 1].[23] By extension, most consumers are also less likely to distinguish the relative impact of any given display parameter on their final viewing experience. Furthermore, display manufacturers do not routinely include all the same specifications for comparison, or confusingly, specifications are shielded by proprietary terminology, making direct comparisons difficult to impossible [Figure 1].[2345] Finally, most adjustable specifications (brightness and contrast being the possible exceptions) are ultimately ignored and stay at default settings when the display is set up by either the user, or more likely, by one's departmental, central, or practice's information technology group.

Figure 1

Example specification sheets for three displays from separate manufacturers. Note that each contains >20 individual specifications, with some specification types listed for only one display and/or being proprietary in nature (most pronounced for the display in the right panel). Specification sheets have been modified from their original format

With this apparent complexity, where should practicing pathologists begin when determining the best display for their use? We have identified three main categories of displays, distilled from manufacturers’ and computing hardware websites, that pathologists should be aware of: (1) medical grade (MG), (2) consumer off the shelf (COTS), and (3) professional grade (PG). MG displays are the current standard in medical imaging. They are typically expensive (thousands of dollars per display), built for multiyear use in different medical environments, and have standardized features in order to provide a uniform experience to their users; unfortunately, their benefit within digital pathology remains unclear.[6789] These are contrasted with COTS displays, which are general-purpose displays used by almost everyone, from home use to office workers to physicians, and are purchased through traditional retail outlets. Most pathologists today use COTS displays as their primary display, provided to them as part of an enterprise's standard core (enterprise) workstation configuration, selected by default with their office computing setup, or included as part of their laptop. PG displays are similar to MG displays in that they are marketed to distinct professions (e.g., graphic design, animation, video production, computer-aided design/drafting, photography, etc.) and typically are made with the goal of being able to provide superior accuracy and precision for specific professional parameters, such as color depth, color gamut, brightness, and contrast ratio.[101112] Of note, while these categories are useful for discussion purposes, they are themselves fluid, imprecise, and at times overlapping; many niche categories exist as well, such as high-end gaming displays, commercial displays, and digital signage.

Many articles have been published in both the popular consumer and academic medical literature discussing the evaluation, purchase, and utilization of displays.[3610131415161718192021] However, articles specifically focusing on appropriate displays for pathology use cases are comparatively rare. Regardless of the use case, there is a paucity of general literature surrounding display specifications, in particular their definitions, how they apply to different display categories (MG, COTS, and PG), and ultimately how one can use these specifications to help choose the right display for one's intended use. Unfortunately, much of the literature regarding displays is highly technical and jargon laden, Unfortunately, much of the literature regarding displays is highly technical and jargon laden. While this is at times unavoidable, one of the goals of this article is to facilitate, for pathologists, the review and understanding of common-place display terminology and features. Following this initial definition section, an up-to-date literature review on the impact of MG, COTS, and PG displays on the practice of digital pathology is presented. Finally, recent events affecting the regulation of digital pathology devices, including the coronavirus disease 2019 (COVID-19) public emergency's effects on display use for remote digital pathology, are discussed.

COMMON DISPLAY SPECIFICATIONS

When purchasing a display, it is important to review the relevant performance specifications of each display in order to compare them with one another. Given the abundance of display specification terms present in the market today, the authors have identified what we feel are the most important specifications to examine. The definitions and units used for these terms are provided in Table 1, with display panel-specific terminology in Table 2. In order to focus more on issues surrounding each specification, the first occurrence of each term in the text is bolded, and we advise the reader to refer to Tables 1 and 2 for further discussion of the definition, primary points, and/or for easy review of each concept. Please note that while generalities about a specification may be discussed, it is out of the scope of this article to provide specific display configuration recommendations for digital pathology use.

Table 1

Important display specifications (medical grade, consumer off the shelf, and professional grade) and definitions, grouped by feature type (original work)

Specification	Definition and salient points
Panel type	The primary technology and design of the display which affects many downstream characteristics. This includes CRT, external backlit, and emissive displays
CRT	Older display type that generates images through the deflection of a beam of electrons to the screen surface, currently considered to be a “dead” technology when compared to LCDs
LCD	Display type that utilizes an array of liquid crystal cells manipulated by electric fields, has largely supplanted the earlier CRT displays in both COTS and MG markets. There are multiple LCD panel technologies, with each having differences in the arrangements and properties of their individual liquid crystal cells [Table 2]. LCDs generally require an LED or CCFL backlight to produce an image with sufficient luminance
Viewing angle	A measurement given in degrees denoting the maximal deviation from the central axis, wherein a viewer can still perceive the image displayed on a display without degradation of brightness and contrast. Note that this is typically given as a symmetrical measurement from a center point. As an example, a 90° would allow the viewer to move 45° in either direction while still maintaining image adequacy
Backlight bleeding	Artifact where the backlight can be seen even while the display is showing a dark color. This is due to a lack of complete opacity in the LCD material and is a characteristic weakness of certain panel types (e.g., IPS)
Luminance	Objective measurement of light intensity (compare with brightness, below), expressed as cd/m2 (candela/meter squared) in the SI measurement system, fL in the US/English system, or nits (deprecated term, however still commonly used in COTS specifications). Typically used when referring to light being generated or reflected, different from illuminance (below)
Brightness	Subjective measure of light intensity by the user, expressed as unlabeled units or as a percentage. Literally denotes the light perceived by the user being generated or reflected from a display. Does not have a linear relationship with luminance
Illuminance	Objective measurement of ambient light intensity present in the surrounding environment, measured in lx
Contrast and contrast ratio	Static contrast refers to the ratio (e.g., 1000:1) of the maximal luminance to minimum luminance possible for the display at a static point in time. Dynamic contrast refers to the same ratio but at different points in time. Reported dynamic contrast ratios are often much higher than static contrast ratios (1,000,000:1) and the popular perception is that, while sounding impressive, dynamic contrast ratios are not reliable specifications for comparing displays
Pixel	A.k.a. picture element, denotes the smallest distinct controllable point of a display panel
High dynamic range	Technology that allows for a more accurate portrayal of color or shades of gray across a wider, extended luminance spectrum (i.e., produces darker shadows and brighter lights)
Color depth (or bit depth)	Number of bits (a binary representation of digital information in the form of on/off, true/false, 1 or 0) allocated to each primary color (red, green, and blue). A higher bit depth allows a greater number of colors to be represented. Modern displays typically have color depths of 8 or 10 bits
Color gamut	The range or spectrum of colors able to be represented on a display. Three common color gamuts are sRGB, CMYK, and adobe RGB
Subpixel	Subregion of a pixel, typically a single-color region of the pixel (red, green, or blue). Manipulating specific subpixels allows a display to render colors more accurately or to make images appear more vivid, muted, etc.
Lookup tables	Precalculated tables used to increase color depth and color accuracy above the native capabilities of the panel. These can also be generated from display calibration tools
Display format/aspect ratio	Width versus height dimensions of the display, typically expressed as a ratio (examples include 3:2, 4:3, 16:9, 16:10, 21:9, 32:9)
Display resolution	Resolution conveys the total number of pixels in the viewable area of the display, typically given as a two-dimensional, width by height, measurement. May be expressed a popular synonym. Example resolutions with synonyms include: 640 × 480 (VGA), 1920 × 1080 (HD), 2048 × 1080 (2K), 2560 × 1440 (WQHD), 3840 × 2160 or 4096 × 2160 (4K), 5120 × 2880 (5K), 7680 × 4320 (8K)
Physical image size (screen size)	Denotes the “size” of the viewable area on the display. Traditionally expressed as the diagonal measurement in inches (lower corner to opposite upper corner, typically should not include the bezels [borders] of the display)
ppi	Number of pixels present in one inch of surface. Analogous to the DPI used in paper printing
Pixel pitch	Denotes the distance between neighboring pixels, usually in millimeters. Pixel pitch is inversely related to display resolution, with a short pixel pitch typically associated with a “clearer” display (e.g., a 15” 4K display will have a shorter pixel pitch, and thus a higher ppi than a 55” 4K display)
Refresh rate	Number of times image is refreshed on the display per second, measured in Hz. Typically, with higher display refresh rates, motion will appear smoother (combined with response time, below). TV manufacturers may report out an “effective refresh rate” that uses proprietary technology plus the native refresh rate of the display to smooth motion even further
Response time	Measurement of how long it takes for a pixel on a display to change from one color to another (e.g., black to white) and is typically given in ms. Shorter response times correlate to less motion artifacts on a display (smoother motion overall)
Adaptive frame synchronization	Set of proprietary technologies that smooth out motion in graphically intensive applications, for example, in PC gaming
Inputs	The cable and port that allows graphical information from a computer or other device to be displayed on the display. HDMI and DisplayPort are currently the two most commonly used formats. Other options are USB, DVI, and VGA. Note that with each new iteration of digital input standards (e.g., HDMI v2.1), new cables and devices will be needed that support the standard
JND	Defined by the DICOM standard as “the luminance difference of a given target under given viewing conditions that the average human observer can just perceive”
Delta E2000	A standardized formula proposed by the CIE for measuring the “distance” between two pixels in color space. Can be used for either gray-scale or color images

MG: Medical grade, COTS: Consumer off the shelf, PG: Professional grade, CRT: Cathode ray tube, LCDs: Liquid crystal displays, CCFL: Cold-cathode fluorescent lamp, fL: Foot-Lamberts, RGB: Red-green-blue, sRGB: Standard RGB, CMYK: Cyan-magenta-yellow-black, VGA: Video graphics array, ppi: Pixel per inch, DPI: Dots per inch, HDMI: High-definition multimedia interface, USB: Universal serial bus, DVI: Digital visual interface, JND: Just noticeable difference, DICOM: Digital Imaging and Communications in Medicine, CIE: Commission on illumination, IPS: In-Plane-Switching, LED: Light-emitting diode, SI: International System of Units, HD: High Definition, WQHD: Wide Quad High Definition, PC: Personal Computer, TV: Television, A.k.a. picture element: Abbreviated form of ‘picture element

Table 2

Commonly used display panel types (original work)

Display type	Description
TN LCD	Liquid crystals within a cell are organized in a helical pattern that become parallel when stimulated with electric current. Generally, these displays are now the least expensive
IPS LCD	Liquid crystals within a cell are organized in a parallel pattern and shift collectively in the same angle when stimulated with electric current. Generally, these displays are more expensive to produce but offer better viewing angles and color reproduction. Another disadvantage is “backlight bleeding,” where a screen displaying darker colors appears unevenly lit
VA LCD	Liquid crystals within a cell are organized at an angle perpendicular to the surrounding polarizer panels. This LCD type was developed as a compromise between TN and IPS displays and has features and tradeoffs in common with both
CCFL LCD	Type of LCD backlight that relies on electrical excitation of a gas to produce light. These are typically less energy efficient than LED backlights and less commonly used today (longer warmup times, may require the use of mercury)
LED LCD	Type of LCD backlight relying on a semiconductor exposed to electric current. LED–LCDs are likely the most commonly used panels today
OLED	A technological alternative to LCDs that involves the generation of light from phosphors exposed to an electric field. These have better viewing angles and color reproduction at both high and low levels of luminance but are expensive, not available in all screen sizes, and are susceptible to burn-in (see text)
MicroLED	Latest display technology where the panel is composed of millions of tiny LEDs that provide both the light source and images seen on the screen. Purported to have better viewing angles, color reproduction, luminance levels, and energy efficiency over both traditional LCDs and OLED displays

TN: Twisted nematic, LCDs: Liquid crystal displays, IPS: In-plane-switching, VA: Vertical alignment, CCFL: Cold-cathode fluorescent lamp, OLED: Organic light-emitting diode

The panel itself-display types

When evaluating a digital display, one of the first specifications a buyer is confronted with is the display or panel type. Display types have continuously evolved over the past 100 years; however, one can easily argue that the past two decades have seen the most significant changes. For example, the plasma screen television's (TV) relatively recent development, aggressive marketing, and subsequent decline from 2000 to 2015 demonstrate how display types represent an area of rapid flux for consumers.[2223] For our purposes, we will only discuss salient points regarding the history of and technological basis for two broad categories of display types: (1) externally backlit, or transmissive, displays that require a separate light source and (2) emissive displays that, by their nature, intrinsically generate their own light source.

External backlit displays

Cathode ray tube (CRT) displays date back to the 1920s and were employed as the first displays in medical imaging in the 1970s–80s, specifically for “soft-copy” diagnosis in radiology.[142425] CRT displays were the norm in both professional- and consumer-grade displays throughout the 20th century, from TVs to computer monitors, and relied on the magnetic manipulation of streams of electrons to produce an image on a phosphorescent screen.[22] Advances in CRT technology, besides ushering in the advent of displays themselves, primarily focused on physical specifications such as display size, color versus monochrome displays, color accuracy, and line resolution.[1424] Over the past 20 years, these devices have been largely supplanted by more energy-efficient, thinner, and brighter liquid crystal displays (LCDs); CRT displays are, in fact, now considered to be a “dead” technology.[2627]

Briefly, LCDs are comprised of a panel of liquid crystal cells sandwiched between two electrodes and two polarizing filters (typically polarizing film on a glass plate).[24] While shining light through the liquid crystal cells, varying the voltage between the electrodes alters the conformation of the liquid crystals within the cell, affecting the amount of light able to pass through to the user and thus producing a viewable image.[24] Additional filters allow for the generation of color by selectively transmitting certain wavelengths of light.[24] The properties of the liquid crystal cells and surrounding polarizers also determine the viewing angle of the display, outside of which the image will appear to lose contrast and darken.[28] Narrower viewing angles, as compared to CRT displays, is one of the few disadvantages of LCDs.[28] A wide viewing angle is important when the display is viewed by more than one person simultaneously or if the viewer will be viewing the display from multiple vantage points (e.g., if switching back and forth between a microscope and display).

LCDs include panel types such as twisted nematic (TN), in-plane-switching (IPS), and vertical alignment (VA), further discussed in Table 2. TN displays were the earliest LCD types and are now generally the least expensive, however they also had relatively narrow viewing angles and poor color reproduction. IPS displays improved upon both of these aspects, however they have been shown to have a tendency toward backlight bleeding.[29] VA panels are the newest of the three and serve as a compromise between the two former types, i.e., less backlight bleeding with wider viewing angles.

Even though LCDs are quite young when compared to the lifespan of CRT displays, they still have gone through multiple iterations with many changes. One particularly significant change has been the external light source, for which LCDs traditionally have been either a cold-cathode fluorescent lamp (CCFL) or light-emitting diode (LED) [Table 2]. While CCFL backlights were popular in early LCD displays and likely can still be found in older devices in most people's homes/offices, LED-–LCD displays are now ubiquitous throughout the industry. LED–LCDs became preferred primarily due to being more energy efficient, thinner, more uniformly lit, and overall brighter over their CCFL backlit counterparts. However, now, with the emergence of emissive display panels, there is potential for, yet again, another shift in display technology.

Emissive displays

Unlike their externally backlit, or transmissive, cousins, emissive display panels are capable of producing their own light – their pixels literally generate the light viewed by the user.[24] Currently, the prototypical emissive display type seen commercially is based on organic LED (OLED) [Table 2]. OLED displays differ from LCDs in that they use only one glass or plastic panel and involve the native generation of light from phosphor exposed to an electric field.[24] OLEDs have been heavily hyped over the past 5 years as the future of display technology as they can provide brighter and sharper images, are more energy efficient, and can be made into thinner panels than the conventional LCDs used today. To date, OLEDs have been produced mostly for mobile devices, however they are now being used in high-end laptops and high-end large form-factor TVs.[3031] Of note, similar to the now deprecated plasma display technology, OLED displays are susceptible to screen burn-in, a process where still images displayed for a lengthy period of time result become permanent “ghost” images left on the screen.[32] How OLED displays will fare in health care where still images can be displayed for hours at a time (e.g., dashboards and repetitive health information system screens) remains to be seen.

Another new emissive technology that has emerged as a potential major disrupter of the display market is micro LED (microLED) technology [Table 2].[3334] As its name suggests, microLED displays are made up of millions of tiny (or micro, 1 mm or less in size) LEDs that act as both the panel's light and image sources.[34] In contrast to OLED and traditional LED–LCD displays, microLED displays are relatively less complex and require fewer material layers to produce, allowing for extremely thin panels. Further, they have the potential to be the brightest displays with the same or better color accuracy, wide viewing angles, no screen burn-in, and longer life spans than current display technology, including OLED.[33] MicroLED displays are manufactured in a modular fashion that allows for easy size manipulation, examples of which include 292” (over 24 feet) to 75” wall displays, most recently demonstrated at the 2020 Consumer Electronics Show.[33] However, until the manufacturing process improves, we should not expect to see microLED displays in the mainstream market anytime soon (earliest general consensus is the mid-2020s).

Optical considerations in digital imaging – luminance, contrast, and color

Luminance

Luminance is the amount of light (or photonic energy) being reflected or generated from a given area and is measured officially in candela (cd/m2).[824] A distinction must be made between luminance and the related, but more frequently used term, brightness. Luminance is an objective measurement of light with standard units, whereas brightness is a subjective impression of light intensity, typically expressed as a percent. The most important point for pathologists to understand is that brightness does not have a linear, strictly defined relationship with luminance – many factors can influence one's perception of brightness, whereas luminance is a property of the display technology itself.[24] For instance, a 50% brightness setting on a display with a max 250 cd/m2 luminance does not mean the luminance is 125 cd/m2.

Most displays allow the user control of only brightness and not luminance directly, although some MG displays allow the current luminance level to be displayed onscreen or manipulated.[824] The luminance of a display is determined by its light source, which can vary at different physical locations on the display and additionally as the display ages, so proper calibration may need to incorporate multiple measurements across its surface at regular intervals throughout the life of the display.[24] However, given that pathologists regularly change the luminance of their microscope's light source throughout histopathologic review of slides, for example to compensate for thicker tissue sections, to what extent strict luminance calibration applies to pathology in general is not known.

The optimum luminance of a display varies with the ambient light present in the surrounding environment, or illuminance, which is measured in lux (lx).[824] If the illuminance incident on a display surface (glare) exceeds the luminance generated from that surface, the viewer will not be able to see the display's contents. In contrast, excessive luminance in a dark room can be both uncomfortable to the viewer and cause a bleached image (i.e., the image is “too bright”). To reduce the effect of background illumination, some display panels are built with antireflective properties.[24] In addition, the human eye alters its sensitivity to changes in ambient light in a process called adaptation, which can take several minutes and has a significant impact on the ability to discern features of an image.[35] In medical imaging, the radiology literature makes recommendations for ambient lighting and the period of time allowed for adaptation, however, the degree to which ambient light in a pathologist's working environment should be controlled is unclear.[36] One working group has developed a mobile (phone based) application for monitoring the effects of ambient light on digital pathology, but this area has not yet been well studied.[37]

MG displays, as well as certain PG, high-end COTS, laptops, and mobile phone displays, have an intrinsically incorporated optical sensor.[82438] This sensor detects ambient light in the environment and makes corresponding adjustments to the luminance of the display in order to ensure that luminance exceeds illuminance. Without this sensor, any compensation for the surrounding ambient light needs to be made manually through a calibration process (either subjectively by the user or through a specific calibration device). Complicating this is the tendency for displays to lose the ability to reach peak luminance as they age. MG displays are typically manufactured with higher potential peak luminance (often 1000 cd/m2), ostensibly granting them more reserve to remain in optimum luminance (e.g., 250 cd/m2) for longer periods of time. Coupled to this concept is the practice of remote calibration, which compares current-state display measurements, for example, luminance, with accepted measures over a network and then make compensatory changes to a MG display's settings to be within a preset range, for example, from medical imaging standards.[39] How often an ordinary COTS display must be calibrated, how long the calibration can be retained, and how regular calibration would affect display use in pathology remains undefined. Of note, even if a COTS display only allows for brightness manipulation, a separate external photometer can be used to manually calibrate luminance indirectly. As with other diagnostic equipment and regardless of the display chosen, luminance calibration may become a part of a laboratory's regular quality control process if the display is being used for clinical diagnostic purposes.[9]

Contrast

Contrast is defined as the difference in luminance between a pixel being 100% lit or “on” relative to it being not lit or “off” and is often expressed as a ratio of maximum to minimum luminance.[24] The contrast ratio of a display is important for the ability to accurately convey a range of light or dark shades, as well as the ability to distinguish between similarly shaded objects. The contrast ratio is typically cited as a single value by manufacturers (either static or dynamic), although this is overly simplistic for multiple reasons, including: (1) contrast can vary at different points on the display; (2) light from surrounding pixels can affect the pixel in question; and (3) ambient light can reduce perceived contrast.[82440] Unfortunately, none of these factors are accounted for in the contrast ratio calculation directly. Furthermore, backlight bleeding artifact in transmissive displays can result in light reaching the screen even when a pixel is showing a dark shade or color (i.e., when “off”).[2941] A more recently marketed technology, high dynamic range (HDR), allows for a more accurate display of colors and shades across a broad range of luminance values, thus enhancing contrast.[424344] While HDR video is becoming more common, creating HDR media requires specialized hardware and software.[45] Therefore, it may be some time before we see HDR regularly used in digital pathology.

Color

The capability of a display to represent color is described through both color depth (bit depth) and color gamut. Color depth denotes the number of bits (information) used to describe the color component of a single pixel on a display, whereas the color gamut describes the range, or spectrum, of colors able to be expressed on a display.[26] The bit depth of a display typically varies between 8 and 12 bits for modern displays. On a color display, each pixel is comprised of multiple (usually three) subpixels calibrated to transmit a certain set of light wavelengths through a set of color filters arranged in a linear fashion (either vertically or horizontally). Given three subpixels, an 8-bit display can display 28 × 3 = 16,777,216 colors. Figure 2 shows an example of how the relative luminance of each subpixel can be used to create colors when viewed from a distance without magnification.[46] Even higher bit depths thus allow for the generation of further colors. Methods of increasing bit depth include flashing multiple colors in a short period to give the impression of additional color depth in a process called frame rate control or through the use of precalculated lookup tables.[4748] Note that some display manufacturers and imaging software will refer to “24-bit” color, treating the 8-bit depth for the three subpixels additively.

Figure 2

The RGB color space as overlapping circles (left) and its representation as a series of pixels on a display (right). “RGB Color Model.svg” by author: Pd4u is licensed under CC BY 1.0

For devices, the color gamut represents the amount of a specific color space, or range of colors, that can be reproduced accurately by a device (e.g., display, printer, etc.). Typical color gamuts supported by COTS displays include sRGB and CMYK. PG displays usually support a wider range of colors than COTS displays, such as the Adobe RGB and P3 color gamuts. While color gamut is more pertinent in the professional photography, video, and animation spaces, it has not been emphasized as much in medical imaging (radiology imaging has been mostly gray scale). There has been some initial discussion regarding color gamut in digital pathology as pertaining to image analysis and color normalization, however the effect of displaying different color gamuts for diagnostic digital pathology has not been fully explored.

Constructing an image – display format, pixels, and resolution

Display format

Display format corresponds to the actual physical structure of the display and includes both a measurement of screen size and the relative dimensions of the viewing area, expressed as an aspect ratio (width: height). In the past, the 4:3 or “standard definition” display was the most common; however, in recent years, 16:9 widescreen “high-definition” displays have become the norm, with the exception of some 16:10 MG, high-end COTS and PG displays. Most recently developed are various ultra-wide and other “nonstandard” displays that can have an aspect ratio of 3:2, 21:9, or even up to 32:9. Some of these displays may have a slight curvature that adds the benefit of more light from the display being directly aimed at the viewer – this has the potential of helping to limit image distortion, improving viewing angles, and decreasing eye strain.[4950] For many, screen size, generally given as a diagonal measurement in inches, is the most important specification considered when choosing a display. However, one must take into account the aspect ratio to get a better idea of the actual display size and shape, as a 49”, 32:9 display will be a very wide, narrow rectangle as compared to a still rectangular, but much taller, 43” 16:9 display.

Of note, the use of smaller screen sizes has gained increased attention with the growth of mobile devices in digital pathology.[515253] While both smartphones and tablets have seen considerable technological advancements in display technology, one must remember that mobile devices should not be viewed as just another display, but instead as an app ecosystem, platform, and overall, an all-in-one device. As such, all aspects of a mobile device must be considered when considering them for specific digital pathology use cases, including computing power, network connectivity, storage capacity, battery life, security capabilities, input methods, operating system, and app platform/availability, in addition to the size and quality of the included display. Given the overall ubiquity of these devices and their increased utility in health care, we expect to see more discussions surrounding both the use and validation of mobile devices in digital pathology in the near future.

Pixels and resolution

A single pixel (or “picture element”) is comprised of one or more liquid crystal cells. When multiple liquid crystal cells make up a single pixel, each of these cells is defined as subpixels. The exact structure of these cells varies between displays, especially those using different panel types [Figure 3]. Display resolution is determined by the total number of pixels present in the panel and is usually given as a series of two dimensions (e.g., 1280 × 1024 pixels), but can also be expressed as their product (1.3 megapixels) or as a density in pixels per inch. Sometimes, only one of the dimensions is given (e.g., 1920 × 1080 is “1080p”) and may be rounded up or down (e.g., 3840 × 2160 and 4096 × 2160 would both be “4K” or “2160p”). As of 2020, the most commonly marketed display resolutions are 1080p and 2160p (4K), however displays with 8K or even 10K resolutions are available. For optimal clarity, both dimensions should be given where possible.

Figure 3

The morphology of pixels/subpixels varies considerably between different types. (a) 2003 Dell Ultrasharp 1703FP (CCFL LCD, VA), 755x; (b) 2014 Samsung QM85D (LED LCD, VA) 762x; (c) 2019 Apple Pro XDR (LED LCD, IPS), 772x; (d) 2020 Samsung Galaxy Chromebook (Samsung OLED), 752x

Pixel pitch is a related measurement denoting the distance between neighboring pixels, usually in millimeters, and is inversely related to resolution for any given screen size (i.e., the lower the pixel pitch, the higher the display resolution for a given display size). The size of each pixel can also vary substantially from display to display (decreasing with technological advancement, actual panel size, or both), meaning the displayed size of the same image on different displays may vary as well. This is readily apparent when viewing 4K displays of different sizes as one may not be able to see the individual pixels on a 4K, 5.5” phone screen, however that same 4K resolution will appear granular and “fuzzy” when viewing a 43” computer monitor close up. In addition, the viewer's distance from the screen is important to consider because that same 43” 4K display will appear sharp and crisp when seated many feet away from the device. Figure 4 highlights the interactions of resolution, pixel pitch, and screen size on four example displays.

Figure 4

Relative pixel density and size of different display resolutions: (a) Dell MR2416 (24”, 1920 × 1200) (b) Philips PP27QHD (27”, 2560 × 1440) (c) Apple Pro XDR (32”, 6016 × 3384) (d) Samsung QM85D (85”, 3840 × 2160). All at effective × 59. As can be seen in a-c, pixels generally decrease in size as resolution increases, however this is offset by the relative screen size (d)

One advantage of using higher resolution displays when viewing WSI cases is that a correspondingly larger field of view of the slide may be appreciated all at once.[54] Of note, the resolution of a display, combined with the optimal viewing distance, may result in a suboptimal field of view (e.g., if one is supposed to sit 10 feet away from the display to view the “best” image). Further, an often-overlooked disadvantage of having a higher resolution display is that, without appropriate scaling, images and text viewed at the highest (native) resolution may appear too small for most, a factor most notable with smaller displays (e.g., a 13” laptop display set at its native 4K resolution). In addition, the interaction of screen resolution, display format, and software settings can result in images that are stretched or squashed. On a personal computer, resolution can generally be adjusted lower, however one cannot adjust the display to above its native resolution, so one should choose a display that meets the upper limits of their potential use. Pathologists viewing digital images should bear in mind the effects size and resolution can have on the appearance of familiar histological and anatomical structures.[50]

Moving pictures – refresh rate, response time, and frame synchronization technology

Refresh rate

The refresh rate of a display is the number of times the screen is updated, or the image refreshed, per second and is given in hertz (Hz). A display's refresh rate determines the smoothness of an image in motion, which could be a video, action within a game, or a WSI being panned or zoomed.[55] A low refresh rate results in flickering or “tearing” of the image, whereas high refresh rates are associated with “smoother” motion. Most displays now have a refresh rate of at least 60 Hz, which is sufficient for most computing purposes. Much higher refresh rates in computer displays do exist (most recently reported up to 300 Hz) and have largely been driven by the console and computer gaming industries rather than health care. One caveat to displays with higher refresh rates (>60 Hz) is that they are often more expensive, have lower native resolutions (720p or 1080p), and/or may achieve these rates by using a TN panel, which can compromise color quality relative to IPS or VA displays.[5657] At this time, it is not known what effect increasing the display refresh rate may have on viewing digital slides.

Response time

Response time is the measurement of how long it takes for a pixel of a display to change from one color to another (e.g., black to white), and is typically given in milliseconds (ms). It should be noted that this is related to, but distinct from, the concept of refresh rate given above. In general, a lower response time denotes better display performance, however for most people, the difference in response times is negligible for their typical use (the exception is for high-end gaming). For reference, IPS panels tend to have a higher (worse) response time than TN or VA panels.[57] Like refresh rate, to date, there have been no studies performed looking at the effects of response time on the practice of digital pathology.

Frame synchronization

Adaptive frame synchronization technology is available on many newer laptops and displays intended for gaming.[58] The two most commonly used proprietary forms of this technology are GSync and Freesync, adopted by the graphics card manufacturers NVIDIA and AMD, respectively.[585960] Briefly, these technologies allow for a smoother pacing of frames transmitted from the graphics processor to the display, resulting in a smoother gaming experience for the user.[585961] Unfortunately, these features traditionally come at an increased cost, require the use of compatible graphics processing hardware, and have only been optimized for use in gaming.[58] Given that these technologies have not been natively enabled for web browsers nor integrated into most WSI viewing software, it is unknown if they will prove to be beneficial in future for smoother experiences when viewing telepathology streams and panning across WSI.

Other considerations – inputs and multiple displays

Input types

Numerous standards for signal input types have been created for digital displays, including high-definition multimedia interface (HDMI), DisplayPort, digital visual interface (DVI), video graphics array (VGA), and universal serial bus (USB). HDMI and DisplayPort are the two most ubiquitous modern input types that allow for the transmission of both audio and video signals to the display device. This can be advantageous when the display also serves as the primary audio generation source for the computer/TV. HDMI, in particular, has undergone multiple revisions, with the latest version (2.1) increasing signal bandwidth up to 48 Gbit/s, thus allowing for very high-resolution signal generation (e.g., 10K, 10240 × 4320).[62] Of the two above standards, DisplayPort is newer, with it being able to support an even higher maximum resolution of 16K (15360 × 8460).[63] DVI and VGA are older, video-only standards that are limited to resolutions of 2560 × 1600 and 2048 × 1536 in their last iterations, respectively. While VGA is an analog standard and is now deprecated, one may still find many audio–visual configurations in older conference rooms around the world, with VGA being the primary PC input – VGA inputs require special adapters to connect to fully digital PC outputs. USB is primarily a connector and communication standard used to link peripheral devices to a computer, however it can also be used to transfer video data from laptops and mobile devices to displays. Of note, the connector (e.g., USB-C) or interface type (e.g., Thunderbolt 3) does not denote the input standard used to transmit the video information (e.g., DisplayPort).

The possible outputs a computer has is dependent on the video controller (card) on the device in question, typically either a graphics card external to the motherboard (or even the PC) or integrated within the motherboard itself. Similarly, some displays are limited to one or two input port types. Note that in all cables and associated adapters, it is imperative to identify the version prior to use, as older cable/adapter versions can have substantially lower specifications and may not be compatible with the computer or display in question. For example, at one of the author's institutions, it was found that in a new reading room setup, even though there were new, high-powered computer workstations, graphic cards, and 55” 4K displays installed, the $19 DisplayPort to HDMI adapter had been chosen incorrectly, limiting the resolution on the 4K displays to 1080p and rendering a lower quality image (switching all adapters to the 4K version fixed the problem).

Multiple displays

Multiple displays can be connected to the same computer, with most basic PCs supporting two displays at once and PCs with more advanced graphics cards supporting three or more displays. For many, using multiple displays is an effective way of increasing one's relative screen size and productive virtual work area. It also allows for easier multitasking when using multiple programs simultaneously, such as a WSI viewer and the EHR/LIS applications. Of note, using multiple displays from different manufacturers may lead to the same image appearing different on each display due to potential variability in screen size, aspect ratio, resolution, color gamut, etc., To avoid this, we recommend purchasing the same model display at the same time and running a calibration step using an external colorimeter to avoid image divergence. In digital pathology, the pathologist's “cockpit” has been typically imagined as a multiple display setup, however no formal studies have been done and multiple displays are not required by the FDA.

The whole-slide imaging pixel pathway and display standards

Whole-slide imaging pixel pathway

The concept of an official WSI system pixel pathway has been informally codified by the FDA, College of American Pathologists, Digital Pathology Association, and American Telemedicine Association in their mutual recommendation and/or requirement that digital pathology setups be validated as complete, unified systems for primary diagnosis.[164656667] The pixel pathway is comprised of two major subsystems: the WSI image acquisition subsystem and the workstation/image viewing subsystem.[1] As defined, it denotes the transportation of a digital signal from an image source (the WSI device) to a workstation (PC) with image viewer/image management system, connected display, and, ultimately, transmission to the user (the viewing pathologist).[1]

Although the display may be the most tangible part of the process for the pathologist, the quality of a WSI can be degraded at any step in the pixel pathway. Typically, WSI are intrinsically large files composed of compressed images tiles and are served to the viewing station in an on-demand, piecemeal fashion.[6869] Depending on one's server, network, and/or viewing station specifications, navigating within a WSI can appear sluggish or one or more tiles may be blurred when attempting to zoom into a higher magnification. In these situations, it is important to note that the display itself is most often rendering the received WSI information accurately and any performance issues in the displayed WSI tiles are most likely due to upstream defects in the pixel pathway. Ultimately, the relationship between slide magnification, the pixel data captured by a WSI sensor, image compression, and the resultant field of view seen on a display is complex and a full discussion on this can be found elsewhere.[546869] Therefore, due to this complexity, any troubleshooting process for WSI performance should include all components of the pixel pathway.

Digital Imaging and Communications in Medicine and display standardization

The Digital Imaging and Communications in Medicine (DICOM) standard is the primary health-care imaging standard and specifies how medical imaging data are managed and transported from their initial acquisition to health information systems.[70717273] Within this standard exists DICOM PS3.14, which specifies the gray scale display function (GSDF) that governs the relationship between pixel signal values and luminance for any given display.[63646566] The GSDF allows for the standardization of the appearance of medical images on different types of displays.[26] From a purchasing standpoint, MG and some PG displays come with a DICOM setting that can be enabled, however DICOM conformance is generally not associated with COTS displays.[97274] Of note, software and external sensors are available to calibrate any type of display to the GSDF.[76869]

Another important concept regarding display standardization is “just noticeable difference.” Just noticeable difference originates in psychology, but is defined by DICOM PS3.14 as “the luminance difference of a given target under given viewing conditions that the average human observer can just perceive.”[87173] In imaging and displays, this correlates functionally to the ability of a person to discern differences in contrast between two pixels. For example, one would want to be able to discern the difference between the darker pixels of heterochromatin and lighter pixels of euchromatin in a nucleus or discriminate eosinophilic granules from the surrounding cytoplasm, despite the fact that both sets of comparisons are similarly colored.[75] These differences, whether in color or gray scale, can be quantified through metrics such as delta E2000 proposed by the International Commission on Illumination.[767778] An extensive discussion of color metrics, color spaces, and their quantification is beyond the scope of this article, but a delta E2000 measurement of < 1 is generally agreed to be imperceptible by a given viewer. It is notable that popular technology media sources and display manufacturers have begun describing products with delta E2000 measurements.[767980] The ability to discern differences between two points will always be affected by the contrast threshold of the observer's eye and will also be impacted by the characteristics of an optical system, such as the contrast ratio in a display or the bulb and lenses of a microscope. Even when delta E2000 measurements are > 1, color science professionals recommend the use of human observers to determine if the discrepancy is significant.[8182] For digital pathology, it may be that small delta E2000 discrepancies would have a greater impact on image-processing algorithms than on clinical diagnosis. Although It is important to note that although a color extension to the DICOM-GSDF has been proposed (aptly named DICOM-CSDF), this does not appear to have been formally codified as of yet.[7583] Therefore, because of pathology's reliance on color imaging, the current role for DICOM PS3.14 in digital pathology is unclear.

THE IMPACT OF DISPLAYS ON DIGITAL PATHOLOGY

We proffer that choosing one's display for digital pathology will be equally as important as currently selecting one's microscope. Like the microscope, the display is an adjustable and tangible “portal” for pathologists and support staff allowing individuals to examine histology and cytology slides for many hours each day. However, even with the current pixel pathway and display recommendations for WSI systems, the selection of displays available for use in pathology is extensive and quickly changing. In order to choose the display best suited for one's needs, current and future pathologists must be familiar with an array of terminology and parameters. Moreover, new or emerging technologies such as emissive displays, frame synchronization technology, and HDR may offer benefits for the practice of digital pathology. In this section, we present and summarize the impact of the pixel pathway and display parameters on digital pathology and pathologist performance, with a review of the existing literature and associated findings summarized in Table 3 and in the text below.

Table 3

Summary of studies in the pathology literature related to display parameters and practices (original work)

Article	Experimental design	Findings	Limitations
Krupinski et al., 2012[100]	Examined impact of color calibration on pathologist interpretation of ROI from breast biopsy WSI	Modest decrease in time to diagnosis but no effect on accuracy when using calibrated display	Single display and organ system; still ROIs and not full slides
D’haene et al., 2013[102] (Note: Replication of earlier abstract by Sharma et al.)	Compared microscope, two COTS displays, one PG display, and one MG display in six performance measures on a selection of surgical and cytological WSI	Image quality, speed, and absence of pixelation were superior on the MG display	COTS and PG displays selected were inferior to MG in resolution, size, and luminance. Measures chosen were subjective
Campbell et al., 2014[94]	Assessed pathologist diagnostic concordance on 85 breast biopsy WSIs at ×20 versus ×40 magnification, while using light microscopy, a 1.3 MP COTS display, or a DICOM calibrated 4MP MG display	No significant difference in diagnostic concordance rates between lower resolution COTS display and higher resolution, DICOM calibrated MG display	Single-organ system, only tested differences in concordance between light microscopy and WSI using each display type without direct display comparisons
Kimpe et al., 2014[75]	Compared COTS display calibrated to sRGB with MG displays calibrated to either DICOM GSDF or CSDF in terms of color contrast of cytologic features from background	MG displays showed greater color contrast than COTS display. CSDF calibration offered greater color contrast than GSDF or RGB calibration	COTS display chosen was inferior in panel type, resolution, size, luminance, and contrast ratio. COTS display was only calibrated to sRGB target, while MG displays were only calibrated to DICOM targets. COTS display was set to lower luminance value to reflect 1 year of use
Avanaki et al., 2015[101]	Assessed intersession agreement of pathologists using surgical and cytological images with and without artificially lowered luminance and chromaticity to reflect display aging	Pathologist intersession agreement decreased when observing the “aged” images relative to the original images	Image manipulation was performed artificially using software rather than actual display aging
Randell et al., 2015[95]	Compared pathologist performance when viewing slides on a microscope, a single display, and a high-resolution, multidisplay setup	Higher resolution display setup led to decreased time to diagnosis and time to complete a searching task compared to single display. Display type did not significantly alter diagnostic accuracy or confidence in diagnosis	Participants were exposed to only three cases, all of which were axillary lymph nodes from breast surgical resections
Campbell et al., 2015[97]	Assessed pathologist performance when viewing breast biopsy WSI on a gray-scale display as compared to original microscopic diagnosis	Comparable performance (92.7% concordance) was seen when using a gray-scale display	Performance was not compared with WSI viewed on color displays
Marchessoux et al., 2016[88]	Assessed the number of interactions (zoom and pan) performed by three pathologists when viewing cytology and histology WSI cases on 2 MP, 4 MP, and 12 MP displays	Reduced number of zoom and pan interactions when using higher resolution displays	Accuracy, time to diagnosis, and pathologist blinding/washout protocol not reported
Norgan et al., 2018[103]	Compared pathologists’ mitotic counts in melanoma specimens and ability to detect Helicobacter pylori in gastric biopsies when using either MG or COTS display	Comparable concordance rates for pathologists using either the MG or COTS display as compared to original microscopic diagnosis	Evaluated only two specific tasks in surgical pathology

ROI: Regions of interest, WSI: Whole-slide imaging, COTS: Consumer off the shelf, PG: Professional grade, MG: Medical grade, DICOM: Digital Imaging and Communications in Medicine, RGB: Red-green-blue, sRGB: Standard RGB, GSDF: Gray-scale display function, CSDF: Color standard display function

Impact of the pixel pathway

Unfortunately, in contrast to the freedom and flexibility offered to the pathologist in microscope selection, the current FDA mandate to adhere to a closed “pixel pathway” locks a practicing pathologists into using the specific (MG) display cleared with their WSI system, unless they wish modify the base system in an off-label fashion and validate it as a laboratory developed test (LDT). Given that the specifications for the displays provided with either FDA-cleared WSI system were already a bit dated at the time of each system's clearance, it is safe to say that these systems do not provide the most up-to-date technology available for digital pathology.[879091] Further, it can be argued solely, based on the supplied vendor specifications, that many comparable MG, COTS, and PG displays could be easily substituted into the pixel pathway without issue.

It is worth noting that the end product of the closed systems cleared by the FDA does not include all potential components comprising the pixel pathway. For instance, while a specific model display is designated as one of the required devices within the FDA-cleared WSI system, a specific model computer workstation and the IT hardware/software hosting the image management system (servers, operating systems, network, etc.) are not so included.[879091] Therefore, one could argue that, because the quality of the displayed image is only as good as the rate-limiting step of the entire pathway, the digital images viewed by the pathologist could be degraded by any component of the pixel pathway, such as inferior network performance, degraded signal cables, graphics adapters, workstations, or even improper configuration of one of the required components of the system.

Further complicating the discussion regarding the role displays play in digital pathology is the topic of what constitutes a major or minor change in the “system,” regardless of whether one implements an FDA-cleared versus an LDT-validated WSI system for primary diagnosis.[16466] Specifically, it is currently unclear what constitutes a significant change in the system and when the lab should revalidate.[6467] Most would interpret current regulatory guidelines to stipulate that a major change in the WSI scanning device itself would meet the threshold of necessitating revalidation, but what about a change in the PC workstation (e.g., performing an operating system update, upgrading a graphics adapter, or even updating a hardware driver)? What about a software update for the image viewer or even the next generation version of the display? Ultimately, the adequacy and validation of any digital pathology setup should be approved by the primary Clinical Laboratory Improvement Amendments (CLIA) laboratory director, prior to clinical use, in addition to reviewing the workflow with one's local compliance and medicolegal groups. One of the author's institutions, as a part of its CLIA validation process for intraoperative consultation telepathology, opted to only require minimum specifications for the display rather than requiring a specific model or brand.

While displays can be compared based on technical specifications, the ultimate question is whether these specification differences result in changes in the diagnostic performance of pathologists. Indeed, the authors can attest from panel discussions at digital pathology conferences with official FDA representation (e.g., Pathology Visions 2015 and 2017); the FDA initially required specific display models for WSI system clearance due to insufficient data in the literature to safely conclude that the display, in fact, did not have an effect on pathologist diagnostic performance. By that same reasoning, they also determined that the pathologist should be the end point of the validation process for the initial implementation and subsequent major changes to digital pathology systems.[92]

A review of the available digital pathology literature regarding displays reveals a number of different metrics that have been used for comparison of different displays. These include subjective measurements of display quality (color, brightness) as well as more objective, yet disparate, measures such as eye movement tracking, time to diagnosis, sensitivity and specificity for lesion detection, and diagnostic concordance (although one could argue that diagnostic concordance is itself a subjective measure given its reliance on human interpretation). A lack of standardized metrics for display comparison, as well as the ever-increasing number of options in display technology, makes conducting formal meta-analyses difficult. By comparison, the radiology literature is rich with examples of primary display studies, recommendations on display selection, and formal guidelines from radiology professional societies, with regard to minimum display specifications for clinical diagnostic use.[6131415161718192026272893] As an example of the latter, a working group between the American College of Radiology, American Association of Physicists in Medicine, and the Society for Imaging Informatics in Medicine jointly publishes guidelines for display assessment and quality control for diagnostic displays in radiology.[27] Notably, specific recommendations are given for minimum luminance, contrast, and pixel pitch-three objective and quantitative specifications that digital pathology, as a field, could begin investigating for clinical diagnostic use.

Impact of resolution and multiple displays

The impact of display resolution on diagnostic accuracy is unclear, with one study showing no difference in diagnostic concordance with light microscopy when using a higher resolution display.[94] However, multiple displays and systems with higher resolution may have an impact on pathologist speed and efficiency. One study comparing three displays with varying resolution (2, 4, and 12 MP) showed a substantial reduction in the number of zooming and panning actions when viewing histology and cytology cases.[88] In another study on WSI performance comparing light microscopy with either a single display or with multiple displays found that pathologists were able to find regions of interest faster using three displays rather than one.[95] The authors pointed out that while 67% of the pathologists subjectively preferred the three-display configuration, some participants expressed concern that a clinically relevant area could fall between the viewable region seen on each display, despite the field of view of the slide being completely rendered on the displays (i.e., no areas were actually “dropped”).[95] In this use case, pathologists may actually benefit from wider displays (e.g., a 20:9, 34” or a 32:9, 49” ultrawide display), from multiple display configurations using displays with very thin bezels (borders), or even future potential custom display configurations based on the use of microLED modular displays (without bezels).

Impact of color in histopathologic diagnosis

Digital pathology differs greatly from the prior primary use cases in medical imaging (radiology) in that histopathological images are natively color based and not gray scale. Attention has been drawn to the importance of accurate color reproduction in WSI and steps to ensure or improve upon it.[7896] However, it was not until relatively recently that pathology articles and textbooks were regularly printed in color and a portion of currently practicing pathologists trained, at least partially, with gray scale images as the norm. In addition, a few studies have been published on color blindness in pathologists. One controversial study showed a mild, but significant difference between pathologists with color vision deficits and those with normal color vision in various tasks, such as the detection of amyloid and acid–fast bacilli.[84] Certain educational programs have worked around this by supplying gray-scale displays to students.[86]

Despite the regular utilization of color-based stains, many histopathological diagnoses and grading schema are contingent upon recognizing architectural, rather than strictly color-based differences. An example of this can be seen with Gleason grading of prostatic adenocarcinoma.[85] Some studies involving pathologists with normal color vision argue for a less important role for color in pathology. In 1997, Doolittle et al. illustrated this fact by showing that pathologists had a subjective preference for 8-bit as opposed to 24-bit color images.[89] A more recent study showed a 92.7% concordance rate for pathologists viewing WSI scanned breast biopsy specimens on 16-bit gray-scale displays.[97] It should also be noted the color rendering of even frequently used hematoxylin, eosin, and immunohistochemical stains can vary considerably between laboratories, staining instruments, or intra-laboratory batches for a given stain.[98]

Studies have shown differences in display color by means of measuring deltaE2000 values with a colorimeter. One such study highlighted the difference in deltaE2000 values between a MG display set to the GSDF standard, a MG display set to the CSDF standard, and a COTS display. The deltaE2000 values were different both for pair-wise comparison of individual pixels and color contrast between clinically important foreground elements and the surrounding background.[7599] Although this illustrates that there is a difference in color as perceived from different displays by a detection device, real-world clinical performance by pathologists on the displays was not assessed. In addition, the COTS display chosen had limited resolution and contrast compared to the MG displays (1280 × 1024 vs. 3280 × 2048, respectively). Another study compared the performance of two identical displays, one color-calibrated display and the other non-calibrated, by showing 250 fields of view (512 × 512 pixels each) selected from 93 breast biopsy specimens to six pathologist observers. No difference was found in diagnostic accuracy, however there was a significant difference in time to diagnosis (4.9 s vs. 6.3 s for calibrated and noncalibrated displays, respectively).[100] Another study design altered the luminance and color of surgical pathology and cytology images to simulate display aging – this study showed a statistically significant difference in intersession agreement between the altered images and those remaining unaltered.[101] Based on the above studies, color depth and gamut may play a role in digital pathology clinical performance, however to what degree is unclear.

Impact of consumer-off-the-shelf versus medical-grade displays

An early (2013) study compared the subjective impressions of pathologists when using different displays of different categories, including a 14.1” IBM Thinkpad laptop, a COTS display (HP ZR24w 24”), two different MG displays (Eizo FlexScan SX2462W 24” and BARCO MDCC-6130 30.4”), and an optical microscope, to view surgical pathology and cytology cases. Significant differences in favor of the MG displays were observed in subjective measures (pathologist opinions) of image quality, speed, and pixelation.[102] A more recent (2019) study had readers performing multiple tasks under various ambient light conditions, such as distinguishing a letter in the foreground from a similarly colored background and identifying a nucleus from a tubular adenoma. Diagnostic accuracy was higher with the MG display; however, technical specifications of the two displays were not equivalent, with the MG having substantially higher resolution and screen size as compared to the COTS display (4096 × 2160, 31” vs. 1600 × 1080, 24”).[37]

In contrast, two studies showed no difference between a MG and a COTS display. One study assessed discordance rates between light microscopic and WSI diagnosis on breast biopsies using either a DICOM calibrated MG display or a noncalibrated COTS display.[94] The other assessed the detection of Helicobacter pylori and the performance of mitotic counts.[103] Further, in the UK and the Netherlands, digital pathology system validations for primary diagnosis have been performed using COTS displays.[65104]

DISCUSSION

One fundamental issue in the evaluation of digital pathology displays is the lack of a strong core base of primary literature assessing the performance characteristics of different display types in this field. Further complicating this is the observation that, of the small number of articles published on the topic [Table 3], the majority are from 2015 or earlier, a fact hampering informed decisions when comparing modern display technologies for digital pathology. Going forward, we recommend that the technical specifications of luminance, contrast, color accuracy, resolution, and just noticeable difference (deltaE2000) are areas in need of further study, with emphasis on how they differ between MG, COTS, and PG displays of similar/equivalent specifications. These technical parameters are not only more objective than pathologist performance alone, but also more easily measured than the current validation standard of using multiple pathologist diagnostic concordance across a wide range of displays. Ultimately, once enough objective observations exist, recommendations can be made from concrete, reproducible data regarding minimum display specification requirements for digital pathology diagnostic use. This assertion is further strengthened by the observation that the field of radiology has largely addressed its own challenges in display device diagnostic performance by switching to these same technical parameters, while at the same time deprecating the involvement of human cognitive interpretive performance.

Being critical of this approach, we fully acknowledge that any study seeking to objectively compare display technical parameters may struggle to find truly equivalent MG, COTS, or PG displays. We explained above that modern displays can have 20 to 30-plus different specifications, and even when efforts are made to ensure maximal compatibility between two displays, they may differ in some unknown aspect that creates the opportunity for hidden variables. However, while prior studies did indeed demonstrate significant intrinsic discrepancies between COTS and MG displays, these were fundamentally unfair comparisons, given the substantially higher average performance specifications associated with MG displays at that time. Fortunately, substantial technological advances in flat-panel display technology now make display comparison studies more feasible, as there are many MG, COTS, and PG displays of similar specifications on the market.

Another obstacle to choice in digital pathology displays is the persistence of the FDA-defined pixel pathway. With respect to this, though, evidence is emerging that the closed nature of the pixel pathway is beginning to open up. Section 864.3700 of the Code of Federal Regulations describes WSI systems and designates them as class II devices, and originally, only one product classification, “Whole Slide Imaging System” (code: PSY), was associated with this regulation.[105106] This class defines the end-to-end pixel pathway which “consists of an image management system, a scanner and associated software, and a display monitor.” The Phillips PIPS (Royal Philips; Amsterdam, The Netherlands) system and the Leica AT2 DX (Leica Biosystems; Buffalo Grove, IL, USA) are both classified as WSI systems and specify a complete pixel pathway including displays.[91107] The FDA has since created two additional product classifications that represent portions of the pixel pathway. One of these, “Digital Pathology Image Viewing and Management Software” (code: QKQ), is a software device and “.intended for viewing and management of digital images….” The Sectra Digital Pathology Module recently received 510k approval under this product classification.[108] While the Sectra Digital Pathology Module submission was only for use with the Aperio AT2 DX scanner, future submissions may gain a broader indication, promoting componentization in the pixel pathway.

The final 864.3700 product classification is “Digital Pathology Display” (code: PZZ), which is “intended for in vitro diagnostic use to display digital images of histopathology slides acquired from FDA-cleared whole-slide imaging scanners.”[106] This product classification appears to be analogous to the “Display, Diagnostic Radiology” classification (code: PGY) which currently lists 79 approved devices. The first 510(k) approval for a “Digital Pathology Display” was the Barco MMPC-4127F1, approved on December 21, 2017, and intended “to be used to view scanned digital images of formalin-fixed, paraffin-embedded tissue slides, using IVD-labeled (in vitro diagnostic) WSI systems.”[109] Notably, the Barco MMPC-4127F1 is not tied to any particular WSI system and like the PP27QHD, its approval required only technical performance testing.[90110] It is worth noting that devices in the “Digital Pathology Display” product class are eligible for the FDA's 3P510k Review Program, which provides an alternative review process for certain low-to-moderate risk medical devices.[111] This program permits manufacturers to use accredited third-party review organizations, which review 510(k) submissions and make recommendations to the FDA.

Muddying the MG versus COTS/PG display waters even more is the relaxation of enforcement policies for both remote review of pathology slides and remote digital pathology devices by CMS and the FDA, respectively, in response to the (SARS-CoV-2, COVID-19) public emergency. On March 26, 2020, the CMS issued a memorandum adopting a “.temporary policy of relaxed enforcement in connection with laboratories located at temporary testing sites.” allowing pathologists to be able to perform primary diagnosis at remote sites through the use of innovative technologies without the need of a separate CLIA certificate or the declaration of temporary sites (only for the duration of the COVID-19 public emergency).[112] Of note, this memorandum did not relax standards around the CLIA validation of FDA-cleared or LDTs, but instead allowed for the primary CLIA certificate and its associated testing to be extended to multiple temporary testing sites (i.e., pathologist homes) while still requiring proper laboratory practices to be followed.

Following the CMS memo, the FDA published a guidance document on April 24, 2020, relaxing enforcement policies on multiple digital pathology devices in order to recognize that “.greater access to remote digital pathology devices may help facilitate the remote reviewing and reporting of pathology slides…” during the COVID-19 public emergency.[113] Multiple components of the pixel pathway for primary diagnosis were referenced, including the Whole Slide Imaging System (code: PSY), the Digital Pathology Image Viewing and Management Software (code: QKQ), and the Digital Pathology Display (code: PZZ).[106113] For these digital pathology devices, the FDA decided to not object to: (1) modifications to the indications for use, functionality, hardware, and/or software of the existing FDA-cleared systems to better allow their remote use and (2) the marketing of new digital pathology devices, not currently 510(k) cleared for any use, intended for use in remote settings.[106113] Further, the FDA cited their belief that these devices would not create an undue risk as long as instructions specific to remote use, related to the performance and product labeling of the device, were present.

Important to this discussion, the FDA provided as an example of proper instructions for remote digital pathology device use that pathologists should “.use their clinical judgment to determine whether the quality of the images from the remote digital pathology devices are sufficient for interpretation of the pathological images.”[113] One can infer from this statement that the FDA is tacitly allowing pathologists to approve not only the quality of the displayed images, but also indirectly, the image-viewing software and display themselves for remote clinical use (which, the reader may make note, is in essence the entire second half of the pixel pipeline). Given the odds of a pathologist having an FDA-cleared MG display at home being extremely low, this guidance can be taken as further affirmation that a “locked-down” pixel pathway is no longer an absolute requirement in the clinical use of digital pathology technology. One vendor went so far as to release a new technical document specifically addressing the expanded use of their FDA-cleared WSI system for remote primary diagnosis, including both minimum technical specifications for acceptable displays and a list of alternate recommended MG, PG, and COTS display models.[114] Indeed, when combined, the relaxed remote digital pathology enforcement policies of both CMS and the FDA only further strengthen the argument that the display type itself (and its associated presence or absence of a medical device designation) may not matter. Efforts instead should be made to enable proper labeling, instructions for use, laboratory validation, and clinical judgment, for each separate pixel pathway component.

The pixel pathway is currently in a state of both technical and regulatory flux. Display technology continues to innovate, with image quality rapidly approaching (and perhaps exceeding) the resolution limit of the human eye, while the regulatory environment has become (in the authors’ opinion) overly conservative in the absence of firm scientific data. With the COVID-19 public emergency forcing flexibility in the FDA's stance on digital pathology devices, the opportunity now exists for pathologists to become more vocal in the regulatory, scientific, and clinical aspects of the pixel pathway, and similarly, far more proactive in bringing about evidence-based transformation as to how the field validates and subsequently deploys candidate display solutions.

CONCLUSIONS

In 2013, the authors of a European WSI validation study noted their preference for COTS displays, stating “It has been found that the pathologists of our department find that the standard desktop displays offer sufficient quality for case reviewing although there are several medical-grade diagnostic displays available throughout the Department.”[104] Based on our review of the limited pathology display-oriented literature, the increased quality of COTS and PG displays, and the recent COVID-19-related relaxation of FDA enforcement of the pixel pathway components for digital pathology, there appears to be a strong argument for deconstructing the pixel pathway, uncoupling it from the WSI scanner apparatus validation process, and increasing the options available to diagnostic pathology for suitable display selection. Indeed, evidence from older display studies are far less relevant, given the widespread release of COTS and PG displays with significantly improved specifications, such as elevated luminance levels, 4K or higher resolution, ≥10-bit color depth, extremely wide color gamut, faster refresh rates, and, finally, their augmentation with high-end features such as self-calibrating sensors, DICOM-GSDF calibration, and adaptive frame synchronization.

As pathology continues its journey from what has been an analog medium to the digital domain, pathologists will need familiarity with the new tools of their trade. For pathologists, we feel that getting to know the fundamental properties of displays is a beneficial step along that path. Ultimately, locking a digital pathology system to a specific display introduces unnecessary friction when upgrading a system as new technology becomes available. Given the important role displays will play in pathology, instead of evaluating all potential display and WSI system combinations in a piecemeal fashion, we recommend pathologists take the lead to establish baseline minimum display specifications for our future “microscope.”

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.