Beyond brightfield: a possible future of slide scanners.

Many diseases involve changes in gross anatomy and microanatomy; the latter is typically viewed on hematoxylin and eosin-stained tissue sections that are central to patient diagnosis and management. Such slides are viewed using brightfield transillumination, usually using standard desktop microscopes but increasingly following whole-slide digital imaging. More information can be available if such slides are imaged with alternative methods to brightfield alone. Our process of dual-mode emission and transmission microscopy, which combines brightfield with fluorescence, can reveal novel contrast. This simple, cost-effective and deployable technology may also serve as a data source that could empower future artificial intelligence-powered digital pathology tools.

Histopathological microscopic examination of tissues is employed almost universally to determine primary diagnoses and to guide patient management; in almost every case, hematoxylin and eosin (H&E)-stained slides are the centerpiece of pathology and constitute the gold-standard method. Direct viewing of these slides using a microscope (the iconic depiction of the pathologist at work) is slowly but seemingly inevitably giving way to the new world of digital pathology. Rather than physical slides being delivered immediately to diagnosticians, the slides are instead first imaged using a whole-slide scanner at diagnostically appropriate magnifications (usually 20 × or higher). This process involves scanning the H&E-stained slides in transillumination (brightfield) mode, creating digital images that can then be directly viewed by a pathologist on a computer screen in an environment that streamlines case management, image sharing and consultation. More useful than simply replicating the experience of direct viewing, having the histology images in digital format facilitates the deployment of computer-based tools: image processing and enhancement and straightforward assessment of quantifiable metrics (area, intensity, molecular marker intensity and distribution) as well as applications of novel machine learning and artificial intelligence (AI) tools. However, to date, virtually all such H&E whole-slide scans have been accomplished only in brightfield mode.

But there is more to life (or histology) than just absorbance contrast signals generated by H&E stains. Notably, there are specific structural elements of great importance that can best be demonstrated using histochemical stains such as trichrome (for collagen), periodic acid–Schiff (for basement membrane), Verhoeff (for elastin) and so on. Some of these so-called special stains are widely used, especially in evaluation of liver and kidney diseases. However, they do involve extra effort, reagents, cost and the preparation of additional slides because they are not performed on the original H&E-stained section. This inevitably means that the image information is not pixel-matched with an image acquired from the H&E slide, and potential registration errors caused by differences inherent in the use of serial sections (if these are even available) could complicate their full utilization by AI-based multisource analysis.

Alternatively, other optical methods beyond brightfield transillumination are under development and are designed to extract more information than what is obtainable from standard views of H&E-stained slides. These include, but are not limited to, quantitative phase imaging [1], near- or mid-infrared absorption imaging [2], brightfield spectral imaging [3], fluorescence lifetime imaging [4] and second harmonic generation [5]. All of these methods reveal potentially important novel contrast and content; some can provide not just structural but also biochemically grounded insights, with the potential to provide information about the metabolic changes occurring within tissues that has possible prognostic import.

However, as with the special stains mentioned above, some of these methods depend on costly optical imaging technologies, involve slow scan speeds or may be optically incompatible with standard glass slide supports. These drawbacks may interfere with transition to use in real-world clinical environments where expense and sample throughput are paramount.

We have observed, mostly by chance, that simple fluorescence imaging of H&E-stained slides can provide a great deal of useful histology content rapidly and cost-effectively [6]. While previous work [7] had noted that fluorescence excitation can generate high-quality contrast that highlights the presence of collagen (Figure 1), for example, it turns out that spectral analysis tools can extract a pure collagen signal away from background with a distribution and pattern that correlates closely with what can be visualized either by chemical stains or second harmonic microscopy. While the fluorescence images usually appeared largely green to the eye, with limited color variation, the application of spectral phasor mathematics revealed the presence of several distinct spectral classes, even when applied to simple color images acquired with a red/green/blue sensor. Going beyond simple red/green/blue images, when multispectral data were acquired using additional optics such as filter wheels or tunable filters, further distinct signals became evident and corresponded to additional physical entities within the specimen.

Figure 1.

Image of hematoxylin and eosin-stained section of human liver imaged under (A) brightfield and (B) fluorescence mode and (C) extracted collagen from fluorescence image. Note the collagen fibers, which are much easier to detect on fluorescence mode. Colors are stretched for better visibility.

It is straightforward to put together a scanner that can conveniently acquire paired brightfield and fluorescence images in area-scanning mode, a process we termed DUET, for DUal mode Emission and Transmission. Our in-house scanner can rapidly scan H&E-stained slides in both modalities within 2–4 min. With the use of agile, spectrally informed tools, several components beyond collagen, including basement membrane and elastin, can be distinguished, extracted and quantitatively evaluated and scored.

There are future applications to be imagined. We expect that DUET images may be particularly useful when paired with AI-based discovery and analysis techniques. Machine learning algorithms or other approaches might be able to take advantage of the enhanced information content and come up with better diagnostic or prognostic readouts than can be obtained when the input is restricted to just brightfield H&E signals. If slide scanners can finally provide images more informative than those pathologists can see through the microscopes on their desks, this may be a further impetus to help move slide-based diagnostics to a digital, and perhaps DUET-powered, world.