Real-time multispectral fluorescence lifetime imaging using Single Photon Avalanche Diode arrays.

Autofluorescence spectroscopy has emerged in recent years as a powerful tool to report label-free contrast between normal and diseased tissues, both in vivo and ex vivo. We report the development of an instrument employing Single Photon Avalanche Diode (SPAD) arrays to realize real-time multispectral autofluorescence lifetime imaging at a macroscopic scale using handheld single-point fibre optic probes, under bright background conditions. At the detection end, the fluorescence signal is passed through a transmission grating and both spectral and temporal information are encoded in the SPAD array. This configuration allows interrogation in the spectral range of interest in real time. Spatial information is provided by an external camera together with a guiding beam that provides a visual reference that is tracked in real-time. Through fast image processing and data analysis, fluorescence lifetime maps are augmented on white light images to provide feedback of the measurements in real-time. We validate and demonstrate the practicality of this technique in the reference fluorophores and in articular cartilage samples mimicking the degradation that occurs in osteoarthritis. Our results demonstrate that SPADs together with fibre probes can offer means to report autofluorescence spectral and lifetime contrast in real-time and thus are suitable candidates for in situ tissue diagnostics.

Introduction

Optical spectroscopy techniques such as fluorescence, Raman or white light diffuse reflectance have been widely investigated and demonstrated potential for tissue interrogation and surgical guidance in many clinical applications[1-10]. In particular, autofluorescence measurements exploit the fluorescence signatures of endogenous molecules to report the biochemical and structural fingerprints of tissues and thus aim to provide label-free contrast between normal and diseased regions, either by means of single-point spectroscopy[11-13] or imaging[14-16].

To date, most clinical studies of tissue autofluorescence are realized in the steady-state regime, by measuring differences in the spectrally-resolved fluorescence signals of normal and diseased tissues. Changes in the autofluorescence spectra of tissues owing to pathological transformations have been reported in osteoarthritis[17], liver fibrosis[18,19], cardiovascular applications[20,21] and many forms of cancer, including in breast[22], cervix[23], brain[24] and skin[25]. While these studies present encouraging results, the specificity of spectrally-resolved autofluorescence measurements is generally limited by the broad overlapping spectra of many tissue fluorophores, such as elastin, collagens, NAD(P)H and flavins[26]. One common method to overcome this limitation is to resolve the autofluorescence measurement both spectrally and temporally, thus increasing the dimension and specificity of the autofluorescence output. Specifically, time-resolved autofluorescence measurements can provide differentiation between endogenous fluorophores with overlapping emission spectra but different fluorescence lifetimes.

For clinical applications, spectral resolution in fluorescence lifetime measurements is typically achieved using a set of dichroic mirrors and optical filters that separate the fluorescence signal in large discrete spectral bands. The resulting spectrally-resolved fluorescence can then be directed to multiple photodetectors[27,28] or through optical fibres of different lengths that temporally separate the signal from different wavelength bands for detection with a single photomultiplier tube (PMT)[29-32]. Alternative configurations include using monochromators[33-35], multi-anode PMTs[36-38] or a filter wheel[39]. Each detection configuration has its own merits and limitations and, thus, their implementation must consider the exact clinical application. For example, while monochromators can provide excellent spectral resolution, they are not suitable for a real-time implementation requiring multispectral lifetime acquisitions, given the long integration times typically required per spectral band to achieve reasonable signal-to-noise ratio (SNR). In this context, large spectral bands are preferred at the expense of spectral resolution. In configurations with multiple discrete detection channels, each with relatively large bandwidth, the spectral range of each channel is defined by the optical transmission of dichroic mirrors and emission filters. This configuration determines that the spectral range and resolution of each channel are fixed and can only be modified by changing filters and/or dichroic mirrors in the detection system. Accordingly, these instruments are typically configured to include large detection channels that overlap with spectral regions where common endogenous fluorophores emit most strongly. This strategy necessarily limits the specificity and versatility of the instrument: if additional spectral range or resolution is required, the entire detection system would have to be modified.

In this context, Single Photon Avalanche Diode (SPAD) arrays have emerged as attractive alternatives to common multispectral lifetime strategies, given that spectral and temporal resolution to the fluorescence measurement can be provided and adjusted at the sensor level using a fixed optical detection setup[40-42]. While the base technology is still at relatively early stage of development, recent studies have demonstrated the potential of SPADs in biomedical research, including super-resolution microscopy[43], measurements of protein-protein interactions[44] and molecular-based diagnosis[45]. In this study, we extend the application of SPAD arrays to single-point measurements, to realize multispectral fluorescence lifetime imaging using fibre optic probes. Specifically, this study expands our previous work[46], where we demonstrated the feasibility of generating fluorescence lifetime maps from single-point measurements at a macroscopic scale and under bright illumination of the specimen using Time-Correlated Single Photon Counting (TCPSC) instrumentation. Here, we explore the versatility of SPAD arrays to realize multispectral fluorescence lifetime acquisitions in real-time using a single-point approach. To enhance potential for clinical deployment, fluorescence intensity and lifetime data are overlaid on white light images acquired with an external RGB CMOS camera, thereby providing spatial and fluorescence feedback in real-time. We present a comprehensive description and characterization of this method, including its validation in articular cartilage samples presenting local degradation following enzymatic digestion of collagen. We believe this technique can provide rapid multidimensional information that could aid tissue diagnosis and thus could have significant impact in many clinical applications.

Methods

Experimental setup

Optical setup

Experiments carried out in this study were realized using a custom-built fibre-based multispectral fluorescence lifetime imaging setup (see Fig. 1a) and partially described elsewhere[42]. Excitation light at 375 nm was provided by a laser diode (BDL-SMN-375, Becker & Hickl GmbH, Berlin, Germany) that produced short optical pulses (<100 ps) at 50 MHz repetition rate. Excitation light was delivered to the sample via a 200 μm core diameter optical fibre embedded in a quadrifurcated fibre bundle (NA = 0.22, length = 4 m, EMVision LLC, Loxahatchee, FL, USA). The average power at the sample plane was kept below 25 μW at all times during measurements, which is below the maximum permissible exposure (MPE) for skin and eye. At a probe-to-target distance of 3 mm, this corresponds to a power density of less than 5 mW/cm[2], which is similar or less than that used for excitation in previous in vivo or ex vivo investigations, using similar excitation strategies[36-38,47]. In none of these studies alterations in the biological characteristics of the sample as a result of laser excitation were reported. The fluorescence signal emanating from the sample was collected by seven 300 μm 0.22 NA optical fibres and delivered to the detection system. In the detection system, fluorescence light was dispersed by a transmission grating (GT50-06V, Thorlabs, Newton, NJ, USA) and the resulting spectrally-resolved fluorescence signal imaged with 1:1 magnification across the long axis of a SPAD array (SPC3, MPD, Bolzano, Italy) consisting of 2048 pixels arranged in 32 rows by 64 columns. A 400 nm long-pass filter (FEL0400, Thorlabs) was added to the emission path to prevent excitation light to reach the detector. We chose this optical configuration given its simplicity and broad wavelength range (400–650 nm) for detection.

Figure 1

(a) Optical layout of the multispectral fluorescence lifetime imaging system. Fluorescence light is dispersed and spectrally-resolved across the long axis of the SPAD array using a transmission grating comprising 600 grooves/mm. L1: Aspheric lens, f = 11.0 mm (A220TM, Thorlabs); L2 and L3: Bi-Convex Lens, f = 60.0 mm (LB1723-B, Thorlabs). (b) Fluorescence emission spectra of reference samples measured with a Horiba microHR monochromator and a Syncerity CCD camera (solid lines) and our instrument (crossed dashed lines). (c) Instrument response function and representative fluorescence intensity decay of a reference fluorophore (50 μM POPOP in ethanol, τ = 1.3 ns).

Augmented fluorescence lifetime and spectral imaging in real time

To generate autofluorescence lifetime and intensity maps out of single-point measurements, we used the aiming beam approach that was previously described by others[30] and expanded by our group in the context of TCSPC measurements[46]. In brief, fluorescence measurements are realized by moving the fibre probe freely across the specimen. A second light source - a continuous-wave (CW) 785 nm laser diode (FC-785-350-MM2-PC-0-RM, RGBLase, Fremont, CA, USA) - is used simultaneously with the 375 nm excitation light, within the same fibre bundle but through an adjacent 200 μm core. The localization of the fluorescence excitation and detection is done by tracking the 785 nm beam in real time using a colour camera (DFK 33UP1300, The Imaging Source, Bremen, Germany). To prevent contamination of the fluorescence signal with 785 nm light, a 700 nm short-pass filter (ET700SP, Chroma Technologies, Bellows Falls, VT, USA) was also added to the detection path.

A significant limitation of photon counting approaches within the visible region of the spectrum refers to their inability to discern fluorescence from background photons (e.g. ambient room light). To overcome this limitation, instead of ambient light, a modulated LED (LED, MNWHL4, Thorlabs) was used to illuminate the field-of-view (FOV) of a sample. The modulation was performed at 50 Hz and 10% duty cycle, resulting in 2 ms illumination time, with an illuminance of approximately 200 LUX. At this frequency, the stroboscopic effect produced by the on-off switching of the LED is not perceived by the human eye and the FOV will appear to be continuously illuminated to the operator. In turn, fluorescence lifetime and spectral measurements are realized at 50 Hz, but within the non-illuminated cycle of the LED, i.e. in the remaining 18 ms. This method guarantees that fluorescence measurements are free from background light originated by the pulsing LED and, thus, from the operator’s perspective, are realized under bright illumination of the FOV. Finally, triggering the colour camera simultaneously with the LED will produce well-illuminated white light images at 50 Hz (with ~2 ms exposure time), which can be used for determining the interrogated point in real-time, as described above. Given that camera and fluorescence acquisitions are synchronized, each fluorescence measurement is associated with a position of the fibre probe. In consequence, fluorescence maps can be generated to provide visual feedback of the measurements in real-time. One particular highlight of this method is that spectral bands can be selected and tuned on-the-fly during processing, so that online feedback is provided for a specific region of the spectrum, and the spectral band can be changed if necessary.

SPAD array and photon time-gating strategy

The sensor used in our experiments is an array of 32 × 64 SPADs (4.8 mm × 9.6 mm) developed using 0.35 µm complementary metal-oxide semiconductor (CMOS) technology. The pixel size is 150 µm × 150 µm, and the SPAD diameter is 30 µm, which results in a fill-factor of 3.14%. The detector features a peak Photon Detection Efficiency (PDE) of 50% at 450 nm and a median Dark Counting Rate (DCR) of 150 cps per pixel. The dead time in each pixel is 50 ns. A complete characterization of the SPAD detector is provided in references[48,49]. Histogram of photon arrival times are generated independently in each pixel by means of time-gating[50,51], at a fixed repetition rate of 50 MHz, which was used for synchronization with the excitation laser. Measurements of the instrument response function (IRF) and fluorescence intensity decays of a fluorescence standard using different gating configurations are illustrated in Supplementary Fig. S1.

Considering the specificities of our implementation, as described above, the specific photon time-gating strategy must balance a number of parameters, namely: (1) photon integration time, since a single fluorescence acquisition must necessarily occur within the non-illuminated period of the white LED (i.e. maximum of 18 ms at 50 Hz with 10% LED duty cycle); (2) photon collection efficiency; (3) temporal resolution. Since the integration time of our application is limited, in order to improve photon collection efficiency while maintaining reasonable sampling of the fluorescence intensity decays, we chose a detection strategy employing long overlapping gates (4 ns width). The detection gate scanned the fluorescence decay in 400 ps steps, for a total acquisition window of 16 ns and 40 sampling points. All measurements presented in this study were realized with these gate settings. Representative fluorescence lifetime and IRF curves are shown in Fig. 1c. For the gate settings used in this study, the measured IRF full-width at half maximum (FWHM) was 4.30 ± 0.04 ns.

Data analysis

Fluorescence lifetime data

To provide feedback of the measurements in real-time, fluorescence lifetime data are analysed immediately upon each single acquisition using the modified phasor approach for time-gated fluorescence lifetime measurements, which is described in detail elsewhere[52]. In brief, each measured fluorescence intensity decay is Fourier Transformed to obtain the corresponding phasor coordinates g and s. In practice, the measured fluorescence signal is a convolution of the sample’s fluorescence response and IRF. The contribution of the IRF can be accounted for by calibrating the measurement using a reference sample, which corresponds to an additional step of rescaling and rotation in the phasor transformation. In our measurements, we used the back-reflected signal measured from excitation light as calibration (τ = 0 ns). Following phasor transformation and calibration, the characteristic phasor of single-exponential emitters falls on the universal circle. A mixture of two fluorophores with distinct single-exponential characteristics lies inside the universal circle, as a linear combination of each individual component. Hence, any combination of the two fluorophores falls along a line connecting their characteristic single-exponential phasors. From the coordinates g and s, phase (τ) and modulation (τ) lifetimes can be calculated. In the measurements below, we report τ, which is given by τ = tan φ /ω, where φ is the angle between the phasor vector and the x-axis (i.e. tan φ = s/g), and ω is the angular frequency (ω = 2πf), where f is the laser repetition rate, i.e. 50 MHz).

Given the high repetition rate of the excitation laser (50 MHz, 20 ns periods) and comparatively short acquisition window (16 ns), the recorded fluorescence decay curves may include contributions from previous excitation periods, particularly for slow decays. Our analysis model does not account for such incomplete recordings, which may lead to slight inaccuracies in the estimation of long fluorescence lifetimes. For two references fluorophores (POPOP in ethanol and flavin adenine dinucleotide (FAD) in purified water) we measured fluorescence lifetimes of 1.36 ± 0.04 ns and 4.02 ± 0.08 ns, respectively. These are comparable to fluorescence lifetimes of 1.32 ± 0.03 ns and 3.79 ± 0.06 ns measured in a TCSPC setup[46].

Fluorescence intensity data

Fluorescence spectra were normalized by dividing the fluorescence intensity measured at each wavelength by the peak intensity. For measurements where data were binned in large spectral bands, the autofluorescence intensity in each band was calculated as a fraction of the total autofluorescence signal, i.e. the sum of all detection channels. Spectral calibration was realized by measuring the back-reflected signal from a 445 nm laser diode (Sacher Lasertechnik GmbH, Marburg, Germany) and LEDs with center wavelength at 470 nm, 530 nm and 630 nm. Spectral measurements of calibration LEDs and laser were initially realized using a gold-standard microHR monochromator (Horiba, Kyoto, Japan) with a Syncerity CCD detector (Horiba) and used to calibrate our custom spectrometer. Spectral measurements of fluorescent slides (Chroma Technologies Corporation, Bellows Falls, VT, USA) were compared and showed a discrepancy of approximately 5 nm between systems, which is equivalent to the spectral resolution of our custom implementation (see Fig. 1b).

Porcine articular cartilage digestion

Articular cartilage specimens (approximate dimensions 3 cm × 3 cm × 3 cm) were obtained from metacarpophalangeal joints of freshly slaughtered pigs and kept in PBS with 0.05% sodium azide for 24 hours to prevent bacterial growth. Samples were thoroughly washed in Phosphate Buffer Saline (PBS) and kept at −20 °C for 24 hours before measurements. Articular cartilage digestion was induced using filter paper soaked in 250 μg/mL bacterial collagenase on the articular surface, as illustrated in Supplementary Fig. S2. Treatment was applied for 5 hours at 37 °C. Samples were thoroughly washed in PBS after treatment to remove any residues of digestive enzyme. Fluorescence measurements were realized before and after treatment over the entire cartilage surface to demonstrate spatial variation of the fluorescence signal.

Results

To demonstrate our implementation in clinically relevant biological tissue, we measured the autofluorescence lifetime and spectral signatures of porcine articular cartilage. Measurements were realized before and after treatment of the articular surface with bacterial collagenase, to induce endogenous contrast derived from the localized digestion of collagen and consequent alteration of its fluorescence characteristics, in an effort to mimic natural degradation occurring in osteoarthritis[47,53]. The photon integration time for each autofluorescence acquisition was set to ~11 ms. For this temporal window, we measured a negligible contribution from background light originating from the LED (see Supplementary Fig. S3), and fluorescence lifetime and image acquisition rates of 24.94 ± 0.03 Hz and 33.33 ± 1.33 Hz, respectively. These acquisition rates are slower that the expected 50 Hz, and this difference results from delays in fluorescence lifetime and image acquisition and analysis that slows down concurrent processes. For 13 ms integration time, we also measured a negligible number of photons associated to background LED light, but the decrease in acquisition rates was more pronounced (see Fig. S4). For integration times longer than 15 ms, there is an increase in the contribution of LED light to the autofluorescence measurement (~20%, see Fig. S3), which also results from delays in data acquisition and processing that cause the fluorescence acquisition to overlap with the illumination cycle of the LED.

Autofluorescence lifetime of porcine articular cartilage pre- and post-treatment with bacterial collagenase are presented in Figs. 2 and 3 for the entire spectral range and for distinct spectral bands: channel 1, 400–450 nm (Fig. 2(c,h)); channel 2, 450–515 nm (Fig. 2(d,i)); channel 3, 515–580 nm (Fig. 2(e,j)). In the spectral region 580–650 nm, we measured a weak fluorescence signal and thus data for this band are not shown. The normalized autofluorescence intensity maps are shown in Fig. 4. Demonstrations of the measurements are also presented in Supplementary Videos S1–S3. We note that, during acquisition, online feedback was provided for the entire spectral range (see Supplementary Video S1).

Figure 2

Autofluorescence lifetime maps of cartilage pre- (top row) and post-treatment (bottom row) with bacterial collagenase. (a,f) White light images of the cartilage specimen. (b, g) Fluorescence lifetime maps generated by binning the entire array into a single channel. Panels (c–e) and (h–j) show fluorescence lifetime maps for different wavelength bands: channel 1 (400–450 nm); channel 2 (450–515 nm); channel 3 (515–580 nm). Wavelengths bands were generated in post-processing by spectrally binning 15 columns of pixels. Scale bar = 10 mm.

Figure 3

Average fluorescence lifetimes for each spectral channel in regions of interest (ROI), as illustrated in Fig. 2(a,f).

Figure 4

Normalized fluorescence intensity maps of cartilage pre- (top row) and post-treatment (bottom row) with bacterial collagenase. (a,d) Channel 1 (400–450 nm); (b,e) channel 2 (450–515 nm); (c,f) channel 3 (515–580 nm).

Before treatment (see Fig. 2(a–e)), we observed a small variation of the autofluorescence lifetime across the articular surface (see Fig. 3), possibly arising from regions with different load-bearing characteristics. In the region where treatment was applied (see Fig. 2a, red square, n = 1088 pixels), we measured pre-treatment average autofluorescence lifetimes of 3.44 ± 0.03 ns, 3.20 ± 0.04 ns and 2.89 ± 0.05 ns, for the wavelengths bands of channels 1, 2 and 3, respectively, which indicate a slight dependency of the autofluorescence lifetime with wavelength. If we merge the data in a single spectral band (Figs. 2b and 3a), we obtain an average lifetime of 3.20 ± 0.05 ns.

Treatment with bacterial collagenase resulted in visible digestion of the articular cartilage surface (see Fig. 2f, red square), which, in turn, resulted in a consistent decrease of autofluorescence lifetime in this region, as compared to non-digested cartilage. While a decrease in autofluorescence lifetime is visible in all spectral bands, this is most evident in channel 2 (see Fig. 2h), which is coincident with the fluorescence emission peak of collagen (see Fig. 5). We also measured a slight decrease in the autofluorescence lifetime of the non-digested region, although this is not as evident as in the region of collagen digestion. In the region of digestion (n = 1740 pixels), we measured autofluorescence lifetimes of 2.82 ± 0.06 ns, 2.51 ± 0.07 ns and 2.36 ± 0.07 for channels 1, 2 and 3, respectively. For the entire spectral range, we measured an average lifetime of 2.42 ± 0.05 ns.

Figure 5

Average fluorescence emission spectra (left) pre and (right) post enzymatic treatment in regions of interest (ROI), as illustrated in Fig. 2(a,f). Solid lines indicate average spectra and shaded regions the corresponding standard deviation.

With respect to autofluorescence intensity (Fig. 4), our data suggest a red shift in the autofluorescence spectrum of the digested region relative to non-digested cartilage. This is most visible in Fig. 4d, as a decrease in the fraction of autofluorescence emanating from channel 1, but also in the average autofluorescence spectra, from which we measured a shift in the peak emission wavelength of approximately 10 nm (see Fig. 5b). We note that, in general, we measured a lower absolute autofluorescence signal after treatment, which generated noisier autofluorescence maps and spectra.

Discussion

In a previous study, we introduced a technique employing TCSPC and hybrid photon counting detectors to realize single-channel fibre-based fluorescence lifetime imaging from single-point measurements in real-time and under bright illumination of the FOV[46]. Here, we aimed to expand this technique by employing SPAD array detectors to provide multispectral resolution to the lifetime measurement and thus increase the specificity of the fluorescence output. As in our previous study with a TCSPC-based setup, interleaving the fluorescence acquisition with external illumination from a pulsed white LED at 50 Hz permitted single-point measurements to be realized at a sufficiently high rate (between 25 and 50 Hz, see Supplementary Fig. S4) to generate fluorescence lifetime and intensity maps in real-time. We note that while measurements were software triggered at 50 Hz, timing delays in data processing and storage following acquisition slowed down the rate at which fluorescence data were acquired. To minimize interference of timing delays in acquisition rate, the integration time for fluorescence acquisitions was adjusted to ~11 ms. Considering the poor photon efficiency of our system, as further discussed below, we aimed for the longest possible integration time within the LED non-illuminated cycle. However, increasing the integration time beyond 11 ms consistently caused the fluorescence acquisition to overlap with the illumination cycle of the LED, thus leading to detection of undesirable LED-background (see Supplementary Fig. S3). For integration times longer than 15 ms, the contribution of LED background to the measured signal was approximately 20%. While such contribution is not sufficient to mask the autofluorescence signal from the sample, the LED illuminance in our measurements was approximately 200 LUX, which is about 10–20% of the illuminance provided by an endoscopic system. Thus, under standard illumination from an endoscope, the background contribution would not be negligible.

The white LED guarantees that the sample is visible to the human eye and sufficiently illuminated during measurements. From the operator’s perspective, the LED provides continuous illumination of the FOV, since 50 Hz switching frequency cannot be perceived by the human eye. This is a key element in our implementation towards clinical translation. So far, clinical translation of optical techniques dedicated to surgical guidance and operating in the visible range of the spectrum has been limited by the intrinsic inability to discern fluorescence photons from ambient background light. Indeed, a number of studies can be found in literature employing analogue detection methods that separate the low frequency background from the high frequency fluorescence decays[34,54,55]. However, such separation is not trivial in photon counting techniques; thus the applicability of such measurements in clinical studies is limited. Our technique, while still requiring room lights to be turned off, allows fluorescence measurements in the visible spectrum to be realized together with bright LED illumination, in a relatively straightforward implementation. This makes our technique compatible with many clinical procedures where LED illumination is becoming standard, such as endoscopy, arthroscopy or robot-assisted surgery, requiring only minimal modifications to the existing clinical setups.

A key point of our implementation refers to the dynamic spectral range for fluorescence lifetime determination that can be tuned according to any specific target. The ability to dynamically modify the range of wavelengths over which fluorescence lifetimes are measured constitutes a great advantage over traditional multichannel instruments with fixed detection spectral range, which are defined by the combined optical transmission of dichroic mirrors and emission filters e.g[27,30,56]. In our implementation, fluorescence is passed through a transmission grating and spectrally-resolved across 64 columns of SPAD detectors, between ~400 nm and 650 nm. If the SNR in one spectral band at maximum resolution is not sufficient to provide robust fluorescence lifetime measurements in real-time, the spectral bandwidth can be increased to improve the SNR by merging adjacent columns of pixels, at the expense of spectral resolution (see Supplementary Video S3). Additionally, data can be spectrally combined to target the emission bands of fluorophores of interest, for any particular clinical application (see Supplementary Video S2). For example, in applications requiring assessment of tissue metabolic properties, the spectral range can be tuned to the bands of key endogenous metabolites NAD(P)H and FAD, i.e. 430–470 nm and 500–550 nm, respectively. In other applications requiring e.g. monitoring collagen autofluorescence, the spectral range can be modified in order to maximize collagen autofluorescence detection, i.e. from 400–460 nm. If additional fluorescence specificity is required, e.g. separate out free- and protein-bound NAD(P)H or to target porphyrin autofluorescence, different spectral bands can be dynamically generated, from which spectrally-resolved fluorescence lifetime data can be retrieved. The benefits of having dynamic spectral range and resolution are also illustrated in Fig. S5c, where we measured the fluorescence lifetime characteristics of two reference fluorophores and mixture solutions. Specifically, our results demonstrate that increasing the number of pixels in each spectral channel, i.e. lower spectral resolution, decreases the specificity of the fluorescence lifetime output, as expected. In summary, in our implementation, spectral bands for lifetime determination can be dynamically selected before or during the acquisition, or in post-processing, and must balance the required specificity for the measurements and the amount of signal at key spectral bands: if one spectral band has low SNR, the spectral range can be increased at the expense of spectral resolution.

The feasibility and potential of our approach to report autofluorescence contrast was demonstrated in cartilage specimens. Specifically, the articular surface of porcine metacarpophalangeal joints was digested with bacterial collagenase, in an effort to mimic extracellular matrix degradation that is characteristic of osteoarthritis (OA)[57,58]. We chose this particular application given the general interest in identifying comprehensive and robust solutions to diagnose and monitor the early onset of OA, which are, to date, inexistent. While the onset of OA is believed to be driven by mechanical factors, some studies suggested that early stages of disease progression are dominated by biochemical alterations in tissue[59], which are not detectable using conventional diagnostic modalities, such as x-ray and magnetic resonance imaging, but make multispectral fluorescence lifetime imaging a suitable candidate technology to detect early development of OA. Overall, our autofluorescence spectral and lifetime data (see Figs. 2–5) reveal some level of heterogeneity in the fluorescence characteristics of normal articular cartilage. This is most evident in the normalized autofluorescence intensity measured in channel 1 (see Fig. 4a), where lateral regions of the joint appear to have a stronger emission compared to medial regions. Variations in the autofluorescence signature of normal articular cartilage may be associated with variations in load-bearing characteristics across the joint. Autofluorescence lifetime maps generated for different spectral bands (see Fig. 2) reveal a well-demarcated region that is coincident with the area of collagen digestion, yielding shorter lifetime relative to non-digested areas. These results are in close agreement with previous studies of cartilage digestion[46,47,53]. The decrease in autofluorescence lifetime is more prominent in channels 1 and 2, i.e. from 400 to 515 nm, and coincident with the measured fluorescence emission peak (see Fig. 5). Interestingly, our results suggest that autofluorescence lifetime of cartilage decreases slightly with wavelength (~500 ps from 400 to 650 nm), which is consistent with previous observations[60]. While collagen is the dominant source of autofluorescence in cartilage, it is possible that other fluorophores contribute slightly to the autofluorescence signal at longer wavelengths where the contribution of collagen is smaller, namely glycosaminoglycans[61], and produce an overall decrease in lifetime with wavelength. This could also explain the smaller decrease in lifetime in channel 3 observed upon digestion of the articular surface (see Figs. 2 and 3d). Changes in the autofluorescence properties of articular cartilage upon digestion with bacterial collagenase were also evident in the fluorescence emission spectra, where we measured a consistent red shift in the fluorescence emission of digested cartilage relative to non-digested and pre-treated cartilage. This is visible in the normalized autofluorescence intensity maps (see Fig. 4) and in the autofluorescence emission spectra (see Fig. 5) averaged over the highlighted ROIs (see Fig. 2(a,f) for identification of ROIs). Specifically, spectral measurements show a ~10 nm shift towards longer wavelengths in the fluorescence emission of digested cartilage relative to non-digested cartilage (see Fig. 5), which is in agreement with previous studies[17,47] but directly contradicts another[53].

One current limitation of this system refers to its low photon efficiency. This is of paramount importance for real-time implementations, since the photon integration time is limited by the overall timing requirements of the application. To some extent, for autofluorescence measurements presented in this study the issue of low photon efficiency was mitigated by binning adjacent columns of pixels to increase SNR, at the expense of spectral resolution. Furthermore, our measurements were realized in collagenous tissue, which typically exhibits strong autofluorescence signal due to the high quantum yield of collagen. In tissues presenting weak autofluorescence signal, as is common, the low detection efficiency could potentially limit the viability of the system for real-time implementations and, therefore, needs addressing in future investigations. A number of factors contribute to the overall poor photon efficiency. 1) The time-gated strategy for fluorescence lifetime measurements is intrinsically inefficient, given that the photon collection window only covers a small percentage of the total acquisition period (20% in our measurements), and all photons arriving at the detector outside the collection window are discarded. Alternative strategies could employ multiple parallel windows to realize fluorescence acquisition with 100% duty cycle[62,63]. 2) The fill factor of each SPAD detector is low (<4%[48]) and can potentially be optimized using microlenses. 3) There are significant losses in the detection arm, given the relatively low efficiency of the transmission grating (~55% at 550 nm) and lenses. To improve light throughput, the spectrometer could be designed using spherical mirrors and a reflective grating, which is a significantly more efficient design compared to ours[64].

Our system is most sensitive to fluorescence signals emitted from the most superficial layers of tissues, due to the strong tissue scattering and absorption at the excitation and emission wavelengths used in this study, that limit the region of interrogation to the first 100–200 μm depth. This can be a disadvantage in cases where disease processes vary with depth and extend to deeper layers, as is the case of some tumours. To some extent, depth information can be provided by resolving the fluorescence emitted from different layers across the rows of the SPAD array, as previously described[42]. In addition, multispectral fluorescence lifetime measurements could be employed together with a technique that can harness information from deep layers of tissues, such as diffuse reflectance[37], Raman spectroscopy[65] or Optical Coherence Tomography[66].

In conclusion, we presented an exploitation of SPAD arrays to realize multispectral fluorescence lifetime imaging in real-time by means of fibre optic probes and under bright visible illumination of the FOV, thus expanding on our previous work using single-channel TCSPC-based instrumentation. In particular, we demonstrated the versatility of SPAD arrays to provide dynamic spectral range and resolution to fluorescence lifetime measurements, which constitutes an advantage over traditional multichannel instrumentation. This can be of interest in tissue measurements with multiple contributing endogenous fluorophores, thus requiring additional specificity to resolve the various fluorescence emitters. Given the simplicity of our implementation to produce multidimensional fluorescence images out of single-point measurements, we believe this method has potential to provide real-time diagnostic information and guidance in many clinical imaging modalities, such as arthroscopy, colonoscopy, endoscopy or robot-assisted surgery.

Supplementary Information.

Supplementary Information2.

Supplementary Information3.

Supplementary Information4.