Intra-instrument channel workable, optical-resolution photoacoustic and ultrasonic mini-probe system for gastrointestinal endoscopy.

There has been a long-standing expectation that the optical-resolution embodiment of photoacoustic tomography could have a substantial impact on gastrointestinal endoscopy by enabling microscopic visualization of the vasculature based on the endogenous contrast mechanism. Although multiple studies have demonstrated the in vivo imaging capability of a developed imaging device over the last decade, the implementation of such an endoscopic system that can be applied immediately when necessary via the instrument channel of a video endoscope has been a challenge. In this study, we developed a 3.38-mm diameter catheter-based, integrated optical-resolution photoacoustic and ultrasonic mini-probe system and successfully demonstrated its intra-instrument channel workability for the standard 3.7-mm diameter instrument channel of a clinical video endoscope based on a swine model. Through the instrument channel, we acquired the first in vivo dual-mode photoacoustic and ultrasonic endoscopic images from the esophagogastric junction of a swine. Further, in a rat colorectum in vivo imaging experiment, we visualized hierarchically developed mesh-like capillary networks with a hole size as small as ~50 µm, which suggests the potential level of image details that could be photoacoustically provided in clinical settings in the future.

Introduction

One of important technical aspects of photoacoustic (PA) tomography (PAT) in terms of clinical application is that related imaging systems can be embodied in a form that is integrated with the conventional ultrasound (US) imaging function [1], [2], [3], [4], [5], [6], [7], [8]. When pursing the clinical application of PAT, the integrability is an attractive aspect, as the US imaging technique has played an indispensable role in numerous clinical areas and also the field of application of the technique is still being broadened, even in minimally-invasive imaging in forms such as endoscopic ultrasound (EUS) [9], [10] and intravascular ultrasound (IVUS) [11], [12]. Thus, if a miniaturized PA imaging device developed for gastrointestinal (GI) endoscopy also possesses the EUS function, it could complement the current limitations of EUS based on the superior microvascular angiographic and functional imaging capability of PAT [1], [2], [3], [4], [5], [6], [7], [8].

After the first idea of applying PAT to endoscopy in 1996 [13], a variety of device concepts have been proposed to translate its benefits into clinical settings [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44]. Among these, the following are notable device concepts that are potentially applicable for GI endoscopy. The first is the application of a built-in scanning mechanism in which a micro-motor is placed at the scanning tip to function as an actuator [17], [18], [19], [20]. The key benefit of this mechanism is that it enables easy embodiment of necessary optical and electrical coupling in the proximal part of the probe, as associated optical and electrical elements can be statically connected to peripheral elements. However, due to the accompanying technical challenge in implementing the key mechanical unit, such an endoscopic probe with a sufficiently short rigid distal section that can freely pass through the instrument channel of a video endoscope has not been demonstrated thus far. To avoid such a technical challenge, a torque coil-based proximal actuation mechanism was introduced in 2014 [26] and has become the dominant mechanism for most follow-up mini-probes for PA endoscopy (PAE) [21], [22], [23], [24], [25] or catheter probes for intravascular photoacoustic (IVPA) imaging application [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39]. Although it is expected that it will take a considerable amount of time for commercialized devices based on the related mechanism to be used for real human subjects, one of the key strengths of this probe is a flexible body along with a short rigid distal section. These two mechanisms are based on single-element transducer-based mechanical scanning. However, since 1998, pure optics-based US detection mechanisms have also been investigated because they exhibit an exceptional sensitivity [40]. Among related embodiments [40], [41], [42], [43], [44], the result reported by Ansari et al. in 2018 demonstrated a 50,000-channel forward-viewing probe concept based on a Fabry–Pérot interferometer [43]. Although the probe was embodied in a rigid form, they achieved the vastest number of array channels, which was never possible with the conventional piezoelectric method.

In this study, by applying the conventional piezoelectric material-based US detection mechanism and the torque coil-based proximal actuation mechanism, we developed a 3.38-mm diameter integrated PAE-EUS mini-probe system, which can be applied immediately when a suspicious tissue is found during a video endoscopic procedure. Although a certain endoscopic application would require a high sensitivity of the optical US detection mechanism, particularly in the case where the applicable optical energy is limited, we considered the traditional piezoelectric material-based US detection mechanism still valuable and important because it offers the irresistible benefit of enabling seamless integration with the EUS technology, thereby accelerating the clinical translation of PAE based on the established platforms of EUS that clinicians are accustomed to. Moreover, we adopted the torque coil-based proximal actuation mechanism because it is a typical mechanism that is applied to current clinical EUS mini-probes [9], [10].

Of course, most of the IVPA devices reported in recent years have been implemented based on the same mechanism [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], and one of the reports even successfully obtained the image of a swine coronary artery in vivo [32]. However, none of them has successfully demonstrated the imaging of the upper GI tract of a large animal via the instrument channel of a clinical video endoscope; only the imaging of a rat colorectum has been reported with regard to in vivo GI tract imaging [22], [23], [24]. Moreover, the PA image qualities have been merely limited to acoustic-resolution (AR) imaging because the devices relied on multi-mode optical fibers with a large core diameter (>100 µm) (note that this is one of the important factors determining the PA transverse resolution of an implemented device). In other words, although the torque coil-based proximal actuation mechanism could be commonly applied to an IVPA and GI endoscopic probe, achieving the latter embodiment is not minor, as it requires resolving the following additional technical issues. First, sealing the probe sheath near the proximal optical and electrical rotary junction as well as at the distal tip must be guaranteed in order to completely isolate the acoustic matching medium filled inside (a temporal injection of a matching liquid like saline, as in the case of IVUS imaging, would be barely applicable when imaging a GI tract). Second, considering the general working condition that the typical diameter of a human GI tract is much larger than that of a mini-probe and, thus, the target distance is not constant as in the case of IVPA imaging (where a narrow-diameter blood vessel provides almost uniform working distance), it is highly desirable to build a probe with a collinearity between the optical axis and acoustic axis over as large a range as possible ([21] discusses the importance of this issue, and [30] is also an example that addresses the collinearity, although the related probe was for an IVPA imaging application). Third, in order to visualize as many diagnostically useful blood vessels as possible through the limited contact area, the transverse resolution needs to be significantly improved compared to the previous AR-based PAE or IVPA probes; this implies that it is essential to embody an optical rotary junction based on an optical fiber with a narrower core diameter, which is not easy in terms of attaining an adequate optical coupling and transmission efficiency based on the narrow core diameter. Fourth, a smooth mechanical rotation with a uniform step size at the scanning tip as well as through an adequate probe length (>1.5 m) must be guaranteed in order to provide a continuous vascular map along the longitudinal direction by passing through the instrument channel. Lastly, it is important to attain an adequate probe strength that ensures that the probe is not easily damaged when it passes through the sharply-bent entrance port of the instrument channel.

In this study, we overcame the technical limits of previous embodiments and achieved optical-resolution (OR) PA imaging as low as ~13 µm even in the presented in vivo imaging demonstrations by implementing a customized optical rotary junction and distal optics that incorporated a quasi-single mode optical fiber and gradient-index (GRIN) lens-based optical focusing mechanism as well as achieving collinearity between the optical and acoustic beam. In this article, we describe the engineering strategies we formulated to surpass existing imaging performances and also present related embodiment results of newly addressed technical issues, such as those pertaining to the coupling of the electric signal near the rotary junction and signal balancing between the PA and US modes when sharing the same digitizer, both of which were successfully proven through the presented in vivo imaging results.

Results

PAE-EUS mini-probe system

System composition and distal structure of the mini-probe

Fig. 1(a) presents the overall view of the lab-made mini-probe system constructed on a movable cart for easy transportation in an endoscopy room. The system consists of a mini-probe, which is a key module designed to be connected to a driving unit in an interchangeable manner and has a flexible section with a length of ~1.8 m and an outer diameter (OD) of 3.38 mm, including the tubing dimension; a driving unit that drives the mini-probe; and their peripheral systems, including a data acquisition (DAQ) computer, laser controller, and DC power supply.

Fig. 1

Integrated PAE-EUS mini-probe system and structure of the distal section of the mini-probe. (a) Photo of the entire system. (b, c) Photo and schematic of the distal section. Note that the Cartesian coordinate system denoted by (x, y, z) is a moving and rotating coordinate system along with the scanning tip. (d) Simulated approximate 2D synthetic acoustic impulse response function of the dual transducers for the receiving mode (i.e., PA only) along the y-z plane. (e) Photo of the scanning tip that emits laser beam and its 2D beam profile (inset) measured at the optical focus.

Fig. 1(b) and (c), respectively, depict a photo and corresponding schematic of the mini-probe’s distal section where rotational scanning is performed. The distal section encapsulated by a stainless steel (SUS) tube had a length of 10 mm and OD of 1.83 mm; moreover, the distal section was connected to a 1.8-m long torque coil with a 1.3-mm OD and 0.7-mm inner diameter (ID), in which a quasi-single mode optical fiber (FG010LDA, 10-µm core dia., Thorlabs, USA) and two micro-coaxial cables from two transducers (40 MHz, 0.6 mm × 0.5 mm × 0.2 mm, Blatek, USA) were placed.

As with conventional EUS mini-probes [9], [10], the distal and torque coil sections must be properly sealed in a sheath (i.e., catheter) to avoid their direct contact with a target tissue during their fast rotation. For the developed mini-probe, the OD and ID of a necessary sheath could be appropriately selected depending on the target application. However, for this study, we created a sheath in our lab by combining two different types of commercial tubing—that is, Perfluoroalkoxy (PFA) for the 1.8-m long torque coil section (OD/ID: 3.0/2.0 mm) and Pebax 7033 for the imaging window (OD/ID: 3.38/3.17 mm) [Fig. 1(b)]; this was done because of the non-availability of such a one-body sheath with two different wall thicknesses among currently available commercial tubing line-ups. We chose the two materials because PFA exhibits low mechanical friction, which is good for the smooth rotation of the torque coil, whereas Pebax has high optical and acoustic transparency. Moreover, applying a tubing with such a thin wall (i.e., ~100 µm) for the imaging window section was necessary to minimize the distortion of acoustic waves as well as a focused laser beam.

Considering the 3.38-mm OD of the imaging window, we targeted a working distance (WD) of 1.46 mm in water [Fig. 1(b)], which was defined by the distance from the surface of the SUS tubular housing with the 1.83-mm OD to the optical focus created by a GRIN lens (0.5-mm diameter, ~0.1 pitch; altered from #64–515, Edmund Optics, USA). The physical length of the GRIN lens was reduced to 0.44 mm from its original length of 1.15 mm by an in-house polishing process; this was done in keeping with a previously reported technical note that recommended an avoidance of direct contact of the GRIN lens to the tip of the optical fiber [20], while achieving the targeted WD. Consequently, the laser beam diverging from the optical fiber tip was propagated in air by a ~1.1-mm interval, converged by the GRIN lens, and finally focused after being reflected at the hypotenuse of a right-angle prism (#66–772, Edmund Optics) by the total internal reflection (TIR).

As mentioned earlier, a collinear alignment between the optical and acoustic axes is highly desired for a PAE probe developed for application in GI imaging. Thus, to collect generated PA waves over a large solid angle without the issue of misalignment between the optical and acoustic axes, we employed the dual transducers that were placed symmetrically with respect to the optical beam, as depicted in Fig. 1(c); the figure reveals that the dual transducers were affixed onto the two inclined surfaces at a 17° angle to the z-axis and with a separation of ~0.5 mm from each other so that the focused laser beam could come out through the aperture in between. Of course, placing only a single transducer on one side would also work to a certain extent when considering only PA imaging. However, in that case, since the acoustic pulse for US imaging was to be fired obliquely from the inclined surface with respect to the probe axis, the echo waves reflected off the lumen wall, which is usually oriented to be parallel to the probe axis, would barely enter the transducer according to the law of reflection. In terms of securing the optical TIR, it was important that one leg of the prism be affixed around the 0.5-mm diameter hole of a brass base [Fig. 1(c)], in order to enable it to function as a sealing point that isolates the associated optics from the acoustic matching liquid filled in the tubing and not merely function as an optical aperture.

Further, although the presented configuration required dual transducers and, thus, the conventional single transducer-based, swapped optical- and acoustic-beam positioning method applied in the previous studies [22], [23], [24] would be more advantageous in terms of number of required transducers, we attempted this new configuration because we expected that it would be a relatively easier method to achieve the high-resolution PAE imaging in terms of implementing the necessary optics; moreover, the oblique acoustic pulse firing strategy would diminish the high acoustic reverberations generated by the imaging window. In addition, the presented optical configuration does not involve any dented optical aperture in the distal optics, which were usually present in the previous configurations [22], [23], [24]—from a practical perspective, such a dented optical aperture can be rather annoying when injecting necessary matching liquid into the catheter because it enables the easier formation of bubbles at the point. In addition, due to the employment of the dual transducers, the synthetic acoustic aperture—that is, the view angle—when detecting PA signals is doubled. Fig. 1(d) depicts an approximate 2D synthetic acoustic impulse response function of the dual transducers simulated for the receiving mode (i.e., PA only) along the y-z plane using the k-Wave MATLAB tool box [45]. For the dual transducers connected parallelly, the total electric impedance that we measured based on the Kirchhoff’s law appeared to be approximately 75 Ω.

After the probe implementation, we measured the laser beam diameter at the optical focus by using a beam profiler (SP620U, OPHIR). Fig. 1(e) depicts the microscope image of the scanning tip that emits a laser beam and a 2D beam profiler image (inset) acquired at the focal plane, from which the FWHM beam diameter was determined to be ~8 µm in air. Thereafter, we performed an optical WD measurement experiment by imaging a surgical blade submerged in water according to the procedure described in Supplementary Fig. S1. Through the experiment, we found that although we targeted an optical WD of 1.46 mm in water, the actual measurement value was ~0.89 mm. In addition, we determined the PA/US image resolutions to be ~17–67 µm/191–355 µm for the transverse direction (x-direction) and ~61–130 µm/66–75 µm for the radial direction (i.e., y-direction) (see Fig. 1(c) for the axis definition).

Although the resolutions determined by the above method revealed a high radial distance dependence and the best PA transverse resolution appeared to be just 17 µm—which is two times greater than the measured optical beam diameter—we were able to validate that the PA transverse resolution of the implemented probe was as low as ~13 µm, while the optical WD was determined to be slightly differently (~0.86 mm) from the above value 0.89 mm when we re-estimated them through the analysis presented in Supplementary Fig. S6. Based on the two determined WD values, we assume that during the rotational scanning of the scanning tip, the focal spot of the laser beam traces a circle of diameter ~3.54–3.6 mm, which is slightly larger than the OD of the imaging window. We attribute the close-shift of the WD to the imperfection of our assembling work for the distal optics.

Unlike a commercial IVUS imaging probe utilized in clinics [11], [12], EUS mini-probes for GI endoscopy are usually manufactured in a form that requires the acoustic matching medium to be permanently filled inside it [9], [10]. Although such a permanent filling was also desirable for the developed mini-probe, we had to inevitably apply the previously reported temporal water injection strategy [20], [21] because there was still a concern regarding the possible degradation of optical and acoustic performance caused by the impurities involved during the in-house assembling process. By using the water injection syringe presented in Fig. 1(a), we filled the inner space of the probe with deionized water from the proximal part to the distal end of the probe and finally sealed the distal tip using a plastic screw with a silicon O-ring [Fig. 1(b)]. After an experiment was conducted, the injected water was drained out due to the abovementioned concern.

Driving unit

To drive the mini-probe, we designed and implemented a driving unit, as depicted in Fig. 2(a), in which an oscillator and delay generator unit, pulse amplifier, RF switch and signal amplifier unit, clamping nozzle, stepping motor, and laser head were integrated. Notably, the proximal part of the mini-probe (i.e., bearing unit) [Fig. 2(b)] was designed to be clamped into the clamping nozzle of the driving unit [Fig. 2(a)]; then, the timing pulley and SMA connector formed around the bearing unit were engaged and connected to the stepping motor and the RF switch, respectively, to transfer the necessary mechanical torque as well as associated electric signals.

Fig. 2

Driving unit. (a) Photo depicting the interior of the driving unit. (b) Entire view of the mini-probe. (c) Schematic presenting the configuration of the integrated optical and electrical rotary junction along with the laser beam path. Note that the Cartesian coordinate system denoted by (X, Y, Z) is a moving but non-rotating coordinate system introduced to translationally trace the position of the center of the scanning tip. Photos depicting the entire view (d) and internal structure (e) of the rotary transformer unit.

One of the key achievements of this study compared to previous studies [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39] is that we applied a rotary transformer concept for the first time to communicate electric signals for PA and US imaging at the associated electric rotary joint, thereby successfully acquiring the required images, which are presented at a later point in the paper. The rotary transformer-based signal coupling mechanism is a common mechanism that is frequently applied to numerous commercial EUS and IVUS probes [9], [10], [11], [12]. However, no previous PAE or IVPA imaging device has been implemented based on this mechanism [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39]. Although the transmission efficiency of a rotary transformer is not as good as the case of a slip ring, which can reach almost 100%, and thus its transmittable power is rather limited, it has the advantage that it does not involve any contact resistance fluctuation and transmission efficiency degradation caused by abrasion, as is the case of a slip ring; the semi-permanent lifespan capability needs to be importantly evaluated because unlike an IVUS probe, an EUS mini-probe is not usually manufactured for single-use purposes. Moreover, it also has an advantage in terms of unit cost and can play the role of an electric impedance matcher between the transducers and amplifier as well as that of a DC blocker.

A rotary transformer unit created on the basis of a set of pot cores was installed inside the timing pulley depicted in Fig. 2(b), and its approximate configuration with respect to coupling optics is depicted in Fig. 2(c). Furthermore, as illustrated in Fig. 2(d) and (e), the rotary transformer unit consisted of the rotor and stator, and associated signals were transferred between them through a pair of coils (turns ratio: 1:1) that were located within the rotor and stator, respectively. A ball bearing installed in the rotor unit [Fig. 2(e)] provided a smooth rotational condition, which enabled a stable signal transfer while the rotor and the torque coil rotate. In addition, we measured one-way electric signal transmission efficiency of the rotary transformer according to the method presented in Supplementary Fig. S2 and observed an efficiency as high as ~95% for the engagement of a 50-Ω load. As depicted in Fig. 2(c), the two micro-coaxial cables from the two transducers were combined inside the bearing unit and connected to the second coil included in the rotor [Fig. 2(e)], which resulted in an electrically parallel connection effect to the employed transducers.

In addition to the electric characteristics, the two ferrite cores of the rotary transformer unit also featured a 1.8-mm diameter hole formed at the center [Fig. 2(e)]; thus, the laser beam could be injected into the entrance tip of an optical fiber via the holes, as depicted in Fig. 2(c), thereby constituting an integrated optical and electrical rotary junction. The laser beam emitted from the laser system (532 nm, ~2 ns pulse length, SPOT 10–200–532, Elforlight) was guided by three turning prism mirrors, whose positions were translationally adjustable along the X-Y-Z directions, and focused on the optical fiber by a focusing lens (f = 30 mm, LA1289-A, Thorlabs). Fig. 2(f) is an en face view of the custom-ordered main drive shaft (OD: 12 mm, SUS) at the center of which the optical fiber that extended up to the scanning tip was placed accurately with the aid of a ceramic optical fiber ferrule (CF126–10, Thorlabs).

In creating such an optical rotatory junction, it is key to provide a stable rotational condition to the optical fiber rotating with a torque coil. To better achieve this, we employed five ball bearings, including four high precision angular ball bearings [Fig. 2(g)], which were modularized in an aluminum (Al) bearing holder through a clapping method [Fig. 2(b)]—this was the simplest method that minimized the runout of the optical fiber’s rotational axis during its operation. However, considering the possible imperfection of our assembling work, we employed an optical fiber with a 10-µm core diameter (i.e., multi-mode) and also applied a broad injection strategy of the laser beam by employing the focusing lens with a low NA (LA1289-A)—the FWHM beam diameter measured at its focal plane was ~20 µm.

Unlike the previous IVPA probes whose device workability was demonstrated based on a temporal injection of a saline solution [32], [37], creating a PA probe whose inner tubular space could be filled with an appropriate acoustic matching medium for a long time was a key aspect in successfully applying it to GI endoscopy via the instrument channel of a commercial video endoscope. To resolve this issue, we employed another SUS shaft with a diameter of 3/16″ that was operated through installation in the main drive shaft with a diameter of 12 mm, while providing the necessary mechanical sealing for the injected deionized water in conjunction with the “shaft seal” [see Fig. 2(c)] during its rapid rotation.

Based on the presented hardware, we performed co-registered PA and US imaging at an A-line acquisition rate of 16 kHz for each imaging mode, which was limited by the maximum pulse repetition rate (PRR) of the employed laser system (the operation sequence is described in the next section). For the presented in vivo imaging experiments, the laser pulse energy that we measured at the probe output showed a variation of between 0.7 and 1 μJ depending on environmental conditions; however, the pulse energy did not exceed 1 μJ, as it was the maximum energy attainable by the employed laser system when measured at the optical output (note that the applied pulse energy is approximately two times greater than that in a previous study [20]).

Circuitry and operation sequence for co-registered PA and US dual-mode imaging

Co-registered dual-mode PA and US imaging based on a commercial US pulser/receiver has already been demonstrated in previous studies, along with the interleaving concept of required laser and electric pulses and detecting corresponding responses in turn [18], [19], [22], [23], [24]. However, in this study, we achieved dual-mode imaging by implementing a customized circuit.

A technical issue when applying a commercially available US pulser/receiver to an integrated PAE and EUS probe implemented for a GI tract imaging is that it is fundamentally difficult to make the signal levels of the two imaging modes comparable to each other because the available electric pulse energy levels of a commercial pulser are limited. In other words, although the lowest possible pulse energy is applied, the signal amplitudes of the US mode are usually much greater than those of the PA mode; this is because PA waves are induced by laser pulses and, thus, subjected to deposited laser pulse energy (note that in the cases of the pulser/receiver models utilized in Refs. [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], an amplitude of ~80 V is the minimum voltage value available for the acoustic pulsing). Of course, when considering an IV imaging application, since the US echo signals from the walls of a blood vessel are usually rather weak due to the softness of the blood vessel, applying the conventional high pulsing voltage—as done in previous IVPA studies [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39]—would not be problematic at all. However, in the case of the GI application, the imaging window itself functions as a hyperechoic object due to its material characteristics of somewhat hard polymer and also its proximity to the engaged transducer (refer to related reverberation signals included in commerical EUS mini-probe images in Refs. [9], [10]); thus, the US echo signals propagating from its exterior regions—that is, target tissues, are mixed with the strong acoustic reverberations generated by the imaging window, even being overwhelmed by them in numerous cases.

Of course, a simple option to make the signal levels comparable would be to add an attenuator between a pulser/receiver and an engaged transducer, thereby attenuating the magnitude of the electric pulse applied to the transducer [18]. However, this method inevitably has the major drawback of attenuating upcoming PA and US signals as well. Another possible solution would be to simply employ dual digitizers whose dynamic ranges then might be set independently for the two imaging modes. However, this method causes an increase in the cost of the system, which is not desirable when considering the commercialization of the related technology. Thus, in this study, as a pilot approach, we addressed the raised issue by operating the implemented circuit at a lowered acoustic pulsing voltage—that is, 24 V.

Fig. 3(a) depicts an approximate block diagram of the first version of the customized pulser/receiver circuit in which a commercial RF switch (ZASWA2–50DR-FA+, 50 Ω, Mini-Circuits, USA) and the high-voltage pulse amplifier concept described in Ref. 46 were applied; Fig. 3(b) and (c) represent the timing charts of the trigger pulses applied to the associated key elements.

Fig. 3

Circuitry of the PAE-EUS mini-probe system and its operation sequence. (a) Block diagram depicting the electric connection of key elements. (b) Digital timing diagrams generated at the marked positions in (a). (c) Magnified view of the dashed area depicted in (b).

In the operation, square-shaped pulses with a 16-kHz frequency (T = 62.5 μs) are first generated by an oscillator that is utilized as a fundamental frequency at which all the sub-electronic and peripheral systems are synchronized, including the stepping motor and laser system. Then, the pulses are sent to a lab-made delay generator (DG) in which the pulses are reshaped into three different types of waveforms with a preset time delay and pulse duration in order to trigger associated electronic elements. Among the three waveforms, the trigger waveform sent to the digitizer (i.e., DG output 1 illustrated in Fig. 3(b)) can be understood as a reference timeline, according to which the dual-mode PAE and EUS imaging sequence and associated DAQ events take place. As illustrated, for every single pulse input to the DG, dual pulses are created with a time lag of 23.8 μs, which respectively correspond to a trigger pulse for PA and US signal acquisitions.

In relation to the timing diagram [Fig. 3(b)], it is a key operation principle of the RF switch that a necessary electric route for an acoustic pulsing with the 24 V excitation voltage is created very shortly only when the RF switch trigger voltage is in a high state; otherwise, the electric condition of the RF switch is immediately changed to the mode to transmit the PA and US response signals to the digitizer (refer to the related RF switch trigger waveform presented in Fig. 3(b)). Fig. 3(c) presents a magnified view of the dashed box included in Fig. 3(b), from which one can better differentiate the relative locations of the trigger waveforms applied to the digitizer and the RF switch with respect to the US pulse firing moment.

As mentioned, the amplitude of 24 V set for the transducer excitation was a voltage value that was much lower than that of a commercial US pulser/receiver, which typically exceeds 80 V. Although the implemented pulse amplifier circuit could also produce such a high voltage output, as it was merely determined by an applied Vcc, we selected an amplitude of 24 V for the excitation because it has been empirically found that even with such a lowered voltage, the signal level of the US echoes coming from the imaging window as well as from the inner wall of the body cavity of a rat was very close to its saturation under the dynamic range of the digitizer that was adapted to the PA imaging mode. In conducting the presented in vivo imaging demonstrations, ensuring that the US echo signals from the imaging window were not saturated was very important because our strategy was to filter out the strong reverberations of the imaging window according to the method described in Supplementary Fig. S4(f); therefore, relatively weak echo signals from the soft tissues, such as colorectal wall (rat) or epithelial tissues (swine), could be more accurately extracted for image presentation. Of course, preventing the signal saturation would not be an absolute direction even in the GI tract imaging application because, even though the conventional high pulsing voltage was to be applied, the signals from the mentioned soft tissues are barely saturated—in this case, the signal saturation of the imaging window could be merely ignored and, thus, the saturated signals could be simply excluded from the final image to be presented.

The trigger sequences depicted in Fig. 3(b) took place continuously, while the scanning tip actuated by the stepping motor continuously rotated in accordance with the fundamental frequency. Thus, the A-line acquisition rate of the endoscopic system was also 16 kHz for each imaging mode, and it was determined by the pulse energy of the employed laser system. However, considering the PA transverse resolution, we set the number of A-lines for one full PA B-scan to be 800, which resulted in a 20-Hz B-scan frame rate; this was done to maximize the rotational scanning speed even though we were unable to fully harness the 8-μm level optical resolving power of the PA imaging mode. Then, the PA or US signals detected by the dual-element US transducers were amplified using dual amplifiers, recorded by the digitizer (200 MHz, 12 bits), and finally displayed on a monitor screen in real time. However, in the monitor display of the current study, we presented the acquired images in a rectangular format [Fig. 1(a)] because if we applied a necessary coordinate transform, intermittent missing of image slices occurred at the set acquisition rate (note that compared to the PA mode, two times longer data points must be recorded for the US mode to cover the same imaging depth). Additional details on the circuitry of the implemented mini-probe and driving unit are presented in Supplementary Fig. S3.

Rat colorectum 3D imaging in vivo

In order to test the in vivo imaging capability of the system, we imaged the colorectum of a Sprague Dawley rat (~450 g; OrientBio, Seongnam, S. Korea) using the setup depicted in Supplementary Fig. S4(a). As the descending colon of a rat typically does not involve serious motion artifacts, it was possible to acquire a 3D in vivo data set by adding a pulling back translation motion to the mini-probe, which resulted in creating a helicoidal scan plane for the scanning tip that constantly rotated inside the tubing [Supplementary Fig. S4(b) and (c)]. In the experiment, considering a possible large variation of a target distance, we set the A-line data length sufficiently high at 400 points and acquired over 4000 B-scan slices for each imaging mode (~3.6 GB) from a C-scan performed at a pullback pitch of 16 µm and speed of ~0.32 mm/s.

Basically, the signal-to-noise ratio (SNR) of in vivo PA data was greater than 30 dB when it was compared to the thermal noise. However, there were substantial effects caused by the interference noise that occurred due to the imperfect shielding to the associated electronic hardware. In a raw B-scan image, interference noise-affected pixels appeared randomly in their positions, but appeared in a rain-like pattern. This was because a range of data points in an A-line were serially affected and, thus, it was not difficult to ascertain whether or not a pixel was affected by an interference noise based on a visual judgement. Supplementary Fig. S4 presents an approximate method that we applied to eliminate the interference noise along with the definition of the coordinate systems applied to the presented images in the paper.

Fig. 4 presents the imaging results of the rat colorectum processed by the method explained above. Fig. 4(a) presents a volume-rendered pseudo color image in which PA (red) and US (green) images are plotted in a merged manner (the rotating movie and a walk-through movie are available in Supplementary Videos 1 & 2), and Fig. 4(b) and (c) depict the US- and PA-radial maximum amplitude projection (RMAP) images, respectively, processed from the same data set. Fig. 4(d) is a magnified image of the dashed box area included in Fig. 4(c); Fig. 4(e)–(g) are respectively an US, merged PA and US, PA cross-sectional image taken at the dashed vertical line in Fig. 4(c), which corresponds to a probe insertion depth of ~6 cm from the anus; and Fig. 4(i) is a histology image (H&E staining) of the tissue harvested around the mid-colorectum.

Fig. 4

In vivo rat colorectum imaging results. (a) Three-dimensionally rendered, merged PAE(red)-EUS(green) pseudo color image acquired from a rat colorectum in vivo over a ~6.4 cm range with a ~5.3 mm image diameter (Supplementary Video 1). The right-hand side of the image corresponds to the anus. The horizontal and vertical scale bars represent 5 mm and 1 mm, respectively. (b, c) US- and PA-RMAP images of (a). The vertical axis corresponds to the angular FOV, covering 360°, and the horizontal axis corresponds to the pullback length of ~6.4 cm. MD, mid-dorsal; MV, mid-ventral; L, left; R, right. (d) Magnified view of the dashed box region in (c). Note that, in the case of (b–d), PA and US signals are mapped on a logarithmic scale. (e–g) Representative US, merged PA-US, and PA B-scan images selected from the vertical dashed line depicted in (c). (h) Magnified view of the dashed area presented in (g). (i) A typical histology image (H&E stain) of the colon.

The following is the Supplementary material related to this article Video S1, Video S2, Video S3, Video S4, Video S5, Video S6.

As revealed in the two RMAP images, the two imaging modes provided different structural information according to their own contrast characteristics. As expected, the US-RMAP image provided the echogenicity distribution for the imaged tissue volume, whose signals mostly originated from the neighboring tissues that surround the colorectum. However, unlike the US-RMAP image, the PA-RMAP image revealed the vascular map of the colorectum in an incredibly detailed manner that had never been achieved previously. As depicted in Fig. 4(d), capillary networks even with a mesh size as small as ~50 µm could be visualized by this new endoscopic device. Through a close image analysis, presented in Supplementary Fig. S5, we could identify that the signals corresponding to the mesh-like pattern originated from the vascular networks distributed in the innermost wall of the colorectum and corresponded to the thorn-like features revealed in the cross-sectional PA image [Fig. 4(g)] and its local magnification image [Fig. 4(h)] (see the white arrow included in Fig. 4(h)). By analyzing the thorn-like feature in the magnified image, we were able to confirm that the PA transverse resolution of the probe was lower than at least 13 µm, even in the in vivo imaging (it must be noted that any capillary scanned along the cross-sectional direction by the probe must appear in such a thorn-like shape in an image because the radial resolution is at an AR level rather than at an OR level. Through the analysis presented in Supplementary Fig. S5, it is conjectured that the two large blood vessels indicated by the pink and blue arrows [Fig. 4(d)] respectively correspond to the artery and vein that supply blood to the colorectum.

From a technical viewpoint, the visualization of such a mesh-like structure was a somewhat unexpected result because the 13-μm transverse resolution of this device was even slightly worse than that (10 µm) of the previous micro-motor-based endoscopic probe reported in Ref. 20. Moreover, there was large gap between the OD of the scanning tip and the ID of the imaging window [Fig. 4(e)], which could cause an unstable fluctuation of the rotation axis unlike the micromotor-based built-in scanning mechanism. We attribute the presented capillary network visualization achievement solely to the increased frame speed of the current system, which was 5 times faster than that of Ref. 20, thereby the connectivity of microvascular networks in adjacent image slices could be secured. However, regardless of the technical basis, considering the transverse resolution (~13 µm) of the probe, typical sizes of RBCs (~7–8 µm), and the apparent hole size of the mesh structure with a fairly uniform hole size (~50 µm), it is presumed that the visualized structure is the “lowest level” capillary network distributed in the colorectum and that a kind of morphological hierarchism exists in the mesh-like capillary networks developed in the colorectum, which refers to the theory presented in Ref. 47 (see the related rationale presented in Supplementary Fig. S5).

In presenting the US-and PA-RMAP images, we applied a logarithmic color scale—through which approximate SNR also could be estimated—to better present imaged structures that revealed a large signal variation caused by the large difference in the echogenicity [Fig. 4(b)] or by the target distance with respect to the fixed optical WD [Fig. 4(c)]. Although there was such a target distance variation, it must be noted that the high-resolution and high-contrast visualization of the capillary networks presented in Fig. 4(d) was possible because the imaged tissue area was fairly positioned around the optical WD. In Supplementary Fig. S6, we present our analysis result that depicts the PA signal variation according to target distance, whose trend fairly agreed with the optical WD measurement result that was performed using a blade, as depicted in Supplementary Fig. S1. Further, in the case of presenting the US [Fig. 4(e)] or merged PA and US B-scan images [Fig. 4(f)], the profile of the imaging window was not excluded, although their magnitudes were still much greater than those generated by intestinal tissues even after the membrane reverberation signal subtraction process [Supplementary Fig. S4(f)]. It is important to note that based on the unsaturated sharp profile of the imaging window revealed by the US imaging mode as well as the location of the superficial PA signals generated by capillaries, we radially adjusted the PA and US data to produce the merged PA and US B-scan image [Fig. 4(f)]. However, in producing the US-RMAP image [Fig. 4(b)], we applied the imaging window signal elimination procedure illustrated in Supplementary Fig. S4(f) to better display the echogenic feature of neighboring tissues that surrounded the imaging window. It must be noted here that as the imaging window signal was eliminated in producing the US-RMAP image [Fig. 4(b)], the US SNR generated by its exterior biological tissues appeared relatively much lower than the PA SNR generated by blood vessels in the PA-RMAP image [Fig. 4(c)]. A pullback movie revealing similar cross-sectional images over the first 1000 slices is presented in Supplementary Video 3.

In the experiment, we induced and maintained anesthesia in the animal by using isoflurane (4% for induction, 1.5–2.0% for maintenance), injected medical US gel into the colorectum for acoustic matching before each introduction of the probe, euthanized the animal by using CO2 gas after acquiring five C-scan data sets, and finally harvested the imaged organ for validation. Before endoscopic imaging, the animal was fasted for 24 h to reduce the amount of feces in the colorectum. All procedures in the experiment followed protocols approved by the Institutional Animal Care and Use Committee at UNIST (UNISTIACUC-20–53).

Swine esophagus imaging in vivo via the instrument channel

Since the major goal of this study was to develop an integrated PAE-EUS mini-probe system that can be promptly applied, just like the conventional commercialized EUS mini-probes [9], [10], when a suspicious tissue is found in a GI tract, we performed an experiment to image the upper GI tract of a swine (Yorkshire, 3 months, 35 kg, OrientBio, Seongnam, S. Korea) to demonstrate the intra-instrument channel workability of the mini-probe system.

As presented in Supplementary Video 4, we were able to successfully acquire a series of PAE and EUS images from the esophagogastric junction of the animal in conjunction with a commercialized video endoscope (GIF-2T240, gastroscope, Olympus) within a mere 20 s from the initial introduction moment of the mini-probe into the entrance port of the instrument channel. Fig. 5(a) and (b) present the still cuts of Supplementary Video 4, taken just before the introduction of the probe to the instrument channel and just after the probe’s projection from the output port. However, before introducing the mini-probe, it was important to ensure there were no air bubbles around the scanning tip by imaging a surgical blade (Supplementary Video 5).

Fig. 5

In vivo imaging results of the swine esophagus. (a, b) Still cuts of Supplementary Video 4, captured just before the probe introduction to the instrument channel (a) and just after the probe’s projection from the output port (b). (c) Video endoscopic image of the esophagogastric junction that we applied our PAE-EUS probe. (d, e) US- and PA-RMAP images acquired from the esophagogastric junction of the animal without any manipulation (no pullback) applied to the gastroscope. (f, g) US- and PA-RMAP images acquired from another spot of the esophagogastric junction by manually pulling back the gastroscope in which the mini-probe was introduced. In (d) and (f), “A” represents the local surface region of the Pebax imaging window that interfaced with air. (h, i) Merged PA and US images selected from the two vertical dashed lines depicted in (f) and (g) along ①–①' (h) and ②–②' (i). These images are views at the proximal of the probe. (j, k) Magnified US- and PA-RMAP images of the regions marked by the dashed rectangular box in (f, g). (l) Histology image (H&E stain) of the esophagogastric junction of the animal. Note that the wall thickness of the specimen rather shrank from its original thickness during the formalin fixation process.

We were able to acquire over 20 data sets from different sites from the animal, such as mid- and lower-esophagus and the greater curvature of the stomach. However, the desired image acquisitions were not always favorable because they were mostly subjected to the associated acoustic coupling condition that solely responded to the body liquid and intermittent waterjet; moreover, there were strong motion artifacts. These two were truly the main aspects that affected the quality of the acquired images. Moreover, due to the tightness of the mini-probe within the instrument channel, we were unable to perform a motorized pullback C-scan, as we did with the rat colorectum imaging. Among the acquired data sets, here we present results processed from two data sets—one from the data set acquired continuously at the same position but with no scope manipulation and another data set that was acquired by manually pulling back the entire scope set, which included useful PA signals acquired from the esophagogastric junction. Fig. 5(c) represents a video endoscope image of the target region before the mini-probe introduction.

Fig. 5(d) and (e) depict the US- and PA-RMAP images, respectively, processed using 550 B-scan image slices that were acquired from the first procedure. As indicated in the PA-RMAP image [Fig. 5(e)], it is evident that the blood vessels were not mapped straight, although the operator did not manipulate the gastroscope in any manner. This was because of the motion artifacts that were mostly caused by the breathing motion. The patch-like regions denoted by the letter “A” in the US-RMAP image [Fig. 5(d)], which represent hyperechoic signal regions generated by a polymer-air interface of the imaging window, manifest the breathing cycle of the animal in intervals of ~4–5 s. As indicated by the white arrow presented in the PA-RMAP image [Fig. 5(e)], the transverse image distortion was as great as 1.3 mm for one breathing cycle.

Even though we attempted the manual pullback for the entire scope set, we were unable to acquire a reliable vasculature map, as presented in Fig. 5(g), which is a PA-RMAP image processed from 320 B-scan slices. By analyzing the PA B-scan images utilized for the PA-RMAP image, we found that the transverse displacement of a blood vessel appeared in two successive images (i.e., in intervals of 50 ms) and was occasionally even greater than 200 µm, which far exceeded the blood vessel diameter that can be resolved by the current system. In the corresponding US-RMAP image [Fig. 5(f)], strong acoustic reflections also appeared at multiple sites because the acoustic coupling was not stable during the manual pullback; this appeared to be attributed to the uneven surface of the imaged tissues under an unstable contact condition of the mini-probe with a small diameter to the esophageal wall that had a larger diameter [Fig. 5(b) and (c)]. To better display only the useful image features included in the reliably imaged region against the strong acoustic reverberations generated by polymer-air interface, we present Fig. 5(j), which corresponds to the dashed box region included in Fig. 5(f), along with the corresponding PA image [Fig. 5(k)].

A close image analysis revealed that hyperechoic responses of the imaging window appeared in approximately two cases: 1) when the corresponding angular portion of the imaging window did not make contact with the tissue but with air or 2) when the membrane surface of the imaging window was perpendicularly oriented to the incident acoustic beam axis. Whether a hyperechoic signal was generated by ambient air or by the vertical incidence of an acoustic beam to the membrane surface could be judged by comparing co-registered PA-RMAP image and also by browsing US B-scan images slice-by-slice. Whichever the case, most often, a corresponding reverberation artifact appeared once more in the US B-scan image at the position corresponding to one round trip travel time of US waves between the US transducers and the imaging window. However, in the first case, no PA signal was observed at the corresponding positions in the PA-RMAP image. Moreover, there was a tendency that such hyperechoic signals were generated regardless of the angle between the US beam axis and the imaging window and the corresponding echogenicity was much greater than that of the latter case.

The two sets of merged PA and US cross-sectional images [Fig. 5(h) and (i)] taken from the two dashed lines depicted in Fig. 5(f) and (g), represent related examples for the first case. As depicted in the first cross-sectional image [Fig. 5(h)] taken at ①–①', a hyperechoic response of the imaging window occurred on account of the polymer-air interface present between the 10 o′clock and 1 o′clock positions, although the acoustic beam axis was not perpendicular to the membrane surface and its reverberation artifact also appeared farther, radial-wise (see the feature indicated by the pink arrow). The blue arrow indicates the direction along which the rotation axis of the scanning tip deviated from the center of the imaging window (see its boundary visualized in the image). In the second cross-sectional image [Fig. 5(i)] taken at ②–②', another type of hyperechoic response of the imaging window caused by polymer-air interface occurred between the 12 o′clock and 4 o′clock positions. In other words, in this case, not only was a polymer-air interface involved but the membrane was also perpendicular to the acoustic axis, as indicated by the blue arrow. Whichever the case, the fact that it was definitely caused by a polymer-air interface was confirmed by “no signal presence” at the corresponding positions in the co-registered PA image.

Furthermore, a hyperechoic signal generation caused only by the vertical incidence of the acoustic beam to the membrane revealed a different pattern. As indicated by the red marker located in the 8 o′clock direction in Fig. 5(i), the acoustic response of the corresponding membrane portion was lower in amplitude than that of the polymer-air interface, and its perpendicularity to the rotating acoustic beam axis could be assured from the orientation of the blue arrow. Although the amplitude was lower, a corresponding reverberation artifact still appeared at a farther distance (see the feature indicated by the white arrow). In this case, since there was no acoustic blockage caused by a polymer-air interface, useful PA and US signals could still be acquired through the region colored in red. Unlike the merged PA and US cross-sectional images, in producing the US-RMAP images [Fig. 5(d) and (f)], we eliminated the signals corresponding to such a reverberation artifact (i.e., signals indicated by the white arrow in Fig. 5(i)) and the imaging window as well by manually segmenting their locations in each B-scan image to better present other image features that appeared weakly.

Unlike the image processing of the rat colorectum, it was difficult to satisfactorily eliminate the reverberation signals generated by the imaging window by applying the simple subtraction method illustrated in Supplementary Fig. S4(f); this is because the number of available US B-scan slices was much lesser and the reverberation pattern was not constant, as the rotation axis of the scanning tip fluctuated dramatically, even seriously deviating from the center of the imaging window (note that the echogenicity of the imaging window varied greatly depending on the radial distance from the imaging window to the transducers). Moreover, as depicted in Fig. 5(i), a certain portion of the imaging window was not even completely visualized because it was not within the stipulated window of the gated time, as the operator increased the angulation of the gastroscope’s distal end to achieve a better acoustic coupling between the mini-probe and the target tissue during the procedure. In terms of the morphological features visualized by PA contrast in Fig. 5(h) and (i), thorn-like shapes, as seen in the rat colorectum image, were barely observed in the epithelium of the imaged esophagogastric junction. Although the target distance was highly fluctuating during the procedure, we attribute this to the intrinsic feature of the vascular pattern and diameter depending on the type of organ [48]; Fig. 5(l) depicts a histology image (H&E staining) of the specimen harvested from the esophagogastric junction. A movie presenting the 320 B-scan slices acquired through the manual pullback is presented in Supplementary Video 6.

Apart from the presented results, we were able to acquire additional data from other sites, as mentioned earlier, until the engaged probe sheath was damaged—there was a water leakage through the UV glued section of the two tubings [see Fig. 1(b)], particularly when we approached the stomach region, which required a higher angulation of the gastroscope. However, based only on acquired B-scan image information, it was difficult to find a notable difference in terms of morphological features for blood vessels that were acquired from different anatomical sites—for example, the esophagus and stomach.

In the experiment, we anesthetized the animal by injecting (IV) a drug cocktail comprising Alfaxalone (5 mg/kg), Azaperone (5 mg/kg), and Xylazine (1 mg/kg); maintained the anesthesia by using isoflurane (2% in 2 L/min, 50% oxygen); and subsequently euthanized the animal with a KCL (2 mmol/kg) injection (IV). Furthermore, prior to endoscopic imaging, the animal was fasted for 24 h to reduce the amount of ingesta in the stomach. All procedures in the experiment followed protocols approved by the Institutional Animal Care and Use Committee at Korea University (KOREA-2017–0128-C1).

Summary and discussion

In this article, we presented the first OR-PAE and EUS mini-probe images acquired from a swine esophagus in vivo, along with a rough scheme for related image processing and interpretation, which could be informative for follow-up studies on GI endoscopy application. Although the implemented system needs to be further improved in terms of hardware and software, this study showcased an integrated PAE and EUS system that inherited the rotary transformer-based signal coupling mechanism as well as the torque coil-based proximal actuation mechanism, which are commonly applied in numerous commercial EUS and IVUS probes [9], [10], [11], [12], along with the potential benefit of the complementary image production provided by the two imaging modes. Moreover, through the rat colorectum imaging study, we showcased an approximate level of 3D vascular visibility that could be attainable by PAE in actual clinical practice in the future. To the best of our knowledge, the presented image is the first to visualize the vascular network distributed in the colorectal wall of a live rat based on the highest PAE-resolving power ever achieved and also over almost the entire area (longitudinal and angular); therefore, this enabled in the in vivo visualization of the arteriolar and venular networks that suggested their morphological hierarchism [47] expressed in the colorectum [Supplementary Fig. S5] (note that conventional capillary-level vascular morphology investigation studies, of course for the colorectum as well, mostly relied on invasive approaches, like vascular corrosion casting [49]).

Early detection of angiogenesis—a phenomenon in which malignant tissues continuously secrete substances that induce neovascularization around them to receive nutrients and oxygen needed for rapid growth—is an important issue in real-world practice [50]. As over 85% of all cancers originate in the epithelium that lines the inner surfaces of organs throughout the body [51] and also since such a vasculature pattern is regarded as an important indicator for pre-malignant or malignant lesions, the OR-PAE-based high-resolution mapping of a vasculature distributed in the epithelium of the GI tract would be diagnostic for early cancer detection as well as for related clinical decisions. Currently, the narrow band imaging (NBI) technique that is frequently utilized for disease screening in routine GI video endoscopic procedures also works based on the above major premise and it is regarded as the best option among currently available tools [52]. Early detection of dysplasia developed in the epithelium is extremely important because if a tumor has already invaded a depth of over 1 mm when it is found, it implies that the cancer might already be rather advanced—that is, it is not a good sign at all in terms of the early diagnosis of cancer. In this regard, our OR-PAE-based approach, which focused on the resolution rather than the penetration depth, would make a feasible contribution because it does not function on such a simple optical filtering-based surface vasculature mapping as that applied in NBI (note that the development of confocal laser endomicroscopy [53] is also being pursued in a similar context). Apart from the clinical perspective, the presented endomicroscopic technique could inspire animal model-based colorectal tumor microenvironment studies [54], [55] on account of the label-free visualization of capillaries in vivo and in situ.

Further, the high-resolution visualization of capillary networks as low as ~13 µm was possible due to the satisfactory operation of the optical rotary junction and the distal optics implemented based on a 10-μm core diameter optical fiber, which was the narrowest among all the previous in vivo-applicable, torque coil-based PAE-EUS probes or IVPA catheters reported thus far [22], [23], [24], [32], [33], [34], [37], [38]. Most of all, as the deposited laser pulse was more tightly focused inside a target tissue, the required laser pulse energy could also be reduced to less than 1 μJ, which enabled a laser dose reduction effect of more than one order of magnitude compared to those applied to the previous IVPA probes that were implemented based on larger core diameter optical fibers (typically >100 µm core) [32], [33], [34], [37], [38]. We believe that this was also possible because the illumination optics and dual transducers were adequately assembled within an acceptable tolerance in terms of their collinearity. Moreover, the hermetic sealing of the acoustic matching medium inside the catheter as well as the applicability of the rotary transformer-based signal coupling mechanism near the proximal has been successfully demonstrated for the integrated PAE-EUS imaging technique, which was also the first in PAE. Supplementary Table S1 provides a comparison among different system features and specifications between the current and previous studies.

However, along with its positive aspects, this study also showed a fundamental limitation—that a continuous 3D vasculature image could not be acquired from the in vivo swine esophagus imaging experiment. Considering that the key benefit of PAT is that it is capable of visualizing 3D vasculature, the presentation of simple B-scan images would not be so useful for an endoscopist. Through the presented first intra-instrument channel-based swine esophagus imaging experience, we recognized that a simple improvement in the scanning speed (e.g., >30 Hz) would not be sufficient to acquire the continuous 3D vasculature map because a stable 3D data acquisition—if it is still based on the manual pullback of the entire sheath of the mini-probe through the instrument channel—would not be fundamentally possible in an actual clinical environment in which strong motion artifacts are frequently involved. In this regard, the development of a next-generation endoscopic system with a rapid self-3D internal helical scanning mechanism just inside an engaged probe sheath would be most essential. In addition, as we also learned from this study that unstable acoustic coupling was the main factor causing the failure of the intended signal acquisition, further equipping the related probe with the conventional medical balloon-based acoustic coupling method typically utilized in the EUS technology [9], [10] is expected to significantly improve the work efficiency for rapid disease screening over virtually the entire angular region by reducing the motion artifacts as well as the instability of the acoustic coupling between the probe and the lumen wall. By doing so, the entire wall of the target lumen can also be positioned at the optimal working distance of the laser beam; thus, all blood vessels can be visualized at the highest resolution. A PAE probe equipped with a capsulized distal housing [25] is being developed on the basis of this context, although the affixation of such a hard case rather than an inflatable balloon will likely not permit its easy intra-instrument channel use.

In undertaking this study, we did not hesitate to diminish the voltage level of the US excitation pulse because our top priority was to ensure that the US echo signals reflected from the imaging window were not saturated and also because we believed that, in terms of the dual-modality-based image interpretation for a target tissue, it would not be necessary to acquire US images over a deeper region than the co-registered OR-PAE image, which has a very shallow image depth—typically less than ~1 mm. Although the outer regions of the rat colorectum still produced reliable US image contrast according to the applied approach, in the swine esophagus imaging experiment, we were unable to acquire useful image features within the implemented imaging FOV mostly because of the weakened SNR and also the fundamental lack of intrinsic contrast sources within the demonstrated imaging depth (<3 mm). Consequently, when pursing the clinical translation of this technique, it would be a better direction to harness the deep imaging capability of US imaging for the purpose of visualizing neighboring anatomical landscape over a radial range that is as wide as possible rather than to focus on the image co-registration over the same imaging depth (refer to [56] as a related example). In this case, the large difference in the transverse resolution between the PA and US modes would not be a serious issue. In addition, the signal balancing issue that came to the fore could possibly be resolved by applying a different gain to each imaging mode while maintaining the conventional excitation voltage level (>100 V) for the US imaging mode, rather than merely sacrificing the excitation voltage (as done in the current study). Moreover, applying a log amplifier rather than a linear amplifier to the US imaging mode would be effective to cover the large dynamic range of the US signals that bounce back from soft tissues and the polymer imaging window. On the other hand, if producing high-resolution PA vasculature images over a depth exceeding 1 mm is still essential for a certain clinical application, the reconstruction-based image production approach that was presented in [57] could be a viable option. In this case, the required high laser energy can be delivered to a target issue by employing a double-clad optical fiber, as utilized in [39].

In this study, the laser pulse energy applied to in vivo experiments was set between 0.7 and 1 μJ. Although the 8-μm beam diameter of our endoscope was less tight than those of typical OR-photoacoustic microscopy systems [58], [59], the applied energy is much greater than the typical values utilized for OR-PAM and also far exceed the ANSI safety limit [60] in terms of the fluence. We set the energy value in this manner to attain the necessary SNR for visualizing the mesh-like capillary network; however, this does not mean that it is an essential value to operate the system; a typical PAE image quality can be achieved using even lower energy. Thus, the applied energy, the beam diameter, and the synthetic detection aperture or solid angle created by the dual transducers need to be understood as reference values when designing an advanced PAE probe in a follow-up study. However, since regulating the safety criterion remains important, future research must further investigate the pulse energy issue. In a wild prediction, employing a more sensitive US transducer made of advanced piezoelectric material [24] or applying a tighter single-mode optical fiber-based optical focus could decrease the required laser pulse energy. In addition to the mentioned issues, identifying a method to minimize interference noise, centralize the rotation axis of the scanning tip in a stable manner, and embody a dedicated probe sheath with different wall thicknesses between the imaging window and the body section (which also includes braided metallic wires) would accelerate the clinical translation of the integrated PAE-EUS technique.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.