Project SWAVE 2.0: An overview of the study design for multimodal placental image acquisition and alignment.

Development of non-invasive and in utero placenta imaging techniques can potentially identify biomarkers of placental health. Correlative imaging using multiple multiscale modalities is particularly important to advance the understanding of placenta structure, function and their relationship. The objective of the project SWAVE 2.0 was to understand human placental structure and function and thereby identify quantifiable measures of placental health using a multimodal correlative approach. In this paper, we present a multimodal image acquisition protocol designed to acquire and align data from ex vivo placenta specimens derived from both healthy and complicated pregnancies. Qualitative and quantitative validation of the alignment method were performed. The qualitative analysis showed good correlation between findings in the MRI, ultrasound and histopathology images. The proposed protocol would enable future studies on comprehensive analysis of placental anatomy, function and their relationship. ● An overview of a novel multimodal placental image acquisition protocol is presented. ● A co-registration method using surface markers and external fiducials is described. ● A preliminary correlative imaging analysis for a placenta specimen is presented.

Specifications table

Method details

Background

Contrary to the historical notion of the placenta as a passive conduit for nutrients, current evidence suggests that the ephemeral materno-fetal organ essentially plays the role of master regulator in the intrauterine environment [27]. Any structural or functional abnormalities during the dynamic developmental process of the placenta would impact the health of the mother and the fetus, both during and beyond the prenatal period [21,36]. The placenta plays a major role in the pathogenesis of many pregnancy complications such as preeclampsia (PE), fetal growth restriction (FGR), and pre-term labor. These complications affect up to one-third of all pregnancies and are leading causes of maternal and perinatal mortality and morbidity globally [40,41].

The placenta, being a dynamic organ, undergoes structural and functional change throughout the pregnancy. Antenatal monitoring of placentas could provide a useful clinical tool to predict pregnancy outcome. Moreover, the placentally mediated complications are often the late manifestations of a chronic pathological process, preceded by a long sub-clinical phase [38]. This sub-clinical phase, potentially associated with specific pathological mechanism, presents a unique opportunity for early detection and intervention enabling improved perinatal outcome. At the same time, this poses the challenge of accessing the placenta in utero with the paucity of clinical tools currently available for in vivo assessment of placenta microstructure [27,33].

Current research efforts directed towards developing non-invasive and in utero placenta imaging techniques offers the promise of identifying early and sensitive biomarkers of placental health [29,41]. Correlative imaging that acquires, aligns and fuses complementary information using multiple multiscale modalities from the same specimen is particularly important to advance the understanding of placental structure, function and their relationship [42]. Multiscale structural imaging including ultrasound (US) and magnetic resonance imaging (MRI) combines the high axial and lateral resolution achievable using US and the large field-of-view in MRI. More importantly, the quantitative counterparts of these modalities, qMRI (quantitative MRI) and QUS (quantitative ultrasound), can further offer complimentary information regarding the placental micro- and macro- structure, perfusion and function [2,10,15]. Among the qMRI techniques, Diffusion Weighted Imaging (DWI) provides a parametric apparent diffusion coefficient (ADC) map, where the ADC value at any voxel represents the degree of water diffusion. IUGR complicated pregnancies are associated with significantly lower ADC measure compared to those in normal controls [19]. T2 relaxometry is another qMRI parameter, that measures the hydrogen proton relaxation, a characterizing property of tissue. T2 relaxation time in the placenta decreases with advancing pregnancy, as well as with the onset of IUGR, as a result of increased fibrin deposition, fibrosis, necrosis and infarcts and reduction in oxygen saturation [3]. Oxygen Enhanced MRI (OE-MRI) and Blood Oxygen Level-Dependent Imaging (BOLD) are two MRI sequences, that have been used to measure oxygen saturation. BOLD MRI has been found to reflect tissue oxygen saturation, which could differentiate placentas in controlled hyperoxia and in IUGR [39]. Susceptibility-weighted imaging, on the other hand, is sensitive to compounds with paramagnetic or ferromagnetic properties, and therefore, can detect deoxyhemoglobin, ferritin, and calcification in tissue. This sequence has been used to enhance visualization of venuous structures [28]. Also, qMRI measures from T2- and diffusion weighted images were found to differentiate placental pathology, including maternal vascular malperfusion, masssive perivillous fibrin deposition and chronic villitis of unknown etiology [4]. QUS, on the other hand, provides the acoustic and mechanical tissue properties. Pathological findings such as infarction, ischemic change, inflammation, and fibrosis have been found to increased elasticity, a QUS parameter, in different organs, such as liver [18]. Among the few studies showing the correlation between placental pathology and local variation in elasticity, it has been found that infarction and collagen deposition are associated with elevated elasticity value [31,37]. Another QUS parameter, attenuation coefficient, has been found to be correlated with the spatial variation of fat in the placenta [13]. The combination of the different properties obtained through multimodal imaging and aligned using a correlative registration process would present a unique signature of a tissue type or its pathological state. Also, ex vivo studies would be important first step towards in utero placental imaging that evaluate the feasibility of the proposed imaging system and enable the establishment of reliable baseline measurements of qMRI and QUS parameters with controlled variables.

Project SWAVE 2.0 was initiated in 2018 with the objective of identifying quantifiable measures of placental health using a multimodal correlative approach. The first phase, SWAVE (shear wave absolute vibro-elastography), initiated in 2015, was designed to measure the mechanical properties of placenta ex vivo using the namesake elastography technology [1]. The ultrasound data acquired in the first phase were later successfully implemented for characterizing and establishing normative QUS values for normal placentas [12]. The initial phase was followed by SWAVE 2.0, where a multimodal image acquisition protocol was designed to acquire and align data from ex vivo placenta specimens derived from both healthy pregnancies and pregnancies with preeclampsia and/ or intrauterine growth restriction. This unique project, which is first of its kind, involved extensive data acquisition, including novel multiparametric quantitative ultrasound (such as elasticity, attenuation coefficient estimate, backscatter coefficient and effective scatterer diameter), multiparametric MRI, and histopathology. A co-registration method was proposed using surface markers and external fiducials to enable alignment and fusion of multimodal data. The proposed method has been utilized in our recent works [8,13]. Though explained within the framework of project SWAVE 2.0, the proposed co-registration approach is versatile and can be adopted in a wide range of experimental setups involving multimodal imaging of ex vivo placentas. Table 1 presents a possible (non-exhaustive) list of commercial US and MR imaging modes, which can be readily acquired with available packages from vendors and aligned using the proposed method. Also, possible applications of quantitative pathology currently being investigated by different research groups have been enlisted. We refer to the quantitative modes obtained using ultrasound as US+, MR as MR+ and pathology as pathology+.

Table 1

Example of commercial imaging modalities for which proposed alignment protocol is applicable.

Modality	Technique	Manufacturing Name
US+	Elastography	ShearWave™ Elastography (SWE) - Super Sonic ImagineShear wave elastography - Canon Virtual TouchIQ - SiemensElastQ and ElastPQ imaging - Philips2D Shear Wave Elastography - GE Healthcare
	Attenuation	Attenuation Imaging (ATI) - CanonUltrasound-Guided Attenuation Parameter (UGAP) - GE Healthcare
MRI+	Inversion Recovery(long r)	FLAIR - GE, Philips, HitachiTIRM - SiemensFastFLAIR - Canon
	Susceptibility-Weighted	SWAN - GE HealthcareSWI - SiemensSWIp - PhilipsFSBB - Canon
	Spoiled Gradient Echo	SPGR - GE HealthcareFLASH - SiemensT1-Fast Field Echo (FFE) - Philips, Canon
	Ultrafast Gradient Echo 3D(with preparation pulse)	3D Fast SPGR - GE HealthcareMPRAGE - Siements, Hitachi3D TFE - PhilipsFastFE 3D - Canon
	Fat-water separation	IDEAL - GE HealthcareDixon TSE - Siemens HealthcaremDixon TSE - PhilipsWFS - Canon
	Diffusion-weighted	Diffusion-weighted Imaging (DWI) - GE, Siemens,Philips, Canon, Hitachi
	Fat & iron evaluation	IDEAL-IQ - GE HealthcareLiverLab - SiemensmDixon Quant - PhilipsMR-Touch -GEElastography - Siemens, Philips, Resoundant
Pathology+	Cellular Phenotyping	Cell classification into classes: cytotrophoblast, fibroblast,Hofbauer, syncytiotrophoblast, vascular [16]
	Gestational age estimation	GestAltNet: deep learning for GA estimationfrom whole-slide images [32].
	Lesion detection	Decidual vasculopathy identification in whole slide images [6]

In this paper, we present an overview of the data acquisition protocol with the aim of supporting multimodal and multiscale placental research. We describe a unique approach for multimodal alignment using surface marker and external fiducial. Finally, we report the alignment accuracy of the proposed method. We also present sample correlative imaging analysis to show our capabilities and ongoing challenges.

Methods

Subjects

In this case-control study, placentas (n = 46) were collected from a group of women (n = 40) who delivered via cesarean delivery at BC Women's Hospital, Vancouver, Canada. We included 26 placentas from pregnancies affected by preeclampsia (PE) and/or intrauterine growth restriction (IUGR). Preeclampsia was defined as the presence of gestational hypertension with proteinuria and a pregnancy was considered to be complicated by IUGR if the fetal abdominal circumference is below the 10th percentile measured from routine ultrasound examination. Of the 26 placentas complicated by PE and/ or IUGR, there were 16 IUGR placentas, 1 PE placenta and 9 placentas affected by both IUGR and PE. Among all cases affected by IUGR, 15 were categorized as early onset (diagnosed at <32 weeks), 2 were late onset (diagnosed at >32 weeks) and the rest were not recorded. Among the PE cases for which the gestational age at diagnosis were recorded, 6 were early onset (diagnosed at <32 weeks) and one was late onset. 4 of these placentas were from pregnancies additionally complicated by gestational diabetes mellitus. We included 10 controls from singleton pregnancies and 10 control twin placentas (from 5 dichorionic twin pregnancies) that were uncomplicated by PE and IUGR. Among our control placentas, one singleton and four twin placentas were from pregnancies affected by gestational diabetes mellitus. Table 2 presents the clinical characteristics of the study cohort by group [35].

Table 2

Clinical characteristics of pregnancies for placentas studied.

Mat. Parameter	Normal Controls (n = 15)			IUGR and/or PE (n = 24)			p-value
Gravidity	Median = 2	25–75% =1	Range = 1–8	Median = 2	25–75% = 1.8	Range = 1–6	0.33
Parity	Median = 1	25–75% = 1.3	Range = 0–3	Median = 0	25–75% = 1	Range = 0–3	0.18
Gestational Age (weeks)	Median = 39.0	25–75% =2.1		Median = 34.3	25–75% =6.9		3.5e-6
Maternal Age (years)	Median = 34.5	25–75% = 4		Median = 32.5	25–75% = 8		0.43
Smoking Status	Ex-smoker	Non-smoker	Unknown	Ex-smoker	Non-smoker	Unknown
	n = 0	n = 12	n = 3	n = 3	n = 18	n = 3	0.64
Hypertension	Yes,	No,	Unknown,	Yes,	No,	Unknown,
	n = 0	n = 15	n = 0	n = 12	n = 8	n = 4	0.62
Diabetes	Yes,	No,	Unknown,	Yes,	No,	Unknown,
	n = 3	n = 12	n = 0	n = 4	n = 16	n = 4	0.41
Neonat. Parameter	Normal Controls (n= 20)			IUGR and/or PE (n= 26)			p-value
Birth weight (gm)	Median = 3395	25–75% = 908		Median = 1539	25–75% = 1406		2.1e-8
Placenta weight (gm)	Median = 480	25–75% = 131		Median = 219	25–75% = 173		1.64e-6
Gender	Female	Male	Unknown	Female	Male	Unknown
	n = 9	n = 10	n = 1	n = 15	n = 9	n=2	0.62

The exclusion criteria included any gestational abnormality identified prior to delivery or pregnancies involving treatment with investigational medication. The study (H17–00331) was performed under written informed consent of all participants after approval by the University of British Columbia Children's and Women's Research Ethics Board. Study data were collected and managed using REDCap electronic data capture tools hosted at British Columbia Women's and Children's Hospital. Clinical data extracted from the participants health record (maternal and neonatal charts) included maternal age, parity, risk factors for placental disease (history of smoking, hypertension, diabetes), birth date and time, gestational age at diagnosis, gestational age at birth, birth weight, cord blood pH, and admission to neonatal intensive care. All 46 placentas underwent the steps in the proposed protocol as was described in the following sections. However, the digital photographs were not available for 5 placenta cases.

Placenta preparation

Placentas were stored at 4°C until the examination. Examinations were initiated between approximately 6 and 48 h after delivery and completed within 5 days of delivery. At the beginning of the examination, a perinatal pathologist (J.T.) removed the umbilical sac and identified externally appreciable placental disk lesions. Layers of acoustic absorbing pad (Aptflex F28, Precision Acoustics, UK) were placed beneath the maternal surface of the disk to reduce reverberation artifacts. The placenta was secured using two rubber bands at the edges to the absorbing pad layers and then placed in a rigid plastic container to minimize any deformation while transporting from one imaging modality to another.

Region-of-Interest identification and annotation

The placenta was scanned using a SonixTouch ultrasound machine (Analogic Corp., Richmond, BC) and an L14- 5/38 linear transducer at the PRIME (Perinatal Research and Imaging Evaluation) center, located at BC Women's Hospital and Health center. A sonographer (V.L.) scanned the placenta to locate two regions-of-interest: (i) a homogeneous region without macroscopic or sonographically visible abnormalities, and (ii) an inhomogeneous region with suspected lesions identified during gross examination or ultrasound scan, such that both regions lie on the same axial-lateral plane. The suspected lesions, if found, were further annotated, measured and recorded using the ultrasound built-in interface. Water, instead of ultrasonic gel, was used as the conductive medium to prevent smearing of ink used for marking the identified locations. India ink was applied using a paint brush to mark the identified regions and the corresponding plane-of-interest on the fetal surface of the placenta. Specifically, while holding the ultrasound probe at each identified region, we marked the location of the probe. For each identified region, we then drew an in-plane line connecting the marks indicating probe edges, drew another line perpendicular to the in-plane line at the mark indicating probe-center and finally extended the two in-plane lines for the two regions (Fig. 1(a)) indicating the plane-of-interest. We then placed cod liver oil pills, considered as MRI-visible fiducials, at the edges of the placenta along the in-plane line and a flexible ruler along the in-plane line marking the plane-of-interest on the fetal surface. Photographs of the fetal surface of the placenta, with the ruler, markings, and the fiducials, were taken for future reference (Fig. 1(a)). The simple but powerful approach, ensuring alignment among multiple modalities, can be adopted to a range of different modalities and imaging of different organs.

Fig. 1

Overview of project SWAVE 2.0. (a) Top: scanning of the placenta to locate two regions-of-interest: a homo-geneous region, and (ii) an inhomogeneous region; mid: an example showing an identified inhomogeneous region-of-interest. A lesion has been identified, annotated, measured and recorded using the ultrasound built-in interface; bottom: an example showing the marking of the placenta using India ink marker, cod liver oil pills, and paper ruler. (b) Top: an overview of the QUS imaging setup using a SWAVE (shear wave absolute vibro-elastography) imaging system. Bottom: the positioning of the transducer to image the placenta at the marked location (left) and the graphical user interface of SWAVE system showing the elastography phasor map (right). (c) An overview of MR imaging setup. (d) An example showing histologic sectioning of the placental at the marked plane-of-interest.

Magnetic resonance imaging

The MRI protocol was reviewed and approved by the BC Children's Hospital (BCCH) MRI Research Facility's Protocol Review Committee. MRI scans were performed on a GE Discovery MR750 3.0T MRI scanner (GE Healthcare, Milwaukee, Wisconsin) using an 8-channel cardiac array (Fig. 1 (c)). The placenta was positioned such that the laser beam localizer aligns to the mark on the fetal surface indicating the plane-of-interest (Fig. 3). Three-plane-localiser using a standard spin-echo pulse sequences were first performed to visualize the overall structure of the placenta and to facilitate the subsequent acquisitions. The placenta was then scanned using the sequences (Fig. 2 (b)) enlisted in Table 3.

Fig. 3

The co-ordinate systems involved in the multimodality imaging system: world coordinate system W, ultrasound coordinate system U, pathology coordinate system P, MRI coordinate system M and the transformations between them.

Fig. 2

Multimodality images obtained from SWAVE 2.0. (a) Ultrasound B-mode image and the Quantitative Ultrasound (QUS) images, including Attenuation, Backscatter Coefficient, Scatterer Size and Elasticity for an example placenta. (b) MR images of the example placenta obtained using different sequences. (c) pathology image of an example placenta.

Table 3

MR Imaging sequences and parameters.

Sequence Name	Echo Time (ms)	Repetition Time (ms)	Flip angle (°)	Slice thickens (mm)	FOV (mm)	Other
T1-weighted fast-spin-echo (T1 FSE)	8.16	850	111	2	220	N/A
T2-weighted fast-spin-echo (T2 FSE)	110	2500	142	2	220	N/A
T2-weighted fluid-attenuation-inversion-recovery (T2 FLAIR)	120	12,000	160	2	220	Inversion Time: 2708 ms
Proton Density fast-spin-echo (PD FSE)	9.3	3000	111	2	220	N/A
3D Susceptibility-weighted Angiography (SWAN)	32.6	40.6	15	2	240	N/A
3D fast spoiled gradient (FSPGR)	3.2	7.9	12	2	240	Inversion Time: 450 ms
Six-echo spoiled gradient echo (SPGR)	1.9, 2.6, 3.3, 4.0, 4.7, 5.4	125	10	4	210	N/A
T2-weighted fast-spin-echo (FSE) FLEX	100	8961	111	2	220	N/A
Diffusion weighted imaging (DWI) using spin echo	62.4	6000	90	4	240	b values: 0, 1000s/mm^2

Except for 3D SWAN and 3D FSPGR, the other sequences were performed in 2D and slices were acquired for covering the pills in axial direction. The 3D sequences, 3D SWAN and 3D FSPGR acquired slices to cover the entire placenta.

Quantitative ultrasound imaging

The quantitative ultrasound (QUS) imaging of the placenta was performed using a SWAVE (shear wave absolute vibro-elastography) imaging system [1]. Elastography is a non-invasive technique where an external force is applied to measure the mechanical properties of tissue, such as shear wave speed and Young's modulus. The ultrasound radio frequency (RF) data acquired in SWAVE can further be processed to compute complementary acoustic properties, such as attenuation coefficient estimate, backscatter coefficient and effective scatterer size [9,10,12]. Young's modulus provides a measure of the tissue stiffness by computing the speed of the shear wave propagation. Attenuation coefficient, backscatter coefficient and effective scatterer size quantify the acoustic properties of the underlying tissue. Attenuation coefficient measures the ultrasound energy loss with propagation depth due to scattering and absorption. Backscatter coefficient, on the other hand, measures the ultrasound intensity that is scattered in the backward direction. Backscatter coefficient can further be used to estimate the effective size of the dominant scatterer, with the effective scale being determined by the incident ultrasound frequency.

The SWAVE system in our project consists of an Ultrasonix SonixTouch ultrasound machine (Analogic, Richmond, BC, Canada), with a 4DL14–5/38 linear array transducer and a m4DC7–3/40 curved array transducer (Ultrasonix, Richmond, BC, Canada) (Fig. 1 (b)). The placenta fixed within the container was placed on a custom-built plexiglass shaker platform. The shaker platform, along with the placenta container, was then submerged in a constant-temperature water bath (Cole-Parmer, Montreal, QC, Canada) with temperature set at 37°C, to ensure a sound of speed value of 1540 m/s. The shaker platform is connected to a voice coil motor through a rod, that generates a multi-frequency (between 40 and 200 Hz) excitation. A set of four springs are affixed to the platforms corner that allow its vibration along the vertical direction. The excitation frequencies and amplitude are controlled by a custom-built control unit. During data acquisition, the vibrating shaker platform generates shear waves within the placenta. The transducer was submerged in the water bath with a few millimeters between the transducer face and the fetal surface unlike traditional contact ultrasound acquisition to avoid deformation. The transducer was positioned to image the placenta at the marked location while being parallel to the platform and the water bath. Volumetric ultrasound radio-frequency data were captured and stored for offline processing to reconstruct QUS maps, including elasticity, attenuation coefficient estimate, backscatter coefficient, and effective scatterer size (Fig. 2(a)). The time requirement for the data acquisition process were 6.3 s and 10.8 s using the curved and linear transducer, respectively.

Histopathology

After imaging, each placenta underwent pathological examination including the measurement of the placental weight and dimensions. The placenta was sampled in such a way that the full-thickness sections included the volumes of interest acquired with SWAVE and aligned with the plane-of-interest marked during placental preparation and imaged with ultrasound and MRI (Fig. 1 (d)). Up to 4 sections (2 µm each) were obtained per placenta. After routine formalin fixation, processing, and hematoxylin and eosin staining, the slides were scanned with the Aperio ScanScope system (Aperio, Vista, CA) at a 400 × microscope resolution at the BC Children's Hospital Department of Pathology (Fig. 2 (c)). The pathologic slides were reviewed by an experienced perinatal pathologist (J.T.) according to a routine protocol [25].

Multimodal registration

The multimodal imaging system involved three coordinate systems, including world (W), MRI (M), ultrasound (U) and pathology (P) coordinate system (Fig. 3). A point p in A coordinate system is transformed to the B coordinate system as follows:where defines the rigid transformation matrix that transforms from coordinates A to B.

The digital photograph (Fig. 5(b)) of the fetal surface along with the marking, ruler and MRI-visible fiducials, taken at the PRIME, provides the locations of ROI 1 and ROI 2 with respect to the world (W) coordinate system. The relative position of the regions-of-interest (ROI 1 and ROI 2) with respect to US (U) and pathology (P) coordinates are known, as the US volumes and pathology images are centered around the ROI locations marked in the PRIME center. To attain the co-registration of the ROIs in different modalities, their relative locations with respect to the MRI coordinate system (M) has to be identified. The MR coronal slice corresponding to the fetal surface shows the fiducials, as well as the placenta boundary. Therefore, the locations of the ROIs with respect to MRI coordinates (M) can be identified by registering the digital photograph and the coronal MRI plane. Finally, the multimodal co-registration can be attained by performing US-MRI and pathology-MRI registration. According to our proposed image acquisition protocol, the center plane in the US volume, MRI volume and histopathology image volumes are aligned. This reduces the multimodal registration to a 2D registration problem, i.e. registering the ultrasound central plane and 2D pathology image to the MRI central axial plane.

Fig. 5

Co-registration between digital photograph taken at the PRIME center and the coronal slice of the MRI localiser sequence. (a) Point clouds before and after registration. Co-ordinates of the cod liver oil pills as well as a portion of the placenta boundary were used to generate the point cloud in the moving and reference images. (b) The digital photograph was used as the moving image. The line connecting the pills indicates the plane-of-interest. Two other lines perpendicular to this line mark the two regions-of-interest ROI 1 and ROI 2 to be imaged in US and pathology. The dot indicates the location with suspected lesion. (c)The coronal slice corresponding to the fetal surface acquired during 3D localiser sequence was used as the reference image. (d) Registration results where ’pink’ denotes the moving point cloud and ’green’ denotes the fixed point cloud. Magnified views show the overlapping of the oil pills and the fetal surface boundaries between the registered images. The distance between the centroids of the pills in the registered digital photograph (130.6 mm) closely matches to the measure indicated by the ruler (132 mm). (e) The corresponding regions-of-interest ROI 1 and ROI 2 in axial MRI have been localized using the registration result.

The US-to-world () transform is known as the coordinates for ultrasound and the pathology are defined using India ink marking in the PRIME center. To attain the co-registration among the multimodal coordinates, the MRI-to-world () transform has to be estimated

The steps for the registration between the MRI and the digital photograph are as follows:

The coronal slice that contains the fiducials, was identified in the MR localiser sequence. This is typically the fetal surface captured in the digital photograph.

On the fetal surface, two point sets were created, the reference point set Y = (yl, …, yM)T (a M × 2 matrix) from the MR image and the moving point set X = (xl, …, xM)T from the digital photograph. The point sets were created using the MATLAB function drawfreehand by outlining the fiducials and corresponding placenta edges (Fig. 5(a)) in the MRI and digital photograph. To perform the alignment between the two point sets according to the coherent point drift (CPD) algorithm [34], we construct a Gaussian Mixture Model (GMM) by considering the points in Y as the centroids of the GMM, and fit the model to the points X by maximizing the likelihood function. An expectation-maximization optimization is performed to solve the registration problem, i.e., compute the transform matrix (code: https://github.com/Farah-Deeba/PlacentaRegistration).

The digital photograph was registered to the MRI using computed during the previous step.

The registration performance was evaluated by measuring fiducial registration error (FRE) and target registration error (TRE). FRE is calculated as the Euclidean distances between the corresponding fiducials (that were used for registration) in the reference MRI and the registered digital photograph. TRE, on the other hand, is the Euclidean distances between corresponding points (which were not used for registration) in the reference MRI and the registered digital photograph. Two points for each placenta were selected by identifying anatomic landmarks visible in both modalities, such as large vessels and cord insertion site.

The ROI locations along the axial MRI plane corresponding to the US and histopathology slides are computed by measuring the distance between the fiducials and the ink marks in the registered digital photograph.

The registration of the center plane in ultrasound volume and the histopathology plane to the MRI central plane are then fine-tuned using anatomic landmarks that were clearly identifiable in all modalities. These landmarks included the specimen contour along the maternal and fetal surface and lesion boundaries. The rigid transformation matrix was generated using the MatchPoint Registration plug-in in MITK [43].

Possible sources of error include the (i) alignment of ultrasound probe during QUS imaging, (b) alignment of laser guide in MR, and (iii) sampling the placenta during histopathology (Fig. 4). Though these alignments are carried out manually, the alignment error is approximated to be limited to a few millimeters in each case.

Fig. 4

Possible sources of registration error during data acquisition. Left: ultrasound probe alignment process during QUS imaging using SWAVE and right: full-thickness placenta sampling, both along the line indicating the plane-of-interest.

Statistical analysis

The registration algorithm implementation and the statistical analysis were performed in MATLAB 2020a (The MathWorks Inc., Natick, MA, USA). Comparison of demographic and clinical characteristics between controls and diseased cases were performed with the non-parametric Mann-Whitney U test, as the sample size was relatively small and the distributions of the estimated measures were not strictly Gaussian. We consider a p value < 0.05 as indicating statistical significance.

Method validation

Registration performance evaluation

Fig. 5 shows qualitative registration results for a representative placenta case. Fig. 6 presents the quantitative performance, where TRE and FRE and their mean values have been shown for 41 placenta cases. The digital photographs were not available for 5 placenta cases. Without the information of the relative positions of the regions-of-interest in MRI, ultrasound and pathology coordinates, the multimodal registration for these cases was infeasible.

Fig. 6

Fiducial registration error (FRE) (left) and target registration error (TRE) (right) of the MRI/digital photograph registration algorithm of 41 placenta samples. The lines indicate the mean FRE (left) and mean TRE (right).

The registration method using the proposed approach was able to achieve mean FRE of 2.66 ± 1.44 mm and mean TRE of 5.49 ± 2.65 mm between MRI and digital image (the marking on which specifies the ultrasound and pathology coordinates). The TRE were comparatively larger which can be contributed to the large point localization error as a result of uncertainty in identifying corresponding features in the MRI and digital photograph. For example, the intersection of two vessels selected from slightly different plane (arising from the placenta height variation along the fetal surface) in the digital photograph and the coronal MRI would correspond to two different locations and thereby result in error.

Correlative multimodality imaging

We present a qualitative demonstration of co-registration among MRI, ultrasound and histopathology images for a placenta specimen (Fig. 7(a)). The particular case included a retroplacental hematoma, which was visible in all the imaging modalities. According to the attending radiologist, the lesion was hypointense in T2-weighted I sequence, whereas hyperintense in T1 FSE and DWI sequences [30]. In ultrasound, an acute hematoma appears slightly hyperechoic, becoming hypoechoic and echolucent with time [29]. In our case, the lesion appears to be hypoechoic, with echogenic border.

Fig. 7

Correlative imaging of a retroplacental hematoma for an example placenta. (a) Registered MRI, ultrasound, and histopathology images containing retroplacental hematoma at the maternal side of the disk (indicated by the arrows). (b) Profiles corresponding to multiparametric QUS and MRI measures along the white line showing in (a). The shaded area highlights the hematoma region.

The aligned multimodal data allows for correlative analysis among different parameters obtained from MRI and quantitative US. As an illustrative example, we show the correlation among quantitative ultrasound and MRI parameters along a vertical line through the hematoma (Fig. 7(b)). Among the quantitative ultrasound parameters, the hematoma region had a smaller scatterer size and elasticity value and larger backscatter coefficient value compared to the surrounding tissue. Three example MRI sequences showed similar trends along the vertical line shown in Fig. 7(a). Further analysis will investigate patterns from the correlative analysis of the quantitative parameters and relationship to diseases.

Method discussion

Multimodal multi-scale imaging of the placenta can potentially provide insight into placental structure and function. The integration of multiple modalities can create a composite and complementary view of the placenta under study. Multimodality imaging also allows the validation of the quantifications obtained from any single modality. Combination of different in vivo modalities measuring different properties of the placenta could be particularly informative. For example, the acoustic and mechanical tissue properties measured in quantitative ultrasound can be related to the functional measures such as diffusion measured in MR diffusion weighted imaging. However, to reap the benefit of multimodal imaging, the development of a structured workflow enabling sample preparation, data acquisition, data processing, and deposition and dissemination of large datasets is critical. This paper presents the standard workflow designed for the data acquisition for the project SWAVE 2.00. The workflow essentially lays the foundation of the subsequent data management, analysis and visualization. In terms of data analysis and visualization, we will investigate analysis and visualization approaches to explore correlations at different scales and different modalities. A possible approach could be to use proprietary software, such as DREAM 3D [23], that uses generic data containers for different length-scales data and represents each voxel in a dataset with multiple features, including features from input data as well as outputs obtained from analysis of the data, an approach that has been described in previous work showing advantage of real-time visualization in correlative imaging [5]. We would apply similar approach to organize the multiscale data as well as to facilitate visualization of the data.

One central problem of multimodality imaging is attaining the alignment among different modalities for enabling correlative analysis. Placenta, unlike many other organs such as the prostate [22], does not contain sufficient anatomic landmarks visible in different modalities. We present a surface-marker and external fiducial (cod liver oil capsule) based method to facilitate the registration process. We attain a mean fiducial registration error (FRE)< 3 mm, which was the mean distance of the corresponding capsule position in the digital photograph and the MRI. This amounts for 1.9% of the placenta dimension along the line connecting the fiducials. We also evaluated the mean target registration error, which was < 6 mm. The target registration error (TRE) was computed by identifying similar landmarks on the fetal surface of the placenta.

One source of error in registration arises from the assumption that the central planes in different modalities are perfectly aligned. Considering the thickness of india ink marking (in the range of a few millimeters) and the best efforts during aligning the laser guide in MR, probe positioning during US and slicing during pathology, the alignment error can be approximated to a maximum of a few millimeters. Also, rigid registration assumes that there were no deformations involved between different modalities. While this assumption holds true for US and MR imaging, the sectioning in pathology may introduce deformation and shrinking of the sampled tissue. Another source of error was due to the assumption that the fetal surface is contained in one coronal plane in the MRI. However, the variable depth of placentas led to different vertical position of fiducials, resulting in slightly different cross-section of fiducials in the selected MRI coronal plane compared to the one in the digital photograph. This limitation could be mitigated in future experiments by recording the placenta height at different points and registering to the plane computed by interpolating the MR coronal planes obtained from the corresponding heights. For evaluating the target registration error, ambiguity in identification of landmarks led to error in localizing points for a few cases (Fig. 6(b)). However, with the current performance metric, we can attain excellent alignment with less than 6 mm TRE among the digital photograph and the MRI. In the future study, we will present further analyses to show the detailed registration results among different modalities. We would also perform experiments to show the effect of the proposed registration approach on the improvement of FRE and TRE measures by including additional fiducials visible in different modalities.

The proposed protocol could serve as a simple yet practical tool for the placenta scientific community to replicate and extend ex vivo placenta imaging using a wide range of commercial and research imaging modalities. The protocol attains the goal of capturing US, MR and pathology images on placentas. The co-registration process with India ink markings and MRI-compatible pills is developed in a way that does not disrupt imaging using any of the three modalities. The alignment protocol includes readily available tools, such as ink markings, oil pills, laser guides, containers, ruler and camera as well as an open-source registration algorithm. Therefore, the protocol offers a promising tool for multimodal placenta image alignment, a crucial step for correlative imaging of placenta.

A major strength of the project SWAVE 2.0 is that it provides multiple quantitative ultrasound and MRI parameters. Unlike their qualitative counterpart, qMRI and QUS extract tissue properties which can be expressed as a number with units or relative to a reference material [24]. The scope of this paper does not cover the detailed analysis of this quantitative aspect. However, we present an example of placenta specimen with a lesion visible in all modalities. We present the correlation of a few representative modalities, including backscatter coefficient, scatterer size, elasticity, DWI, and proton density water fraction. The multimodality approach will open new opportunities for characterizing placental tissue under normal and pathologic conditions (i.e., IUGR and/or PE) and therefore identifying subtle change in placenta microstructure indiscernible in qualitative counterparts.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. The authors would like to thank the support Microsoft corporation, Schlumberger foundation, Canadian Institutes of Health Research (CIHR), and the Natural Sciences and Engineering Research Council of Canada (NSERC). Dr. Septimiu Salcudean and Dr. Robert Rohling hold director, executive and equity positions in Sonic Incytes, the commercial licensee of the S-WAVE technology and are co-inventors on S-WAVE patents.

Subject Area;	Engineering
More specific subject area;	Placental Imaging
Method name;	Multimodality Imaging of the Placenta Ex Vivo
Name and reference of original method;	NA
Resource availability;	NA