Deep learning-based reconstruction of in vivo pelvis conductivity with a 3D patch-based convolutional neural network trained on simulated MR data.

PURPOSE: To demonstrate that mapping pelvis conductivity at 3T with deep learning (DL) is feasible.
METHODS: 210 dielectric pelvic models were generated based on CT scans of 42 cervical cancer patients. For all dielectric models, electromagnetic and MR simulations with realistic accuracy and precision were performed to obtain B 1 + and transceive phase (ϕ± ). Simulated B 1 + and ϕ± served as input to a 3D patch-based convolutional neural network, which was trained in a supervised fashion to retrieve the conductivity. The same network architecture was retrained using only ϕ± in input. Both network configurations were tested on simulated MR data and their conductivity reconstruction accuracy and precision were assessed. Furthermore, both network configurations were used to reconstruct conductivity maps from a healthy volunteer and two cervical cancer patients. DL-based conductivity was compared in vivo and in silico to Helmholtz-based (H-EPT) conductivity.
RESULTS: Conductivity maps obtained from both network configurations were comparable. Accuracy was assessed by mean error (ME) with respect to ground truth conductivity. On average, ME < 0.1 Sm-1 for all tissues. Maximum MEs were 0.2 Sm-1 for muscle and tumour, and 0.4 Sm-1 for bladder. Precision was indicated with the difference between 90th and 10th conductivity percentiles, and was below 0.1 Sm-1 for fat, bone and muscle, 0.2 Sm-1 for tumour and 0.3 Sm-1 for bladder. In vivo, DL-based conductivity had median values in agreement with H-EPT values, but a higher precision.
CONCLUSION: Anatomically detailed, noise-robust 3D conductivity maps with good sensitivity to tissue conductivity variations were reconstructed in the pelvis with DL.

INTRODUCTION

The omnipresence of electromagnetic (EM) technologies in telecommunication and medical applications has increasingly stimulated research on the electrical properties (EPs) of the human body. Electrical properties (permittivity and conductivity) regulate the interaction between EM fields and human tissues. Accurate dielectric models of the human body are crucial in EM dosimetry , , (e.g. for safety purposes in telecommunication and MR scanning) and in treatment planning for therapeutic heating of cancer. , At RF frequencies, these properties reflect tissue water content and ionic concentration , and have been shown to change between healthy and pathologic tissues, , , , , suggesting potential use as a biomarker.

The majority of these EP studies and current computational dielectric models are based on ex vivo EP measurements. However, differences between ex vivo and in vivo EPs were reported. , Hence, non‐invasive techniques for patient‐specific EP detection were developed. , , One such technique, termed “MR Electrical Properties Tomography” (EPT), measures in vivo EPs at the Larmor frequency of the MR system (e.g. at 128 MHz for a 3T MR scanner). EPT exploits the fact that a subject’s EPs perturb the spatial distribution of the transverse magnetic field produced by the RF coil system for spin excitation, the field. The field is measurable with MR sequences. Therefore, tissue EPs can be reconstructed from measurements. , ,

EPT reconstruction algorithms to disentangle EPs from field measurements include physics‐based direct and inverse methods, and learning‐based techniques. Direct methods retrieve EPs by applying Maxwell’s equations to the measured field. , Second order spatial derivatives in these equations are typically computed as finite difference kernels, which are convolved with maps. Despite this straightforward implementation, the differentiation operation—normally a Laplacian kernel—is extremely sensitive to noise in measured fields and introduces numerical errors at physical tissue interfaces and/or acquisition artefacts. , , Moreover, several direct EPT methods rely on piece‐wise constant EPs and transceive phase assumptions. , , Denoising strategies, for example, , , were proposed and magnitude‐driven bilateral filters , or reformulations of full Helmholtz equation , were used to reduce boundary errors. Direct EPT techniques were employed in recent clinical studies evaluating the potential value of EPT‐based conductivity in discriminating tumours , and in hyperthermia treatment planning. Reconstructing good quality permittivity maps with clinical scanners and within acceptable times remains challenging: the necessary high precision requirements are unmet with standard mapping techniques.

Inverse approaches reconstruct EPs by iteratively minimizing a cost function comparing the true with a modelled field. , , , , These reconstruction techniques bypass the assumption of piece‐wise constant properties, reduce boundary errors and mitigate noise impact on EP maps by avoiding differentiation on measured data. Their challenges are represented by accurate modelling of 3D incident magnetic and electric fields and high computational cost, for example, to update the contrast source in CSI‐EPT. Promising reconstructions with inverse approaches were shown on simulated data, but no study to date has reported in vivo EP maps reconstructed with these techniques.

Learning‐based methods, namely dictionary‐based EPT and deep learning EPT (DL‐EPT), infer EPs from large datasets of fields and their corresponding EPs via machine/deep learning. The basic idea of DL‐EPT, for example, is to train a convolutional neural network (CNN) to learn the mapping relation between and EPs. Typically, the CNN is trained in a supervised fashion with multiple data and their corresponding EP distributions as input. So far, CNNs were trained on image or local level based on directly accessible MR quantities ( amplitude, and/or transceive phase, ϕ) from simulations or in vivo measurements. When tested in brain at 3T, the trained network produced conductivity , and permittivity maps with higher precision than in direct EPT maps. Nonetheless, an additional anatomical input (e.g. an MR magnitude image) was required besides the map for accurate EP reconstruction in Mandija et al. In Hampe et al the reliability of conductivity reconstruction in diverse brain geometries was strongly limited by the lack of large representative datasets for training.

Building on the challenges of previous DL‐EPT works, we introduce new methodological aspects for DL‐based conductivity mapping at 3T. Our DL‐EPT method employs a novel 3D patch‐based CNN which was trained exclusively on simulated B 1 fields with realistic accuracy and precision. Realistic accuracy and precision in B 1 fields were reproduced by implementing the framework developed in our previous study which combines EM and MR simulations. This enables training on realistic measurable datasets for which ground truth EPs are available, and stands as a valid alternative to training on in vivo MR measurements for which the true EP values are not available. In this study, we consider DL‐EPT for female pelvic anatomies with and without cervical cancer, because patient‐specific conductivity maps in this region would be relevant for hyperthermia treatment planning of cervical cancer patients. Thus, the training dataset comprised in silico fields of diversified human pelvic models. These pelvic models were derived from segmentation of CT scans of cervical cancer patients who underwent hyperthermia treatment and possessed realistic tissue EPs variations at 128 MHz, including EPs of both healthy and malignant cervical tissues. By examining two network configurations, that is, using complex B 1 data ( and ϕ) or only transceive phase information as input, we assess the conductivity reconstruction accuracy and precision of the proposed method in silico. We also discuss its performance regarding typical challenges for pelvis conductivity mapping: (i) greater size, which limits the validity of transceive phase assumption already at 3T ; (ii) high degree of anatomical variations (e.g. bladder filling, different inter‐patient organ positions, inter‐patient variation in body mass index); (iii) organ motion (bowel peristalsis and breathing); (iv) low SNR. Finally, we show DL‐based in vivo conductivity maps and compare them to the status‐quo EPT method for pelvis, that is, Helmholtz‐based EPT.

METHODS

Database construction

Human pelvic models

Forty‐two human models were built from CT scans of 42 cervical cancer patients who had undergone hyperthermia treatments. Patients’ CT scans were semi‐automatically segmented based on Hounsfield unit intensity with an in‐house developed software, resulting into fat, muscle, bone, and air segmentations. Tumour and bladder were manually segmented for all pelvic models by a radiation oncologist.

Construction of dielectric models

Tissue‐specific uniform distributions of realistic conductivity, σ, and permittivity, εr, values at 128 MHz (see Table 1) were defined for each tissue as explained in Supporting Information Appendix. Then, a total of 210 dielectrically different human models were generated out of the 42 anatomically segmented models. For every anatomy, 5 different combinations of piece‐wise constant conductivity and permittivity were randomly assigned to each segmented tissue from the predefined tissue‐specific EP uniform distributions.

TABLE 1

Electrical properties and relaxation times of tissues in pelvic models. Minimum and maximum values for the uniform distributions of both conductivity and permittivity are reported. T1 and T2 values assigned as input to MR simulations are listed as well. T1 and T2 values were taken from the following references: 54 for fat, bone and tumour; 55 for muscle, 56 for bladder

Segmented tissue	Tissue in dielectric model	σ (Sm−1) [min, max]	εr (rel. units) [min, max]	T1 (ms)	T2 (ms)
Air	Air	0	1	20	1
Fat	Fat	[0.04, 0.08]	[5.92, 8.89]	382	68
Muscle
Metal clips*	Muscle	[0.73, 1.07]	[49.77, 73.21]	1233	37.2
Plastic catheter*
Empty bladder
Full bladder	Bladder content	[1.35, 2.60]	[69.60, 72.62]	2400	100
Bone	Cancellous	[0.14, 0.18]	[21.03, 26.28]	586	49
	Cortical	[0.07, 0.08]	[14.72, 17.66]	586	49
Tumour	Tumour	[0.66, 1.45]	[37.35, 82.34]	1616	83

Non‐biological materials (metal clips and plastic catheters), present during CT scan session in some cervical cancer patients and manually segmented in the corresponding pelvic model, were considered dielectrically equivalent to muscle, because their segmentations were physically surrounded by muscle. These non‐biological materials are typically absent in human body.

To represent both healthy subjects and cancer patients in our dataset, one out of five tumour EP permutations was enforced to have muscle EPs for every anatomical model. Both empty and full bladder cases were included in the dataset. Empty bladders (present in 70/210 dielectric models) were given muscle EPs, as commonly adopted in hyperthermia treatment planning studies. , Full bladders were assigned bladder content EPs. Figure 1 illustrates the final EP distributions for each tissue in our complete dataset.

FIGURE 1

Final distribution of both conductivity (top row) and permittivity (bottom row) for all pelvic tissues in the complete dataset, after EPs were randomly assigned to all 210 dielectric pelvic models. Tissue‐specific conductivity and permittivity distributions are displayed together in “All tissues” (first column), and separately in the remaining columns. Red dashed lines represent EP limits (maximum and minimum of uniform distribution, also reported in Table 1) imposed a priori for every tissue. Note that the resulting EP distributions for muscle and fat were higher than for bone, tumour and bladder content, because of their higher number of voxels in pelvic models

EM simulations

EM simulations with realistic MR setup and patient positioning were performed in Sim4Life (v3.4, ZTM AG, Zurich, Switzerland) for all 210 dielectric models. First, a 3T birdcage coil (bore diameter = 70 cm) was modelled and tuned at 128 MHz. Subsequently, each dielectric model was positioned inside the birdcage coil such that the cervix/tumour location was axially centred with the isocenter. Furthermore, the model’s back was 13.5 cm distant from the isocenter, to mimic a patient lying on the MR table. A 2.5 × 2.5 × 2.5 mm3 mesh‐grid was enforced within a volume of 550 × 360 × 260 mm3 centred at the isocenter. This volume was chosen based on the models with largest dimensions, such that it contained the anatomy between L3‐4 vertebra to the perineum for each model. Outside this volume, the rest of the human model and the body coil were voxelized with a default resolution. Two FDTD simulations, i.e. in quadrature and anti‐quadrature modes, were run per dielectric model (15.755 × 106 cells) on a GPU (NVIDIA Titan X Pascal, 12 GB RAM) to calculate both transmit, , and receive, , fields. From these complex fields, amplitude, , and transceive phase, ϕ, were retrieved for all 210 dielectric models. was normalized to its mean value in the central slice. The total simulation time per model was 3.25 hours.

MR simulations

MR simulations emulating and ϕ acquisitions used in MR experiments were run as described in Gavazzi et al to account for sequence‐specific accuracy and precision. We implemented AFI and SE sequences in a Bloch simulator adopting experimental MR parameters (see MR experiments) and using T1, T2 (Table 1), and ϕ as inputs. In AFI, the nominal flip angle was scaled by . In SE, ϕ was applied as a phase shift to the RF pulses. Spatial encoding gradients were not included. MR signals were computed voxel‐wise at TE. For both AFI and SE, the resulting noiseless MR signals were used to retrieve noiseless and ϕ. was calculated from AFI signals as described in Yarnick and ϕ corresponded to the phase of SE signal.

To mimic a realistic image precision, real and imaginary parts of AFI and SE noiseless signals were corrupted by realistic Gaussian noise levels, and subsequently used to reconstruct noisy and ϕ. Realistic noise levels were chosen such that the simulated image SNR for both AFI and SE closely resembled experimental image SNRs obtained in the volunteer experiment for most tissues (see e.g. muscle, bladder and bone SNRs in Supporting Information Figure S1).

MR simulations were performed on dielectric models of dimensions 550 × 360 × 130 mm3, which consisted of the pelvic region only. Such smaller axial dimension was based on the FOV used for MR experiments and was adopted to reduce the computational cost when training the network. Both AFI and SE simulations were run in Matlab (R2015a; The MathWorks, Inc., Natick, MA) on a workstation with Intel Xeon CPU E3‐1220 v3 at 3.1 GHz and lasted in total 2 minutes per model.

MR experiments

A healthy volunteer and two cervical cancer patients were scanned on a 3T clinical MR scanner (Ingenia, Philips, The Netherlands) with a 28‐channel torso coil for reception. Patient 1 was diagnosed with IIB squamous cell carcinoma. Patient 2 had IIA2 neuroendocrine cervical carcinoma. Our imaging protocol was approved by the local ethical committee and all subjects gave written informed consent. The was acquired with AFI sequence (flip angle = 60°, TE/TR1/TR2 = 2.5/30/210 ms). The transceive phase was obtained by averaging phase images from two SE acquisitions with opposed gradient polarities to minimize eddy current effects. , SE settings were: imaging flip angle = 90°, TE/TR = 6.2/12000 ms. FOV was 370 × 259 × 120 mm3 for volunteer and 400 × 280 × 120 mm3 for patients. Resolution was 2.5 × 2.5 × 7.5 mm3 for volunteer and patient 1 and 5 × 5 × 7.5 mm3 for patient 2 (due to scan time limitations). In all scans, the vendor‐specific CLEAR option was used to replace the phase contribution of receive array with the receive phase of the body coil operated in anti‐quadrature.

For both patients, the tumour was delineated by a radiation oncologist on an ADC map aided by a T2‐weighted image. , For all subjects, muscle and bladder ROIs were manually delineated on 5 and 3 adjacent slices of SE magnitude image, respectively. Tumour delineation was transferred to SE magnitude image too.

Conductivity mapping

DL‐EPT

EPs locally distort the RF magnetic field in all three dimensions. To handle this physical problem, we sought a 3D network architecture for regression tasks trainable at a patch level. The compact, 3D CNN architecture by Li et al satisfied these requirements, thus was used for DL‐EPT conductivity reconstruction. Li et al’s network consists of 20 convolutional layers of 3 × 3 × 3 voxels, which are residually connected in pairs and are progressively dilated with increasing dilation factors (for more details on network architecture, see Ref. 49). The last layer had no activation function, differently from the original design in Ref.49 which was meant for classification problems. This network is currently implemented under the name of “highres3dnet” in NiftyNet (NiftyNet is an open‐source, TensorFlow‐based platform with modular structure for DL research in medical imaging (https://niftynet.io/)).

Tissue conductivity perturbs both and transceive phase, but it is predominantly encoded in the curvature of the latter. , , Based on this prior knowledge, we trained highres3dnet in both “full ” and “transceive phase only” configurations in order to assess whether differences in conductivity reconstruction exist between configurations. In “full B 1” configuration, input data comprised real and imaginary parts of a complex field calculated as · exp(i·ϕ). We refer to “NetEM‐B 1” when these input data were derived from (noiseless) EM simulations (i.e. · exp(i·ϕ)) and “NetMR‐B 1” when they came from MR simulations (i.e. · exp(i·ϕ)) corrupted by noise. The “transceive phase only” variant employed ϕ from noisy MR simulations as input and is indicated hereafter with “NetMR‐ϕ”. In both configurations, a binary mask of the pelvis was provided as additional input. Ground truth conductivity (σ GT) was given in input as regression target during training.

Before entering the network, input data were rescaled by subtracting their mean value, which resulted in input data with values not exceeding [−3, 3] units. Physically, removing this mean value has no effect on conductivity estimation with EPT. , Unlike usual DL endeavours, the standard deviation of input magnetic field data was not normalised to 1, because it holds information on spatial modulations from which the conductivity is retrieved in EPT. Before performing the aforementioned pre‐processing steps, in vivo and ϕ were interpolated to 2.5 × 2.5 × 2.5 mm3 (same resolution of in silico models used in training). Target conductivity σ GT was normalized within [−3, 3] units with fixed factors before training ( , a = 1.5, b = 0.5). These factors were used at inference (or test) stage to convert the inferred conductivity map from normalized to absolute values.

For both configurations, highres3dnet was trained and tested in 7‐fold cross‐validation on 180 (36 pelvic models × 5 EP permutations) and 30 simulated data, respectively. (In k‐fold cross‐validation (k = 7 in this study), the complete dataset (210 dielectric models) is randomly partitioned into k subdatasets of equal size (fold). One fold is retained for testing the CNN, whereas the remaining k − 1 subdatasets are employed for training. This process is repeated k times, such that each subdataset is tested once and serves as the validation data on which the accuracy is assessed. The overall accuracy is then obtained by averaging the accuracy results obtained for each of the validation k subdatasets.) All 210 dielectric models were tested once among all 7 folds. Training was performed with mean absolute error (MAE) loss function, L1 regularization with weight decay of 5.0 × 10−5 and Adam optimisation. MAE loss function and L1 regularization on network weights were chosen to reduce sensitivity to outliers and promote sparsity. During training, input data were randomly sampled into patches of 80 × 80 × 24 voxels. We chose this patch size after preliminary empirical tests with different sizes, taking also into consideration the FOV of input data and the depth of the network, which benefitted from relatively big patches for accurate reconstruction. The learning rate was 5 × 10−3 for 10000 iterations and was decreased to 8 × 10−4 thereafter. Training was stopped after 42500 iterations for NetEM/MR‐B 1 and after 35000 iterations for NetMR‐ϕ because the loss function had converged and the conductivity MAE averaged over all 7 folds was smaller than in other iterations for the majority of pelvic tissues. Training and testing were run on a GPU (NVIDIA Tesla P100, 16GB RAM). Training lasted 1‐2 days. Inference time was ~10 s for a single model/subject. Further details regarding parameter tuning are listed in Supporting Information Table S1.

The accuracy and precision of conductivity reconstruction were evaluated for NetMR‐B 1 and NetMR‐ϕ with the 7‐fold cross‐validation. Accuracy and precision were calculated for each pelvic tissue. The accuracy was indicated by the mean error (ME), defined as:where N is the number of voxels within the tissue. The precision was denoted by the difference between conductivity 90th and 10th percentiles, S 90 10:where σ (P) indicates the i th percentile for conductivity.

Helmholtz‐based EPT (H‐EPT)

For comparison, Helmholtz‐based conductivity was reconstructed by convolving a noise‐robust Laplacian kernel (7 × 7 × 5 voxels) with simulated and measured data. The transceive phase assumption was used. , ,

RESULTS

In silico results

Conductivity maps reconstructed with NetEM‐B 1 on noiseless simulated EM data displayed detailed pelvic anatomy (Figures 2 and 3B). In particular, tissue interfaces were sharply reconstructed. Only a few boundary voxels were assigned an incorrect conductivity value. This is a striking improvement over conventional Helmholtz‐based conductivity, which exhibited the well‐known “boundary errors” at tissue interfaces. These boundary errors prevented correct estimation in small tissue volumes, for example, bone in Figure 2. Moreover, anti‐symmetric errors in Helmholtz‐based conductivity (e.g. muscle) caused by degradation of the transceive phase assumption were absent in DL‐based conductivity. Nevertheless, DL‐based conductivity could display slight over‐/under‐estimation within a tissue: in the test model in Figure 3B, for example, conductivity was underestimated by 0.23 ± 0.27 Sm−1 (mean ± SD) in the bladder volume. Note that the general reconstruction performance, assessed with the cross‐validation, is discussed below. Figure 2 also shows that the tumour embedded in muscle was correctly “detected” when muscle/tumour conductivity contrast was sufficiently high (0.35 Sm−1) but was partly assigned muscle conductivity for low contrast (0.13 Sm−1).

FIGURE 2

Conductivity reconstruction for three in silico pelvic models (that share same anatomy but different tissue conductivity, pelvic model #15). Pelvis conductivity from a healthy subject (i.e. no tumour/muscle contrast) is displayed in top row. Conductivity from two cervical cancer patients is also shown: one with good tumour/muscle contrast (0.35 Sm−1, middle row) and one with poor tumour/muscle contrast (0.13 Sm−1, bottom row). Maps for ground truth conductivity (first column), Helmholtz‐based conductivity (σ H, second column), DL‐EPT conductivity tested with NetEM‐B 1 (third column) and the difference between NetEM‐B 1 conductivity and ground truth conductivity (fourth column) are reported

FIGURE 3

Impact of noiseless and noisy MR simulated input data on DL‐EPT conductivity reconstructed with NetEM ‐B 1, the network trained on EM simulations ( and ϕ). A, Ground truth conductivity. DL‐EPT conductivity maps tested with NetEM ‐B 1 using input data obtained from: B, (noiseless) EM simulations ( and ϕ). This is similar to the result reported in Figure 2. C, noiseless MR simulations (noiseless and ϕ), to determine the impact of sequence accuracy on DL‐EPT reconstruction. D, noisy MR simulations with realistic noise level on MR images (noisy and ϕ), to determine the impact of sequence noise propagation on DL‐EPT reconstruction. E, Conductivity map reconstructed with NetMR ‐B 1, the network trained on noisy MR simulated data (noisy and ϕ), to show the benefit of retraining with realistic MR simulated data. Magenta arrows in (C) and (D) point to an air pocket, the conductivity reconstruction of which is disturbed

Figure 3 illustrates the impact of using and ϕ ± simulated with AFI and SE sequences on DL‐EPT conductivity reconstruction with NetEM‐B 1. Figure 3C shows that the reconstructed conductivity was disturbed by the strong underestimation in air (e.g. in bowel, Supporting Information Figure S2) when noiseless simulated MR data were given in input. However, small errors, occurring in fat and bone due to their low T1 to which AFI is sensitive , (Supporting Information Figure S2), did not influence the conductivity. Using noisy simulated data as input for NetEM‐B 1 further perturbed the conductivity reconstruction (Figure 3D), especially in tissues with low precision, for example, air, muscle and bladder (see Supporting Information Figure S1). Conductivity reconstructions robust to both sequence‐specific errors and noise (Figure 3E) were obtained when the CNN was trained with input noisy simulated MR data (i.e. NetMR‐B 1).

Figure 4 shows that both NetMR‐B 1 and NetMR‐ϕ reconstructed comparable conductivity maps in two patients with distinct anatomies. This finding was also confirmed by the results of 7‐fold cross‐validation: as reported in Table 2, conductivity reconstruction accuracy (ME) and precision (S 90 10) in all tissues were comparable for both network configurations. ME and S 90 10 are also visualized for all test data and for all tissues in Supporting Information Figure S3: on average, ME was below 0.1 Sm−1 in all pelvic tissues, with minimum/maximum ME of approximately ±2.5 × 10−3 Sm−1 for fat, ±5.0 × 10−3 Sm−1 for bone, ±0.15 Sm−1 for muscle, ±0.23 Sm−1 for tumour and ±0.4 Sm−1 for bladder content. In general, S 90 10 was below 0.1 Sm−1 for fat, bone and muscle. For tumour and bladder content, S 90 10 was on average 0.2 and 0.3 Sm−1, respectively, with maximum peaks of 0.5 Sm−1 for tumour and 0.6 Sm−1 for bladder content. Interestingly, bladder content and tumour displayed the highest absolute values for ME and S 90 10 (Table 2). Note that they were the least represented tissues in our dataset (140/210 and 168/210 dielectric models showed bladder content and tumour EPs, respectively). Moreover, the comparable ME and S 90 10 among all test folds in Supporting Information Figure S3 denote that both network configurations were robust to data with different geometries and EPs. It is also worth mentioning that NetMR‐B 1 retrieved sharper anatomical details, such as tissue infiltrations and interfaces, than NetMR‐ϕ (as can be appreciated in Figure 4). This was not accounted for in ME and S 90 10 calculations, as these metrics were calculated in tissue ROIs obtained by erosion of 1 voxel from the original tissue segmentation.

FIGURE 4

Comparison between conductivity maps reconstructed with NetMR‐B 1 and NetMR‐ϕ, the networks trained on noisy MR simulated data using both and ϕ or only ϕ, respectively. Maps for ground truth conductivity (first column), DL‐EPT conductivity reconstructed by NetMR‐B 1 (second column), DL‐EPT conductivity reconstructed by NetMR‐ϕ (third column) and the conductivity difference between NetMR‐B 1 and NetMR‐ϕ (fourth column) are shown for two pelvic models with different anatomies (one thin patient, model #07, and one fat patient, model #21)

TABLE 2

Mean ± SD values for global ME and S 90 10 over all dielectric models in all 7 test folds for both NetMR‐B 1 and NetMR‐ϕ

	NetMR‐B 1		NetMR‐ϕ±
	ME (Sm−1)	S 90 ‐ 10 (Sm−1)	ME (Sm−1)	S 90 ‐ 10 (Sm−1)
Fat	−6.28 × 10−3 ± 0.0260	0.0107 ± 0.0063	−0.0101 ± 0.0246	0.0126 ± 0.0248
Muscle	−0.0106 ± 0.0663	0.0938 ± 0.0374	0.0207 ± 0.0620	0.0994 ± 0.0307
Bone	7.16 × 10−3 ± 0.0228	0.0676 ± 0.0262	−1.03 × 10−4 ± 0.0242	0.0766 ± 0.0308
Tumour	0.0113 ± 0.1292	0.1886 ± 0.1360	0.0437 ± 0.1061	0.1654 ± 0.1171
Bladder Content	0.0607 ± 0.1593	0.3160 ± 0.1430	0.0912 ± 0.1107	0.3037 ± 0.1211

Supporting Information Figure S4 explores the reconstruction capability of NetMR‐B 1 in presence of tissues with conductivity values outside the predefined tissue conductivity ranges (outliers). For example, the tumour in outlier 1 (σ GT = 2.9 Sm−1) was assigned a mean σ = 2.2 Sm−1, belonging to bladder distribution, (Figure 1). The tumour in outlier 3 (σ GT = 0.6 Sm−1, a value in between bone and muscle conductivity distributions, Figure 1) was predominantly assigned muscle‐like conductivity although conductivity values of cancellous bone were also present. In outlier 2, fat tissue presented fat‐like conductivity values, with a mean σ ≈ 0.1 Sm−1 (vs σ GT = 0.18 Sm−1). In all examples, the conductivity of adjacent tissues was reconstructed within the abovementioned accuracy and precision.

In vivo results

Conductivity maps obtained with NetMR‐B 1 and NetMR‐ϕ are shown in Figure 5 (transversal view) and Supporting Information Figure S5 (sagittal view). The conductivity maps reconstructed in vivo with DL‐EPT confirmed the results obtained in silico: first, the underlying anatomy was clearly reconstructed (Figures 5 and 6 for a zoomed view); second, conductivity values were quite homogeneous within tissues. For both NetMR‐B 1 and NetMR‐ϕ erroneous conductivity values were predominantly caused by imaging artefacts in the underlying and ϕ ± (Figure 7), such as ghosting arising from motion (e.g. breathing and flow) during acquisition. These were visible, for example, in proximity of bladder and bone. Note that ghosting artefacts were not included in training data.

FIGURE 5

In vivo conductivity results on a healthy volunteer and two cervical cancer patients (transversal view). Magnitude image from SE acquisition (first column), Helmholtz‐based conductivity map (σ H, second column), DL‐EPT conductivity map reconstructed with NetMR‐B 1 (third column) and DL‐EPT conductivity map reconstructed with NetMR‐ϕ (fourth column) are shown. Tumour delineation is displayed in magenta colour

FIGURE 6

Zoomed view of in vivo conductivity results on a healthy volunteer and two cervical cancer patients shown in Figure 5. Magnitude image from SE acquisition (first column), Helmholtz‐based conductivity map (σ H, second column), DL‐EPT conductivity map reconstructed with NetMR‐B 1 (third column) and DL‐EPT conductivity map reconstructed with NetMR‐ϕ (fourth column) are shown. Tumour delineation is displayed in magenta colour

FIGURE 7

(first column) and ϕ (second column) maps of healthy volunteer and two cervical cancer patients. Tissue ROIs used for statistics were manually delineated on 5 adjacent slices, by avoiding visible Helmholtz‐based EPT boundary errors, and are displayed on a slice of the magnitude image (third column)

The quality of DL‐based conductivity outperformed that of Helmholtz‐based EPT conductivity (Figures 5 and 6), as already observed in silico. Figure 8 reports a quantitative comparison of NetMR‐B 1 and NetMR‐ϕ with H‐EPT, depicting median conductivity values calculated in 3D tissue ROIs (illustrated in Figure 7), together with 10th and 90th percentiles. Previous simulation and experimental studies , showed that accurate H‐EPT estimates can be obtained within boundary‐free ROIs. Because the broad boundary errors in H‐EPT in fat and muscle did not leave enough voxels for reliable statistics, the comparison was not reported for these tissues. For all three subjects, median conductivity values differed from H‐EPT median values by less than 8% in NetMR‐ϕ and ~11% in NetMR‐B 1 in muscle and tumour. In bladder, differences in median values up to 35% were found in NetMR‐ϕ for all subjects, whereas in NetMR‐B 1 differences were less than 5% for volunteer and patient 2, and 40% for patient 1. DL‐EPT values were also in the range of values found in an earlier study. Moreover, the conductivity spread in DL‐EPT was smaller than in H‐EPT. The relatively low 10th percentiles found for DL‐EPT in the tumour were caused by aforementioned motion‐related errors entering the tumour ROI (Figure 5).

FIGURE 8

Quantification of pelvic tissue conductivity reconstructed with Helmholtz‐based EPT and DL‐EPT (both NetMR‐B 1 and NetMR‐ϕ) for both healthy volunteer and cervical cancer patients. Median values are reported. Error bars denote 10th and 90th percentiles. These values were calculated within the tissue ROIs illustrated in Figure 7. Literature mean values, along with maximum and minimum values found in the population examined in Ref.42 are also shown. V = volunteer; P1 = patient 1; P2 = patient 2; Lit = literature

DISCUSSION

A novel deep learning‐based method for reconstruction of subject‐specific conductivity in the pelvic region was presented. This method comprises a fully open source three‐dimensional (3D) patch‐based convolutional neural network (CNN) architecture (highres3dnet) and an in silico training dataset of MR accessible quantities ( and transceive phase, NetMR‐B 1, or transceive phase only, NetMR‐ϕ), which were obtained from realistic MR simulations at 3T of pelvic models with representative anatomies and electrical properties. Accuracy and precision of conductivity reconstruction were validated in silico. The trained network reconstructs in few seconds a 3D pelvis conductivity map with sharp tissue interfaces, robust to experimental noise levels and relatively sensitive to tissue conductivity variations. When tested in vivo, the CNN produces conductivity maps with similar quality as in silico conductivity maps, demonstrating that the adopted simulation‐based training dataset sufficiently represents measured B 1 fields.

A merit of our approach is the ability to accurately reconstruct the conductivity at tissues interfaces, revealing the underlying pelvic anatomy (Figures 4 and 5). This shows that the network learnt to extract the anatomical information directly from the input B 1 data alone, unlike a previous study, where an MR magnitude image was required in input to obtain 2D conductivity maps with correct brain anatomy. Anatomically detailed conductivity maps, as achieved by the proposed DL‐based method, are particularly appealing for MR‐based hyperthermia treatment planning of cervical cancer patients: for example, to enable localization of hot spots, which typically occur at muscle/fat interfaces.

Conceptually, the capability of accurate boundary reconstruction is attributable to highres3dnet architecture, and in particular to the synergic combination of (1) patch‐based training and (2) dilated convolutions. Highres3dnet was initially chosen for conductivity reconstruction because of its flexibility in dealing with 3D regression problems in patch‐based fashion, which we expected to capture the 3D, local nature of the EPT problem. Patch‐based approaches allow to capture local features from input data, while diminishing the risk of overfitting global spatial context. Besides, they reduce the computational cost on GPU memory. Dilated convolutions in our CNN architecture efficiently expand the receptive fields, which relate to wider spatial contexts of the input patch, while preserving the spatial resolution of features in the input patch throughout the layers. This appears to be more advantageous than using down‐ and up‐sampling layers, as found, for instance, in the U‐net architecture employed by Hampe et al. Down‐sampling layers reduce the resolution of the input features, which is only partially recovered by the subsequent up‐sampling layers. This might potentially explain the smoother boundaries in Ref. 38.

Our approach does not depend on geometrical similarities among training human models and tested subjects, as supported by the consistent DL‐EPT results for diverse pelvic tissue anatomies (Figures 2, 3, 4, 5): all the relevant information is recovered from the . This is a substantial difference with a recent DL‐EPT finding, where only test geometries similar to training geometries were well reconstructed. The network from this prior study likely over‐fitted the anatomical information in training from the input transceive phase. This issue was attributed to the lack of geometrical variability in training, being the training dataset based on geometrical transformations of few human brain models. Differently, we expect that our CNN properly exploited the diversified training dataset to learn the mapping function between input magnetic field and conductivity in the pelvis.

The training of our supervised DL‐based method was deliberately founded on simulated data, as simulations intrinsically connect the ground truth EPs to and ϕ and can reproduce the magnetic field with high fidelity. Direct training on MR data, as explored in Ref. 38 is another viable option, which could be motivated by the final practical application of the method on in vivo MR measurements. Nonetheless, in vivo MR data lack ground truth EP information, thus target EPs must be obtained by an arbitrarily chosen EPT reconstruction method (e.g. processed H‐EPT ). A possible downside of training on in vivo MR data is that potential inaccuracies in these EP maps deriving from the selected EPT algorithm would be incorrectly learnt as true values. Furthermore, simulations allow to include knowledge of the RF coil system and to expand the training samples: for instance, by creating several EP permutations and/or by retrieving anatomies from different imaging modalities. In this study, CT scans of real patients were used to create human models but segmenting clinical MR images, which offer greater tissue contrast, might also be considered. In that case, we believe that highres3dnet would be able to reconstruct most of the tissues’ anatomy for the reasons discussed above. Moreover, the EP permutations were intended to cover the spectrum of possible inter‐subject EP variations as much as possible (see Supporting Information Appendix) but, for simplicity, each pelvic tissue was assigned a single EP value from the predefined EP distributions. Thus, our in silico training dataset currently does not account for intra‐tissue EP variations, which instead would be naturally encoded in measured B 1 data, even though B 1 maps from a rather large population should be acquired to sufficiently sample the inter‐subject EP variability. Finally, simulations permit to emulate the accuracy, precision and acquisition artifacts of the and ϕ maps measured with different MR sequences, which could be learnt and potentially compensated by the CNN.

Based on a framework using EM and MR simulations, the propagation of sequence‐specific systematic deviations (accuracy) and noise (precision) occurring from MR images to and ϕ during the acquisition is realistically included in our training and ϕ maps. On the contrary, it is “common practice” in DL‐EPT to train on simulated EM data of magnetic fields to which Gaussian noise is directly superimposed. This does not account for sequence‐related inaccuracies and it implicitly assumes a possibly unrealistic noise distribution in the and ϕ maps (which, in fact, is regulated by the mapping sequence), although it might approximate the noise level obtainable in measurements. However, our results demonstrated that the conductivity reconstruction with NetMR‐B 1, that is, the network trained on simulated MR data, was more robust to errors (e.g. in air, Figure 3C) than the reconstruction performed with the network trained on EM simulations alone (NetEM‐B 1). DL‐based reconstruction with NetMR‐B 1 was also noise‐robust.

The comparable performance of the two network configurations (NetMR‐B 1 and NetMR‐ϕ) suggests that the CNN can reliably reconstruct the conductivity from the transceive phase alone (Table 2 and Supporting Information Figure S3). NetMR‐B 1 better reconstructed the tissue interfaces in silico; this advantage was not clear in vivo. In general, the contribution of for DL‐based conductivity mapping seems minor; the two configurations differed only noticeably in the bladder for two subjects. A transceive phase‐only approach would save scan time and avoid artefacts strictly related to acquisitions.

All the technical aspects discussed thus far alleviated some problems of pelvis conductivity mapping at 3T. For example, the validity of the transceive phase assumption, adopted in many EPT methods, degrades at 3T for large body sizes such as the pelvis (see e.g. H‐EPT in Figure 2). Training with the transceive phase circumvents this limitation. Furthermore, the low pelvis SNR, which is typically compensated using large voxel sizes, was mitigated in this study by training on simulated MR data with SNRs predicting the experimental precision at a desired resolution (Supporting Information Figure S1). Nevertheless, our DL‐based method does not currently handle the imaging artifacts caused by motion (breathing, bowel peristalsis, flowing blood) that inevitably affect and ϕ maps (Figure 5). Handling these motion‐induced artefacts is still an unresolved challenge in EPT. We envisage that including these artefacts in the simulated training dataset would improve in vivo DL‐based pelvis reconstructions.

The network performs consistently when presented with unseen pelvic data, as confirmed by its stable behaviour with respect to test data of different folds (Supporting Information Figure S3), which gives confidence on the reliability of the overall accuracy and precision predicted by the in silico cross‐validation. The cross‐validation revealed on average a low mean error (ME) in all tissues (Table 2), which indicates an overall good sensitivity to conductivity variations. Nevertheless, MEs up to ~20% from the ground truth conductivity could occur for muscle, bladder and tumour, therefore further optimization is necessary to render our method competitive with the accuracy of other physics‐based EPT reconstruction methods (e.g. CSI‐EPT approaches , exhibiting errors below 10% in simulated pelvis conductivity maps). Those MEs might be possibly decreased by manipulating the loss function or augmenting the number of EP permutations; yet, we cannot predict the number of training samples needed for this improvement. Moreover, bladder content and tumour were less represented in our database (approximately one order of magnitude lower than the other tissue types, Figure 1), which likely explains the higher uncertainty in ME and S 90 10 in absolute values in these two tissues (Supporting Information Figure S3). Solutions to the issue of imbalanced data were proposed for DL classification problems, but they are still focus of research in regression problems. We expect that the accuracy in these tissues will be enhanced by handling the problem of imbalanced data.

Our data‐driven approach retrieves the conductivity for the specific body region, body coil and field strength, because this knowledge is imprinted during training. Its applicability to other body sites or field strengths cannot be assured yet. Moreover, Supporting Information Figure S4 suggests that the network tends to favour anatomically realistic conductivity values (seen in training) if outlier conductivity values, that is, outside the training range, are present. Thus, we deduce that training with a larger range of conductivity values would likely be more appropriate to generalize our method. We acknowledge also that the current implementation fails to indicate such type of error. We believe that combining our DL‐based reconstruction with an inverse EPT reconstruction method (which guarantees data consistency), similarly to the hybrid approach adopted by Leijsen et al, could increase the confidence in the accuracy of reconstruction of outlier cases.

To emphasize the feasibility of the proposed DL‐EPT method, we also reported conductivity maps reconstructed on experimental in vivo data. These maps reproduce the quality observed in the simulation study, thus we conclude that our training simulated fields are sufficiently representative of realistic (excluding imaging artefacts, which were not simulated). Furthermore, we compared in vivo DL‐EPT conductivity maps with Helmholtz‐based EPT (H‐EPT) conductivity maps. Note that H‐EPT is currently the only method shown in vivo for pelvis conductivity mapping; the aforementioned CSI‐EPT approaches, , despite showing good accuracy and precision in pelvis conductivity simulations, have not yet been presented for measured pelvic data. In vivo, DL‐based conductivity represents a remarkable improvement over the state‐of‐art H‐EPT conductivity (Figure 5), presenting no boundary errors, higher precision and rather small differences with H‐EPT values in boundary‐free tissue ROIs (Figure 8). However, we do realize that the presented comparison does not provide validation for our in vivo results, since H‐EPT values cannot be considered as ground truth. Therefore, we advise caution in interpreting quantitative in vivo values obtained with DL‐EPT without additional validation. We surmise that the accuracy predicted with the cross‐validation also applies experimentally, but a separate study is warranted to validate this. Similar to previous conductivity works, an effective validation could be performed, for example by testing the CNN on phantoms with known EPs, although the network might need re‐training on different digital phantoms for this scope.

In the perspective of oncologic applications, several implications ensue from the fact that MEs up to ~0.20 Sm−1 could occur in tumour conductivity with NetMR‐B 1 and NetMR‐ϕ. In the context of hyperthermia treatment planning, for example, an under‐/over‐estimation of this magnitude in the conductivity of a deep‐seated tumour like cervical cancer has a limited effect on the tumour temperature. Nevertheless, the tumour temperature is also affected by the conductivity of tissues surrounding the tumour, like muscle and bladder. Thus, errors up to 0.15 and 0.4 Sm−1 in muscle and bladder, respectively (Supporting Information Figure S3), might profoundly affect the tumour temperature achieved during the treatment. Moreover, it is currently unclear whether this reconstruction accuracy would prohibit usage in longitudinal studies to monitor the response of a treatment because inter‐treatment tumour conductivity changes are unknown. For diagnostic applications based on conductivity, the tumour volume would be detected only if the dielectric contrast of normal/cancerous tissues were above ~0.25 Sm−1. For lower conductivity contrasts, we observed that tumour and muscle (/healthy) tissues were generally given the same conductivity value (Figure 2).

CONCLUSION

We showed, for the first time, that in vivo conductivity maps in the human pelvis characterized by anatomical details, robustness to noise and relatively good sensitivity to tissue conductivity variations are feasible at 3T using deep learning. In particular, we demonstrated that the combination of (i) a broad training dataset, with realistic pelvic anatomies and EPs, (ii) a 3D, compact, high resolution CNN architecture (the implementation and configuration of which are made publicly available) trained at a patch level on (iii) only measurable and ϕ quantities obtained from MR simulations with realistic accuracy and precision was sufficient to guarantee good quality, 3D maps of pelvis conductivity.

The quality of DL‐based conductivity maps shown in this work represents a dramatic improvement over the status quo methodology used for pelvis EPT, that is, conventional Helmholtz‐based EPT, and may prove useful for clinical oncological applications.

FIGURE S1 Comparison between measured and simulated MR data with realistic noise levels. For both AFI (left) and SE (right) sequences, measured and simulated data are reported in top and bottom rows respectively. Image SNR for both sequences is shown in first column. For measured data image SNR were retrieved with Kellman and McWeigh’s method. for AFI and ϕ for SE are shown in second column. The standard deviations in ( ) and in ϕ ( ), derived "as described in Refs. 30 and 45 are shown in third column

FIGURE S2 Comparison between (top row) and ϕ (bottom row) obtained from EM and noisy MR simulations. First column: and ϕ maps. Second column: and ϕ maps. Third column: Difference maps Δ and Δϕ , obtained as difference between data from noisy MR simulations (in second column) and data from EM simulations (in first column). Magenta arrows in difference maps point to an air pocket

FIGURE S3 Mean error (ME) and spread S 90‐10 for all pelvic tissues, resulting from 7‐fold cross validation for NetMR‐B1 (first and second rows) and NetMR‐ϕ ± (third and fourth rows). ME and S 90‐10 in every tissue were calculated within a tissue ROI that corresponded to the tissue segmentation without the outermost voxels (i.e. 1 voxel was eroded from the original segmentation). Each colour represents one (test) fold, where 30 dielectric models were used as test data. Each circle represents one dielectric model belonging to the test dataset. All 210 dielectric models were tested once within all 7 folds. The dashed blue lines represent average ME and S 90‐10 (over all test folds) for each pelvic tissue

FIGURE S4 DL‐EPT conductivity reconstruction (with NetMR‐B1) in presence of outlier EP values, i.e. values that were not included in our EP database. Three outlier cases are presented: outlier 1 (top row), with tumour σ = 2.9 Sm−1 and εr = 85 rel. units; outlier 2 (middle row), with fat σ = 0.18 Sm−1 and εr = 8.27 rel. units; outlier 3 (bottom row), with tumour σ = 0.6 Sm−1 and εr = 35 rel. units. Maps for ground truth conductivity (first column), DL‐EPT conductivity retrieved with NetMR‐B1 (second column) and conductivity difference (third column) are reported. Conductivity histograms for the tissue with outlier EPs are shown in fourth column. Solid and dashed lines in red colour represent ground truth conductivity and mean DL‐EPT conductivity value in the tissue, respectively

FIGURE S5 In vivo conductivity results on a healthy volunteer and two cervical cancer patients (sagittal view). Magnitude image from SE acquisition (first column), Helmholtz‐based conductivity map (σ H, second column), DL‐EPT conductivity map reconstructed with NetMR‐B1 (third column) and DL‐EPT conductivity map reconstructed with NetMR‐ϕ (fourth column) are shown. Tumour delineation is displayed in magenta colour. (Transversal view is shown in Figure 5)

FIGURE S6 Permittivity‐Conductivity scatterplot of human tissues at 128 MHz. These values for electrical tissue properties were taken from Ref. 55. The dash grey line represents the polynomial fit to the data and the shaded area represents the ±20% error

TABLE S1 Parameters used to tune both full (“NetEM/MR‐B 1”) and transceive phase only (“NetMR‐ϕ ”) network configurations. This table is shown in the form of the configuration file as provided in NiftyNet for this study (for more details on configuration file, see https://niftynet.readthedocs.io/en/dev/config_spec.html)

Click here for additional data file.