Making three-dimensional echocardiography more tangible: a workflow for three-dimensional printing with echocardiographic data.

Three-dimensional (3D) printing is a rapidly evolving technology with several potential applications in the diagnosis and management of cardiac disease. Recently, 3D printing (i.e. rapid prototyping) derived from 3D transesophageal echocardiography (TEE) has become possible. Due to the multiple steps involved and the specific equipment required for each step, it might be difficult to start implementing echocardiography-derived 3D printing in a clinical setting. In this review, we provide an overview of this process, including its logistics and organization of tools and materials, 3D TEE image acquisition strategies, data export, format conversion, segmentation, and printing. Generation of patient-specific models of cardiac anatomy from echocardiographic data is a feasible, practical application of 3D printing technology.

Introduction

Imaging techniques such as computerized tomography (CT), magnetic resonance imaging (MRI), and transesophageal echocardiography (TEE) are commonly used for diagnosis, monitoring, and decision-making in patients with cardiovascular disease (1, 2). In cardiac surgery, TEE is the imaging modality of choice due to its portability and availability at the point-of-care, and because it is capable of generating high-quality two- and three-dimensional (2D, 3D) images with high temporal and spatial resolution (3, 4). Recently, there has been an increase in use of 3D TEE as an adjunct for percutaneous cardiac interventions. Intraoperative TEE-derived information has significant value in the evaluation of procedure suitability, real-time guidance, and post-procedure assessment (5).

Currently, 3D images are analyzed and evaluated on flat displays, on which different shades, opacity, and color tones are used to create a perception of depth. While these images provide superior spatial orientation compared to traditional 2D TEE images, they are limited by parallax error (3). Therefore, accurate measurements require generation of 2D slices through multi-planar reconstruction. While 3D images have been helpful in surgical planning, they lack the haptic feel and true 3D perspective that would be obtained with an actual, patient-specific physical model.

Rapid prototyping (i.e. 3D printing) has several potential applications in cardiac surgery (6, 7, 8, 9, 10, 11). Traditionally, visual assessment of the heart valves during open-heart surgery has been performed on an empty and paralyzed heart. Furthermore, in minimally invasive surgery and percutaneous valve interventions, there is limited or no direct valve exposure during the procedure. Therefore, availability of a tangible high-fidelity 3D model of a valve prior to intervention could provide an opportunity for ‘ex vivo’ valve analysis. In addition, such models could potentially be used for procedural training, to aid in communication with patients or among members of the surgical team, and for designing patient-specific procedures and prostheses. With the rapid development of 3D printing technology and the growing range of available materials, it is now possible to print models that mimic mechanical properties of certain tissues and that could be sterilized and potentially implanted. With improvements in technology, clinical applications of 3D printing are likely to expand significantly in the near future (12).

Patient-specific models of anatomical structures such as the skull or aorta can be generated using 3D printing (8, 13). The majority of 3D printing in medicine to date has been based on CT and MRI data. The high spatial resolution, relatively high signal-to-noise ratio, and Digital Imaging and Communications in Medicine (DICOM) standardization of data formats for these modalities simplify their processing into printable models. However, CT and MRI’s use for point-of-care imaging in cardiac procedures is limited by their low temporal resolution, limited accessibility in the operating room, and other factors such as magnetic interference and use of ionizing radiation.

Recently, echocardiographic data have been used to create patient-specific 3D-printed anatomical models of intracardiac structures (12, 14, 15, 16). The ‘image to print’ time for this process is currently on the order of hours for most structures of interest. However, with the rapid advances currently taking place in this field, this technology will likely become ‘point-of-care’ in the near future (17, 18). There are multiple software, hardware, and material choices for echocardiography-derived 3D printing, ranging from low-cost desktop systems to industrial-level ones.

Given that 3D printing from echocardiographic data is relatively novel and didactic materials are sparse and technical, choosing appropriate hardware and software equipment can be difficult and confusing. Therefore, drawing upon our experience we have described the workflow for 3D printing of cardiac anatomical structures from 3D TEE data.

Workflow

The workflow for echocardiography-based 3D printing can be broadly divided as follows (Fig. 1):

Figure 1

Summarized workflow for 3D printing of a 3D TEE data set of a mitral valve after a MitraClip procedure. Examples of file formats used are included where applicable. From left to right: After export from the ultrasound system, the data set is converted to the Philips Cartesian DICOM format (first panel). Using segmentation software the voxels in the region of interest are labeled, creating a ‘solid’ voxel model (second panel). A triangular surface mesh model is generated based on the voxel model (third panel). The mesh model is processed by the slicing software, generating a printer code, which directs the printing of the final model (fourth panel). (File formats: NIFTI, Neuroimaging Informatics Technology Initiative; NRRD, Nearly Raw Raster Data; STL, Stereolithography; PLY, Polygon; OBJ, Wavefront Object.)

Logistics

Echocardiographic image acquisition (seconds to minutes)

Data export (minutes)

Format conversion (minutes)

Segmentation (minutes to few hours)

Generation and refinement of a mesh surface model (several minutes)

Printing (minutes to hours)

Logistics

Echocardiography-derived 3D printing is dependent on the acquisition of high-quality 3D data sets of intracardiac anatomical structures and their subsequent conversion to printable formats. Besides a 3D printer, accomplishing this task requires a trained echocardiographer, a 3D echocardiography system, a computer workstation, and software for data conversion, segmentation, and modeling. With currently available ultrasound systems it is not possible to directly convert a 3D data set into a 3D printable file. Therefore, the ultrasound system must be able to export the volumetric data in a format that can be accessed by offline post-processing software. Despite the processing being semi-automated, significant user input is still required at most steps and most of software packages required have steep learning curves in the initial stages.

Software options for 3D echocardiography-derived printing include Free/Libre and Open Source (FLOS) options as well as proprietary commercial packages (Table 1) (6). In our opinion, the quality of digital models achievable with these tools is comparable. Differences primarily involve the user interface and secondary features. We have not found a significant advantage in usability or learning curve between the proprietary and FLOS packages. Multiple printer options are available in the market with wide price variation. The main determinants of this price difference are printing resolution, speed, and the ability to simultaneously print using different materials. It is important to note that the real spatial resolution of the printed model is ultimately limited by the resolution of the original data set, which is significantly lower than that of most desktop 3D printers. Therefore, while higher resolution printers can potentially print more aesthetically appealing models, these models are not more accurate in their representation of the underlying imaging data.

Table 1

Two sample configurations for a 3D echocardiography printing system. Price estimates are provided in US dollars as of December 2015.

Workflow step	Low-cost solution	High-cost solution
Conversion DICOM to Philips Cartesian DICOM	Philips QLAB (~$9000 one-time)
Segmentation and generation of STL file	3D Slicer/ITK-Snap ($0)	Mimics Innovation Suite ($10,000 annual license)
Modification and refinement of STL file (3D modeling)	Blender/MeshLab ($0)	Mimics Innovation Suite ($10,000 annual license)
3D printing	Desktop FDM printers e.g. Lulzbot Taz, Ultimaker, Makerbot ($2000–$4000)	Polyjet printers such as Stratasys Objet series ($60,000–$500,000)
Total cost of software and printing hardware	$11,000–$14,000	>$80,000

Echocardiographic image acquisition

With the exception of 3D color flow Doppler, all 3D imaging modes can be currently used for rapid prototyping. Optimal 3D image acquisition for printing follows many of the general principles of 3D TEE image optimization, with some subtle differences (Table 2). Ideally, probe position should be adjusted so that the structure of interest is close and relatively perpendicular to the ultrasound beams. Occasionally, this may require the use of nonstandard reference views, depending on specific patient anatomy and artifacts. Higher frequencies (i.e. resolution mode) should be used when possible. Harmonic imaging may provide significant reductions in noise and artifact. Sector width and imaging depth should be reduced as much as possible to maximize frame rate. R-wave-gated imaging is particularly well suited to acquire 3D data sets with high temporal and spatial resolution, but can only be performed in patients with a regular cardiac rhythm and in the absence of patient or probe motion. In patients with arrhythmias, the wide-angle ‘live’ zoom mode or single beat full volume acquisition can be used. The automatic gain and dynamic range adjustments available on most ultrasound systems are generally adequate. If manual optimization is required, the image should be adjusted in order to minimize grayscale variations within the structure of interest and maximize the contrast at the boundaries.

Table 2

Goals of image optimization for 3D modeling (ROI, region of interest).

Parameter	Optimization goals
Cropping	Adjust sector size and depth to minimize extraneous structures unless required for orientation
Gain and compensation	Optimize tissue boundaries• Maximize contrast between ROI and surrounding area• Minimize texture variation within ROI
Spatial vs temporal resolution	Maximize spatial resolution but allow enough temporal resolution to• Avoid motion blurring• Allow capture of desired point in cardiac cycle

Data export

In general, the on-cart analytical software does not permit the export of data in a format appropriate for post-processing and printing. Most ultrasound systems are capable of exporting 3D data in the DICOM format, either via disc drive or a Universal Serial Bus drive. However, unlike more traditional modalities such as CT or MRI, 3D ultrasound data within the DICOM object is encrypted in proprietary vendor-specific formats, which prevent direct access to the data set by the segmentation software (19).

Format conversion

Currently, only Philips Medical Systems’ 3D data sets can be converted to a format accessible to segmentation software. This process requires export of data in the proprietary ‘Philips Cartesian DICOM’ format from QLAB (Philips Medical Systems). Raw 3D ultrasound images are composed of anisotropic (i.e. having different dimensions in each axis) voxels that are frustum-shaped. Additionally, the dimensions of these voxels vary with distance from the transducer. During conversion to the Cartesian DICOM format the image is resampled into uniform rectilinear voxels, which are not completely cube-shaped (i.e. they are also anisotropic, but all of uniform size). As a result, the data is typically resampled again by the processing software prior to segmentation. It is important to note that the exported data set is not affected by any post-processing done in QLAB (e.g. 3D cropping, alterations in gain or compression).

Segmentation

Segmentation of the 3D data set is the process of identifying the voxels that are part of the structure of interest. The relevant voxels are labeled, creating a ‘label mask’ (Fig. 1). While this labeling can be done manually, it is a laborious and time-consuming process. Typically, various semi-automated techniques are used first, and followed by manual editing. Three available segmentation software packages are the Mimics Innovation Suite (Materialise, Leuven, Belgium) and the FLOS packages 3DSlicer (http://www.slicer.org), and ITK-Snap (http://www.itksnap.org).

The functional capabilities of Mimics Innovation Suite and 3DSlicer are broadly similar, and both software packages have steep learning curves (Table 3). In the authors’ experience, ITK-Snap combines the simplest interface with a set of sophisticated tools and is well suited for the novice user. Importing Cartesian DICOM files into 3DSlicer and ITK-Snap requires the SlicerHeart extension for 3DSlicer (https://www.slicer.org/wiki/Documentation/Nightly/Extensions/SlicerHeart), which converts the data into a standard DICOM encoding, permits anonymization, and allows conversion of the data set into the open Nearly Raw Raster Data (NRRD) format, which can be imported into ITK-Snap or further processed in 3DSlicer.

Table 3

Comparison of FLOS and commercial segmentation software.

	Commercial software	Free/Libre and open source software
Software	Mimics Innovation Suite	3D Slicer	ITK-Snap
Pros	• Wide selection of manual and automatic segmentation tools• Capable of correcting mesh files	• Wide selection of manual and automatic segmentation tools and extensive collection of extensions• Free	• Simple interface• Free
Cons	• Complex interface• Expensive license	• Complex interface• More complex ultrasound data importation	• Limited selection of manual segmentation tools• More complex ultrasound data importation

The development of reliable automated and semi-automated segmentation algorithms for cardiac structures is an active area of research. Customized algorithms are being developed for specific structures, using various pattern recognition techniques to improve performance and overcome noise (20). However, robust algorithms are still not widely available for most anatomical structures and significant input from an operator is still required. Therefore, most segmentation is performed with a combination of manual and semi-automated tools such as thresholding and region growing. In thresholding, the system labels all voxels that possess a grayscale value between an upper and lower limit that is set by the user (i.e. the threshold values). Region growing, also known as active contour, requires the user to select ‘seed’ points within the region of interest and a threshold. The algorithm then iteratively expands the selection from the seed voxels to include adjacent voxels with values within the threshold. The user has to monitor the process in order to stop it at an appropriate point before the region expands beyond the boundaries of the structure of interest. ITK-Snap is particularly noteworthy for its sophisticated and relatively intuitive active contour segmentation tool.

It is frequently difficult to achieve adequate segmentation results in a single step. In most cases, structures have to be segmented in a few steps with multiple masks that are then combined or subtracted. For example, it is often easier to label the blood pool around a valve and then invert this ‘negative’ mask to obtain a mask of the valve itself. Masks generated by the automated algorithms can be refined with secondary tools. For example, masks can be expanded or shrunk by a specified number of voxels, allowing elimination of isolated voxel clusters or small gaps.

The higher level of noise typical of ultrasound imaging often makes segmentation of data from this modality more difficult than from CT or MRI. While the techniques used for segmentation are generally the same regardless of imaging modality, the use of noise filtration and ‘de-speckling’ algorithms prior to segmentation, as reported by Olivieri and colleagues (16), can be especially useful for processing of ultrasound data.

Generation and refinement of a mesh surface model

There are two broad categories of digital representation for 3D images: voxel and geometric. In a voxel representation, the form is approximated as a conglomeration of discrete building blocks called voxels (volume elements). In a geometric representation, the form can be approximated as a surface mesh consisting of vertices and edges (more common for complex forms) or as a combination of simple geometric primitives such as spheres, prisms, and polyhedrals. The voxel and geometric families of representations are respectively analogs of pixel (raster) and vector-based representations of 2D data.

The process we have described so far, from the original 3D echocardiographic data set to the label map, is based entirely on voxel representations of the 3D data. These voxel representations generate very large files and require significant computational resources for processing. In order to facilitate further processing, a geometric surface mesh model is generated based on the final label mask (Fig. 1, third panel). The surface mesh approximates the surface of the object using nonuniform triangles. The primary variable determined by the user is the minimum edge length of the triangles. If the minimum edge length is set automatically, the triangles may be smaller and more numerous than the resolution of the original image, generating large data files and prolonging processing times without preserving any meaningful information. All of the software packages mentioned previously provide mechanisms for adjusting the triangle size and number. As a general rule, the minimum triangle edge length should be no smaller than the resolution of the original image. In most cases values 2–3 times this can be used without loss of relevant detail. Optimal values depend on the purpose of the model.

Ideally, the mesh surface generated should be ‘manifold,’ a topological property roughly described as ‘water tight.’ The surface should be continuous with no gaps, and each triangle should have distinct internal and external surfaces. Due to the computational complexity of the process, the created mesh model often contains errors. Common errors include holes and edges being shared between more than two triangles (leaving one triangle either completely inside or outside of the form). All of the software packages described here include mechanisms for detecting and correcting mesh errors. Mimics Innovation Suite includes an integrated error detection and correction module that typically needs to be run iteratively to correct the most critical errors. Some of the errors can be disregarded, as they have no impact on later stages of the process. In addition to the structure of interest, the resulting mesh surface may also include numerous small ‘islands’ that result from noise in the original image. These can be removed in a variety of ways, such as by automatic selection of the largest surface for export, or by manual deletion.

The most powerful FLOS tools for editing and correcting mesh files are Blender (Blender Foundation, Amsterdam, the Netherlands; https://www.blender.org) and MeshLab (Visual Computing Lab, National Research Council of Italy, Pisa, Italy; http://www.meshlab.net/). Blender is primarily designed as a 3D animation suite but has significant capabilities for constructing and editing 3D models from any source. Blender’s CellBlender add-on (http://www.mcell.org/tutorials/install_cellblender.html) contains functionality for correction of mesh errors. MeshLab is specifically designed for editing and manipulation of mesh files and has a somewhat less complex interface. Additionally, it possesses a full range of manual and semi-automated filters for post-processing of meshes, but notably lacks an undo function.

After optimization, the mesh file can be exported for printing in a variety of formats. The stereolithography (.stl) file format is currently the most popular, but has several important limitations. First, stereolithography files do not contain measurement units. Most software and hardware tools assume the values to be in millimeters, but this is not universal and can result in scaling errors. Second, these files only describe a surface geometry and contain no information about the material, density, color, or surface texture. As printers become increasingly sophisticated, more robust formats such as Wavefront object (.obj), Polygon (.ply), or the Additive Manufacturing (.amf) may become standard.

The process of creating a printable digital model from the original data set involves multiple stages and transformations. Most of these steps have significant potential for the introduction of noise, processing artifacts, and unintended loss of relevant information. All automated steps can generate different results depending on the specific algorithm and settings used. Processing artifacts can interact with the original data and its imaging artifacts in complex ways, generating clinically significant errors in the final model. A clinically significant anatomic defect, for example, can be ‘corrected’ by a generous smoothing algorithm, or an imaging artifact, amplified through processing, can masquerade as a calcification or other abnormality (21). We recommend careful visual comparisons of the results with the original data set at every stage of processing.

Printing

The final model, represented in a 3D geometric format (e.g. .stl, .ply, or .amf) is processed by the printer software, which will generate instructions for building the model layer by layer, a process known as slicing. Depending on the printer and the control software, various levels of user control are permitted. The most important user-adjusted parameters associated with slicing are layer thickness, thickness of the surface walls, and the pattern and density of filling. Thinner layers produce smoother prints at the cost of increased printing time. Thicker surface walls and denser filling increase the strength of the printed model, again at the cost of increased printing time.

Printing technology is evolving rapidly. Current technologies have been reviewed elsewhere (6, 13). The systems most relevant for visualization and modeling of cardiac structures are fused deposition modeling (FDM), stereolithography, and PolyJet printing (Table 4). FDM printers currently have a more limited range of print materials and resolutions on the order of 100 μm, while PolyJet systems can achieve up to 16 μm resolution. However, as noted above, the resolution of most medical imaging data rarely exceeds 500 μm, far lower than that of desktop FDM printers. More advanced systems are capable of simultaneously printing a range of materials and colors, therefore generating fairly accurate models of anatomical structures (Figs 2 and 3). All current printing systems require some level of post-processing. This can include removal of supports; soaking the print in alcohol, water, or other solvents to remove excess printing materials; soaking in other substances to reinforce, seal, or color the model; and ultraviolet curing of resin prints.

Table 4

Comparison of current printing technologies.

	Fused deposition modeling (FDM)	Stereolithography (SLA)	Polyjet
Pros	• Low cost• Widely available• Capable of printing either flexible or rigid materials including biodegradable plastics	• Better resolution than FDM• Low cost	• Highest resolution• Capable of printing a combination of materials of varying color and consistency
Cons	• Lower resolution• Removal of supports	• Limited material selection• Toxicity of resin• More complex postprocessing	• Expensive (>$60,000)• Toxicity of resin• More complex postprocessing

Figure 2

3D-printed model of the blood volume in the left atrial appendage (LAA). The model on the far right is a negative mold of the LAA. The 3D TEE data set was exported as a Philips’ Cartesian DICOM and converted to NRRD format using 3D slicer’s SlicerHeart extension. The NRRD file was then segmented using ITK-Snap. Model was exported as STL. Clipping and inversion to the negative model were performed using openSCAD. Models were printed using a desktop fused deposition modeling printer (Aleph Objects Taz 5).

Figure 3

Aortic and ventricular views of an aortic root model obtained from 3D transesophageal echo data. This model was printed with transparent photo-resin using the Formlabs Form 2 printer.

Conclusion

Echocardiography-derived 3D printing of intracardiac structures is an evolving technology. We have presented an outline of the workflow required for 3D printing, as well as a series of software tools that can be used for this purpose. While the overall procedure is certainly feasible and relatively inexpensive, it is likely to be simplified, automated, and significantly faster in the future. In our opinion, a single competent echocardiographer can execute the whole procedure after some basic exposure and hands-on practice with the software. Moreover, currently the process needed to obtain high-resolution printed models from intraoperative 3D TEE data is not fast enough for point-of-care use.

The most significant current limitation for developing a simple, inexpensive, and vendor-independent workflow is the need for specifically licensed software to access vendor-specific encrypted encodings, a problem unique to 3D ultrasound imaging.

The workflow described was developed for prototyping heart valve models for surgical planning and education, however, the process is essentially similar for all 3D ultrasound data. However, applications requiring exceptionally detailed, high-fidelity models may require a more in-depth understanding of the processes and transformations beyond the scope of this review. We have attempted to highlight key concepts and outline the workflow in sufficient detail to guide trained echocardiographers, with little prior exposure to 3D printing, in approaching this technology. The ability to 3D print patient-specific models of cardiac anatomy is a feasible, practical application of 3D printing technology.

Declaration of interest

The authors declare that there is no conflict of interest that could be perceived as prejudicing the impartiality of this review.

Funding

This research did not receive any specific grant from any funding agency in the public, commercial, or not-for-profit sector.