A systematic review of image segmentation methodology, used in the additive manufacture of patient-specific 3D printed models of the cardiovascular system.

BACKGROUND: Shortcomings in existing methods of image segmentation preclude the widespread adoption of patient-specific 3D printing as a routine decision-making tool in the care of those with congenital heart disease. We sought to determine the range of cardiovascular segmentation methods and how long each of these methods takes.
METHODS: A systematic review of literature was undertaken. Medical imaging modality, segmentation methods, segmentation time, segmentation descriptive quality (SDQ) and segmentation software were recorded.
RESULTS: Totally 136 studies met the inclusion criteria (1 clinical trial; 80 journal articles; 55 conference, technical and case reports). The most frequently used image segmentation methods were brightness thresholding, region growing and manual editing, as supported by the most popular piece of proprietary software: Mimics (Materialise NV, Leuven, Belgium, 1992-2015). The use of bespoke software developed by individual authors was not uncommon. SDQ indicated that reporting of image segmentation methods was generally poor with only one in three accounts providing sufficient detail for their procedure to be reproduced. CONCLUSIONS AND IMPLICATION OF KEY
FINDINGS: Predominantly anecdotal and case reporting precluded rigorous assessment of risk of bias and strength of evidence. This review finds a reliance on manual and semi-automated segmentation methods which demand a high level of expertise and a significant time commitment on the part of the operator. In light of the findings, we have made recommendations regarding reporting of 3D printing studies. We anticipate that these findings will encourage the development of advanced image segmentation methods.

Introduction

The care of those with congenital heart disease has evolved rapidly over the past fifty years.[1] For the novel surgical and interventional options that are increasingly employed,[2] the ability of non-invasive imaging modalities to define structural abnormalities has become paramount.[3] However, full appreciation of the complex 3D structures involved in a congenital heart abnormality remains hampered by presentation on a 2D computer screen.

Accordingly, researchers have explored the use of 3D printed models in various clinical and non-clinical aspects of paediatric cardiology.[4-8] Physical models are not limited by their dependence on computer workstations, provide the clinician with a tactile experience and allow simulation of surgical or interventional procedures.[9] However, for all its promise, 3D printing is yet to become part of routine practice. We suggest that shortcomings in the 3D printing pipeline, specifically in the image segmentation process prohibit its wider uptake.

Image segmentation is frequently laborious[10,11] and user dependent[12] due to its reliance on expertise in both congenital heart disease morphology and image processing. The clinician, though having a wealth of anatomical knowledge, is less familiar with image processing than the medical physicist and vice versa. Additionally, the hours that can be spent completing a complex segmentation are often incompatible with the workload of clinical staff. Until these problems are solved, 3D printing will remain limited to a select number of research facilities that have the expertise and resources necessary to perform complex image segmentation.[3]

These observations motivate the development of improved 3D printing pipelines with an express focus on faster and simpler image segmentation. We conducted a systematic review to examine existing methods of segmentation and the way that methods were reported, attempting to evaluate their success. It is hoped that the findings of this review can inform research into novel image segmentation procedures. Explicitly stated, the questions addressed by this review were:

‐ What methods have been used to perform image segmentation in the development of patient specific, physical models of human cardiovascular anatomy from medical images? and

‐ How much time do these methods require to complete their respective image segmentations?

Methods

Eligibility criteria

We searched for studies that reported the fabrication of patient-specific models of cardiovascular anatomy, derived from medical images. The eligibility criteria in Table 1 expand on this statement. The scope was not limited to modelling of congenital heart disease specifically, as we considered that the segmentation methods used in other cardiovascular applications may prove insightful for the development of novel 3D printing pipelines. Reviews, articles and case reports published in peer reviewed journals, books and conference proceedings from the grey literature were all considered. English language publications from any setting and time frame were eligible.

Table 1.

The properties of eligible resources that were included in the systematic review of the literature.

Eligibility criteria
Consider human subjects
Fabricate cardiovascular structures
Manufacture patient specific structures
Derive model properties from medical images
Use additive manufacturing methods to fabricate the model

Study identification and selection

Each of the Cochrane Library (1992 to present), Medline (1946 to present), EMBASE (1974 to present), PubMed (1946 to present), Web of Science (1970 to present) and Scopus (1823 to present) were searched on 27 January 2016 without limitation on date of publication or article type. Preliminary searches showed that the use of subject headings or search terms was obstructive to the process of gathering a broad and unbiased library of sources. This may be a consequence of the immaturity of this field and an inconsistent association with key terms. Therefore, a set of free text searches were tailored for each database. These captured variants of the terms “cardiovascular” and “additive manufacturing”. The search used in each case can be found in Supplementary material 1.

Records were screened for eligibility at two levels. Initial assessment of citation titles and abstracts preceded retrieval of full text sources which was then followed by further screening.

Data extraction

To ensure information was consistently recorded, we manually extracted data from full text records using a standardised spreadsheet prepared prior to reading. The details included are shown in table 2. Multiple entries for imaging modality, segmentation method and software, and model subject were recorded as necessary. Extracted data were validated against the set of options listed in Supplementary material 2.

Table 2.

A list of the data that were extracted from the full text sources that were retrieved.

Data extracted from full text items
First author
Year
Title
Imaging modality
Segmentation method
Segmentation descriptive quality (SDQ)
Segmentation software
Segmentation duration
Model subject
Type of modelling
Clinical application

Segmentation descriptive quality (SDQ) was scored on a novel three point scale: (1) No description of segmentation procedure; (2) Mention of the segmentation methods used, but no description of how these were applied; (3) Full description of how the segmentation methods used were applied, such that the procedure could be understood and reproduced. Where the full text description of image segmentation referred to a separate resource (whether the citation was identified in the original searches or not) the cited source was also retrieved and considered alongside the original report to extract data.

Risk of bias and strength of evidence assessment

Given the immaturity of this area of study, anecdotal and case reports with a small number of patients predominate. There is no common way in which studies have been carried out or reported. Consequently, there are no neither recognised, nor obvious ways to assess the risk of bias and strength of evidence, as there would be for more established studies such as randomised controlled trials (RCTs). In summary, we do not think that this technology, nor reporting of its use, is advanced enough to warrant rigorous interrogation on these matters.

We have resorted to separating reports by publication type, making rudimentary use of the hierarchy of evidence to infer reliability. Data extracted from case reports, conference proceedings and technical notes; journal articles and journal reviews; and clinical trials are presented separately.

Results

Included sources

The results of free text searching and eligibility assessment are shown in the flow diagram in Figure 2. The citations of all eligible resources are provided in Supplementary material 3.

Figure 2.

A flow diagram summarising the identification, screening, retrieval, eligibility and inclusion of records and full text resources within the systematic review.

The publication of research on the application of additive manufacturing to cardiovascular modelling has increased over the past 20 years (see Figure 1). Despite this rise, only 136 records (see Figure 2) were included in this systematic review, and fewer than 30 items of any sort were published in 2015. Figure 1 also shows a rapid increase in the number of publications from 2014 onwards, thanks largely to a jump in the number of conference submissions, technical and case reports (the grey literature) on this topic. A single clinical trial was retrieved from the Cochrane Library,[13] however this examined the use of 3D printing in Marfan syndrome. No clinical trial of this technology in congenital heart disease was found. Most records returned were journal articles (see Figure 1). These observations are consistent with an immature technology that is beginning to be used in larger teaching hospitals and research centres.

Figure 1.

A graphical history of publications on the topic of additive manufacturing in cardiovascular applications. Note that data for 2016 are only correct up to 27 January 2016.

Medical imaging modalities

The medical imaging modalities used for additive manufacturing were nominally reported throughout all sources included except for the sole clinical trial[13] and in one conference abstract.[14]

Figure 3 shows the use of different imaging modalities across included records. Most commonly (in 121 cases, approximately 90% of included records), CT- and MRI-based images are used. This ratio is observed in journal publications and in the grey literature. The 13 remaining reports rely on 3D echocardiographic images. In nine of these cases, ultrasound-derived models depicted heart valves only. When compared with only four out of 108 sources that describe CT- and MRI-derived valvular models, a distinction between different scan types becomes clear. CT and MRI data have primarily been used to model intracardiac anatomy of the ventricles and atria and extracardiac anatomy of the great vessels; whereas echocardiography data have been more often used to depict valvular anatomy.

Figure 3.

A summary of the different imaging modalities used to acquire data from which 3D models can be developed. Values represent the fraction of journal publications (left) and conference, technical and case reports (right) that use each modality. Note that as a single publication can report the use of more than one modality, the fraction of publications using each method need not sum to 1. CT: x-ray computed tomography; CTA: x-ray computed tomography angiogram, MRI: electrocardiogram- (ECG) and / or respiratory-navigated balanced steady state free precession; MRA: contrast-enhanced magnetic resonance angiogram; PC: phase contrast magnetic resonance imaging, US: ultrasound, Echo: echocardiogram.

Regarding cardiac magnetic resonance, the image modality used was either an ECG-gated whole-heart balanced SSFP sequence or a non-gated gadolinium-enhanced magnetic resonance angiogram (MRA). The ratio of ECG-gated MRI to non-gated MRA acquisitions being approximately 3:1, for both intracardiac and extracardiac structures.

In approximately 20% of records, authors explicitly reported the production of models from more than one type of scan. Most frequently this means the use of CT and MRI modalities. In all but three of these cases, however, separate models are fabricated from distinct scans, without true combination of multi-modal information into a single model. Conference abstracts reported the combination of ultrasound-derived valvular morphology within a whole heart model developed from MRA[15] and CT[16] data. The potential benefit of an echocardiographic-tomographic combined approach was discussed by Kurup et al.[17] in their recent journal publication on hybrid 3D printing.

Image segmentation software

Although a number of different computer programs have been used to perform image segmentation, a piece of software called Mimics (Materialise NV, Leuven, Belgium, 1992–2015) has proven the most popular. Generically, or with details of software version (7.3, 8.11, 9.0 and 15.0) the use of this software is reported in 49 records. No other piece of proprietary software has been mentioned in more than six publications. In 18 records, the authors have developed their own algorithm or relied on a procedure developed by a collaborator.

Image segmentation reporting standards and methodologies

Figure 4 characterises the standard of image segmentation reporting (using the SDQ score) and summarises the different methods that have been used. The data represent journal publications only. It is inappropriate to compare the reporting of methods used between longer journal articles, which are able to provide more detail, and shorter conference, technical and case reports or abstracts.

Figure 4.

A summary of the SDQ and segmentation method data extracted from the journal publications (both reviews and articles) included in the review. The top pie breaks down the SDQ score characteristics of the 80 publications. The methods used within publications with SDQ = 2 or 3 are then summarised in the two lower pies. Note that as a single publication can report the use of more than one method, the fraction of publications using each method need not add up to 1.

The upper chart in Figure 4 suggests that reporting of segmentation procedure is generally poor. Only 34% of journal publications provided an account with sufficient detail to be reproduced (SDQ = 3). A further 38% mentioned the methods that they had used, but did not explain how these had been applied (SDQ = 2). The remaining 29% of publications did not provide any description of their method whatsoever (SDQ = 1).

The bottom plots provide a breakdown of the different methods that were reported in papers with SDQ scores of 2 and 3. A similar distribution emerges in each chart: segmentation is dominated by the use of manual and semi-automatic methods. Brightness thresholding, region growing and manual editing are the three most frequently used methods. The combination and order of these methods reflect the segmentation workflow encouraged by the developers of Mimics, returning an accurate, but time-consuming result. The remaining techniques listed in Supplementary material 2 are all found within this review, but are less popular. There were five publications in which a specialised method was developed that could not be categorised: two were suited to the segmentation of limited vascular structures to validate computational models of haemodynamics,[18,19] two were adapted for segmentation of ultrasound images[20,21] and one referred to a conference abstract which lacked sufficient detail to be fully appreciated.[22]

Segmentation time

We had hoped to perform a thorough statistical analysis of segmentation time to establish whether there was a combination of image modality, software and segmentation method that achieved the fastest results. However, segmentation time is not reported frequently or consistently enough to allow for statistical scrutiny.

Only 20 of the 136 sources record the segmentation duration (all of these are journal articles). Furthermore, there are a number of factors which make this limited data set highly heterogeneous. These include: (1) imprecise reporting (such as between 2 and 3 h[8]); (2) different segmentation targets (for example all the intracardiac structures of the heart compared with a part of the vasculature); (3) the different motivations for modelling, and the different segmentation requirements inherent to each of these; (4) the operator dependency of image segmentation.[12] Given that these data are not measuring the same quantity, it is unrealistic to synthesise a result using descriptive statistics. The most that can be said is that reported times ranged between 15 min[7] and 120 h[23] to segment models of the whole heart, although between 2 and 3 h[8] is perhaps more representative.

Discussion

Summary of findings

The use of brightness thresholding, region growing and manual editing to segment cardiovascular structures from primarily CT and CMR images dominate the development of patient-specific 3D printed models. Perhaps unsurprisingly, these tools compose the segmentation pipeline associated with the most commonly used software, Materialise’s Mimics. However, the use of bespoke software is not uncommon.

Perhaps the most significant finding of this review is that image segmentation methods are generally reported poorly, with only one in three authors providing an account from which their work could be reproduced. Unfortunately, due to infrequent, inaccurate and imprecise reporting, we are unable to present rigorous results to summarise the amount of time that is generally required to complete image segmentation. Our best estimate is that this process takes a time on the order of hours.

These findings are relevant for those attempting to incorporate 3D printing into their clinical practice and for those researching whether image segmentation methods can be improved for this application.

Findings surrounding the choice of CMR acquisition

The review found that the selection of imaging data for the fabrication of valvular models reflected the superior ability of echocardiography to visualise heart valves compared to either CMR or CT. A similar relationship, derived from the relative strengths and weaknesses of different CMR acquisitions, was not observed. Our experience is that the selection of CMR data impacts greatly on the geometry of the resulting segmentation, particularly where small structures, often in close proximity to one another, are clinically relevant to the patient’s condition. These include the pulmonary veins and atrial septum. This choice also influences the length of time needed to complete segmentation.

Anecdotally, we find that the high blood contrast properties of a gadolinium-enhanced MRA type acquisition are favourable for segmentation, but that the lack of cardiac gating and therefore lower spatial resolution limit its use to the definition of extracardiac vasculature. Conversely, the display of intracardiac details necessitates high spatial resolution, as provided by ECG-gated and respiratory-navigated balanced steady state free precession data. This distinction was not observed, with the ratio of ECG-gated MRI to non-gated MRA acquisitions being approximately 3:1, for both intracardiac and extracardiac structures.

Implications and further work

The results of this review suggest that although well-established image processing techniques are widely used throughout the literature, they cannot deliver a suitable segmentation of cardiovascular structures without considerable operator-dependent input. A robust, automated approach to CMR image segmentation does not exist. Although many researchers and clinicians may have anticipated this result, this is the first systematic review of this topic that can rigorously confirm this assertion.

Furthermore, these observations are consistent with the concerns raised about the amount of time[10,11] and expertise[12] demanded by existing, manual segmentation methods. The finding that image segmentation requires on the order of hours of the operator’s time may prevent the clinician from completing this task. Ultimately, the results of this review agree with the assertion of Kim et al.[23]: the failings of existing image segmentation methods prohibit the introduction of 3D printing to the routine care of those with congenital heart disease, at least outside of larger teaching hospitals and research centres.

Along with this observation, the findings of this study motivate the development of advanced image segmentation procedures. Ideally, these would use automated processes to reduce both the time and expertise involved.

If this is to become a reality, we recommend that significant improvements in the reporting of 3D printing pipelines, and in particular, image segmentation methods are made. At a minimum, we think authors need to specify: (1) The imaging from which models were derived; (2) Any software that was used; (3) The image segmentation tools used; (4) Provide a brief account of the way that these were applied; (5) And indicate how long the image segmentation process took. We also encourage authors to reflect on the clinical feasibility of the methods they adopt in addition to the clinical impact of the 3D printed models they produce. These details would provide valuable data to inform the development of improved 3D printing pipelines through future research. This may include the formal introduction of hybrid models, derived from and exploiting the relative strengths of multi-modal imaging data.

Study limitations

Our inability to rigorously assess the risk of bias and the strength of findings is a limitation of this review. We have argued that this was largely enforced by the immaturity of the field addressed and the nature of the review question posed. For the majority of reports, the implicit primary outcome is measured by a binary dependent variable that characterises either success or failure of their ability to reproduce patient-specific anatomy and disease morphology in a physical model. When coupled with the largely anecdotal studies of a small number of patients, this makes it difficult to contextualise the methods used. A strong publication bias to include only those cases where 3D printing is successful is apparent.

Given that the objectives of this review were to establish the different methods that can be used to perform image segmentation, we do not think that these shortcomings preclude interpretation of the review’s findings. They do, however, mean that we cannot evaluate the performance of different methods, only report those which are commonly employed.

Conclusion

The segmentation of cardiovascular structures from medical images (CT and CMR) is an unavoidable step in the development of patient-specific, 3D printed models of congenital heart disease morphology. This review finds a reliance on manual and semi-automated segmentation methods which demand a high level of expertise and a significant time commitment on the part of the operator. This result is consistent with the assertions of previous authors who have considered the reasons why 3D printing is yet to become part of routine care. In light of these findings, we have made recommendations regarding reporting of 3D printing studies. We anticipate that these findings will be useful to the development and motivation of advanced image segmentation methods.