High-throughput digitization of analog human echocardiography data.

Echocardiographic imaging has been acquired in historical longitudinal cohorts of cardiovascular disease. Many cohorts were established prior to digital recording of echocardiography, and thus have preserved their archival imaging on Video Home System (VHS) tapes. These tapes require large physical storage space, are affected by physical degradation, and cannot be analyzed using modern digital techniques. We have designed and implemented a standardized methodology for digitizing analog data in historical longitudinal cohorts. The methodology creates a pipeline through critical steps of initial review, digitization, anonymization, quality control, and storage. The methodology has been implemented in the Framingham Offspring Study, a community-based epidemiological cohort study with echocardiography performed during serial examinations between 1987 and 1998. We present this method as an accessible pipeline for preserving and repurposing historical imaging data acquired from large cohort studies. The described technique:•Outlines a generalizable pipeline for digitization of analog recordings of echocardiography stored on VHS tapes•Addresses research concerns including quality control, anonymization, and storage•Expresses the authors' individual experience regarding observed image quality, training needs, and potential limitations to help readers understand the costs and benefits of this method.

Introduction

The Framingham Heart Study (FHS) has provided fundamental insights into the epidemiology of risk factors and cardiovascular disease. A substantial portion of this seminal work has arisen from echocardiography [1], [2]–3]. The Framingham Offspring cohort study is one of the nation's earliest second-generation epidemiologic studies [4,5]. The Offspring cohort study began in 1971, as an expansion from the original 1948 Framingham Heart Study, by enrolling the original cohort's offspring and their spouses [6]. Participants underwent regular examinations on average every four years for the duration of the participant's life. A total of 5,124 participants were originally enrolled in this study, with currently 2,675 living participants still enrolled. As part of the Framingham Offspring Study, echocardiography was performed and recorded at four visits by the end of 1998. By the year 2002, all echocardiography was recorded digitally for the cohort, however at the time of preceding image acquisition, digital imaging was not broadly available. Initial imaging studies were recorded on Video Home System (VHS) tapes. VHS tapes typically undergo up to 10-20% structural deterioration over a period of 10-25 years, depending on storage conditions [7]. The information contained on the Framingham Offspring Study tapes deserves preservation of participant effort and accessibility for contemporary digital and quantitative methods of analysis that may derive new information from this historical data.

We have established a standardized process for converting analog echocardiographic videos captured on VHS tape into digital files. This will enable access of the historical data to multiple users in digital format, prevent physical deterioration of data, and reduce the physical storage space requirements of VHS tapes. Cardiac disease remains the most common cause of death in the United States [8]. While multiple modern cohorts collect digital cardiovascular imaging data, historical cohorts with analog recorded imaging have accrued patient outcomes that are only accessible through long-term follow up. Analog quantitative echocardiography data from Framingham Offspring Cohort has been assessed using conventional image analysis methods, as reported in multiple prior publications [[9], [10], [11]–12]. Digital conversion of images will allow for alternative methods of analysis including texture analysis and artificial intelligence approaches [13]. We present our scalable analog-to-digital conversion methodology, which may be generalizable to other historical longitudinal cohorts with analog echocardiography (Table 1). A team of core laboratory investigators developed and designed our analog-to-digital imaging data conversion process following a careful review and evaluation of previously reported methods. The finalized protocol was then approved by the FHS Executive Committee and our institutional review board. The protocol involves a multi-step workflow as follows (Graphical Abstract).

Table 1

Large longitudinal cohorts in cardiovascular disease with echocardiography performed prior to 2000 and ≥ 1000 participants. MACE = Major Adverse Cardiovascular Events. PAD = Peripheral Artery Disease.

Study Name	Start Date	End Date	Echo Years	Participants with Echo	Goals
Cardiovascular Health Study (CHS)	1989	2009	1988-19891994-1995	5888 [21]	Determine causes of incident MACE, PAD, and mortality in adults > 65 years old
Coronary Artery Risk Development in Young Adults (CARDIA)	1983	Ongoing	1990-1991	4111 [22]	Identify causes of MACE in youth of multiple ethnicities
Chin–Shan Community Cardiovascular Cohort (CCCC)	1990	Ongoing	1992-19931994-1995	3602 [23,24]	Investigate cardiovascular health transition from a developing to developed nation
Hypertension Genetic Epidemiology Network (HyperGen)	1995	2008	1995-1997	2539* [25]	Determine interaction of genetic factors and hypertension and MACE
Atherosclerosis Risk in Communities Study (ARIC)	1987	Ongoing	1993-1995,2011-ongoing	2445 (Jackson Cohort) [26]	Investigate cardiovascular disease in adults from four separate communities
Tromsø Study	1974	Ongoing	1994-2010	2406 (Tromsø 4) [27]	Identify causes of MACE and other systemic disease in men in Norway
Framingham Heart Study (FHS)	1948	Ongoing	1979-Ongoing	2291 (Original Cohort) [28]	Identify common factors or characteristics that contribute to cardiovascular disease
Austrian Stroke Prevention Study (ASPS)	1991	Ongoing	1991-1994	1998 [29]	Determine vascular risk factors on brain function in elderly participants
The Study of Health in Pomerania (SHIP)	1997	Ongoing	1997-2001	1955 (SHIP-0) [30]	Assess health differences in East Germany post-reunification
Monitoring Trends and Determinants in Cardiovascular Disease (MONICA)	1979	1996	1995-1995	1467 (Glasgow Cohort) [31]	International study of communities for risks of cardiovascular disease

*HyperGEN has digitized 2150 studies.

Original image acquisition and storage

Echocardiographic imaging using analog video recording methods was included in cohort examinations for Framingham Offspring participants beginning in 1987 (Fig. 1). The echocardiographic image acquisition protocol was standardized for each examination of the Offspring cohort examinations 4, 5 and 6 using an HP SONOS 1000 ultrasound machine equipped with a Panasonic AG6300 VHS video recorder (Table 2). The VHS tapes were stored onsite in a secured, temperature and humidity-controlled facility. Corresponding logbooks containing participant name, cohort study identification number, exam number, tape number, date of exam, and exam beginning and ending time were maintained. Each VHS tape contains a total of 8 to 24 studies, which vary in duration and image quality. Lower quality studies typically included longer recording time in order to compensate for limitations of image quality. There were differences in the number of standard echocardiographic views recorded at exam visit 5, compared to the preceding visit 4 and the later visit 6 (Table 3).

Fig. 1

Timeline of Framingham Offspring Cohort and digital storage standards. In 1983 ACR (American College of Radiology) and NEMA (National Electrical Manufacturers Association) form ACR-NEMA Committee to create an imaging standard that satisfied the needs of both physicians and medical imaging equipment manufacturers. Three versions of the standard were released. The first in 1985, the second in 1988, and the third in 1993. DICOM = Digital Imaging and Communications in Medicine.

Table 2

Dates of original examinations, exam conversion, details, and considerations. PW = pulse wave. CW = continuous wave.

	Original Study Recording Dates	# Cardiac Cycles	Variable Considerations
Exam 4	1987-1991	• 4 to 6 cardiac cycles per 2D view• 3 cardiac cycles per m-mode, PW, and CW image	• Quality of study - affected # of cycles obtained and images obtained• Length of study - affected # of cycles obtained• Availability of images - affected which images we could obtain
Exam 5	1991-1995
Exam 6	1995-1998

Table 3

Exam-specific Protocols: LV = left ventricle, RV = right ventricle, TV = tricuspid valve, MV = mitral valve, TR = tricuspid regurgitation, AR = aortic regurgitation, MR = mitral regurgitation, LVOT = left ventricular outflow tract CW = continuous wave, PW = pulse wave.

	Exam 4 and 6 Digitization Protocol	Exam 5 Digitization Protocol
Parasternal Long Axis View	2D of Left Ventricle	2D of Left Ventricle
		RV Inflow - CW doppler, if TR is present and TR envelope is seen/measurable
Parasternal Short Axis	M-Mode of Aortic Valve, Aortic Root, and Left Ventricle	M-Mode of Aortic Valve, Aortic Root, and Left Ventricle
	2D of Left Ventricle at level of the Papillary Muscle	2D of Left Ventricle at level of the Papillary Muscle
	M-Mode of LV at level of the Papillary Muscle	M-Mode of LV at level of the Papillary Muscle
Apical 5 Chamber	n/a	PW doppler of LVOT (1 fast frame, 1 slow frame)
		CW Doppler of LVOT, if AR is present
Apical 4 Chamber	2D of all 4 chambers	2D of all 4 chambers
		Zoomed view of 2D of Left Ventricle (if available)
		PW doppler of MV inflow, regular frame rate
		CW doppler, if MR is present
	Zoomed view of 2D of Left Ventricle (if available)	PW doppler of TV inflow regular frame rate
		CW doppler, TR is present
Apical 2 Chamber	2D of Left Ventricle and Left Atrium	2D of Left Ventricle and Left Atrium
	Zoomed view of 2D of Left Ventricle and Left Atrium (if available)	Zoomed view of 2D of Left Ventricle and Left Atrium (if available)
		CW doppler if MR is present
Apical Long Axis	2D of Left Ventricle and Left Atrium	2D of Left Ventricle and Left Atrium
	Zoomed view of 2D of Left Ventricle and Left Atrium (if available)	Zoomed view of 2D of Left Ventricle and Left Atrium (if available)
		CW doppler if MR is present

Analog to digital conversion

All technical staff involved in digitization had received an intensive and direct hands-on process training including training on standard echocardiographic views, basic echocardiography physics, and cardiovascular anatomy. Previous experience in clinical echocardiography or medical imaging, or advanced degree work was not required prior to training. For all involved technical staff, training on the basic operation of the analog to digital recording application was completed within a half day; however, development of complete understanding of overarching goals for the digitization process within the framework of the study required a longer timeframe. All training activities were personalized to individual trainee needs, and all trainees were able to become self-sufficient and demonstrate reliable process performance within 2 weeks of training initiation. Essential milestones that were successfully achieved by each trainee included ability to: recognize artifacts and causes for reduced image quality, identify key elements of standard echocardiographic views to avoid foreshortened images and anatomically incomplete views, and distinguish the phases of the cardiac cycle to ensure adequate recording of echo loops. Quality control checks were performed by technical supervisors at regular time intervals over the course of the digitization process; these quality control checks involved assessing fully digitized studies in person with digitization staff with direct educational feedback provided as needed.

All VHS tapes are manually reviewed on a VHS tape player, cross-checked against original logbooks to verify which participant exams are recorded on a given tape, and then physically labeled. This step confirms study identity, corrects historical annotation errors during initial recording, and optimizes the ability to locate studies. Following confirmation of participant and study, a Mitsubishi MD3000 VCR connected to a TIMS 500 medical computer system (Foresight Imaging, Chelmsford, MA, USA) is used to convert VHS data to digital format. Images are digitized into Digital Imaging and COmmunications in Medicine (DICOM) format given its acceptance as a universal standard format for medical imaging and the ability to share DICOM image files within a Picture Archiving and Communications System (PACS). The highest-quality 4 to 6 cardiac cycles of each standard echocardiographic view are manually identified. Each view recorded digitally and saved as a separate DICOM file within the corresponding study. The DICOM header is populated during the TIMS conversion process with participant name, ID number, and study date. Each study requires 15-20 minutes to digitize manually, depending on the quality and technical issues pertaining to each individual echocardiographic study. During the digitization process, the quality of image output from digitizing the VHS image was directly observed and compared (i.e. original source image on VHS tape compared to digitized output image produced by the DICOM file generator) for each image and each study to ensure minimal to no observable differences in quality.

Corresponding digitization logbooks containing participant ID, digitization date, digitization quality (graded on a scale of 1 to 4), tape number, and any relevant notes are maintained. Views with limited quality were noted in the digitization logbook under their corresponding study; however, 4 to 6 cardiac cycles per view are still recorded to maintain study record consistency. For each study that is fully processed via the digitization workflow, the final digitized images are sent to a PACS system residing within a firewall-secured, networked server environment. Overall, a total of 10 to 20 cine clips are recorded per study, depending on the availability of images. We observed that age-related degradation manifested as subtle reductions in image clarity, and the magnitude of this degradation was directly related to the time from acquisition. Despite the reductions in clarity, 20-30 years after VHS recording at the time of image acquisition, no studies were completely excluded due to degradation. The most common limitations were related to technical issues pertaining to image acquisition (e.g. limited acoustic windows and artifact). We note that the VHS tapes on which images were recorded were stored in environment-controlled rooms (with actively managed temperature, relative humidity, and air filtration) for the duration of their existence and this level of storage environment control may not be generalizable to all longitudinal cohorts.

Research anonymization workflow

Healthcare Insurance Portability and Accountability Act (HIPAA) mandates removal of protected health information from any medical records before use in a research context. In order to generate a research-ready database, anonymization of the DICOM files is necessary including the text-based header and any text within images. Although multiple DICOM anonymization software programs are available; performance of the different software packages vary widely [14]. The anticipated total quantity of Framingham Heart Study studies is greater than would be feasible for manual deidentification. Proprietary software (coded in Python v.3.8.1) is used by technical staff to anonymize images in batches of 20, whereby all 20 studies are stripped of headers at the same time. Then, 2D images and M-mode/doppler images are stripped of burned-in text separately, due to variation in the location of the text on the image itself. We remove unreplaced identities, descriptions, series description, acquisition protocol name, patient characteristics, all unique image identifiers, unsafe private attributes, device identifiers, institution identifiers, clinical study attributes, all structured content, and unsafe structured content. Anonymized studies are subsequently stored in a secured research server. Corresponding anonymization logbooks containing participant ID, anonymization date, and any relevant notes are maintained.

Image quality control

Ensuring image quality control, throughout each step of the digitization pipeline, is critical. For our pipeline, we recognize an inherent limitation in the inability to apply a quantitative measure of quality control to compare precision of measurements performed on the original analog recorded images versus precision of measurements that could be performed on the output images generated by digitization. Distance measurements were not performed directly on source images at the time of original image acquisition, therefore direct quantitative comparisons are not possible without manual caliper-based measurement. Additionally, although DICOM headers can encode the relationship of physical distances to pixel distances, this information is not directly extractable from analog VHS recordings. We opted to not manually measure pixel spacing distances and encode the header, given the risk of introducing additional sources of measurement error during such a process. Because the burned-in markers are required to calibrate image spatial information for analog images, these same burned-in markers can be used for any post-digitization measurements so long as there has been no image distortion introduced during the digitization process (e.g. retained aspect ratio). In situations where measurements were made at the time of original image acquisition, a formal quantitative comparison of measurement precision performed on original analog images versus measurement precision performed on digitized images should be considered.

Quantitative quality control of the anonymization step was performed. Loss of image quality during anonymization is possible due to direct image editing to remove “burned-in” patient identifiers within the image. Quantitative comparisons were performed by converting DICOM to AVI with no compression using MicroDicom Viewer (MicroDicom, Sofia, Bulgaria) with and without anonymization. Both of these files are then imported into QC Tools (V1, by MediaArea, Curienne, France) where they are analyzed for major discrepancies using the metrics listed in Table 4. From quantitative and qualitative comparisons, no significant differences were observed between anonymized vs non-anonymized studies. To ensure quality, two DICOM files from 10 patients every 250-500 patient batches were quantitatively and qualitatively compared for changes between pre-anonymized and anonymized images.

Table 4

Image quality assessment variables. PSNR = Peak signal to noise ratio, MSE = Mean square error.

IMAGE QUALITY ASSESSMENT
Metric	Information Given
Y-Value	Information about brightness and luminance
U-Value	Information about video color
V-Value	Information about video color
Y, U and V Diff	Indicates the extent of visual change from one frame to the next
PSNR	Comparison as a ratio of the peak signals expressed as dB
Y-Range	Indicative of contrast range
Y, U, V averages	Average of Y, U and V values
MSE	Mean Square Error for each (YUV) plane. Higher values may be indicative of differences between 2 images being compared.
File Size	Differences in file size could be indicative of differences in video quality

Current status of full digitization project

The study pipeline consists of multiple steps in order to ensure adequate quality, accurate labeling, and complete anonymization. While digitization itself has been straightforward, quality assurance with respect to image quality and complete anonymization comprise the most time and effort intensive steps. We recognize the essential priority of ensuring adequate subject protection and data quality. To this end, project pipeline including quality assurance procedures are ongoing for the full compendium of available studies.

Access of anonymized studies and future directions

Current and future plans are in place to perform quantitative analysis on the data including conventional and advanced image analysis using approaches such as texture analysis, machine learning, and deep learning. We anticipate that multiple projects will be feasible using these data. Data sharing will be enabled from a centralized secure server via standard institutional data sharing agreements.

Conclusions

Conversion of archival analog echocardiography to research-ready digital storage is possible using a combination of proprietary and open source methods. Physical storage differences are substantial with associated cost savings, given the resources required for both space and maintenance of both analog media as well as analog reading devices. Digitally stored data allows for access by multiple users and locations and is easily duplicated for analyses. Current methods for ensuring digital fidelity and protection against data loss via hardware degradation include storage of servers and hard drives in temperature-controlled rooms, performance of regular maintenance on storage hardware, protection from electrical surges, and use of high quality equipment [[15], [16]–17]. All of these protections ensure that the lifetime of the Framingham Offspring Study digital data will extend far beyond the initial storage methods for future research.

This approach is scalable and generalizable. There have been multiple historical cohorts that have collected analog echocardiography data (Table 1). However, few studies have published digitization of their historical data. The HyperGEN Archeological Echocardiography study has had the largest digitization process to date. This study examined association between hypertension genetics and left ventricular hypertrophy, digitized 2150 of 2234 echoes and applied strain analysis [18]. The HyperGEN study used a similar TIMS setup with offline storage and no formal anonymization process described. Other alternative approaches to digitize echocardiograms stored on analog tapes exist using alternative conversion hardware and electronic storage methods [19,20]. We offer our experience at the FHS as a proof of concept that preservation of legacy data can be cost-effective, consistent, and scalable. Data can be preserved in a research-ready durable storage format.

We recognize limitations within our workflow. All processes were performed at a single site with the same equipment by a dedicated research team. The inability to perform retrospective quantitative quality control of the analog images as discussed in the “Image Quality Control” section should be noted. The quantity of time and use of manual processes remains a limitation, as this required a full-time team of research staff and multiple years of effort. The manual process of logging and labeling is at risk for error, which could affect data labeling. At this time, we have completed digitization, quality assurance procedures, and anonymization for only a portion of the total planned number of studies and, thus, additional limitations may become apparent with further progress.

Large community-based epidemiological cohort studies with historically collected data continue to offer a multitude of opportunities to derive new insights relevant to modern medical questions. As the accessibility to and support for analog data decrease over time, a number of large cohort studies with historically collected analog imaging data will face similar challenges related to data conversion, anonymization, quality control, research preparation, and storage, for which we offer an approach. Further automation of this process is being pursued. Preservation of the data, and storage in a format amenable to novel research methods is a worthy goal for investigators. Our method is scalable and maintains image quality across contemporary storage formats and may provide a blueprint for future efforts to maintain and update historical imaging data collected from clinical research studies.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Subject Area:	Medicine and Dentistry
More specific subject area:	Echocardiography
Method name:	High Throughout Digitization of Echocardiography
Name and reference of original method:	N/A
Resource availability:	Mitsubishi MD3000 VCR (Mitsubishi Corporation, Chiyoda City, Tokyo, Japan)TIMS 500 medical computer (Foresight Imaging, Chelmsford, MA, USA)MicroDicom Viewer (MicroDicom, Sofia, Bulgaria)QC Tools (V1, by MediaArea, Curienne, France)