Automated versus manual B-lines counting, left ventricular outflow tract velocity time integral and inferior vena cava collapsibility index in COVID-19 patients.

Background and Aims: The incorporation of artificial intelligence (AI) in point-of-care ultrasound (POCUS) has become a very useful tool to quickly assess cardiorespiratory function in coronavirus disease (COVID)-19 patients. The objective of this study was to test the agreement between manual and automated B-lines counting, left ventricular outflow tract velocity time integral (LVOT-VTI) and inferior vena cava collapsibility index (IVC-CI) in suspected or confirmed COVID-19 patients using AI integrated POCUS. In addition, we investigated the inter-observer, intra-observer variability and reliability of assessment of echocardiographic parameters using AI by a novice.
Methods: Two experienced sonographers in POCUS and one novice learner independently and consecutively performed ultrasound assessment of B-lines counting, LVOT-VTI and IVC-CI in 83 suspected and confirmed COVID-19 cases which included both manual and AI methods.
Results: Agreement between automated and manual assessment of LVOT-VTI, and IVC-CI were excellent [intraclass correlation coefficient (ICC) 0.98, P < 0.001]. Intra-observer reliability and inter-observer reliability of these parameters were excellent [ICC 0.96-0.99, P < 0.001]. Moreover, agreement between novice and experts using AI for LVOT-VTI and IVC-CI assessment was also excellent [ICC 0.95-0.97, P < 0.001]. However, correlation and intra-observer reliability between automated and manual B-lines counting was moderate [(ICC) 0.52-0.53, P < 0.001] and [ICC 0.56-0.69, P < 0.001], respectively. Inter-observer reliability was good [ICC 0.79-0.87, P < 0.001]. Agreement of B-lines counting between novice and experts using AI was weak [ICC 0.18, P < 0.001].
Conclusion: AI-guided assessment of LVOT-VTI, IVC-CI and B-lines counting is reliable and consistent with manual assessment in COVID-19 patients. Novices can reliably estimate LVOT-VTI and IVC-CI using AI software in COVID-19 patients. Copyright:

INTRODUCTION

Ultrasound imaging of the lung and associated tissues helps in informed clinical decision-making in patients with coronavirus disease (COVID)-19 and management of their associated respiratory failure and lung injury. In addition, myocardial injury is expected to present in more than a quarter of patients with critically ill COVID-19 patients.[1] Thus, the rapid bedside assessment of the lungs, heart, and blood vessels by point-of-care ultrasound (POCUS) becomes a useful tool for the frontline warriors to aid rapid diagnosis in the fight against the COVID-19 pandemic. POCUS is usually performed by a treating physician/expert sonographer to obtain real-time information for the diagnosis, triaging, management and prognostication. Pneumonia caused by COVID-19 leads to alteration in lung tissues causing localised vertical artifacts to appear on the ultrasound images in addition to the alterations of the sub-pleural tissue. These artifacts have generally been called B-lines.[2] The myocardium may also be involved in COVID-19 as a result of cytokine storm, myocarditis or stress cardiomyopathy.[34] Being a non-invasive imaging technique, echocardiography can help quickly identify the circulatory status of COVID-19 patients and guide haemodynamic management. Left ventricular outflow tract velocity time integral (LVOT-VTI) is one of the echocardiographic parameters to calculate cardiac output (CO), which may be a surrogate for cardiovascular function. One of the potential causes of hypotension in haemodynamically unstable COVID-19 patients is hypovolaemia. The identification of latent/overt hypovolaemia offers the clinician an opportunity to implement fluid resuscitation. Concerning volume status, the variability of the diameter of the inferior vena cava with respiration in a spontaneously breathing subject known as the “inferior vena cava collapsibility index” (IVC-CI), is considered to be a valuable predictor of volume responsiveness in cases of circulatory failure even in the presence of nonfatal cardiac arrhythmias.[567] Artificial intelligence (AI) allows automatic real-time detection and quantification of lung B-lines, LVOT-VTI and IVC-CI which potentially could make ultrasound assessment less operator dependent. In this present study, we aimed to test the agreement between manual and automated B-lines counting, LVOT-VTI and IVC-CI in COVID-19 patients. We hypothesised that there would be good agreement between the two methods in all three parameters in COVID-19 patients. The primary aim was to compare and validate automated with manual calculations of all three parameters in expert hands and the secondary aim was to compare AI-based acquisition by a novice with those of experts.

METHODS

This single centre prospective observational study was conducted in a tertiary health care facility in an urban setting. All adult patients of age more than 18 years, with confirmed or symptomatic COVID-19 disease admitted to the hospital for observation were enroled for the study. All patients were breathing spontaneously without ventilatory support either invasive or non-invasive After obtaining ethical approval from the institutional ethical committee, written informed consent was obtained from 90 patients. Patients with a severe form of COVID-19, those on ventilatory support, haemodynamically unstable patients, psychiatric patients and patients who were not willing to participate in the study were excluded. The two expert sonographers independently and consecutively performed and stored video clips for B-lines counting, LVOT-VTI and IVC-CI calculations of each patient in a blinded fashion to determine inter- and intra-observer variations. To avoid potential bias, manual method was performed first followed by the automated method. First round was followed by the second round after 15 minutes duration. The sonographers recorded the clips and interpreted them offline which were blinded to the findings of the other expert. Similarly, novice practitioners performed automated B-lines counting, LVOT-VTI and IVC-CI in two rounds with 15 minutes gap. The novice was defined as a medical professional who graduated from nursing or medical resident other than cardiology or sonography who has no substantial training in ultrasound except for a ‘hands-on’ crash course. Crash course consists of a 2-hour training session for a period of 4 days with experts in ultrasound scanning. After the initial training, novice learners performed hands-on scanning independently for all three parameters in COVID-19 patients. Images were stored and the quality of images was examined by experts. The image obtained by the novice was graded as excellent, good, moderate or weak and the decision on intervention/treatment was performed only after consultation with a professional with appropriate credentials and expertise in the unit. A dedicated AI installed ultrasound machine (Venue Go™ Point of Care Ultrasound, General Electric Healthcare, Wauwatosa, WI, United States) was used for this study. To prevent contamination, an ultrasound machine was utilised only for COVID-19 patients. All those who performed POCUS donned and wore personal protective equipment (PPE) and performed hand hygiene precautions as per national recommendations Probe cover was used to prevent probe contamination. High level disinfection was performed after each usage of ultrasound to prevent pathogen transmission.[8]

For manual assessment of B-lines counting, the ultrasound scanning was performed using high frequency linear array probe (L4-12t, 4-12 MHz). Four zones in each hemithorax, respectively located at the upper and lower front and the upper and lower lateral chest were selected for the evaluation and marked beforehand to guide the performer of ultrasound examination in probe positioning. For the anterior chest, 3rd and 5th intercostal spaces were selected. For lateral chest, 4th and 6th intercostal spaces were selected. Each sonographer performed manual B lines counting followed by automated B lines counting by activating the auto B-lines tool [Figure 1]. Novice performed automated B-lines counting by using the same tool.

Figure 1

Artificial intelligence software showing B-lines count

For assessment of LVOT-VTI, the ultrasound investigation was carried out using real-time scanners equipped with phased array sector probes (3Sc, 1.6-4.5 MHz. In all cases, manual LVOT-VTI was computed on an apical five-chamber view using pulse-wave Doppler in the left ventricle outflow tract and tracing it manually. For auto LVOT-VTI, the investigator achieved a five-chamber view. By activating the auto-VTI tool, LVOT-VTI measurement was derived automatically [Figure 2].

Figure 2

Artificial intelligence software showing left ventricular outflow tract velocity time integral (LVOT-VTI)

For assessment of IVC-CI, the IVC was imaged in a longitudinal plane with the phased array sector probe (3Sc, 1.6-4.5 MHz) in the sub-xiphoid position. The intra-hepatic segment of the IVC was visualised as it entered the right atrium. The IVC diameter was measured at 2 cm caudal to the hepatic vein-IVC junction, or approximately 3–4 cm from the junction of the IVC and right atrium. This measurement location was preferred as IVC collapsibility in the intra-hepatic segment, which was not influenced by the activity of the muscular diaphragm compared to one at the IVC-right atrial junctionM-mode was used to capture a 10 seconds cine loop of the IVC over two or three respiratory cycles. The maximum IVC diameter (IVCdmax) was measured as the maximum anterior-posterior dimension at end-expiration using the leading-edge technique (inner edge to inner edge of the vessel wall). In addition, the minimum IVC diameter was measured at end-inspiration (IVCdmin). The IVC –CI was the difference between the maximum and minimum IVC diameters divided by the maximum IVC diameter, expressed as a percentage ([IVCdmax – IVCdmin]/IVCdmax ×100%). For automated measurement of IVC-CI, the sonographer achieved the same sub-xiphoid IVC view in the longitudinal plane. Using the Auto-IVC tool, the ultrasound device was automatically detecting the IVC maximum and minimum dimensions and provided, IVC-CI [Figure 3].

Figure 3

Artificial intelligence software showing inferior vena cava collapsibility index (IVC-CI)

The sample size was calculated using nMaster software version 2.0 using single group continuous outcome. Based on a population reliability value of 0.5 and expected agreement between automatic and manual modes of more than 0.7 with power 80%, alpha error of 5% and number of replicates as two, the minimum required sample size was 80 patients. For possible attrition, it was decided to include 90 patients. The distribution of the continuous data was checked with the Kolmogorov–Smirnov one-sample test and the Shapiro-Wilk test. Continuous variables with a normal distribution were expressed as mean±standard deviation (SD). Dichotomous data were expressed as numbers and percentages. Intraclass correlation coefficients (ICCs) were used to study agreement between the visual and automatic system of B-lines counting, LVOT-VTI and IVC-CI in the two scanning rounds using the Spearman correlation coefficient. Similarly, ICC was used for intra-observer and interobserver reliability. For all statistical tests, a P value of <0.05 was considered to be significant. All the statistical tests were two-sided and were performed at a significance level of a = 0.05. Data were analyzed by using a statistical package for social sciences (SPSS Inc., Chicago, IL version 22.0).

RESULTS

A total of 90 patients were included in the study out of which, 7 patients were excluded because of incomplete data. Data from the remaining 83 patients were included in the study. The mean age of the patients was 54.46 ± 16.60 years. Of 83 patients, 66.2% were males, remaining 33.8% were females. Descriptive statistics of the study patients and observed parameters are given in Table 1.

Table 1

Demographics and clinical data of study cohort (n=83)

Variable	Descriptive statistics
Age (years)	54.5±16.6
Gender: Male/Female	55/28 (66.2/33.8)
Weight (kg)	67.5±9.0
Height (cm)	162.2±9.1
CAD	20 (20.1)
HTN	32 (38.5)
CKD	7 (8.4)
CVA	5 (6)
DM	35 (42)
LVEF (%)	52.2±8
Manual B-lines counts	1 (0-5)
Auto B-line counts	1 (0-5)
Manual LVOT-VTI (cm)	18.1±3.3
Auto LVOT-VTI (cm)	18.9±3.34
Manual IVC-CI (%)	41.0±6.7
Auto IVC-CI (%)	42.1±6.6

CAD=coronary artery disease, HTN=hypertension, CKD=chronic kidney disease, CVA=cerebrovascular accident, DM=diabetes mellitus, LVEF=left ventricular ejection fraction, LVOT VTI=left ventricular outflow tract velocity tine integral, IVC-CI=Inferior vena cava collapsibility index. Data are provided as Mean (Standard deviation), Proportions (percentages) or Median (interquartile range)

Agreement between the manual and automated LVOT-VTI measurement

Agreement between manual and automated LVOT-VTI assessment was excellent [ICC 0.98, P < 0.001]. Intra-observer reliability and Inter-observer reliability were also excellent [ICC 0.96-0.99 and ICC 0.95-0.99, P < 0.001]. Moreover, an agreement between novice and experts AI based LVOT VTI assessment was excellent [ICC 0.95-0.97, P < 0.001] [Table 2].

Table 2

Agreement between the manual and automated LVOT-VTI, IVC-CI and B-lines counting

Parameter	Test	No.		ICC (95% of CI)	P
LVOT-VTI	Manual vs Automated (Total)	332		0.987 (0.980-0.99)	<0.001
		Round 1	166	0.987 (0.98-0.99)	<0.001
		Round 2	166	0.986 (0.98, 0.99)	<0.001
	Expert vs Novice (Total)	332		0.96 (0.94-0.98)	<0.001
	Intra-observer reliability in manual method (Round 1 vs. Round 2)	Observer 1	83	0.99 (0.99-0.99)	<0.001
		Observer 2	83	0.98 (0.98-0.99)	<0.001
	Intra-observer reliability in automated method (Round 1 vs Round 2)	Observer 1	83	0.99 (0.99-0.99)	<0.001
		Observer 2	83	0.99 (0.98-0.99)	<0.001
	Inter-observer reliability in manual method (Observer 1 vs Observer 2)	Round 1	83	0.98 (0.97-0.990)	<0.001
		Round 2	83	0.99 (0.98-0.99)	<0.001
	Inter-observer reliability in automated method (Observer 1 vs Observer 2)	Round 1	83	0.99 (0.98-0.99)	<0.001
		Round 2	83	0.99 (0.98-0.99)	<0.001
IVC-CI	Manual vs Automated (Total)	332		0.99 (0.987-0.991)	<0.001
		Round 1	166	0.99 (0.98-0.99)	<0.001
		Round 2	166	0.99 (0.987-0.993)	<0.001
	Expert vs Novice (Total)	332		0.98 (0.97-0.99)	<0.001
	Intra-observer reliability in manual method (Round 1 vs Round 2)	Observer 1	83	0.987 (0.98-0.99)	<0.001
		Observer 2	83	0.982 (0.97-0.99)	<0.001
	Intra-observer reliability in automated method (Round 1 vs Round 2)	Observer 1	83	0.984 (0.98-0.99)	<0.001
		Observer 2	83	0.982 (0.97-0.99)	<0.001
	Inter-observer reliability in manual method (Observer 1 vs Observer 2)	Round 1	83	0.984 (0.98-0.99)	<0.001
		Round 2	83	0.987 (0.98-0.99)	<0.001
	Inter-observer reliability in automated method (Observer 1 vs Observer 2)	Round 1	83	0.988 (0.98-0.99)	<0.001
		Round 2	83	0.99 (0.985-0.99)	<0.001
B-lines counted	Manual vs Automated (Total)	2656		0.52 (0.49-0.56)	<0.001
		Round 1	1328	0.52 (0.47-0.57)	<0.001
		Round 2	1328	0.53 (0.47-0.58)	<0.001
	Expert vs Novice (Total)	2656		0.18 (0.11-0.24)	<0.001
	Intra-observer reliability in manual method (Round 1 vs Round 2)	Observer 1	664	0.64 (0.58-0.69)	<0.001
		Observer 2	664	0.69 (0.64-0.73)	<0.001
	Intra-observer reliability in automated method (Round 1 vs Round 2)	Observer 1	664	0.60 (0.54-0.66)	<0.001
		Observer 2	664	0.56 (0.48-0.62)	<0.001
	Interobserver reliability in manual method (Observer 1 vs Observer 2)	Round 1	664	0.84 (0.81-0.86)	<0.001
		Round 2	664	0.79 (0.76-0.83)	<0.001
	Interobserver reliability in automated method (Observer 1 vs Observer 2)	Round 1	664	0.87 (0.84-0.88)	<0.001
		Round 2	664	0.79 (0.76-0.82)	<0.001

LVOT-VTI=left ventricular outflow tract velocity time integral; IVC CI=Inferior vena cava collapsibility index; ICC=intraclass correlation coefficient, CI=confidence interval, vs=versus.

Agreement between the manual and automated IVC-CI measurement

Agreement between manual and automated IVC-CI assessment was excellent [ICC 0.99, P < 0.001]. Intra-observer reliability was excellent [ICC 0.98, P < 0.001]. Inter-observer reliability was also excellent for both systems in both rounds [ICC 0.97-0.99, P < 0.001]. Similar to LVOT-VTI, agreement between novice and experts AI based IVC-CI assessment was excellent [ICC 0.98, P < 0.001] [Table 2].

Agreement between the visual and automated B-lines counting

Agreement between manual and automated B-lines counting was moderate [(ICC) 0.52-0.53, P < 0.001]. Intra-observer reliability was moderate [ICC 0.56-0.69, P < 0.001]. Inter-observer reliability was good [ICC 0.79-0.87, P < 0.001]. However, auto B-lines assessed by novice had weak agreement with auto B-lines assessed by experts [ICC 0.18, P < 0.001] [Table 2].

DISCUSSION

AI technology when incorporated into ultrasound and echocardiography, has a great potential to decrease the burden on the healthcare professionals by allowing novice learners to perform basic ultrasound examination by the ability to count B-lines, LVOT-VTI, IVC-CI and incorporate findings into their clinical decisions in patients with COVID-19 Our study showed that there is moderate to excellent agreement between manual and automated measurement of B-line counting, LVOT-VTI and IVC-CI in COVID-19 patients. Moreover, this study demonstrated that the measurement of LVOT-VTI and IVC-CI using AI software in COVID-19 patients is found to have an excellent agreement between novice and expert sonographers.

AI is a machine’s ability to mimic human intelligence. In practice, it is a segment of computer science that involves designing computer applications to perform tasks that typically have required human intelligence such as visual perception, speech recognition, and decision making. In a recent proposal aimed at regulating AI software in medical devices, the U.S. Food and Drug Administration (FDA) stated that “artificial intelligence-based technologies have the potential to transform healthcare by deriving new and important insights from the vast amount of data generated during the delivery of healthcare every day”.[9] Instead of instructing the computer to evaluate a given condition, or to perform a specific task according to detailed programmed instructions, AI algorithms learn from exposure to numerous similar conditions previously, which is very useful in dealing with COVID-19 patients. Recently, FDA accelerated the process to provide rapid clearance for AI-based medical imaging technologies to help clinicians in the COVID-19 pandemic.

The excellent correlation between manual and AI-based measurement of LVOT-VTI and IVC-CI in this study indicates the reliability of the AI software for these measurements. Evidence shows that COVID-19 can affect the heart directly or indirectly through cytokine storm causing a decrease in left ventricular ejection fraction and cardiac output (CO).[4] Hence, auto-VTI is helpful in calculating CO rapidly. LVOT-VTI is a doppler-derived parameter to measure stroke volume, which may be used as a surrogate marker of cardiac function in adults.[10] Bobbia et al.[11] showed a good correlation between manual and automated VTI calculation in piglets.[11] Shaikh et al.[12] evaluated the feasibility of using AI in POCUS assessments of CO and compared the accuracy of automated versus manual measurements of left ventricular outflow tract diameter (DLVOT) and LVOT-VTI. They reported automated VTI measurement by AI is feasible and allows for quick and accurate measurement of CO in novice operators. Further, they suggested AI in POCUS diagnostics may limit expertise dependence and improve accuracy in non-experienced users. LVOT-VTI though not a direct measure of myocardial injury, is a useful parameter to determine the cardiac function, especially in terms of stroke volume and CO. The LVOT-VTI is a simple, feasible and reproducible measurement to serially track the stroke volume and CO and hence LVOT-VTI is highly valuable in the haemodynamic monitoring of critically ill patients in point-of-care settings. In addition, the LVOT-VTI is able to predict outcomes in selected populations.[13] Similarly, in one internal study conducted by GE healthcare, the IVC measurements were equivalent to an expert user’s ability, 90% of the time for minimal diameters and 97% for maximal diameters.[14] With B-lines counting, we observed moderate agreement between the manual and AI-based assessments. Short et al.[15] investigated the reliability and consistency of B line counting using manual and automated methods in four critically ill patients, in which they found good agreement between the two. Brusasco et al.[16] compared automatic quantification of B-lines by AI with semi-quantitative scores in the measurement of extravascular lung water as determined by standard thermodilution. They concluded that AI based B-line quantification is faster than conventional method of extra lung water estimation.

In this study, we observed excellent agreement between novice and experts in LVOT-VTI and IVC-CI measurement. However, a moderate agreement was observed in B-lines counting. This could possibly be secondary to technical variations in probe positions or lung inflation status between the manual and automatic assessment. Additionally, the AI software could only provide a maximum score of five B-lines at one time. These observations prove that novice operators can be used for POCUS study using AI software, as it decreases operator dependency. This can be helpful in the current COVID-19 pandemic, where experienced sonographers may not be available round the clock. Moreover, in highly populated countries like India, this technology helps in the rapid bedside assessment of COVID-19 patients with less fatigue and manpower. In their study, Gundersen et al.[17] showed that after adequate training, nurses can perform high-quality ultrasonography with good agreement with experts. Similarly, after adequate training, medical students can acquire and interpret diagnostic imaging.[181920]

Our study has several strengths. Our study emphasised the reliability of AI in POCUS for measuring the LVOT-VTI, IVC-CI and B-lines in COVID-19 patients. Moreover, our study shows novices can be utilised for assessing these parameters using AI incorporated POCUS which may help treating physician at beside.

There are some major limitations associated with our study. First, our study cohort had less severe disease. We suggest that the study must be conducted in patients with more severe and critically ill COVID-19 patients who require ventilators. Second, we did not calculate the time duration required between the two methods. Another important limitation of our study was the exclusion of posterior lung segments for B-lines counting. We chose eight zone method instead of 12 zone method to make the scan easy, feasible and friendly for patients, expert sonographers and novice practitioners. Since COVID-19 predominantly affects posterior segments of lungs, exclusion of these segments would decrease the number of B-lines counting made, which could affect the results.

CONCLUSION

Our study showed AI guided assessments of LVOT-VTI, IVC-CI and B-lines counting are reliable and consistent with manual assessments in COVID-19 patients. Novices can reliably estimate LVOT-VTI and IVC-CI but not B-lines counting using AI software in COVID-19 patients.

Declaration of consent

The authors certify that they have obtained all appropriate patient consent forms. In the form the patient(s) has/have given his/her/their consent for his/her/their images and other clinical information to be reported in the journal. The patients understand that their names and initials will not be published and due efforts will be made to conceal their identity, but anonymity cannot be guaranteed. Also, study investigators were not having any direct administrative control or supervision over the hospital employees who are engaged as participants in this current research project.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.