Assessing left ventricular systolic function by emergency physician using point of care echocardiography compared to expert: systematic review and meta-analysis.

Assessing left ventricular systolic function (LVSF) by echocardiography assists in the diagnosis and management of a diverse range of patients presenting to the emergency department (ED). We evaluated the agreement between ED-based clinician sonographers and apriori-defined expert sonographers. We conducted a systematic review and meta-analysis based on Preferred Reporting Items for Systematic reviews and Meta-Analysis guidelines. We searched Medline, EMBASE, Cochrane, ClinicalTrials.gov, TRIP and Google Scholar for eligible studies from inception to February 2021. Risk of bias was evaluated using Quality Assessment Tool for Diagnostic Accuracy Studies-2 tool. The level of agreement between clinician and expert sonographers was measured using kappa, sensitivity, specificity, positive and negative likelihood ratio statistics using random-effects models. Twelve studies were included (1131 patients, 1229 scans and 159 clinician sonographers). Significant heterogeneity was identified in patient selection, methods of assessment of LVSF, reference standards and statistical methods for assessing agreement. The overall quality of studies was low, with most being small, single centre convenience samples. A meta-analysis including seven studies (786 scans) where visual estimation method was used by clinician sonographers demonstrated simple Kappa of 0.68 [95% confidence interval (CI), 0.57-0.79], and sensitivity, specificity, positive and negative likelihood ratio of 89% (95% CI, 80-94%), 85% (95% CI, 80-89%), 5.98 (95% CI, 4.13-8.68) and 0.13 (95% CI, 0.06-0.24), respectively, between clinician sonographer and expert sonographer for normal/abnormal LVSF. The weighted kappa for five studies (429 scans) was 0.70 (95% CI, 0.61-0.80) for normal/reduced/severely reduced LVSF. There is substantial agreement between ED-based clinician sonographers and expert sonographers for assessing LVSF using visual estimation and ranking it as normal/reduced, or normal/reduced/severely reduced, in patients presenting to ED.

Introduction

Point-of-care (POC) echocardiography is widely used in the evaluation and resuscitation of patients in the emergency department (ED) [1,2]. Acute breathlessness, cardiac arrest and shock are common presentations to the ED where POC echocardiography performed by clinicians provides rapid and focused findings to assist in the initial assessment. Data suggest that emergency physicians’ clinical assessment fails to accurately estimate the patient’s cardiac function and hemodynamic status [3], potentially leading to incorrect diagnosis and treatment with associated increases in length of stay and mortality [4,5]. A recent randomised control trial showed that using point-of-care ultrasound (POCUS) did not improve outcomes for patients presenting the ED with undifferentiated shock [6]. However, this trial excluded patients with conditions that had evidence of rapid diagnosis with POCUS, the interventions in both groups were similar and the trial evaluated a diagnostic test as therapy. Studies, including one systematic view and meta-analysis, have shown that rapid assessment with POC echocardiography early in the patient’s ED journey reduces time to diagnosis, improves diagnostic accuracy and changes treatment potentially improving outcomes in critical illness, shock and acute heart failure [7-16]. Consequently, there is an increasing interest in emergency physicians performing POC echocardiography with more detailed studies performed urgently on selected patients or following admission by an expert.

Assessment of left ventricular systolic function (LVSF) is one of the most common indications for echocardiography [17]. In 2013, the American Society of Echocardiography published a recommendation to consider POC echocardiography as a bedside adjunct to physical examination to assist in diagnosis and guide treatment [18]. The American Society of Echocardiography and American College of Emergency Physicians 2010 consensus statement for use of POC echocardiography in the ED stated that POC echocardiography had a role in the assessment and diagnosis of pericardial effusion, right ventricle dilatation, intravascular volume status and left ventricular performance [1]. The latter was suggested as visually assessed and categorised as normal (ejection fraction >50%), reduced (ejection fraction 30–50%), or severely reduced (ejection fraction <30%). In 2014, the American College of Emergency Medicine published a revised policy statement [19] stating that the assessment of LVSF is one of three primary indications for POC echocardiography. In 2014, the International Evidence-Based Recommendations for Focused Cardiac Ultrasound stated that POC echocardiography can accurately assess left ventricle global function, narrow differential diagnosis and improve outcome in the setting of shock and is superior to physical examination alone [20].

The ability of emergency physicians to evaluate LVSF has been previously assessed in observational studies and one narrative review [21]. The authors were unable to identify any previous systematic review and meta-analysis. The aim of this review was to assess the level of agreement between emergency department-based clinician sonographers as compared to expert sonographers for the assessment of LVSF using POC echocardiography. The PICO question is depicted in Table 1.

Table 1

Review question details

Review question (PICO)
Population	Adult patients (18 years or over) in the emergency department (ED), no limits on presenting complaint or physiology
Intervention	POC echocardiography performed in ED by clinician sonographer working in the ED as emergency physician who had limited ultrasound training defined as: Level 1 (core training) or level 2 (advanced training). Clinicians who had completed a defined local training program instituted for study. AND who do not meet the criteria below for expert sonographers.
Comparison	Echocardiogram performed or reported by expert sonographer defined as: Graduated sonographer. Graduated board-certified cardiologist. Clinician with completed fellowships in echocardiography (the training standards are defined by country of training and not standardised worldwide, so we did not define experts by hours in training or number of scans performed).
Outcome	Level of agreement between the clinician sonographer and the expert sonographer for the assessment of left ventricular systolic function (LVSF) where clinician sonographer uses one of the following three methods: Visual Estimation. E-Point Septal Separation (EPSS). Velocity Time Integral (VTI).The method for expert sonographers to assess LVSF was not defined a priori.

Methods

This systematic review was designed in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analysis statement and was registered on the International Prospective Register of Systematic Reviews (CRD42020179209). All authors reviewed the article and T.H. acts as a guarantor.

Data sources and search strategy

The medical literature was searched using PubMed (MEDLINE), OvidSP (EMBASE), Cochrane library and ClinicalTrials.gov. The grey literature was searched using Google Scholar and TRIP database. A combination of Medical Subject Headings terms and search terms was used and are listed in Appendix 1 in Supplemental Digital Content 1, http://links.lww.com/EJEM/A318. In addition to the database search, the bibliographies of the selected articles were reviewed for further references. Supplementary searching was performed using snowballing and hand searching of key medical journals. The publication period was defined by the database searched. The literature search was performed independently by a medical librarian, B.A. and M.H. in April 2020. A repeat search was performed in February 2021.

Study selection

The search was restricted to studies on humans and published in English. Studies that recruited adult patients where POC echocardiography was performed in the ED were included, regardless of physiological parameters, diagnoses or presenting complaints. Clinician sonographer was defined by the study authors as a trainee or specialist in emergency medicine with level 1 (core) or level 2 (advanced training or local training). A priori studies that focused on patients outside the ED were excluded (i.e. prehospital, ICU or ward) as well as studies using sonographers other than emergency physicians (e.g. intensivist or medical student). We defined expert sonographers as sonographers who had completed a training program, cardiologists who had completed training or ultrasound fellowship-trained clinicians who had completed a fellowship in echocardiography. We did not assess the experience of expert sonographers as this varies between countries and was not described in detail in any article under review.

We included studies that assessed LVSF using transthoracic echocardiography by visual estimation, E-point septal separation (EPSS), or an assessment of stroke volume/cardiac output using velocity time integral (VTI). We excluded studies where the clinician sonographer used fractional shortening to assess left ventricle ejection fraction (LVEF), reflecting its inherent inaccuracies. We imposed no restrictions on the method used by expert sonographers or machine used by either clinician or expert sonographer. We included studies performed using hand-held and free-standing ultrasound machines and any transducer type but excluded studies using transoesophageal echo. We included studies that used the review of recorded videoclips or stored images by expert sonographers as the reference standard and those where the expert sonographer’s echocardiogram was performed using either the same or a different machine to the clinician sonographer. Where there was a lack of clarity in the article, we wrote to the corresponding authors.

Three reviewers (T.H., B.A. and M.H.) worked independently in two groups to review all eligible titles and abstracts using inclusion criteria defined a priori. We included studies regardless of their designs. Randomised trials and observational studies (prospective and retrospective cohort studies) were included. Conference abstracts with sufficient methodological description for quality assessment and data for analysis were also included. Case studies, case reports, guidelines, editorials, letters and review articles were excluded. The full text was then read, and consensus was achieved for inclusion. When consensus agreement could not be reached between the authors, an independent third-party adjudicated (B.J.).

Data extraction and quality assessment

Two reviewers (B.A. and M.H.) independently extracted data using a standardised form (Table 2). The revised Quality Assessment Tool for Diagnostic Accuracy Studies-2 (QUADAS-2) was used to evaluate the overall quality of the included studies [22-33]. The tool was applied by the two reviewers independently, and disagreements were resolved by a third reviewer (T.H.). The QUADAS-2 tool allows modification of the signalling questions to customise to the topic under review. Our signalling questions are described in Appendix 2 with details for the process of rating as low, high or unclear risk of bias in Supplemental Digital Content 1, http://links.lww.com/EJEM/A318.

Table 2

Summary of included studies

AuthorYearCountryDesignsTitle	Study characteristics	Clinician sonographer/training	Participant characteristics	Intervention and expert sonographer	Outcome (95% confidence interval)
Moore et al. [22]2002USAPOSDetermination of Left Ventricular Function by Emergency Physician Echocardiography of Hypotensive Patients	Machine type: Index: Apogee CX-100 Floor-based US Machine Transducer PA Reference: Recorded images and clips Windows: PSLA, PSSA, A4C, A2C, SC. Method: Index: visual estimation Reference: visual estimation Categorization: (>50%) normal function, (30–50%) reduced function, (<30%) severe dysfunction	Clinician sonographer: four EPs Level of experience: highly experienced with various levels. Must have completed 10 h of basic US instruction and 100 documented non-cardiac ultrasound examinations prior to echocardiography training. Additional training:Theoretical: 6-h consist of video clips reviewPractical: 10-h observing and performing Echo Supervised by cardiologist.	Convenience sample, n = 51 ED adult patients Inclusion: symptomatic hypotension, systolic Blood pressure <100 mmHg for two times and 15 min apart, no traumaExclusion: Asymptomatic hypotension, CPR, ECG suggestive of myocardial infarction	Intervention: POC echocardiography by clinician sonographer, then the scan repeated immediately by another clinician sonographer in convenience sample n = 8 (blinded) Expert sonographer videoclips reviewed by 1# cardiologist then 2# cardiologist reviewed some scans in convenience sample n = 20 (blinded) Interval: N/A	Agreement of expert sonographer with clinician sonographer, n = 50 (1 patient excluded due to poor image quality):•kw = 0.61 (0.39–0.83)•Weighted percent agreement = 84%•Overall agreement = 68%•Pearson’s correlation, r = 0.86 (+-8%)•Modified Bland–Altman plot showed the difference falls within 1 SE of the estimate 96% of the patients.Correlation between the clinician sonographer, n = 8•Pearson’s correlation, r = 0.94.Interobserver reliability between the expert sonographer, n = 20•Pearson’s correlation, r = 0.84•κ = 0.7•Modified Bland–Altman plot showed agreement within 1 SE of the estimate in 95% of patients
Randazzo et al. [23]2003USAPOSAccuracy of Emergency Physician Assessment of Left Ventricular Ejection Fraction and central venous pressure Using Echocardiography	Machine type: Index: Hewlett-Packard Sonos 1000 Floor-based US MachineReference: Sonos 5500 Floor-based US Machine Transducer PA Windows: SC, PSSA, and A4C Method: Index: visual estimationReference: Not Described Categorization:(>55%) normal function, (30%–55%) reduced function, (<30%) severe dysfunction.	Clinician sonographer: 7 EPs, (three residents, 4 attendings) and 1 physician assistant. Level of experience: completed an emergency medicine US workshop (didactic and practical sessions, with a minimum of 150 examinations of non-echo scans) Additional training:Theoretical: 3 h session of series review of normal and abnormal.Practical: >5 scans.	Convenience sample, n = 115 ED adult patients Inclusion The formal echocardiogram should be performed within 4 h of the ED echocardiogram.Most reason for echo: chest pain 45.1%, CHF 38.1%, SOB 5.7%, suspected endocarditis 10.6% Exclusion: >4 h delay between echocardiograms, or if the investigator was aware of the result of previous Echo.	Intervention: POC echocardiography by clinician sonographer Expert sonographer: TTE by certified cardiac sonographer and interpreted by cardiologist Interval: <4 h. clinician sonographer first then expert sonographer	Categorical LVSF agreement for clinician sonographer with expert sonographer n = 115•κw = 0.71 (CI = 53%–89%)•Overall agreement = 78.3%Calculated LVSF correlation for clinician sonographer visual estimation with expert sonographer n = 102 (calculated ejection fraction was available for 102 patients. Thirteen cases had no formal ejection fraction reported on the final Echo report by cardiology)•Pearson’s Correlation, r2 = 0.72Raw agreement for each category of LVSF•Normal 92.3%•Poor 70.4%•Moderate 47.8%•Overall agreement = 86.1%
Secko et al. [24]2011USAPOSCan Junior Emergency Physicians Use E-Point Septal Separation to Accurately Estimate Left Ventricular Function in Acutely Dyspneic Patients?	Machine type:Index: Philips HD11XE Floor-based US machineTransducer PA. Reference: Recorded images and clips.Windows: SC, PSSA, PSLA, A4C. Method: Index: EPSS Reference: visual estimationCategorizations: normal or low	Clinician sonographer: 12 emergency medicine residents (PGY3 and PGY4) Level of experience: 70–150 total ED US scans. And <25 Echo and had been instructed on EPSS through informal teaching during their day-long Introduction to emergency ultrasound course. Additional training: No	Convenience sample, n = 70 ED adult patients Inclusion: dyspneoa, ED length of stay <2 h, no trauma, and normal mental status. Exclusion: known history of mitral valve repair or replacement, aortic insufficiency, or mitral stenosis.	Intervention: POC echocardiography by clinician sonographer Expert sonographer: videoclips. All clips reviewed by 2 EP US fellows at bedside but were not blinded to the EPSS measurements by clinician sonographer.Half of the clips reviewed by two cardiologists.Both used visual estimation methodInterval: N/A	Correlation between measurements of EPSS by resident EPs and visual estimations of LVSF by the EP US fellows, n = 58:•Spearman’s correlation, ρ = −0.844The agreement between EP US fellow and cardiologists for visual estimation (expert to expert), n = 58:•κ = 0.75 (0.48–1.00)
Weekes et al. [25]2011USAPOSComparison of Serial Qualitative and Quantitative Assessments of Caval Index and Left Ventricular Systolic Function During Early Fluid Resuscitation of Hypotensive Emergency Department Patients	Machine type:Index: Philips CX50 Floor-based US machineTransducer PAReference: Recorded images and clips Windows: Not Described Method: Index: visual estimation and fractional shorteningReference: Not DescribedCategorization:increased function, normal function, reduced function, or severe dysfunction	Clinician sonographer: EPs number not Described Level of experience: not well described Emergency US division physicians Additional training: no standardized training. training on EPSS and the fractional shortening methods and on the details of image acquisition, technique, details of scoring criteria, and the sequence of measurements to standardize the protocol.	Convenience sample, n = 24 ED adult patients Inclusion: symptomatic hypotension systolic Blood pressure ≤100 mm Hg, sinus rhythm, the EP had intent to administer IV fluid challenges.Exclusion: unstable arrhythmia or CPR, suspected CHF, inability to obtain adequate quality US images, pregnancy, trauma, inability to tolerate the positioning, predictable ED stay of <45min.	Intervention: POC echocardiography by clinician sonographer Expert sonographer: videoclips. US EP fellowshipInterval: N/A	Agreement between clinician sonographer and expert sonographer scan for visual estimation, n = 24 patients (72 scans):•κ = 0.46 (0.29–0.63)•κw = 0.57 (0.41–0.72)•Overall agreement = 65.3%US scoring and measurements were performed at three defined points per enrolment. Time 0 was defined as the start of the fluid challenges. Time 1 was defined as immediately after the first fluid challenge. Time 2 was defined as 15–20 minutes after Time 1. Seventy-two videos of LVSF and caval index were obtained.
Dinh et al. [26]2012USAPOSMeasuring cardiac index with a focused cardiac ultrasound examination in the ED	Machine type:Index: Z. One Ultra Floor-based US machineReference:1.Sonographer: same machine.2.Cardiologist: stored images.Transducer PAWindows: PSLA, A4C Method: Index and Reference: cardiac index by VTI and LVOT diameter and HR.Categorization: normal Cardiac index was defined as 2.5–4.0 L/min/m2Rating for the images by cardiologist: Optimal, suboptimal, and unobtainable images	Clinician sonographer: two EPs Level of experience: Not Described Additional training: Practical: 20 h by a certified sonographer of acquiring adequate images with PSLA, A4C.	Convenience sample, n = 100 ED adult patients Inclusion: 5 days per month when the study team was available, no described indication for the Echo, cardiac indication in (30%) of sample. Exclusion: congenital heart disease, aortic valvular disease, inability to lie supine or inability to lie in the left lateral decubitus during the scan.	Intervention: POC echocardiography by clinician sonographer. Expert sonographer:Echo by sonographer (expert sonographer #1) for 23/100 patientsAll US images by the expert sonographer and clinician sonographer were stored and reviewed by a cardiologist (expert sonographer #2) Interval: clinician sonographer first then expert sonographer <10 min.	Agreement between clinician sonographer and sonographer (expert sonographer #1) (n = 23):Agreement for VTI:•κ = 0.40 (0.04–0.76)•Pearson’s correlation, r = 0.87 (0.70–0.94)•Bland-Altman −0.17 (−4.08 to 18.01)Agreement for cardiac index:•κ = 0.56 (0.23–0.90)•Pearson’s correlation, r = 0.82 (0.60–0.92)•Bland–Altman −0.11 (−1.06 to 0.83)Cardiologist (expert sonographer #2) rating of cardiac US images for Optimal VTI measurements with:•clinician sonographer: Overall agreement = 78.4% of subjects (n = 97)•Sonographer: Overall agreement = 78.3% of subjects (n = 23)
Bustam et al. [27]2014MalaysiaPOSPerformance of emergency physicians in point-of-care Echocardiography following limited training	Machine type: Index and reference: Logiq-e portable on a trolley USTransducer PA Windows: PSLA, PSSA, A4C, SC Method: Index and Reference: visual estimation, Teichholz method Categorization:(>50%) normal function, (30–50%) reduced function, (<30%) severe dysfunction	Clinician sonographer: nine emergency medicine Residents (PGY1–PGY2) Level of experience: novice with the least experience in Echo, all had < 10 informal Echo Additional training:Theoretical: web-based learning which included images and videos. Practical: 3 h delivered at the bedside by a cardiologist.	Convenience sample, n = 50 ED adult patients Inclusion: random sample. no described indication for the scan. Stable. Exclusion: if they were unstable (not described) or were still receiving acute treatment.	Intervention: POC echocardiography by clinician sonographerExpert sonographer: Echo by cardiologist.Interval: expert sonographer first then clinician sonographer, directly.	Agreement between clinician sonographer and expert sonographer for visual estimation, n = 100 scan:•κ = 0.79 (0.773–0.842)•Overall agreement of 93% of scansAgreement between clinician sonographer and expert sonographer for quantitative LVSF measurements, n = 92 scan:•Pearson’s correlation, r = 0.82 (0.734-0.881).•Bland–Altman: 2.1 (−9.5% to 13.7%)Agreement between clinician sonographer quantitative LVSF and expert sonographer visual estimation n = 84•κ = 0.80 (0.636–0.882)•Overall agreement of 92.9% of scans
McKaigney et al. [28]2014Canada/USAPOSE-point septal separation: a bedside tool for emergency physician assessment of left ventricular ejection fraction	Machine type: Index: Logic P6 Floor-based US Reference: Phillips iE33 Floor-based USTransducer PAWindows: SC, PSSA, PSLA, A4C. Method: Index: visual estimation, EPSS. Reference: Teichholz methodCategorization:(>55%) Normal function, (30%–55%) reduced function, (<30%) severe dysfunction.	Clinician sonographer: 3 EPs (EP US fellows – 7 months into a 1-year fellowship). Level of experience: 100 bedside echocardiograms. And had training in both visual and calculated LVSF estimation as part of their fellowship Additional training:Theoretical: 10-min presentation on EPSSPractical: three separates supervised EPSS measurements with the principal investigator.	Convenience sample, n = 80 ED adult patients7 (8.8%) subjects were enrolled in the ED, 11 (13.8%) in ICU, and 62 (77.5%) in ward. Inclusion: treating physician had ordered TTE. Exclusion: pregnancy.	Intervention: POC echocardiography by clinician sonographer in the ED, ICU, inpatient ward. Expert sonographer: TTE by certified cardiac sonographer and interpreted by cardiologist Interval: expert sonographer then clinician sonographer <24 h.n = 23 patients were randomly selected to have a second bedside ultrasound study performed independently byanother clinician sonographer, blinded to the results of the first examination.	Agreement between clinician sonographer visual estimation and the calculated LVSF in TTE (expert sonographer)•Overall agreement 69%•κw = 0.58.•for subjects with severe systolic dysfunction (n=8) overall agreement 100%Correlation of EPSS by clinician sonographer to the calculated LVSF by expert sonographer:•Pearson’s correlation, r = 0.73EPSS measurement of >7 mm for LVSF ≤30% n = 71•Sensitivity = 100.0% (62.9–100.0)•Specificity = 51.6% (38.6–64.5)•Positive LR = 2.07•Negative LR = 0.00EPSS measurement of >8 mm for LVSF <55%, n = 71•Sensitivity 83.3% (62.6–95.2)•Specificity 50.0% (29.2–70.9)•Positive LR = 1.67•Negative LR = 0.33Interobserver correlation for EPSS measurements, n = 23•Spearman’s correlation, ρ = 0.87Interobserver agreement for visual estimation, n = 23•κw = 0.49
Ünlüer et al. [29]2014TurkeyPOSVisual Estimation of Bedside Echocardiographic Ejection Fraction by Emergency Physicians	Machine type: Index and reference: M7R model ultrasound portable on a trolley US, Transducer PA Windows: PSLA Method: Index: visual estimationReference: SBT Categorization: normal or low threshold of 55%	Clinician sonographer: two EPs attending with 1-year experience. Level of experience: certified on focused abdominal sonography for trauma by Emergency Radiology Association in Turkey. Additional training: 3 h of didactic training; performed in the presence of an experienced echocardiographer.	Convenience sample, n = 133 ED adult patients Inclusion: dyspneoa. Exclusion: intubated, elevated cardiac biomarkers, pregnancy, atrial fibrillation, valvular pathology or surgery, if technical limitations.	Intervention: POC echocardiography by clinician sonographer Expert sonographer: Formal Echo by experienced cardiologist Interval: clinician sonographer first then expert sonographer within 2 h	Correlation between clinician sonographer to expert sonographer:•Pearson’s correlation, r = 0.77 for clinician sonographer #1 and 0.78 for clinician sonographer #2•Spearman’s correlation, ρ = 0.87 (0.82–0.90) clinician sonographer #1 and 0.88 (0.84–0.91) clinician sonographer #2•Intra-class correlation coefficient of both clinician sonographer = 0.952 (0.934, 0.966)Clinician sonographer POC echocardiography scans (cardiologist reports as reference standard):•Sensitivity 98.7%•Specificity 86% (86.2–87.9%)•Positive PV = 0.902•Negative PV = 0.980•Negative LR = 0.015•Positive LR = 7.153
Farsi et al. [30]2017IranPOSFocused cardiac ultrasound (FOCUS) by emergency medicine residents in patients with suspected cardiovascular diseases	Machine type: Index: SonoAce X8, Samsung Medison Co, Floor-based US Reference: EKO 7 device, Samsung Madison Co, Floor-based US Transducer PA Windows: PSLA, PSSA, A4C, SC Method: Index: EPSS or Quinones equation, or both Reference: visual estimation Categorization: (≥50%) normal function, (30–49%) reduced function, (<30%) severe dysfunction	Clinician sonographer: 17 emergency medicine Residents Level of experience: Not Described Additional training: Theoretical: 2 h for movies and pictures Practical: 2 h.Workshop designed by the American Society of Echocardiography	Convenience sample, n = 205 ED adult patient Inclusion: cardiac emergency requiring POC echocardiography (chest pain 45%, SOB 33%, syncope 10%) Exclusion: if interfere with treatment.	Intervention: POC echocardiography by clinician sonographer Expert sonographer: cardiologist. Interval: clinician sonographer first then expert sonographer, as soon as possible.	The agreement between clinician sonographer and expert sonographer, n = 205:•κ = 0.85 (0.79–0.91)•Overall agreement = 91%Clinician sonographer POC echocardiography scans (cardiologist reports as reference standard):•Sensitivity 89% (81–99)•Specificity 96% (90–99)•Postive PV 96% (89–99)•Negative PV 89% (81–99)•Accuracy 92% (85–99)•Negative LR 0.12 (0.07–0.2)•Positive LR 22 (8–58)For wall motion abnormalityκ = 0.83 (0.76–0.90)
Balderston et al. [31]2019USAROSDiagnostic Yield and Accuracy of Bedside Echocardiography in the Emergency Department in Hemodynamically Stable Patients	Machine type: Index: Fujifilm Sonosite X Porte US machine, Floor-based USReference: not describedTransducer PA Windows: Not Described Method: Index: visual estimationReference: SBTCategorization: (≥50%) normal function, (30-49%) reduced function, (<30%) severe dysfunctionFor the analysis: normal ≥50% and reduced <50%	Clinician sonographer: 90 EPs from all grades (40 residents, fellows, 50 attendings). Level of experience: US certificates in accordance with American College of Emergency Physicians guidelines.Additional training: no	Consecutive sample, n = 276 ED adult Inclusion: stable patients, had TTE < 2 days of POC echocardiography indications were mainly for chest pain or SOB, or both and syncope and lower limps oedema. Exclusion: unstable systolic Blood pressure <90 mm Hg, need for positive pressure ventilation, or cardiac arrest, Paediatric.	Intervention: POC echocardiography by clinician sonographer Expert sonographer: TTE by certified cardiac sonographer and interpreted by cardiologist Interval: clinician sonographer first then expert sonographer, <2 days	Clinician sonographer to expert sonographer for the reduced LVSF and pericardial effusion, n = 270•Overall agreement of 77%•κ = 0.53Clinician sonographer POC echocardiography scan (cardiologist reports as reference standard), n = 224:•Overall agreement 78.1% Sensitivity 80% (66.6–89.1)•Specificity 77.5% (70.3–83.4)•Positive PV 53.7% (42.4–64.6)•Negative PV 92.3% (86.2–95.9)•Negative LR 0.26 (0.15–0.44)•Positive LR 3.65 (2.61–4.85)
Monsomboon et al. [32] 2019Thailand Clinician sonographer SAgreement between Emergency Physicians and a Cardiologist on Cardiac Function Evaluation after Short Training	Machine type: Index: Philips HD15 PureWave US, Floor-based USTransducer PA Reference: Recorded images and clips Windows: PSLA, PSSA, A4C, A5C, SC Method: Index: visual estimation Reference: Not described Categorization:(>55%) Normal function, (30%–55%) reduced function, (<30%) severe dysfunction.	Clinician sonographer: seven emergency medicine Residents (PGY2 - PGY3) Level of experience: The residents had been working in the ED for less than 3 years. They had not participated in any formal US training courses because emergency ultrasound was not part of the emergency medicine curriculum.Additional training: Theoretical: 2 h lecturePractical: 1-h hands-on workshop conducted by cardiologist.	Consecutive sample n = 101 ED adult Inclusion: patient needed a volume status or LV function assessment 1) shock (systolic blood pressure < 90 mmHg or diastolic blood pressure < 60 mmHg); and 2) suspected heart failure, defined as having at least 1 of the Framingham criteria.Exclusion: uncooperative. STEMI. Intubated. Patients for whom POC echocardiography might delay their standard treatment.	Intervention: POC echocardiography by clinician sonographer Expert sonographer: videoclips review by one cardiologist Interval: N/AAfter the course, each resident had to practice on at least 1 patient in the ED. All recorded images and videos were subsequently sent to the cardiologist to assess the residents’ ability to perform POC echocardiography. The cardiologist had to approve each participant’s performance before he or she was permitted to participate	Agreement between clinician sonographer visual estimation and expert sonographer review, n = 92.•Overall agreement = 79.4%•κw = 0.73 (0.58–0.89)The agreement between clinician sonographer and expert sonographer on the pericardial effusion diagnoses = 100%.
Lafon et al. [32]2019FranceComparative Early Hemodynamic Profiles in Patients Presenting to the Emergency Department with Septic and Nonseptic Acute Circulatory Failure Using Focused Echocardiography	Machine type: Index: CX 50, Philips HealthcareTransducer PA Reference: CX 50, Philips HealthcareWindows: PSLA, PSSA, A4C, SC Method: Index: visual estimation Reference: visual estimation Categorization:(30%–50%) reduced function (<30%) severe dysfunction.	Clinician sonographer: five EPsLevel of experience: no previous experience in USAdditional training:validated curriculum was fully conducted and supervised by two experienced intensivists who were board-certifiedin echocardiography (level 3 competence)Theoretical: total 9-h (6-h of didactics, 3-h of interactive clinical cases)Practical: 3-h of hands-on	Consecutive sample n = 114 ED adultInclusion: Acute respiratory failure, hypotension (systolic blood pressure <90 mmHg or mean blood pressure <65 mmHg), altered tissue perfusion (skin: cold, clammy, mottling; kidneys: urine output <0.5 mL/kg/h; brain: altered mental status), and cellular dysoxia (lactate >2 mmol/L).Exclusion: moribund status or pregnancy.	Intervention: POC echocardiography by clinician sonographer Expert sonographer: expert in critical care echocardiographyInterval: <1 h, in random order, depending on the availability of investigators.	Agreement between clinician sonographer and expert sonographer for: n = 100LV failure κ = 0.74 (0.59–0.89)Hyperdynamic LV associated with vasoplegia κ = 0.61 (0.43–0.78)

A2C; apical 2 chambers, A4C; apical 4 chambers, A5C; apical 5 chambers, CI; confidence interval, ED; emergency department, EM; emergency medicine, EP; emergency physician, EPSS; E-point Septal Separation,; LR; likelihood ratio, LV; left ventricle, LVOT: left ventricle outflow tract, LVSF; left ventricle systolic function, PA; phased array transducer, POC; point-of-care, PGY; post-graduation year, POS; prospective observational study, PSLA; parasternal long axis, PSSA; parasternal short axis, PV; predictive value, ROS; Retrospective observational study, SBT; Simpson’s biplanar technique, SC: subcostal, SOB; Shortness of breath, STEMI; ST-segment elevation Myocardial infarction, TTE; transthoracic echocardiography, US; ultrasound, VTI; velocity time integral.

Meta-analysis

To determine the level of agreement between the clinician sonographer and expert sonographer for LVSF assessment a meta-analysis was performed using four effect measures of interest. Cohen/Conger’s Kappa statistics (simple and weighted) [34-36], sensitivity, specificity, positive and negative likelihood ratio with 95% confidence interval (CI) were independently pooled. Forest plots were used to summarise and compare the included studies. Heterogeneity was quantified using Higgin’s I2 statistics; any value of I2 >50% was indicative of significant heterogeneity. To account for heterogeneity random effect models were used to pool the study effect sizes. The presence of publication bias was assessed using Deeks’ funnel plot [37]. The meta-analysis was conducted on each method of assessment of LVSF when there was enough data provided (visual estimation, EPSS, VTI). Information obtained from the quality assessment tool was used for bias assessment and subgroup analysis. Analysis of subgroups was performed to explore the potential sources of heterogeneity in the agreement. The following subgroups were evaluated: level of experience of clinician sonographer, training on POC echocardiography for clinician sonographer prior study enrolment and year of publication (before and after 2010). Analyses were conducted using kappaetci, meta and Midas packages in Stata software (StataCorp, 2019, Stata Statistical Software: Release 16, StataCorp LLC, College Station, Texas, USA).

Results

A total of 1673 studies were identified and screened, and 25 full articles were eligible for review (Fig. 1). Thirteen studies that did not meet the inclusion criteria were excluded. Consequently, 12 studies [22-33] published between 2002 and 2020 (1229 scans on 1131 patients) were included in this systematic review. Eleven studies enrolled 159 clinician sonographers (88 emergency medicine residents/fellows, 70 emergency medicine attendings and one study enrolled one physician assistant [23]). One study did not specify the number of clinician sonographers [25]. An attempt was made to contact all authors to clarify aspects of the included studies; four replies were received [22,24,27,31].

Fig. 1

PRISMA flow diagram for the selection process for the systematic review studies. PRISMA, Preferred Reporting Items for Systematic reviews and Meta-Analysis statement.

Characteristics of the included studies

Overall, 12 studies were included in this review (Table 2). Eleven studies [22-30,32,33] were prospective observational cohort design, and one study [31] was a retrospective chart review. All studies were single centre, included only patients presenting to hospitals via the ED, and used floor-based ultrasound machines with a phased array transducer. In all studies, POC echocardiography was performed solely in the ED except one study which included POC echocardiography performed in ED, ICU and wards [28]. The inclusion criteria for patients varied considerably, two studies included only patients with hypotension [22,25], two recruited patients with dyspnoea [24,29], five studies recruited patients who required inpatient echocardiography for any reason [23,26-28,31], one study included any patient with suspected cardiac disease [30], one study included patient with acute circulatory failure [33] and one study included patients who required volume status or LVSF assessment due to hypotension or suspected heart failure [32].

Visual estimation of left ventricular systolic function

Visual estimation of volume changes between diastole and systole using ranked categories was the most commonly used method by clinician sonographers to assess LVSF (nine studies) [22,23,25,27-29,31-33]. Cardiac function was categorised as normal (ejection fraction ≥ 50%), reduced (ejection fraction 30–49%) or severely reduced (ejection fraction <30%) function in four studies [22,27,31,33], or as normal (ejection fraction >55%), reduced (ejection fraction 30–55%) or severely reduced (ejection fraction <30%) function in three studies [23,28,32]. One study did not use numerical cutoff values [25]. Two studies also included the category hyperdynamic function in addition to the three previously mentioned categories [25,33]. LVSF was ranked simply as normal or abnormal in two studies with cutoff ejection fraction value of 50% [31], or 55% [29]. Both the clinician sonographer and expert sonographer used visual estimation in three studies [22,27,33], in two studies clinician sonographers used visual estimation and the expert sonographer measured LVSF using Simpson’s Biplanar Technique (SBT) [29,31]. Three studies did not describe the expert sonographer method of LVSF estimation [23,25,32].

There was heterogeneity in the methods used to assess agreement between clinician sonographer and expert sonographer for visual estimation of LVSF (Table 2). Eight studies [22,23,25,27,28,31-33] reported a simple or weighted Cohan’s Kappa (0.46–0.79), one reported Pearson’s correlation [29] (for the two recruited emergency physicians values were 0.77 and 0.78). Five studies [22,23,27,28,31] reported raw/overall agreement, two reported agreement using Bland–Altman [22,27] and four calculated specificity and sensitivity for LVSF as normal or abnormal [28-31]. Overall, there was a high agreement in identifying LVSF as normal and with severe dysfunction. However, the agreement was moderate in identifying moderate dysfunction.

E-point septal separation

The second most frequently used method to assess LVSF was EPSS (three studies including 355 patients) [24,28,30]. No studies compared EPSS by both clinician sonographer and expert sonographer. Two studies compared EPSS by clinician sonographer with visual estimation by expert sonographer [24,30], one reported a Kappa (κ = 0.85) and the other reported a Spearman’s correlation (−0.84). One study [28] compared the EPSS by clinician sonographer to ejection fraction assessed using the Teichholz method by an expert sonographer, reporting with Pearson’s correlation of 0.73.

Velocity time integral

Cardiac index calculation using VTI and Left Ventricular Outflow Tract (LVOT) diameter was assessed in only one study [26], which reported the moderate agreement (κ = 0.40).

Image acquisition

There were some variations in the methods used to obtain images. In five studies [23,26,29-31], the clinician sonographer performed POC echocardiography prior to the expert sonographer while in two the expert sonographer scanned first [27,28]. In one study, the POC echocardiography was performed based on investigator availability [33]. In five studies [22,24-26,32], the expert sonographer reviewed video images taken by the clinician sonographer; in three studies [23,28,31], the cardiologist reviewed videos recorded by a sonographer and in three studies [27,29,30] expert sonographers (cardiologists) performed their own echocardiograms. Expert sonographer used the same ultrasound machines as clinician sonographer in four studies [26,27,29,33] and different machines in four [23,28,30,31]. The expert sonographer were emergency physicians in two studies [24,25], cardiologists/sonographers in 10 [22-24,26-32] (one study combined emergency physician and cardiologists as expert sonographer [24]) and intensivists expert in echocardiography in one [33]. In 11 studies [22,23,25-33], the second sonographer was blind to the findings of the first, while in one study, the expert sonographers were not blind to EPSS measures by clinician sonographers [24]. In four studies, there was a time difference of greater than 1 h between clinician sonographer and expert sonographer scans [23,28,29,31].

Quality assessment of studies

The QUADAS-2 quality assessment is depicted in Table 3. Studies were rated as low (), high () or uncertain (?) risk of bias. All studies scored at least one high-risk domain and seven studies scored two or more. Thus, the quality of the included studies was low, and all studies were assessed as high risk of bias. All studies excluded a proportion of their acquired POC echocardiography data, most commonly as a consequence of poor image quality and were consequently considered high risk (total enrolled sample 1309 patients and analysis sample 1131 patients). Eleven were rated as high risk of bias in patient selection due to convenience sampling or due to exclusion criteria, or both (body habitus or technical difficulties obtaining images) [22-30,32,33]. The protocol for the POC echocardiography performed by clinician sonographers varied widely between the studies with differing windows, machines, and transducer frequencies. Seven studies included low numbers of clinician sonographers selected from a larger clinician group with unclear reasons for clinician sonographer selection [22,23,26,28,29,32,33]. Five studies involved time intervals between POC echocardiography performed by clinician sonographers and the echocardiogram performed by expert sonographers, reflecting high risk of bias in flow and timing [28-31,33].

Table 3.

Quality assessment using QUADAS-2 of the included studies: ; high risk of bias, ; low risk of bias,?; unclear

	Bias				Applicability
Author	Patient selection	Index test	Reference standard	Flow and timing	Patient selection	Index test	Reference standard
Moore et al. [22]
Randazzo et al. [26]
Secko et al. [27]
Weekes et al. [28]		?	?				?
Dinh et al. [29]
Bustam et al. [30]
McKaigney et al. [31]
Ünlüer et al. [32]
Farsi et al. [33]				?
Balderston et al. [23]
Monsomboon et al. [24]
Lafon et al. [25]
Percentage %	Summary of QUADAS-2 results
	Bias				Applicability
	Patient selection	Index test	Reference standard	Flow and timing	Patient selection	Index test	Reference standard
	8.5%	33.5%	91.5%	50%	91.5%	91.5%	91.5%
	91.5%	58%	0%	41.5%	8.5%	8.5%	0%
?	0%	8.5%	8.5%	8.5%	0%	0%	8.5%

QUADAS-2, Quality Assessment Tool for Diagnostic Accuracy Studies-2.

In seven studies (688 patients, 786 scans, >119 clinician sonographers; 60 attendings, and 59 emergency medicine residents/fellows) [22,23,25,27,29,31,32], clinician sonographers used the visual estimation method to assess LVSF and reported adequate data to conduct the meta-analyses. The primary data were extracted from the original articles for statistical analysis (Supplementary Table 5 in Supplemental Digital Content 1, http://links.lww.com/EJEM/A318). The remaining five studies [24,26,28,30,33] had inadequate data for meta-analysis and we were unable to obtain sufficient data from the authors. There were insufficient studies to conduct a meta-analysis for EPSS or VTI assessment of LVSF. Weighted kappa was calculated in five studies [22,23,25,27,32] where the LVSF was categorised as normal, reduced, or severely reduced. In one study [25] where LVSF ranked as hyperdynamic, we include this as normal function for analysis. In two studies [29,31] where LVSF was categorised as normal or abnormal, simple kappa was calculated from primary data. Then, an overall simple kappa was calculated for the seven studies [22,23,25,27,29,31,32] by combining the reduced and severe reduced function as abnormal. Combined data were then used to calculate sensitivity and specificity to identify normal or abnormal LVSF. A ‘positive’ finding is defined as abnormal LVSF and a ‘negative’ finding is defined as normal LVSF.

The meta-analysis of the seven studies [22,23,25,27,29,31,32] yielded simple Kappa, sensitivity, specificity, positive and negative likelihood ratio of 0.68 (95% CI, 0.57–0.79), 89% (95% CI, 80%–94%), 85% (95% CI, 80%–89%), 5.98 (95% CI, 4.13–8.68), and 0.13 (95% CI, 0.06–0.24), respectively. The weighted kappa for five studies [22,23,25,27,32] was 0.70 (95% CI, 0.61–0.80). The forest plots of the previous results are shown in Figs. 2 and 3, and in Supplementary Figs. 4 and 5 in the Supplemental Digital Content 1, http://links.lww.com/EJEM/A318.

Fig. 2

Forest plot of agreement of POC echocardiography by clinician sonographer for the assessment of LVSF as normal/abnormal using Simple kappa statistics as compared to expert sonographer. LVSF, left ventricular systolic function; POC, point-of-care.

Fig. 3

Forest plot of agreement of POC echocardiography by clinician sonographer for the assessment of LVSF as normal, reduced or severely reduced using weighted Kappa statistics as compared to expert sonographer. LVSF, left ventricular systolic function; POC, point-of-care.

The subgroup meta-analysis regarding the year of publication before or after 2010, yielded simple Kappa of κ = 0.70 (95% CI, 0.55–0.84) for the studies published after 2010 [25,27,29,31,32] and κ = 0.64 (95% CI, 0.46–0.84) for those published before 2010 [22,23]. There was no evidence of publication bias with significant asymmetry in Deeks’ funnel plot of the seven studies (P = 0.95) (Supplementary Fig. 6 in Supplemental Digital Content 1, http://links.lww.com/EJEM/A318).

Discussion

The results of this systematic review suggest that ED-based clinician sonographers (56% trainees, 44% attendings, with or without study-specific training) are able to interpret visual estimation of LVSF with a substantial agreement as compared to expert sonographer, with a pooled κ = 0.68 (seven studies) and κw = 0.70 (five studies). Visual estimation of LVSF is a rapidly performed, widely acceptable method and in experienced hands may be superior to formal measurements for assessment of LVSF [38-41], but requires considerable experience to perform accurately. The sensitivity (true positive rate for detection of abnormal LVSF), specificity (true negative rate for detection of normal LVSF), positive and negative likelihood ratios in seven studies were 89%, 85%, 5.98, and 0.13, respectively, suggesting that clinician sonographer can correctly identify normal or abnormal LVSF sufficiently for clinical practice when compared to expert sonographer’s interpreted echocardiography as the reference standard.

For every 100 patients classified as having normal LVSF by expert sonographer, and clinician sonographer identified 85 patients. For every 100 patients identified as having abnormal LVSF by the expert sonographer, the clinician sonographer correctly identified 89 patients. Clinician sonographer classified 11% wrongly as abnormal and 15% wrongly as normal. A narrative review of the twelve included studies suggests that there is a high agreement in identifying LVSF as normal as compared to severely reduced function. However, the agreement was moderate in identifying mildly/moderately reduced LVSF function.

Other methods to assess LVEF (not considered in this review) such as mitral annular plane systolic excursion and tissue doppler imaging may offer more precise assessment and have been reported as having good interobserver reliability between untrained sonographers and experts [42-44].

Measuring EPSS is a much simpler and quicker method to assess LVSF than the SBT technique and is easily taught to novice sonographers [45]. However multiple factors may cause an over or underestimate in the LVSF measurement, such as valvular disease or septal hypertrophy. In this systematic review, EPSS was used in three studies [24,28,30]. The level of agreement and correlation were variably expressed, and no meta-analysis was possible. These three individual studies reported a moderate to a high level of agreement between clinician sonographers and expert sonographer.

VTI assesses the volume of blood ejected via the aortic outflow tract and may be used to calculate the stroke volume, cardiac index and guide fluid therapy [46]. VTI depends on ejection fraction, LV filling pressures and vascular resistance, and so offers different data to clinicians as compared to ejection fraction and EPSS (and other assessments of LV function). One study included in this systematic review assessed cardiac index using VTI and LVOT diameter by clinician sonographer and expert sonographer (κ = 0.40) [26].

SBT method is widely used by cardiologists, but this technique is time-consuming to perform in the resuscitation phase of care and includes A4C and A2C views, with the latter not widely taught to clinician sonographers. None of the included studies used this method by clinician sonographer for LVSF assessment.

A 2019 narrative review [21] described the assessment of LVSF by emergency physicians and general practitioners with varying POC echocardiography experience. The authors reported an overall raw agreement of 84–93% (10 studies) [22-24,26-30,47,48], based on visual estimation, EPSS and VTI. This review had no a priori defined research question, clear protocol, critical appraisal, or meta-analysis of included studies. The authors included eight studies which are included in the systematic review reported here [22-24,26-30] and two studies that were excluded [47,48]. Shah et al. [47] enrolled adult and paediatric patients in ED and outpatient settings with POC echocardiography performed by internal medicine and social service doctors followed by expert review of the recorded images (cardiologist or emergency medicine ultrasound fellow). This study reported almost perfect agreement for LVSF (κ = 0.98). Dehbozorgi et al. [48] studied the diagnostic accuracy of thoracic and cardiac POCUS for acute heart failure as compared to clinietween clinician sonographers and expert sonographers. Therefore, these two articles did not meet the inclusion criteria for the work reported in this article.

Visual estimation of LVSF is well described in the cardiology literature [39,40,49,50]. However, there are few studies that investigate the interobserver agreement between cardiologists or between cardiologists and sonographers. One study including 136 patients with suspected left ventricular dysfunction explored agreement between echocardiograms performed by cardiology trainees using portable ultrasound machines and hospital-based echocardiograms performed by senior cardiac technicians using a floor-based ultrasound machine. The authors reported the excellent agreement (κ = 0.87) for the detection of LV systolic dysfunction [51]. Testuz et al. compared echocardiography performed by certified cardiologists using hand-held devices to departmental echocardiography interpreted by blinded cardiologists, again with excellent agreement for LVSF (κ = 0.89) [52].

Several studies have evaluated POC echocardiography for the assessment of LVSF in an ICU setting. Amiel et al. reported the substantial agreement (κ = 0.75) in assessing LVSF between two experienced intensivists trained in critical care echocardiography using hand-held POC echocardiography as compared to cardiology reported departmental echocardiography [53]. Biais et al. reported excellent agreement (κ = 0.87) in assessed LVSF between two different expert intensivist sonographers using hand-held POC echocardiography in ED and departmental echocardiography [54]. Two studies assessed the ability of trainees in internal medicine to assess LVSF. One study reported substantial agreement (κ = 0.77) while a second study reported moderate agreement (κ = 0.51) between POC echocardiography performed by internal medicine trainees and cardiology performed echocardiography [55,56]. An ICU-based pilot study reported that following 8 h of focused training to assess LVSF using POC echocardiography by noncardiologist trainees (two anaesthesia and two internal medicine) recorded substantial agreement (κ = 0.76) as compared to expert intensivist sonographers using floor-based devices [57].

There was significant heterogeneity identified among the included studies. The level of POC echocardiography experience for clinician sonographers ranged from no previous significant experience [27,32,33], <25 scans [24], 100 scans [28], completed an US workshop [23], ACEP certified [31], and not clearly described [22,25,26,29,30]. However, the level of experience of clinician sonographers did not predict the level of agreement with expert sonographers in this systematic review. Three studies enrolled only novice clinician sonographers with no significant previous experience in ultrasound and reported substantial agreement for visual estimation, κ = 0.61 to 0.79 [27,32,33]. In three studies [22,25,29], the clinician sonographers were all attending emergency physicians but the level of POC echocardiography experience was not clearly defined, and these studies reported widely different levels of agreement for visual estimation of LVSF. In two studies the clinician sonographers were a mixture of trainees and attendings [23,31]. In this systematic review, emergency physician attending did not have higher levels of agreement with expert sonographers than emergency medicine trainees.

There was also heterogeneity in additional training in POC echocardiography offered by the study teams prior to enrolment. Ten studies [22,23,25-30,32,33] offered study-specific training in POC echocardiography to clinician sonographer participants ranging from 10 min to 16 h (theoretical and hands-on training). In one study, the clinician sonographer did not receive any additional training with a calculated κ = 0.49; however, the clinician sonographers were credentialed to the standards defined by ACEP [31]. Implementing additional, targeted training in POC echocardiography is associated with an improvement in image acquisition and interpretation [58-61].

POC echocardiography technology has advanced considerably in the past two decades, with better quality machines at a reduced cost. Five studies that used visual estimation by clinician sonographers and published after 2010 [25,27,29,31,32] showed higher pooled agreement compared to those two studies before 2010 [22,23]: κ = 0.7 (95% CI, 0.55–0.84) vs. κ = 0.64 (95% CI, 0.46–0.84). This may reflect the improvement in technology and training for clinician sonographers [62].

Limitations

This systematic review has several limitations. All the data are from small, single centre observational studies. The study design, patient population, statistical analysis, clinician sonographer selection, POC echocardiography training and experience varied considerably. The risk of bias for included studies was high, mainly a consequence of convenience sampling, patient exclusion and selected clinician sonographers. However, this reflects the clinical use for POC echocardiography where images are not always attainable and not all clinicians are trained in ultrasound/POC echocardiography. ED performed POC echocardiography involves critically unwell patients who may be challenging to position optimally. Many emergency medicine training schemes do not include assessment of LVSF and in 10 studies [22,23,25-30,32,33] additional training was provided for clinician sonographers. This work is not generalisable to clinician sonographers without additional POC echocardiography training. Two studies [29,31] classified LVSF as normal or abnormal, a very blunt tool for clinical assessment. Classification of LVSF as normal, reduced, and severely reduced function may offer a more useful clinical approach. Only two studies assessed the interobserver agreement between the expert sonographer [22,24], which was similar to the agreement between clinician sonographer and expert sonographer. There were insufficient data to allow meta-analysis for EPSS and VTI. The reference standard in this systematic review was expert sonographer interpreted echocardiograms, which is not a gold standard for LVSF assessment. This limits the value of reported sensitivity and specificity. However, this is considered an acceptable reference standard for clinical practice. A multicentre observational study comparing clinician sonographer to expert sonographer, interobserver expert sonographer/interobserver clinician sonographer reliability and intra-observer reliability using multiple methods of LVSF assessment would provide valuable data on understanding the reliability, precision, and reproducibility of each technique.

Conclusion

Clinician sonographer (trainee or attending) had a substantial agreement with an expert sonographer for the POC echocardiography assessment of LVSF as ‘normal or abnormal’ and as ‘normal, reduced and severely reduced’ by visual estimation. Few studies assessed the agreement between clinician sonographer and expert sonographer for EPSS and VTI methods and meta-analysis was not possible. There is considerable heterogeneity among studies with limited generalisability. A standardised training curriculum for LVSF assessment is needed. The quality of the included studies was low. Further studies are warranted.

Acknowledgements

Open Access funding provided by the Qatar National Library.

Conflicts of interest

There are no conflicts of interest.