Measuring quality of pre-hospital traumatic shock care-development and validation of an instrument for resource-limited settings.

BACKGROUND AND AIMS: Improving the quality of pre-hospital traumatic shock care, especially in low- and middle-income countries, is particularly relevant to reducing the large global burden of disease from injury. What clinical interventions represent high-quality care is an actively evolving field and often dependent on the specific injury pattern. A key component of improving the quality of care is having a consistent way to assess and measure the quality of shock care in the pre-hospital setting. The objective of this study was to develop and validate a chart abstraction instrument to measure the quality of trauma care in a resource-limited, pre-hospital emergency care setting.
METHODS: Traumatic shock was selected as the tracer condition. The pre-hospital quality of traumatic shock care (QTSC) instrument was developed and validated in three phases. A content development phase utilized a rapid literature review and expert consensus to yield the contents of the draft instrument. In the instrument validation phase, the QTSC instrument was created and underwent end user and content validation. A pilot-testing phase collected user feedback and performance characteristics to iteratively refine draft versions into a final instrument. Accuracy and inter- and intra-rater agreement were calculated.
RESULTS: The final QTSC instrument contains 10 domains of quality, each with specific criteria that determine how the domain is measured and the level of quality of care rendered. The instrument is over 90% accurate and has good inter- and intra-rater reliability when used by trained pre-hospital provider users in South Africa. Pre-hospital provider user feedback indicates the tool is easy to learn and quick to use.
CONCLUSION: We created and validated a novel chart abstraction instrument that can reliably and accurately measure the quality of pre-hospital traumatic shock care. We provide a systematic methodology for developing and validating a quality of care tool for resource-limited care settings.

INTRODUCTION

Assessing quality of care, especially in the dynamic pre‐hospital (ambulance) setting, remains a major challenge and impediment to understanding and improving the quality of emergency and trauma care. , , , , Quality audits have historically been tedious and conducted using expert observation, physician chart review, or case discussion. However, the paucity of pre‐hospital experts, especially in resource‐limited settings, results in the majority of pre‐hospital cases remaining un‐reviewed. , In addition, there are no standardized, validated measurement tools that focus on assessing quality in pre‐hospital emergency care.

Pre‐hospital care is the earliest formal opportunity of a trauma system to identify life‐threatening injuries and initiate timely resuscitative care. One multi‐national comparative study of severe trauma concluded that mortality correlates inversely with country economic level, and the majority of trauma mortality occurred in the pre‐hospital setting, further justifying the need to strengthen pre‐hospital trauma care in low‐ and middle‐income countries (LMICs). Trauma is a leading cause of global mortality, and LMICs experience a disproportionately large share of all global trauma‐related mortality. , Populations in resource‐limited settings face over twice the injury mortality compared to high‐resource settings. , , , At the extreme, South Africa, for example, has eight‐times the global mean rate of trauma mortality due to interpersonal violence. ,

In LMICs, pre‐hospital trauma care is often of poor quality or delayed, and in‐hospital resuscitation is often too late, further necessitating timely identification and high‐quality management of trauma in the pre‐hospital setting. , Quality of care is not easy to define or measure and is even more challenging to assess quality within the heterogeneous trauma population. Therefore, defining and measuring the quality of care delivered, and evidence‐based opportunities to improve care, remain poorly explored in the pre‐hospital trauma care literature.

The objective of this study is to develop and validate an instrument that can be consistently and objectively applied to pre‐hospital trauma cases to measure the quality of care provided in low‐resource settings.

METHODS AND MATERIALS

Summary

Development and validation of the QTSC instrument occurred in three main phases: content development, instrument validation, and pilot testing (Figure 1). Investigators selected traumatic shock as the tracer condition. Traumatic shock is identifiable and intervenable by pre‐hospital providers, and if poorly managed, contributes to poor patient outcomes—it is a priority pre‐hospital condition worldwide. , The location selected for instrument testing and validation was in the Western Cape Province of South Africa, due to the high trauma caseload seen by the pre‐hospital system. , ,

FIGURE 1

Phases of Quality of Traumatic Shock Care (QTSC) instrument development and testing. Content development phase was led by investigators and expert panel. Instrument validation and pilot testing phases involved collecting data from investigators and trained chart abstractors. The final validated instrument is intended for end‐users

In the content development phase, there was a literature search to identify domains of quality in traumatic shock, and the findings informed an expert consensus process to define the study population and clinical domains for a quality of traumatic shock care (QTSC) assessment instrument. In the instrument validation phase, the QTSC instrument was created and underwent end user and content validation. In the pilot testing phase, the instrument was applied to pre‐hospital clinical records to generate a quality score and analyzed against an expert EMS physician panel quality of care score.

Content development phase

Literature review

A rapid literature review of indexed and “grey” literature was performed to identify evidence‐based clinical components of pre‐hospital traumatic shock care appropriate for LMIC health settings. The literature search included references in English from January 1, 2000 to December 31, 2016. Indexed articles were identified in PubMed using the following search terms: trauma; injury; pre‐hospital; EMS; shock; hemorrhage; resuscitation; and quality. Retrieved articles were manually reviewed for relevance. The “grey” literature search used snowball sampling, starting with well‐established pre‐hospital trauma care references from January 1, 2000. Investigators sorted relevant findings into two categories: components to recognize pre‐hospital shock, and components to manage shock in an ambulance.

Expert consensus process

Investigators assembled a panel of experts—the panel was tasked to reach consensus on which components from the literature were relevant to the objectives, while adhering to the following principles: measurable in a low‐resource setting, highly relevant to pre‐hospital care, applicable to all traumatic shock cases, and reflect quality. The panel included eight multi‐disciplinary content experts with overlapping research and clinical expertise in shock, trauma care, pre‐hospital medicine, emergency care, survey instrument design, and chart abstraction methodology. Five members were located in the USA, and three in South Africa. The panel was asked to agree on the case definition for traumatic shock and to vote on components of shock care (per the literature review). The process was conducted in SurveyMonkey (San Mateo, California), and voting was done using Likert scale scores of 1 (worst) to 5 (best), with anchor descriptions provided to help calibrate responses and yield consistency in voting. There was a final ratification step in which panelists were asked to “agree/disagree” on the core and non‐core components. Final consensus was defined a‐priori as ≥75% concordant “agree” votes.

Instrument validation phase

From the expert consensus process, a draft QTSC instrument was created in an electronic format in REDCap (Research Electronic Data Capture, Nashville, Tennessee) and enhanced via three cycles of pilot testing and refinement (Figure 1). Since the inaugural application of the QTSC instrument was planned for the Western Cape Government (WCG) Emergency Medical Services (EMS) system in South Africa, de‐identified clinical charts from that EMS system were used to test the QTSC instrument.

Selection of clinical cases for testing

WCG EMS clinical charts were eligible for inclusion for adult patients who met the traumatic shock case definition. The following case types were excluded: (a) severe head injury; (b) burns; (c) drowning; (d) electrocution; and (e) strangulation.

Study investigators (J.D., T.B.) manually retrieved and reviewed chronologically occurring available trauma charts from the WCG EMS medical records office. Trauma charts that satisfied inclusion criteria were manually reviewed, assigned a unique study number, de‐identified, and uploaded to our online study database in REDCap. The principal investigator (N.M.) selected a subset for pilot testing that was representative of WCG EMS patient acuity, mechanisms of injury, and documentation style.

The aim of pilot test cycle one was to collect user feedback about the structure and content of the QTSC instrument, for which 20 charts were selected. The goal of cycles two and three was to calculate content validity indexes. Sample size calculations estimated 50 and 200 WCG EMS trauma charts were needed for cycles two and three, respectively.

Pilot testing by investigators

In cycles two and three, three investigators (J.D., N.M., T.B.) independently reviewed each trauma case, abstracted data into REDCap, then recorded feedback about the abstraction experience. Abstracted data and feedback were downloaded from REDCap, reviewed, and discussed by investigators. Investigators refined the QTSC instrument to improve structure, wording, formatting, flow, and data input options while a training manual was simultaneously updated.

Accuracy validation

Investigators consented, recruited, and trained three local (South African) paramedic chart abstractors for pilot testing. Paramedics then independently abstracted 50 trauma charts into REDCap, while noting relevant feedback. One day later, each paramedic re‐abstracted 10 (20%) of charts to provide data for intra‐rater reliability calculations. Paramedic data were downloaded from REDCap and compared to reference standard (investigator) data. Data and feedback were used to make improvements to the QTSC instrument. For all abstractions, paramedics were blinded to identities of patients, providers, outcomes, and to other abstractors' reviews.

Reference standard quality of care determination by expert reviewers

In pilot cycle two, two clinical reviewers (a WCG EMS physician and a WCG EMS manager, not part of the study team) jointly reviewed each of the 50 cases and provided a consensus quality of care score (1 = low quality, 2 = average quality, and 3 = high quality) for the same 50 charts previously abstracted by the paramedics and two study investigators. Anchor definitions for Likert scores were provided (see Data S1). Clinical reviewers' Likert scores were coded dichotomously as “low quality” (if 1) or “not low quality” (if 2 or 3). To generate a quality of care score using paramedic abstracted data, logic was created to assign a quality score (1‐3, with three being the highest) for each domain (see Data S2).

Analysis

We assessed the content validity of the QTSC instrument by calculating the following relevant indicators: content validity index, accuracy, agreement, and criterion validity.

Content validity index

The Content validity index (CVI) for each item is the proportion of experts who rate the domain as relevant (ie, ≥3 on a 5‐point Likert scale). A‐priori, it was determined that ≥75% of experts scoring a domain ≥4 out of 5 would qualify the domain as a “core” component of shock care; 2.5 to 3.9 would qualify as a “non‐core” component; and <2.5 would be considered irrelevant (ie, an unnecessary quality component of pre‐hospital traumatic shock care in a resource‐limited setting). Missing data were handled as such, and no imputation or deletion methods were used. Missing data were handled as such, and no imputation or deletion methods were used.

Accuracy

Accuracy was defined as the proportion of paramedic chart abstractor data that matched reference standard data, per case. The proportion of paramedic data concordant with reference data (ie, crude accuracy) were descriptively summarized using means and proportions.

Agreement

Agreement of abstracted data within and among paramedics (ie, intra‐ and inter‐rater reliability, respectively) was calculated using a Cohen's kappa test. A multi‐rater kappa statistic was generated for inter‐reliability analyses. For intra‐rater reliability, a kappa statistic was calculated to test the agreement between a rater's first chart evaluation and repeated evaluation, for each measure. The proportion of crude agreement between paramedic data entry and physician reference standard helped to assess for any possible skewed prevalence between ratings, which was important to ensure that skewed data were not entered into the Kappa calculation to yield distorted findings. Kappa coefficients (k) were interpreted agreement as follows: <0.4 = poor; ≥0.4 = moderate; ≥0.6 = substantial and crude proportional agreement ≥0.7 are the threshold values, consistent with similar validation studies.

Criterion validity

Criterion validity is the extent to which a measure is related to an outcome. We assessed criterion validity by comparing paramedic‐abstracted data with reference standard quality care. The quality scores resulting from paramedic chart abstraction were compared to the scores from the EMS experts' review using a weighted kappa and non‐parametric comparison of the distributions (F‐test assesses whether two populations have equal variances; the Mann‐Whitney test compares whether there is a difference in the dependent variable for two independent groups; and the Kolmogorov‐Smirnov test is a nonparametric test that compares the cumulative distributions of two data sets). Squared weights were used. All analyses were performed in SAS, 9.4 (SAS Institute, Inc, Cary, North Carolina). Significance was set at .05 and statistical tests were two‐sided.

Ethical approval

This study was approved via a waiver of informed consent by the relevant human research boards in South Africa (University of Cape Town Human Research Ethics Committee ‐ UCT HREC Ref# 080/2017) and in the USA (Colorado Multiple Institutional Review Board ‐ COMIRB# 17‐0284). Written approval was obtained from WCG EMS.

RESULTS

The case definition for pre‐hospital traumatic shock agreed upon by the expert panel was “An adult patient with a high‐risk trauma mechanism with one or more vital sign findings of shock and one clinical symptom or picture of shock.” Study investigators converted the shock definition into a traumatic shock recognition algorithm (Figure 2). The panel's rationale for a blood pressure cut‐off of 100‐mmHg was to promote earlier and more conservative identification of shock cases, considering early shock is often missed in the pre‐hospital setting, and noting that resource‐limited trauma systems could benefit from earlier triggers.

FIGURE 2

Bundle of EMS traumatic shock care (EMS‐TruShoC). Mechanisms of injury placing patient at high risk for shock are as follows: Penetrating: Gunshot wound (head, neck, torso, groin, proximal extremity). Blunt: Fall from height (>6 m); Motor vehicle collision (high speed, ejection); Motor cycle crash; Pedestrian struck by vehicle; Assault (with high energy transfer). Amputation: Of limbs (proximal to wrist and ankles). Active Bleeding: Uncontrollable external bleeding; Physical signs of contained (internal) hemorrhage

Content validity index

Ten components (“domains”) of the quality of traumatic shock care emerged from the literature review. For five domains, 7 out of 8 (87.5%) experts scored ≥4 out of 5 Likert points qualifying them as core domains, including: control of external hemorrhage, short on‐scene time, insert appropriate large bore intravenous (IV) catheter, trauma hospital as destination, and oxygen delivery. For five other domains, 6 out of 8 (75%) experts scored 2.5 to 4 Likert points, qualifying as non‐core domains (Table 1). Two domains—facility pre‐arrival notification and vasopressor administration—scored <2.5 out of 5 Likert points, and were excluded.

TABLE 1

Domains, criteria, and Likert scale scores

	Domain of Care	Criteria for high quality	Experts rating as relevant; n (CVI)	Mean ofLikert scores
Core components	[1] Control external hemorrhage	Use 1 or more methods of external hemorrhage control when applicable.	8 (1.0)	4.5 ± 0.53
	[2] Short scene time	Scene arrival to departure in less than 10 minutes.	8 (1.0)	4.3 ± 0.46
	[3] IV catheter	Place 14‐, 16‐ or 18‐gauge IV catheter in the antecubital fossa or external jugular.	7 (0.875)	4.2 ± 0.99
	[4] Hospital destination	Patient transported to a capable trauma hospital or designated trauma center.	6 (0.875)	4.1 ± 0.83
	[5] Oxygen	Any route and concentration of oxygen was delivered.	6 (0.75)	4.1 ± 0.83
Non‐core components	[6] First set of vitals	Initial heart rate, systolic blood pressure and capillary refill time recorded.	8 (1.0)	3.9 ± 0.14
	[7] IV fluid management	Any volume of IV fluids given if systolic blood pressure < 100‐mmHg.	6 (0.75)	3.9 ± 0.88
	[8] A‐B‐C‐D assessment	Documented a trauma A‐B‐C‐D assessment.	7 (0.875)	3.8 ± 0.67
	[9] A‐B‐C‐D interventions	Documented trauma A‐B‐C‐D interventions (when applicable).	7 (0.875)	3.4 ± 0.69
	[10] Last set of vitals	Final heart rate, systolic blood pressure, and capillary refill time all recorded.	6 (0.75)	2.9 ± 0.58

Abbreviations: IV, intravenous; A‐B‐C‐D, airway, breathing, circulation, and disability; CVI, content validity index (n/8).

Accuracy

Overall, accuracy was strong and improved with each progressive stage of pilot testing (Table 2). In cycle one, the mean accuracy was 86% ± 7.8 and all 10 domains were over 70% accurate. Pilot cycle three had the highest mean accuracy (94% ± 7.6) with seven domains scoring >90% accuracy. The A‐B‐C‐D assessment domain performed poorest attaining 78% accuracy. The total number of chart abstraction elements was 58.

TABLE 2

Raters' accuracy compared to reference standard, and inter‐ and intra‐rater reliability statistics

Instrument section	Domain of care	Number ofquestions per domain (N)			Proportion matching(raters' average vs reference standard) (%)
		Cycle 1	Cycle 2	Cycle 3	Cycle 1	Cycle 2	Cycle 3
Core bundle	[1] Control external hemorrhage	8	5	6	7/8 (88%)	4.7/5 (94%)	5.2/6 (87%)
	[2] Short scene time	2	2	2	1.9/2 (97%)	1.9/2 (95%)	2/2 (100%)
	[3] IV catheter	5	5	5	4.3/5 (85%)	4.6/5 (91%)	5/5 (100%)
	[4] Hospital destination	1	1	1	0.8/1 (83%)	0.9/1 (90%)	1/1 (99%)
	[5] Oxygen	2	2	3	1.5/2 (73%)	1.9/2 (97%)	2.9/3 (98%)
Non‐core bundle	[6] First set of vitals	14	11	11	13.3/14 (95%)	10.2/11 (93%)	10.6/11 (96%)
	[7] IV fluid management	4	4	4	3.4/4 (85%)	3.5/4 (87%)	3.4/4 (86%)
	[8] A‐B‐C‐D assessment	4	5	5	3/4 (75%)	4.5/5 (89%)	3.9/5 (78%)
	[9] A‐B‐C‐D interventions	6	4	10	5.5/6 (92%)	3.5/4 (87%)	10/10 (100%)
	[10] Last set of vitals	12	10	11	10.4/12 (87%)	9.5/10 (95%)	10.6/11 (96%)
Total		58	49	58	‐	‐	‐
Mean		‐	‐	‐	86% ±7.8	92% ±3.5	94% ±7.6

Abbreviations: IV, intravenous; A‐B‐C‐D, airway, breathing, circulation, and disability.

Agreement

Raters had substantial to moderate inter‐rater reliability on 64% (n = 46) of items on average (see Data S3). Five poor performing (kappa <0.4) items were revised, all located within the “Shock Signs” and “ABCD Assessment and Management” sections. Re‐testing of the subsequent version of the instrument indicated those sections improved to moderate agreement as evidenced by the majority of P‐values to be greater than .05 for the Mann‐Whitney, Kolmogorov‐Smirnov, and weighted Kappa tests, which together indicate similar responses between paramedic raters and gold standard experts (see Data S3 and Table 3).

TABLE 3

Weighted kappa agreement and nonparametric distribution comparison between chart abstraction quality score and expert clinician quality score

	P‐value
Domains of care	F‐test for equal variance a	Mann–Whitney test a	Kolmogorov–Smirnov test a	Weighted Kappa
[1] Bleeding control	.23	.01	.03	0.32
[2] Scene time	.94	.75	1.00	0.97
[3] IV size/placement	.74	.39	.77	0.85
[4] Trauma center	.66	.19	.28	0.80
[5] Oxygen	.16	.31	.10	0.77
[6] First vitals	.26	.33	.92	0.50
[7] IV fluids	.25	.83	.99	0.64
[8] A‐B‐C‐D assessment	.22	<.001	.01	0.17
[9] A‐B‐C‐D intervention	.01	<.001	<.001	−0.05
[10] Last vitals	.38	.15	.17	0.70

A significant P‐value means two samples come from two different populations.

Criterion validity

Pilot test cycle two found substantial or moderate agreement between the chart abstraction and clinician reference standard for all domains of care except external bleeding control, A‐B‐C‐D assessment, and A‐B‐C‐D intervention (Table 3). For each nonparametric distribution calculation (ie, the Kolmogorov‐Smirnov test and the Mann‐Whitney test), there were no significant differences between scores determined by chart abstraction and scores determined by expert physicians (Table 3). The distribution of quality scores determined by abstraction as compared to physicians is graphically presented in Figure 3 ‐ each dot represents the paramedics and experts agreement for a given case, and perfect correlation is defined as the paramedics scoring “1” when the experts scored “1” for the same case, for example.

FIGURE 3

Distribution of paramedic quality with reference standard quality (1 = low, 2 = average, 3 = high) for each domain. Each unique case is represented as a dot and clustering around a like number indicates agreement

Chart abstractor experience

Training required approximately 3.5‐hours per abstractor. The mean duration per chart abstraction was 10.5 minutes (standard deviation, SD = 1.24). Paramedic abstractors' feedback indicated the instrument was relatively straightforward to learn to use, with the case screening (inclusion/exclusion) criteria being the most conceptually challenging aspect, and the ABCD assessment and management sections being the most difficult to abstract given poor documentation of airway and disability assessments. The final QTSC instrument is shown in Data S3.

DISCUSSION

We developed, iteratively refined, and validated a novel chart abstraction instrument that accurately and reliably quantifies the quality of pre‐hospital traumatic shock care in a resource‐limited trauma system. The QTSC instrument is over 90% accurate and has good inter‐ and intra‐rater reliability. In addition, the final data capture instrument is practical and time‐efficient to apply. Three noteworthy points emerge regarding the process, the product, and the public health relevance of this work.

While a novel application in the field of pre‐hospital trauma care, our development and validation methodology are similar to several published studies that aimed to produce similar tools to assess quality of care in ischemic stroke, geriatrics, heart failure, and palliative care. , , , Similar to Kergoat and colleagues who selected geriatric falls as the tracer condition for measuring geriatric quality of care, we selected pre‐hospital traumatic shock care as the tracer condition. , , We combined the literature review with expert consensus to create a case definition and care domains appropriate for resource‐limited settings. Lastly, we drafted and iteratively tested our chart abstraction tool using members of our investigator team, external EMS experts, and trained chart abstractors. This stepwise approach allowed us to cyclically optimize the performance of the instrument, and validate the instrument using four standard measures of validity. While time‐consuming, our multi‐sequence approach may serve as a template for others seeking to develop and validate similar quality tools.

The product (ie, our validated QTSC instrument) is innovative, evidenced by being the first quantitatively validated quality of care tool purposefully designed for pre‐hospital trauma care. We note three other published validated pre‐hospital tools, from Australia, Brazil, and the USA. , , However, those tools are not exclusively focused on traumatic shock care. The earliest published tool, the System Input Severity Measure, by Headrick and colleagues in 1978, appraised the overall quality of the EMS system rather than the quality of patient care. In 2004, Smith and colleagues in Australia modified the Maryland Practitioner Clinical Medical Record Audit tool and qualitatively validated their instrument for measuring quality of documentation in pre‐hospital report forms of any case type, using trauma cases as a feasibility proof‐of‐concept. In Brazil in 2015, Dantas and colleagues developed an instrument for assessing the quality of overall pre‐hospital care services (QA‐PHC), which included clinical and non‐clinical indicators. Therefore, our QTSC instrument helps to fill a scientific gap as the only validated pre‐hospital trauma quality of care assessment tool in the literature, albeit limited to traumatic shock.

Our final instrument had good performance characteristics; domains achieved at least moderate reliability and high accuracy which are comparable to the quality of care instruments from other medical disciplines. , , , Agreement with reference standard could not be calculated for a few rare variables (eg, tourniquet application) and procedures done without variation (eg, IV fluids) ‐ for those variables, we relied on accuracy and reliability data and abstractor feedback. The poorest performing sections were in the A‐B‐C‐D assessment and intervention sections (specifically, airway, breathing, or neurologic issues). As evidenced by lack of agreement (Table 3) and poor rater correlation with gold standard (Figure 3) in these specific sections, both sections proved challenging for our chart abstractors, primarily explained by poor clinical documentation.

There are several limitations of this work. The largest limitation is inherent in chart review methodology because assessment of quality relies on the accuracy and completeness of clinical documentation. Second, we had limited data points to robustly appraise performance of a few items in our instrument due to the low frequency with which those elements are performed in routine pre‐hospital trauma care (eg, pelvic binding). Lastly, the external validity of our instrument may be limited and necessitates testing in other EMS systems.

This work has notable clinical and public health implications. Delivery of early, high‐quality resuscitation is central to improving the global burden of trauma, a role that is achievable by EMS systems. Clinical care audits by record reviews are one effective method to improving quality of care, which is challenged in resource‐constrained EMS systems by the lack of requisite tools and physician time. In the Western Cape EMS system, for example, our QTSC tool can be used by trained paramedics to assess quality of care accurately, reliably, and rapidly in traumatic shock cases thereby circumventing the need for physicians to determine quality. We postulate that our QTSC instrument can identify relevant care gaps, which if targeted for improvement, may help improve trauma outcomes in the Western Cape trauma care system. We also intend to use this QTSC instrument for assessing clinical outcomes in traumatic shock studies.

CONCLUSION

In conclusion, we systematically developed and validated the QTSC instrument, a chart abstraction tool that enables a trained paramedic to assess QTSC from an EMS clinical chart in a resource‐limited pre‐hospital setting. Additionally, we have described the process and methodology used to develop and validate the QTSC instrument to enable others to conduct similar work focused on assessing quality of care.

FUNDING

This work was supported by a Pilot Grant awarded to Dr. Mould‐Millman by the University of Colorado School of Medicine, Department of Emergency Medicine (DEM‐Pilot‐2017‐MM). REDCap (online software for data collection and storage housed at the University of Colorado) was provided by the Colorado Clinical and Translational Sciences Institute via a National Institutes of Health grant (NIH/NCRR Colorado CTSI Grant Number UL1 TR001082). Neither of these two funding sources played any role in the study design, data collection, analysis, interpretation of findings, or publication.

CONFLICT OF INTEREST

The authors declare that there is no conflict of interest.

AUTHOR CONTRIBUTION

Conceptualization: Nee‐Kofi Mould‐Millman, Adit Ginde.

Data Curation: Nee‐Kofi Mould‐Millman, Julia Dixon, Taylor Burkholder, Jacob Thomas.

Formal Analysis: Jacob Thomas, Julia Dixon, Nee‐Kofi Mould‐Millman.

Funding Acquisition: Nee‐Kofi Mould‐Millman.

Investigation: Nee‐Kofi Mould‐Millman, Julia Dixon, Michael Lee, Taylor Burkholder.

Methodology: Nee‐Kofi Mould‐Millman, Adit Ginde.

Project Administration: Nee‐Kofi Mould‐Millman, Shaheem de Vries, Fabio Moreira, Beatrix Bester.

Software: Jacob Thomas.

Supervision: Nee‐Kofi Mould‐Millman, Shaheem de Vries, Lee Wallis, Adit Ginde.

Validation: Nee‐Kofi Mould‐Millman.

Visualization: Nee‐Kofi Mould‐Millman, Julia Dixon, Halea Meese.

Writing ‐ Original Draft Preparation: Nee‐Kofi Mould‐Millman, Julia Dixon, Halea Meese, Fabio Moreira, Beatrix Bester.

Writing ‐ Review & Editing: Nee‐Kofi Mould‐Millman, Julia Dixon, Michael Lee, Halea Meese, LM, Taylor Burkholder, Fabio Moreira, Beatrix Bester, Jacob Thomas, Shaheem de Vries, Lee Wallis, Adit Ginde.

All authors have read and approved the final version of the manuscript.

Nee‐Kofi Mould‐Millman had full access to all the data in this study and takes complete responsibility for the integrity and the accuracy of the data analysis.

TRANSPARENCY STATEMENT

Nee‐Kofi Mould‐Millman affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

Data S1. Likert Scale score definitions for EMS physicians' gold standard reviews.

Click here for additional data file.

Data S2. Algorithm to create quality of traumatic shock care score from paramedic abstracted data.

Click here for additional data file.

Data S3. Reliability and crude agreement of items in version 1 of the QTSC instrument.

Click here for additional data file.

Data S4. Final QTSC instrument.

Click here for additional data file.