Telementoring of Healthcare Teams in the Care of Miners.

Background: Given the reemergence of pneumoconiosis in the United States, there is a tremendous need to train rural professionals in its multidisciplinary management. The Miners' Wellness TeleECHO (Telementoring Extension for Community Health Outcomes) Program in New Mexico, United States, provides longitudinal multidisciplinary telementoring to professionals taking care of miners. The impact of this approach has not been previously evaluated. Objective: To examine the change in self-efficacy of professionals taking care of miners and participating in the TeleECHO Program.
Methods: This is a 12-month longitudinal study involving clinical and nonclinical professionals caring for miners. The study outcome was the change in self-efficacy scores, using a customized instrument of 14 measures grouped into three domains: clinical, medicolegal, and soft skills. The primary outcome used a retrospective pre-post design that collects "pretest" data at the postintervention timeframe.
Results: Participants reported significant improvements in 10 of 14 items (P < 0.05) and a significant decline in 1 of 14 items (with respect to their ability to interpret pulmonary function test results, P < 0.001) since their start dates in the program. Subjects also reported significant improvement with respect to their scores for all three domains and for the 14-item total score (P ⩽ 0.01). Existing participants and clinical professional groups demonstrated greater improvement in selected items than fresh participants and nonclinical professional groups, respectively.
Conclusion: This study is the first in a stepwise approach to determine the benefit of participating in a multidisciplinary telementoring intervention by improving participant self-efficacy in caring for miners with complex mining-related diseases. Our study finding represents a potential solution to a growing access-to-care gap for miners with pneumoconiosis.

The recent reemergence of pneumoconiosis (i.e., dust-related lung diseases), particularly among coal miners (i.e., black lung or coal workers’ pneumoconiosis [1]), in the United States has created a challenge for rural mining communities. Although the number of miners with pneumoconiosis requiring complex multidisciplinary care has increased, the expertise required to provide the care has decreased in rural United States (2). There is a tremendous need to train rural professionals caring for miners across the multidisciplinary aspects of the management of complex mining-related diseases, particularly in the pneumoconiosis-mortality hot-spot regions of Appalachia and the Mountain West (3). These multidisciplinary skills include clinical, medicolegal, and “soft” skills, with the latter category including interpersonal skills needed to navigate highly collaborative work in the care of miners. The existing inadequacy of expertise demands innovative solutions to support quality multidisciplinary teams to improve the health and well-being of miners in rural and remote communities.

The Miners’ Wellness TeleECHO (Telementoring Extension for Community Health Outcomes) Program, set up in July 2016 in New Mexico, United States, provides longitudinal multidisciplinary telementoring to various groups of professionals taking care of miners (4). Employing virtual techniques of engagement, the program uses a “community-of-practice” approach, defined in the literature as “groups of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly” (5). Our group has recently published a preliminary cross-sectional evaluation of our community-of-practice approach, which demonstrated that the community was regarded highly by participants in relation to trust, willingness to help, and being closely knit (4). Participants reported low baseline self-efficacy (defined as the belief that one can actually perform a behavior or attain specific outcome [6]) with respect to diagnosing miners’ conditions, interpreting chest radiographs using the International Classification of Radiographs of Pneumoconiosis (i.e., B reads), and determining eligibility for compensation. The longitudinal impact of the virtual community-of-practice approach on the change in self-efficacy of participants has, however, not been previously evaluated, constituting a critical gap in knowledge. Because lack of clinician self-efficacy is a barrier to patient care (7), improvement in self-efficacy of TeleECHO participants may lead to improved care of miners. Addressing this gap in knowledge may allow for new evidence-based rural interventions for providing complex multidisciplinary care for miners. The objective of this longitudinal study was to examine the change in self-efficacy of participants, using a virtual community-of-practice approach in telementoring rural professionals taking care of miners in the United States. The outcome corresponds to level 3 of the Moore expanded framework for continuing medical education (CME) activities (8). If effective, it provides an exciting opportunity for greater investment in interprofessional telementoring to foster collaborative healthcare practices in rural and remote mining communities as an approach toward countering the resurgence of pneumoconiosis.

Methods

Study Design

This is a 12-month longitudinal study of participants involved in the TeleECHO Program, a novel community–university partnership between the University of New Mexico School of Medicine, Albuquerque, New Mexico, and its rural partner, the Miners’ Colfax Medical Center, Raton, New Mexico. Together, these New Mexico sites constitute the hub site of experts. The spoke partner sites, located across the pneumoconiosis-mortality hot-spot regions of the United States, include clinicians, respiratory therapists, home health professionals, benefits counselors, lawyers or attorneys, and others, including policy-makers, administrators, and mine-safety officers (4). The hub and the spoke partners together form a virtual community of practice.

Program Description

As published previously (4), the TeleECHO sessions are scheduled at the same time twice every month for 75 minutes, adhering to a standard format. After an initial 10 minutes of introduction and announcements, an invited expert delivers a 15-minute didactic presentation followed by a 20-minute, facilitated question–answer session and a 30-minute, interactive, facilitated case discussion. This format, employing adult-learning principles, focuses more on active learning through discussion than on didactic training. Participants can avail CME credits without charge upon completing the CME survey. A multidisciplinary curriculum committee frequently reviews the feedback provided to formulate a structured curriculum.

Irrespective of whether they present the case or not, participants can view the didactic and case discussions, provide information and insight from their own experience, and receive mentoring from the expert panel and their peers. Outside the program sessions, participants have access to experts or peers at the hub or other spoke sites, by telephone or e-mail, for urgent consultation requests. Over time with iterative practice and feedback, participants gain additional expertise and become more independent. Recorded and archived sessions are available from the program’s website without charge.

Program Development

Since July 2016, the program has used the ECHO model to provide structured long-term telementoring in the care of miners. This approach contrasts with telemedicine, in which experts provide short-term care to individual patients. This approach also differs from traditional lectures or webinars because it provides a real-time, interactive discussion of cases with expert panels. By contextualizing discussions, the program fulfills key learning theories, such as deliberate practice (9), social cognitive theory (10), and situated learning and communities of practice (11). As published previously (4), the five key principles of the ECHO model include the following: 1) use of Internet technology to leverage resources; 2) use of a disease-management model shown to be effective in other diseases by sharing best practices (12–15); 3) use of the principle of case-based learning; 4) creation of a community of practice, which emphasizes reciprocity in knowledge transfer, acknowledging that all participants bring some unique expertise; and 5) use of an Internet-based database to monitor outcomes. For the curriculum outlining individual didactics during the study timeframe, see Table E3 in the data supplement.

Outcomes

The study outcome was the change in self-efficacy scores, using a customized instrument created by a multidisciplinary curriculum and evaluations committee, based on the review of the literature and feedback from individual stakeholder groups. Self-efficacy, or a provider’s perceived ability to recognize and manage essential components of care of complex miners’ diseases and to serve as a local expert in these diseases for other community professionals, was assessed, with assessment adapted from previously validated instruments (16). Participants rated each of the 14 self-efficacy measures on a scale of 1–7, in which “1” indicates “none” or no skill and “7” indicates “expert” or ability to teach others. The 14 measures were grouped into three domains: a six-item clinical-skills domain, a three-item medicolegal-skills domain, and a five-item soft-skills domain. The self-efficacy instrument, provided in Table E4, was not fully validated in the current study. The change in self-efficacy items, separately calculated since the subject’s start date in the program (using the retrospective pre-posttest method with both pretest and posttest data obtained at the 12-month study time point) and over a 12-month study timeframe (using the traditional pre-posttest data method with pretest and posttest data obtained at the 0- and 12-month study time points, respectively) was calculated. For our primary study outcome, we chose a priori the retrospective pre–post design that collects “pretest” data during the postintervention timeframe at 12 months for the following two reasons. 1) By encouraging participants to reflect back and rate self-efficacy before participation in the ECHO Program with the benefit of hindsight, the retrospective pretest evaluation helps reduce the bias of self-evaluation (13). This approach is useful, as ECHO evaluators have found that participants often do not realize how much they did not know until after they have participated in the program (17). 2) Several prior ECHO outcome evaluations have demonstrated that, as compared with traditional pretest scores, retrospective pretest scores have lower mean and/or standard-deviation values, yielding greater power for analysis for small-sized comparison studies (18, 19).

Data Collection

Data were collected during the 1-year timeframe of September 12, 2018, to September 18, 2019, using Research Electronic Data Capture (Vanderbilt University), a secure Web application for building and managing online surveys and databases.

Analytic Strategy

Data collected were analyzed using Statistical Analysis Software version 9.4 (SAS Institute, Inc.). Wilcoxon signed rank sum tests for paired measurements were used to analyze change in continuous outcomes. Subgroup analyses for change scores were analyzed using t tests. Comparison groups included clinical professional groups versus nonclinical professional groups and existing versus fresh participants. Clinical professional groups include clinicians, respiratory therapists, and home health professionals (mostly nurses). Nonclinical professional groups include lawyers or attorneys, benefits counselors, and others. Fresh participants were defined as those who first attended the community of practice in or after the summer of 2018 (defined as May 9, 2018, and onward) versus existing participants (defined as those who had first attended any time between July 1, 2016, and May 8, 2018). Existing participants thus had greater cumulative participation and experience with the TeleECHO Program than fresh participants (11.4 ± 9.8 vs. 4.6 ± 4.6 total sessions before and/or during the study timeframe; P = 0.03). The cut-point date was chosen on the basis of the date of funding by the sponsor, which allowed the frequency of the TeleECHO Program to be raised from monthly to twice a month. A two-tailed P value < 0.05 was considered statistically significant without adjusting for multiple comparisons.

Ethics Approval

Approval was obtained from the institutional review board, Human Research Protections Office, at the University of New Mexico Health Sciences Center (18-386).

Results

As shown in Figure 1, 71 subjects completed the baseline survey (which collects information for the traditional pretest evaluation), and 37 of these completed the 12-month survey (which collects information for the retrospective pretest and posttest evaluation). Another 10 subjects joined the 12-month survey but did not complete the baseline survey. A total of 47 participants were therefore available for analyses. As shown in Table 1, most respondents reported caring for miners for 10 years or fewer (55.3%) and caring for at least a 40% proportion of miners living in rural areas (59.6%); 53.2% of subjects were existing participants. Clinical professional groups constituted the largest group of subjects (74.4%), with the largest subgroup being clinicians. Despite a high level of reported professional satisfaction, a significant minority described professional isolation.

Figure 1.

A flow diagram showing an overview of participation by study subjects.

Table 1.

Characteristics of surveyed participants at 12-month study time point (n = 47)

Participant Characteristics	n	%
Duration of miner care, yr
1	7	14.9
2–5	11	23.4
6–10	8	17.0
11–15	3	6.4
16–20	2	4.3
21+	6	12.8
Not reported	10	21.3
Gender identity
Male	13	27.7
Female	24	51.1
Not reported	10	21.3
Race
White	32	68.1
Asian	2	4.3
Two or more races	2	4.3
Other	1	2.1
Not reported	10	21.3
Ethnicity
Hispanic	4	8.5
Non-Hispanic	31	66.0
Not reported	12	25.4
Age, yr
≤30	3	6.4
31–40	8	17.0
41–50	6	12.8
51–60	9	19.1
>60	9	19.1
Not reported	10	21.3
Proportion of rural miners served, %
81–100	10	21.3
61–80	10	21.3
41–60	8	17.0
21–40	2	4.3
0–20	7	14.9
Not reported	10	21.3
Existing vs. fresh participants
Fresh	22	46.8
Existing	25	53.2
Individual stakeholder groups
Clinical professional groups
Clinician	17	36.2
Respiratory therapist	9	19.1
Home health professional	9	19.1
Nonclinical professional groups
Lawyer/attorney	2	4.3
Benefits counselor	6	12.8
Other	4	8.5
Satisfaction in professional practice
Strongly agree to agree	44	93.6
Neutral to strongly disagree	3	6.4
Lack of professional isolation
Strongly agree to agree	34	72.3
Neutral to strongly disagree	13	27.7

Using the retrospective pre-posttest method, with both pretest and posttest data obtained at the 12-month study time point, subjects reported significant improvements in 10 of 14 items (P < 0.05, positive average change scores) and a significant decline in 1 of 14 items (with respect to their ability to interpret pulmonary function test results; P < 0.001, negative average change scores) since their start dates in the program (Table 2 and Figure 2). Subjects also reported significant improvement with respect to their scores for each of the three domains of clinical skills (P < 0.001), medicolegal skills (P = 0.04), and soft skills (P = 0.01) and for the 14-item total score (P = 0.002). As compared with fresh participants, existing participants demonstrated significantly greater improvement in their self-reported ability to diagnose health conditions (i.e., clinical-skill item 1; estimate, 0.78; 95% confidence interval [95% CI], 0.29–1.26; P = 0.003), in their self-reported ability to help manage common health conditions (clinical-skill item 2; estimate, 0.59; 95% CI, 0.01–1.17; P = 0.046), and in their clinical-skills-domain score (estimate, 0.33; 95% CI, 0.06–0.59; P = 0.02). As compared with nonclinical professional groups, clinical professional groups demonstrated significantly greater improvement in their self-reported ability to assess the quality of the pulmonary function test (i.e., clinical-skill item 4; estimate, 0.56, 95% CI, 0.08–1.04; P = 0.03; Table 3).

Table 2.

Change in self-efficacy items, separately calculated since the subject start date in the program and over a 12-month study timeframe

All Subjects	Score at 12-mo Study Endpoint		Change in Score since the Subject Start Date in the Program, Using the Retrospective Pre-Posttest Method			Change in Score since the Study Start Date Using the Traditional Pre-Posttest Method
	n	Mean ± SD of 12-mo Score	n	Mean ± SD Change in Score	P Value	n	Mean Change in Score SD of Change	P Value
Clinical skills
Ability to diagnose common health conditions in miners	43	4.35 ± 1.63	43	0.47 ± 0.91	<0.001	30	0.47 ± 1.07	0.02
Ability to help manage common health conditions in miners	44	4.66 ± 1.58	44	0.61 ± 1.04	<0.001	34	0.56 ± 1.13	0.01
Ability to interpret pulmonary-function test results	46	4.20 ± 1.98	45	−0.38 ± 0.72	0.001	36	−0.31 ± 0.86	0.04
Ability to assess the quality of the pulmonary-function test	46	4.43 ± 2.04	46	0.41 ± 0.78	0.001	36	0.14 ± 1.02	0.36
Ability to interpret arterial blood-gas test results	47	4.55 ± 2.01	46	0.22 ± 0.63	0.04	37	0.03 ± 0.90	0.88
Ability to interpret B-read reports of chest radiographs	47	3.62 ± 1.84	47	0.30 ± 1.02	0.04	36	0.33 ± 1.59	0.17
Average 6-item clinical-skills-domain score	47	4.31 ± 1.59	47	0.27 ± 0.50	<0.001	37	0.20 ± 0.68	0.07
Medicolegal skills
Ability to determine eligibility for compensation under specific miners’ compensation programs	46	4.43 ± 1.70	44	0.39 ± 1.24	0.049	35	0.63 ± 1.63	0.01
Ability to advocate for your patient/client to help them navigate the compensation process	45	4.53 ± 1.60	45	0.40 ± 1.30	0.043	33	0.24 ± 1.32	0.27
Ability to collect information required under the miners' compensation programs	46	4.52 ± 1.67	44	0.30 ± 1.30	0.16	36	−0.14 ± 1.53	0.84
Average 3-item medicolegal-skills-domain score	47	4.49 ± 1.58	47	0.37 ± 1.17	0.02	36	0.24 ± 1.33	0.14
Soft skills
Ability to demonstrate empathy toward miners with work-related diseases	47	5.53 ± 1.28	45	0.16 ± 1.11	0.35	37	−0.35 ± 1.18	0.11
Ability to identify social, linguistic, cultural, economic, and educational barriers to care for miners	47	4.89 ± 1.20	47	0.43 ± 1.04	0.01	36	−0.44 ± 1.50	0.09
Ability to serve as the miners’ expert in your community/region	46	4.48 ± 1.71	46	0.48 ± 1.30	0.01	34	0.38 ± 1.44	0.15
Ability to collaborate with and educate other team members about miners’ diseases	47	4.87 ± 1.56	47	0.45 ± 1.08	0.001	37	0.30 ± 1.41	0.14
Ability to refer patients with diseases to appropriate experts when you do not possess the relevant expertise	47	4.87 ± 1.45	47	0.23 ± 1.25	0.29	35	−0.20 ± 1.37	0.45
Average 5-item soft-skills-domain score	47	4.94 ± 1.24	47	0.34 ± 0.89	0.001	37	−0.07 ± 0.96	0.40
Average 14-item total score	47	4.59 ± 1.29	47	0.31 ± 0.64	<0.001	37	0.10 ± 0.73	0.27

Definition of abbreviation: SD = standard deviation.

The change in self-efficacy items, separately calculated since the subject start date in the program, was determined using the retrospective pre-posttest method with both pretest and posttest data obtained at the 12-month study time point. The change in self-efficacy items over a 12-month study timeframe was calculated using the traditional pre-posttest method, with pretest and posttest data obtained at the 0- and 12-month study time points, respectively. Improved item scores are indicated by positive change scores, and negative change scores indicate a decline in item scores. Absolute values for effect sizes for statistically significant differences ranged from 0.31 to 0.59, indicating only small-to-medium clinically meaningful changes (34). The data for the change in scores for those participating for both types of testing strategies, limited to 37 subjects who had both 0- and 12-month data measured, showed similar results, as provided in Table E5.

Figure 2.

Self-efficacy items at baseline (pretest), estimated retrospectively (retrospective pretest), and at 12 months (posttest). Error bars are 95% confidence intervals. The symbols inside the left graph margin indicate results from nonparametric Wilcoxon signed rank tests for change measured retrospectively (retrospective pre-posttest; n = 47) and from baseline (traditional pre-posttest; n = 37) using paired measurements (-P ≥ 0.05, *P < 0.05, **P < 0.01, and ***P < 0.001). Raw data for this figure are provided in Table 2. “Diagnose” indicates the ability to diagnose common health conditions in miners; “manage” indicates the ability to help manage common health conditions in miners; “interpret PF tests” indicates the ability to interpret pulmonary function (PF) test results; “assess PF test quality” indicates the ability to assess the quality of the PF test; “interpret BG tests” indicates the ability to interpret arterial blood-gas (BG) test results; “interpret X-rays” indicates the ability to interpret chest radiograph reports using the International Classification of Radiographs of Pneumoconiosis (i.e., B reads); “determine eligibility” indicates the ability to determine eligibility for compensation under specific miners’ compensation programs; “advocate/navigate” indicates the ability to advocate for your patient/client to help them navigate the compensation process; “collect information” indicates the ability to collect information required under the miners’ compensation programs; “demonstrate empathy” indicates the ability to demonstrate empathy toward miners with work-related diseases; “identify barriers” indicates the ability to identify social, linguistic, cultural, economic, and educational barriers to care for miners; “community/regional expert” indicates the ability to serve as the miners’ expert in your community/region; “collaborate” indicates the ability to collaborate with and educate other team members about miners’ diseases; “make referrals” indicates the ability to refer patients with diseases to appropriate experts when you do not possess the relevant expertise; “Avg clinical skills” indicates the average (Avg) 6-item clinical-skills-domain score (diagnose, manage, interpret PF tests, interpret BG tests, and interpret X-rays); “Avg medicolegal skills” indicates the Avg 3-item medicolegal-skills-domain score (determine eligibility, advocate/navigate, and collect information); “Avg soft skills” indicates the Avg 5-item soft-skills-domain score (demonstrate empathy, identify barriers, community/regional expert, collaborate, and make referrals); and “Avg total score” indicates the Avg 14-item total score.

Table 3.

Subgroup analysis of change in self-efficacy items since the subject start date in the program (using the retrospective pre-posttest method with both pretest and posttest data obtained at 12-mo study time point)

Change in (Item, Domain, or Total) Score	Fresh Participants			Existing Participants			Clinical Professional Groups			Nonclinical Professional Groups
	n	Mean ± SD	P Value	n	Mean ± SD	P Value	n	Mean ± SD	P Value	n	Mean ± SD	P Value
Clinical skills
Ability to diagnose common health conditions in miners	20	0.05 ± 0.69	0.75	23	0.83 ± 0.94	<0.001	33	0.52 ± 0.94	0.004	10	0.30 ± 0.82	0.28
Ability to help manage common health conditions in miners	22	0.32 ± 0.72	0.05	22	0.91 ± 1.23	0.002	34	0.65 ± 1.01	<0.001	10	0.50 ± 1.18	0.21
Ability to interpret pulmonary-function test results	22	−0.32 ± 0.78	0.07	23	−0.43 ± 0.66	0.01	33	−0.48 ± 0.76	<0.001	12	−0.08 ± 0.51	0.59
Ability to assess the quality of the pulmonary-function test	22	0.41 ± 0.67	0.01	24	0.42 ± 0.88	0.03	34	0.56 ± 0.79	<0.001	12	0.00 ± 0.60	>0.99
Ability to interpret arterial blood-gas test results	22	0.09 ± 0.68	0.54	24	0.33 ± 0.56	0.01	34	0.29 ± 0.72	0.02	12	0.00 ± 0.00	>0.99
Ability to interpret B-read reports of chest radiographs	22	0.00 ± 1.02	>0.99	25	0.56 ± 0.96	0.01	35	0.31 ± 1.08	0.09	12	0.25 ± 0.87	0.34
Average 6-item clinical-skills-domain score	22	0.09 ± 0.33	0.20	25	0.42 ± 0.57	0.001	35	0.31 ± 0.53	0.002	12	0.13 ± 0.35	0.22
Medicolegal skills
Ability to determine eligibility for compensation under specific miners' compensation programs	21	0.14 ± 1.11	0.56	23	0.61 ± 1.34	0.04	33	0.36 ± 1.27	0.11	11	0.45 ± 1.21	0.24
Ability to advocate for your patient/client to help them navigate the compensation process	21	0.24 ± 1.04	0.31	24	0.54 ± 1.50	0.09	33	0.36 ± 1.39	0.14	12	0.50 ± 1.09	0.14
Ability to collect information required under the miners’ compensation programs	21	0.33 ± 1.24	0.23	23	0.26 ± 1.39	0.38	32	0.28 ± 1.40	0.26	12	0.33 ± 1.07	0.31
Average 3-item medicolegal-skills-domain score	22	0.23 ± 1.05	0.32	25	0.49 ± 1.27	0.06	35	0.35 ± 1.22	0.1	12	0.42 ± 1.06	0.20
Soft skills
Ability to demonstrate empathy toward miners with work-related diseases	21	0.19 ± 0.51	0.10	24	0.13 ± 1.45	0.68	34	0.15 ± 1.16	0.46	11	0.18 ± 0.98	0.55
Ability to identify social, linguistic, cultural, economic, and educational barriers to care for miners	22	0.41 ± 0.85	0.04	25	0.44 ± 1.19	0.08	35	0.31 ± 1.05	0.09	12	0.75 ± 0.97	0.02
Ability to serve as the miners’ expert in your community/region	22	0.50 ± 1.01	0.03	24	0.46 ± 1.53	0.16	34	0.41 ± 1.44	0.1	12	0.67 ± 0.78	0.01
Ability to collaborate with and educate other team members about miners’ diseases	22	0.41 ± 1.05	0.08	25	0.48 ± 1.12	0.04	35	0.43 ± 1.20	0.04	12	0.50 ± 0.67	0.03
Ability to refer patients with diseases to appropriate experts when you do not possess the relevant expertise	22	−0.05 ± 0.90	0.82	25	0.48 ± 1.48	0.12	35	0.11 ± 1.35	0.62	12	0.58 ± 0.90	0.046
Average 5-item soft-skills-domain score	22	0.29 ± 0.67	0.05	25	0.38 ± 1.07	0.09	35	0.27 ± 0.98	0.11	12	0.53 ± 0.55	0.01
Average 14-item total score	22	0.19 ± 0.56	0.12	25	0.41 ± 0.70	0.01	35	0.30 ± 0.69	0.01	12	0.34 ± 0.51	0.04

For definition of abbreviation, see Table 2.

Positive change scores indicate improved item scores, and negative change scores indicate a decline in item scores.

In the alternate analysis of 37 subjects, using the traditional pre-posttest method, with pretest and posttest data obtained at the 0- and 12-month study time points, respectively, a significant improvement in 3 of 14 items (P < 0.05) and a decline in 1 item (with respect to their ability to interpret pulmonary function test results; P = 0.04) was noted over the 12-month study timeframe (Table 2). Subjects showed no significant improvement in any of the three domain scores or their total score over this timeframe. As compared with the nonclinical professional groups, clinical professional groups demonstrated significantly greater improvement in their self-reported ability to determine eligibility for compensation (i.e., medicolegal-skill item 1; estimate, 1.67; 95% CI, 0.25–3.08; P = 0.03), their self-reported ability to advocate for the patient/client to help them navigate the compensation process (i.e., medicolegal-skill item 2; estimate, 1.23; 95% CI, 0.06–2.40; P = 0.045), their medicolegal-skill domain score (estimate, 1.35; 95% CI, 0.29–2.41; P = 0.02), and their total score (estimate, 0.62; 95% CI, 0.06–1.18; P = 0.04; Table 4).

Table 4.

Subgroup analysis of change in self-efficacy items over a 12-month study timeframe (using the traditional pre-posttest data method with pretest and posttest data obtained at 0- and 12-mo study time points, respectively)

Change in Item	Fresh Participants			Existing Participants			Clinical Professional Groups			Nonclinical Professional Groups
	n	Mean ± SD	P Value	n	Mean ± SD	P Value	n	Mean ± SD	P Value	n	Mean ± SD	P Value
Clinical skills
Ability to diagnose common health conditions in miners	13	0.23 ± 1.24	0.51	17	0.65 ± 0.93	0.01	27	0.44 ± 1.12	0.05	3	0.67 ± 0.58	0.18
Ability to help manage common health conditions in miners	17	0.53 ± 1.18	0.08	17	0.59 ± 1.12	0.046	29	0.69 ± 1.07	0.002	5	−0.20 ± 1.30	0.75
Ability to interpret pulmonary-function test results	17	−0.41 ± 0.94	0.09	19	−0.21 ± 0.79	0.26	29	−0.31 ± 0.93	0.08	7	−0.29 ± 0.49	0.17
Ability to assess the quality of the pulmonary-function test	17	0.12 ± 1.05	0.65	19	0.16 ± 1.01	0.51	29	0.14 ± 0.99	0.46	7	0.14 ± 1.21	0.77
Ability to interpret arterial-blood-gas test results	17	0.00 ± 0.87	>0.99	20	0.05 ± 0.94	0.82	30	0.13 ± 0.90	0.42	7	−0.43 ± 0.79	0.20
Ability to interpret B-read reports of chest radiographs	16	−0.06 ± 1.77	0.89	20	0.65 ± 1.39	0.05	29	0.48 ± 1.70	0.14	7	−0.29 ± 0.76	0.36
Average 6-item clinical-skills-domain score	17	0.07 ± 0.63	0.65	20	0.31 ± 0.71	0.07	30	0.28 ± 0.69	0.03	7	−0.16 ± 0.55	0.48
Medicolegal skills
Ability to determine eligibility for compensation under specific miners’ compensation programs	15	0.93 ± 1.39	0.02	20	0.40 ± 1.79	0.33	30	0.87 ± 1.36	0.002	5	−0.80 ± 2.49	0.51
Ability to advocate for your patient/client to help them navigate the compensation process	15	0.47 ± 1.46	0.24	18	0.06 ± 1.21	0.85	28	0.43 ± 1.26	0.08	5	−0.80 ± 1.30	0.24
Ability to collect information required under the miners' compensation programs	16	0.00 ± 1.55	>0.99	20	−0.25 ± 1.55	0.48	30	0.07 ± 1.34	0.79	6	−1.17 ± 2.14	0.24
Average 3-item medicolegal-skills-domain score	16	0.45 ± 1.29	0.19	20	0.07 ± 1.36	0.83	30	0.46 ± 1.12	0.03	6	−0.89 ± 1.80	0.28
Soft skills
Ability to demonstrate empathy toward miners with work-related diseases	17	−0.06 ± 1.34	0.86	20	−0.60 ± 0.99	0.01	30	−0.20 ± 0.92	0.25	7	−1.00 ± 1.91	0.22
Ability to identify social, linguistic, cultural, economic, and educational barriers to care for miners	17	−0.59 ± 1.73	0.18	19	−0.32 ± 1.29	0.30	30	−0.27 ± 1.41	0.31	6	−1.33 ± 1.75	0.12
Ability to serve as the miners' expert in your community/region	16	0.75 ± 1.53	0.07	18	0.06 ± 1.30	0.86	29	0.55 ± 1.38	0.040	5	−0.60 ± 1.52	0.43
Ability to collaborate with and educate other team members about miners’ diseases	17	0.41 ± 1.23	0.19	20	0.20 ± 1.58	0.58	30	0.27 ± 1.34	0.28	7	0.43 ± 1.81	0.56
Ability to refer patients with diseases to appropriate experts when you do not possess the relevant expertise	16	−0.44 ± 1.59	0.29	19	0.00 ± 1.15	>0.99	30	−0.20 ± 1.45	0.46	5	−0.20 ± 0.84	0.62
Average 5-item soft-skills-domain score	17	0.00 ± 1.14	0.99	20	−0.12 ± 0.80	0.51	30	0.02 ± 0.89	0.89	7	−0.45 ± 1.22	0.36
Average 14-item total score	17	0.12 ± 0.76	0.52	20	0.08 ± 0.73	0.61	30	0.22 ± 0.65	0.08	7	−0.40 ± 0.89	0.28

For definition of abbreviation, see Table 2.

Positive change scores indicate improved item scores, and negative change scores indicate a decline in item scores.

Discussion

Telementoring multidisciplinary professional groups in the care of miners improved participants’ self-efficacy with respect to clinical, medicolegal, and soft skills. The retrospective pre-posttest method used since the participant start date with the program demonstrated improvement in a greater number of self-efficacy items and aggregate scores than the traditional pretest and posttest method over a 12-month study period. Existing participants and clinical professional groups demonstrated greater improvement in selected items than fresh participants and nonclinical professional groups, respectively. Our study demonstrates that the ECHO model can be successfully applied to professionals providing complex multidisciplinary care to miners.

Mining is a key economic activity in the rural United States, with potential for health hazards. With the United States having 28% of the world’s recoverable coal reserves, mining employment in the United States is likely to remain significant for many years, despite recent mine closures (20). It is anticipated that pneumoconiosis will continue to be seen in U.S. miners for several decades because of the long latency period of the disease. Miners constitute an underserved, isolated, medically vulnerable, and often underinsured rural population (2). In the southwestern United States, miners are predominantly Hispanic or American Indian high-school dropouts (2). The key challenges for caring for miners are the lack of adequately trained rural providers, perceived isolation among existing rural providers, and the complexity of the miners’ compensation systems (based on our preprogram launch needs assessment, A. Sood and colleagues, unpublished results). Addressing these challenges is a first step in fighting the emerging epidemic of pneumoconiosis but requires access to multidisciplinary expertise and training. The ECHO model leverages the power of information technology and case-based learning to provide state-of-the-art training and mentorship for such professionals in underserved areas.

The Miners’ Wellness TeleECHO Program has successfully recruited stakeholder groups from mining communities in pneumoconiosis-mortality hot-spot regions of rural Appalachia and the Mountain West. The target audience for the TeleECHO Program includes both clinical and nonclinical professionals who diagnose, treat, manage, and assist miners with pneumoconiosis. Evidence from our needs assessment demonstrates that treating pneumoconiosis is a complex process involving a difficult-to-navigate miners’ compensation system in the United States. Our multidisciplinary telementoring approach is essential to enable both clinical and nonclinical professionals to collaboratively develop comprehensive plans for providing the best possible care to miners with pneumoconiosis.

We found different results using our two analytic strategies: retrospective pre-posttest assessment since participant start date with the program versus traditional pretest and posttest assessment over a 12-month study period. The effect sizes tended to be greater, and changes were more likely to be statistically significantly different, using the former versus the latter approach, as also described by prior studies of ECHO outcomes (18, 19). By presenting the pre- and postassessments simultaneously in the retrospective pre–post design, respondents are forced to rate their status from the same frame of reference, which can help control for “response-shift bias” (21). This bias occurs when respondents use one frame of reference at the pretest but use a different frame of reference at the posttest, rendering it difficult to compare responses and resulting in erroneous inferences about program effectiveness. On the other hand, because the retrospective pre–post design queries participants “after” they have received the intervention, the participants may intentionally misrepresent the degree of impact they received from the intervention. Furthermore, given the possibility of recall bias in a retrospective pre–post design, this study design may replace one set of biases (vis-à-vis the traditional pre-posttest study design) with another (22).

Despite improvement in multiple self-efficacy parameters, our analysis indicates a decline in self-efficacy with respect to interpreting pulmonary function tests. This may relate to the confusion among rural professionals from multiple pulmonary-function-test updates by professional organizations since 2017 (3, 23–26). This also implies a need for greater attention in the future to this competency gap identified by our telementoring program. The fact that existing participants show greater improvement than fresh participants indicates that there may be a dose effect of telementoring on change in self-efficacy. This conclusion is supported by the finding that additional adjustment for participation frequency reduced the magnitude of differences between fresh and existing participants (see Figure E1 and Table E6). In addition, greater improvement in outcomes among participants belonging to the clinical versus nonclinical professional groups may reflect the disproportionate emphasis of clinical didactics and case discussions in the program (see Table E3). This implies a need for greater attention to nonclinical topics by our multidisciplinary program.

A strength of the current study is that our ECHO intervention is located in a rural and poor but mining-intense state, designated by the U.S. Health Resources and Services Administration as a medically underserved area. We believe that this increases the generalizability of the study findings to other rural mining regions. In addition, we tested the feasibility of applying a model of healthcare delivery that is already well established in other diseases, such as hepatitis C, by reducing variation in processes of care and sharing best practices (12–15). It is likely that this model of healthcare delivery may be similarly useful in the management of other lung diseases requiring multidisciplinary discussion, such as interstitial lung diseases (27). Because the ECHO model has been adopted nationally and globally to improve access to care in a number of disciplines, there is substantial infrastructure already in place to allow for rapid scaling of the Miners’ Wellness TeleECHO model. This telementoring approach was particularly beneficial during the coronavirus disease (COVID-19) pandemic, during which multiple face-to-face professional educational programs were canceled, resulting in its recognition as a rural COVID-19 innovation by the U.S. Health Resources and Services Administration’s Rural Health Information Hub. The American Thoracic Society also recognized the program as a 2019 innovation in fellowship education.

There are also limitations to the current study. We are unable to correlate improvement in professional self-efficacy with patient outcomes. The literature, however, indicates that professional self-efficacy influences behavior or performance and improves patient outcomes (28). For example, self-efficacy is a good predictor of asthma-guideline adherence among primary care clinicians (29) and of clinical performance of student physician assistants (30). Investigators report positive findings using randomized controlled designs to test the efficacy of electronic-learning interventions on novice surgeons’ surgical performance of laparoscopic cholecystectomy (31) and nurses’ ability to detect child abuse in an emergency department (32), respectively. A listing of qualitative changes that our ECHO participants reported they were going to make in their practice, obtained as part of a CME survey requested at the end of each TeleECHO session, are provided in Table E2 (with other quantitative outcomes provided in Table E1). A small sample size and multiple comparisons in the current study raise the possibility of type I error. High-risk individuals who drop out of the program or do not volunteer to participate in the study do not provide information in the estimation of the program effects, thus introducing an element of potential participation bias. The self-efficacy instrument was not fully validated in the current study. Standardized Cronbach α-coefficient values for the instrument, computed from the baseline measurement, however, showed high levels of internal consistency (values ≥ 0.85 for all items and each of the three domains).

This study is the first in a stepwise approach to determine the benefit of participating in a multidisciplinary telementoring intervention by improving participant self-efficacy in delivering high-quality care to miners with complex mining-related diseases. The acknowledgment of provider self-efficacy as a common barrier to optimal care of complex diseases (33) and as a predictor for provider behavior or performance and for patient outcomes (28) in the literature underscores the relevance of employing the TeleECHO intervention in professions caring for miners in medically underserved regions. Although future studies are required to determine any improvement in participant behavior or performance or miner outcomes, our study finding represents a potential solution to a growing access-to-care gap for miners with pneumoconiosis. This provides the rationale for using systems that are already in place for rapid scaling of the Miners’ Wellness TeleECHO Program at other institutions nationally and globally.