Evaluating a Muscle Ultrasound Education Program: Theoretical Knowledge, Hands-on Skills, Reliability, and Satisfaction of Critical Care Physiotherapists.

OBJECTIVE: To evaluate learning results of critical care physiotherapists participating in a muscle ultrasound (MUS) educational program.
DESIGN: Cross-sectional study.
SETTING: A custom-made 20-hour MUS course was performed over a 2-week time period, including knobs familiarization, patient positioning, anatomic landmarks, image acquisition, and limb muscle measurements. PARTICIPANTS: Nineteen critical care physiotherapists with little to no prior experience in ultrasound (N=19).
INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Theoretical knowledge, hands-on skills acquisition, and satisfaction were assessed. Inter- and intrarater reliability on landmarks, thickness, and pennation angle of quadriceps between participants was evaluated using intraclass correlation coefficients (ICCs). Reliability among instructors measured prior to the course was also reported as a reference.
RESULTS: The percentage score (mean±SD) of knowledge questionnaires was 69±11 (pre-course), 89±10 (post-course), and 92±9 (hands-on skills). Course satisfaction scores ranged from 90%-100%. Pooled interrater reliability of participants (median ICC [interquartile range]) was good (0.70 [0.59-0.79]) for thickness, moderate (0.47 [0.46-0.92]) for landmarks, and absent (0.00 [0.00-0.05]) for pennation angle and the intrarater reliability was good (0.76 [0.51-0.91]) for thickness and weak (0.35 [0.29-0.52]) for pennation angle. Interrater ICC values for instructors were excellent (0.90) for thickness, good (0.67) for landmarks, and moderate (0.41) for pennation angle and intrarater ICC values were excellent (0.94) for thickness and good (0.75) for pennation angle.
CONCLUSIONS: Although our sample was quite small and homogeneous, increased theoretical knowledge, high hands-on performance acquisition, and good satisfaction of physiotherapists were observed. Reliability was moderate to excellent for thickness and landmarks and absent to weak for pennation angle. Landmarking and pennation angle remain challenges for physiotherapist training in the application of MUS. Further studies are needed to identify variables that could modify reliability during MUS training.

Muscle wasting occurs during a variety of disease and illness states, including in patients with critical illness admitted to the intensive care unit (ICU).1, 2, 3 Studies using movement sensor technology have reported that critically ill patients remain inactive for 92%-98% of their ICU stay.4, 5, 6, 7 Immobility and disuse are primary reasons why patients in ICU suffer rapid and early muscle wasting in rectus femoris muscle size during the first 10 days of an ICU admission.1, 2, 3 Outcome measures that allow early identification of musculoskeletal and physical dysfunction are increasingly used in critically ill patients.8, 9, 10 However, most outcome measures are volitional in nature, requiring the patient to be alert and cooperative with testing, which often results in a delay in identifying those individuals at highest risk of musculoskeletal and physical dysfunction.

In recent years, muscle ultrasound (MUS) has gained traction as a potential tool that can be used early during critical illness without the need for volitional patient effort to assess changes in skeletal muscle. MUS can be performed at the bedside and is noninvasive and readily available, showing high clinical and research utility. MUS can be used to detect changes in the trajectory of muscle mass quality and quantity, and, when combined with physical functioning measures, may enable tracking of progress and evaluation of the effectiveness of rehabilitation interventions for critically ill patients., MUS measurements have robust clinimetric properties in critically ill patients,,14, 15, 16 including excellent validity and reliability when standardized training has been performed.15, 16, 17, 18 MUS allows the evaluation of muscle quantity (muscle thickness, cross-sectional area) and quality/biomechanical properties (fiber pennation angle, fascicle length, echogenicity)., Good to excellent reliability has been reported for measurement of muscle thickness, cross-sectional area, and echogenicity within critically ill patients.,,

However, reliability of ultrasonographer landmarking and measurement of fiber pennation angle has not been reported for patients with critical illness. Patient setup including standardized postioning and the correct anatomic landmarks are vital components of acquiring an accurate ultrasound image. This is particularly important for skeletal muscle because small variations in setup and landmarking may lead to significant differences in anatomic location and affect the acquisition process. Thus, standardized patient positioning and identification of landmarks are essential to minimize differences between evaluators and when assessing for change over time.

Owing to operator dependence of MUS, international recommendations suggest that the accurate use of ultrasound requires standardized training combining theoretical knowledge and hands-on practical skills.20, 21, 22 There are 8 international ultrasound training programs for the ICU setting primarily focused on physicians and radiologist, not including ultrasonography for physiotherapists. Ultrasound training for acute care physiotherapists has been previously conducted;, however, formal training on peripheral skeletal muscle mass assessment does not exist in Chile. For this reason, there is scarce evidence on learning outcomes of physiotherapists receiving MUS training that can be applied within the ICU setting.

Accordingly, the aim of this study is to evaluate the change in pre- and post-course theoretical knowledge, practical skills, and satisfaction of ICU physiotherapists during a custom-made MUS education program focused on patients in the ICU and to determine the interrater and intrarater reliability measurement of landmarks, muscle thickness, and fiber pennation angle.

Methods

Study design

A cross-sectional observational study was conducted to evaluate the results of the first Chilean MUS education program focused on patients in the ICU (eMUSICS: Education in Muscle Ultrasound for Intensive Care Setting) performed between August 19 and 31, 2019, in the faculty of medicine of a Chilean private university, which has a structured continuing education platform. This study received review and approval after course development by the research ethics committee of Universidad del Desarrollo (Registration No. 2020-106), and informed consent was waived.

Instructors

Five critical care physiotherapists (A.C.M., J.J.P., F.R.C., A.S.G., F.G.S.) with more than 2 years of ultrasound clinical experience led the eMUSICS program. Instructors had received formal training from national or international ultrasound courses of at least 8 hours. Two months before the course, all instructors carried out two 3-hour training meetings among themselves to standardize the methodology of image acquisition and measurement. The aim of this study was to evaluate the participants; however, the intra- and interrater reliability of instructors performed before course initiation is presented in the Results section for transparency. Reliability measurement procedure of instructors is presented in table S1 (available online only at http://www.archives-pmr.org/).

Program description

A 20-hour MUS education program was developed based on international training recommendations.20, 21, 22, Selected content and materials from an ultrasound post-congress workshop developed in the 2017 World Confederation of Physical Therapy Conference were used in this program with prior authorization (S.M.P.), involving ultrasound physics, knobology (familiarization with the ultrasound knobs), muscle anatomy and physiology, patient positioning, landmarks, image acquisition, and lower limb measurements of muscle quality and quantity. The course was designed for rehabilitation clinicians with at least 1 year of working experience with patients in the ICU. The ultrasound training course was delivered in 2 parts: the first was online learning (eLearning) and the second, in-person learning including a combination of lectures and hands-on practice (table S2, available online only at http://www.archives-pmr.org/, provides a detailed program curriculum).

eLearning training and assessment

The eLearning was available for participants 2 weeks before the in-person course using the Moodlea online platform with an estimated dedication time of 6 hours. The objectives of the eLearning were to identify the normal anatomic structures with real ultrasound images and to understand initial concepts of muscle structure measurements using ultrasound. The online platform included recommended pre-reading material, ultrasound machines technical manuals, an instructor-led discussion forum to answer questions, and 4 training videos (for more details on training videos see table S3, available online only at http://www.archives-pmr.org/). To guarantee participants’ knowledge before the in-person course, a formative questionnaire with 20 open-ended questions based on the observation of normal MUS images was performed, which was answered using the prereading material provided. Participants submitted their answers until 1 day before the in-person course using the online platform.

In-person course

After the eLearning, the in-person course was performed during 2 consecutive days including 5 hours of didactic lectures and 9 hours of hands-on training led by 5 trained instructors. Lectures were conducted with a projector/screen using standard PowerPointb presentations or real-time ultrasound muscle scanning of a participating volunteer. During hands-on training, 10 ultrasound machines were available, including 9 wireless devices (Philips Lumifyc, Sonus SL-2Cd, Sonus DUO LCPd), each connected to an iPade, and 1 portable device (Philips InnoSightc, including linear and curvilinear array transducer). To optimize participant learning, a maximum of 20 participants was defined to achieve an instructor-to-trainee ratio of 1:4 and ultrasound machine–to-trainee ratio of 1:2. Knobology, patient positioning, landmarks, image acquisition, muscle thickness, and fiber pennation angle measurements were studied through lectures and hands-on training. The hands-on training was designed for each participant to perform 15 landmarks identifications and 25 supervised muscle scans including all ultrasound parameters involved in this study using the quadriceps measurements of the same participants.

Participant assessments

Theoretical knowledge was evaluated using a pre-course diagnostic questionnaire at the beginning of the in-person course. At the end of the course, theoretical knowledge was reevaluated through a post-course formative questionnaire using the same questions in a randomly assigned order. Both questionnaires included 25 multiple-choice questions, of which 11 were obtained from the questionnaire of Ntoumenopoulos et al. Hands-on assessment was performed during the last 3 hours of the course to assess the practical skills of participants. Owing to class size and timing convenience, participants were split into groups to perform the hands-on assessment and interrater reliability. Using the Intemodino RNGf as random number generator software, participants were distributed into 5 groups of 4 people. The participants were blind to each other's measurements to assess quadriceps landmarks, rectus femoris (RF), vastus intermedius (VI), quadriceps complex (QC) thickness, and/or vastus lateralis (VL) pennation angle in 6 healthy individuals. Each instructor qualitatively evaluated 4 participants through direct observation using a non-middle answer category Likert scale from 1 to 4 points (1=poor, 2=fair, 3=good, 4=excellent). This scale was used to evaluate the performance of the following items: patient positioning, landmarks identification, knobology operation (accurate use of gain compensation, zoom, focus, depth, freeze function, caliper), image acquisition (anatomy identification), transducer placement, and quadriceps measurements. Participants measured the landmarks and muscle thickness in centimeters and pennation angle in degrees on a prespecified written record document. Participants were previously trained to capture all ultrasound images directly on the ultrasound machine, and subsequently instructors exported without any adjustments to a computer for analysis using a data storage device. This data were used to evaluate inter- and intrarater reliability of participants according to the assigned group. Reliability measurement procedure of participant is presented in table S1 (available online only at http://www.archives-pmr.org/) and was informed following the COnsensus-based Standards for the selection of health Measurement INstruments standards to assess the quality of studies on reliability and measurement error.

Course measurement protocol of landmarks, muscle thickness, and pennation angle

Participants were examined in the supine position with neutral rotation and passive extension of lower limb. Anatomical landmarks were marked using an erasable skin marker and measured in centimeters using a flexible tape measure. Quadriceps landmark was defined as the midpoint between the anterior superior iliac spine and the superior patella border., Tibialis anterior landmark was defined as one-third of the distance from the tibial plateau to the inferior border of lateral malleolus., Depending on specific thigh size, a B-mode with linear (4-12 MHz) and curvilinear (5-2 MHz) array transducers was used. When required, participants adjusted gain compensation, zoom, focus, depth, and freeze function. A generous amount of contact gel was used to minimize the required pressure of the transducer on the skin, allowing the minimal compression technique. Scans were performed with the transducer in neutral tilt using a transverse cross-sectional view for the muscle thickness and a sagittal view for pennation angle. The transverse cross-sectional view was acquired to measure RF, VI, and QC muscle thickness.30, 31, 32 Muscle thickness was reported in centimeters using the caliper of the ultrasound machine as the inside height measured between epimysial borders of each muscle (figure S1, available online only at http://www.archives-pmr.org/). To acquire an accurate sagittal view for VL pennation angle, the transducer was moved laterally 5 cm from the site where RF/VI was obtained. Pennation angle of the VL was reported in degrees as the angle between the direction of muscle fibers and force line action represented by external tendon or aponeurosis—that is, the vertical inclination of fibers from the long axis of muscle—using the average of 3 consecutive separate attempts (figure S2, available online only at http://www.archives-pmr.org/).

Course satisfaction

At the end of the second day of the in-person course, participants were asked to voluntarily answer a standardized anonymous satisfaction survey predesigned by the local university educational program and used for all courses related to medicine. This survey evaluates the participant's perception including the overall assessment of the course, academic scope, instructor quality, eLearning, prereading material, and course coordination. Each item was scored using a non-middle answer category Likert scale from 1-4 points (1=strongly disagree, 2=disagree, 3=agree, 4=strongly agree). Additionally, this survey included the following 2 yes-no questions: Would you recommend these instructors for a future course? and Would you recommend this program to other people?

Statistical analyses

Descriptive data were analyzed using STATA SE 15.0g. The normality of the data of each variable was analyzed with the Shapiro-Wilk test. Data were described as mean ± SD or median (interquartile range [IQR]), depending on the normality of the data. The limit of statistical significance was set at a 2-sided P value of ≤.05. The scores of knowledge and practical skills questionnaires and satisfaction survey were reported as percentage correct. Reliability of participants and instructors was calculated using the intraclass correlation coefficient (ICC) and 95% confidence interval according to Koo and Li. Repeated measurements by the same rater on the same subject were used to calculate intrarater reliability and included a brief period with the instructor removing previous test results. Repeated measurements by different raters on the same subject were used to calculate interrater reliability, and raters were blinded to the test and results of the other raters. Interrater reliability was calculated using the 2-way random effects, absolute agreement, and average measure of the number of records, and intrarater reliability was calculated using the 2-way mixed effects, absolute agreement, and single measure (see table S3, available online only at http://www.archives-pmr.org/). Group comparisons were not performed, because this was not the focus of the study. To obtain the overall interrater reliability of the 5 groups of participants, the median and IQR of the ICC values of each ultrasound parameter were calculated. The qualitative interpretation of the ICC was classified as 0.00 (absent), 0.00-0.19 (poor), 0.20-0.39 (weak), 0.40-0.59 (moderate), 0.60-0.79 (good), and ≥0.80 (excellent). The SEM was calculated for each ICC value as the product of SD and the square root (1−single measures ICC). The coefficient of variation (CV) was calculated by dividing the SD by the mean value of the repeated measures (CV=[SD/mean] × 100), interpreted as CV<10 (very good), 10-20 (good), 20-30 (acceptable), and CV>30 (not acceptable).,

Results

Participants

Nineteen physiotherapists were enrolled in the course (fig 1), and demographics of instructors and participants are provided in table 1. Only 1 (5.3%) had previously received specific MUS training and 11 (57.9%) had never participated in any ultrasound training prior to this workshop.

Fig 1

Flow diagram of participants during eMUSICS training program. LM, landmarks; MT, muscle thickness; PA, pennation angle.

Table 1

Characteristics of instructors and participants in the eMUSICS training program

Variables	Instructorsn=5	Participantsn=19
Age, y	31 [29-32.5]	32 [29-34]
Female	0 (0.0)	6 (31.5)
Years working in ICU	8 [4-8.5]	5 [4-9]
Type of hospital
Private	4 (80)	10 (52.6)
Public	1 (20)	9 (47.4)
Highest academic degree
Bachelor's	0 (0.0)	2 (10.5)
ICU diploma*	3 (60)	14 (73.7)
Master's	2 (40)	2 (10.5)
PhD	0 (0.0)	1 (5.3)
ICU PT or RT certification	3 (60)	1 (5.3)
Previous ultrasound training	5 (100)	7 (36.8)
Previous muscle ultrasound training	2 (40)	1 (5.3)

NOTE. Data are presented as median [interquartile range] or as n (%).

Abbreviations: PT, physiotherapy; RT, respiratory therapist.

Includes any 1-year postgraduate course related to intensive care.

eLearning assessment

The eLearning questionnaire (mean ± SD) showed 89.1%±7% percentage score when participants had the option to respond using the prereading material before the in-person course.

In-person course assessments

The percentage score (mean ± SD) of theoretical knowledge was 69.0%±11% and 88.9%±10% for the pre-course and post-course questionnaires, respectively. The percentage score (mean ± SD) of practical skills during hands-on assessment was 91.5%±9%, and the best and worst evaluated items were positioning of the patient (100%±0%) and quadriceps measurements using caliper or angle function (84.2±8), respectively (table 2). The results of the theoretical knowledge and practical skills of participants are presented in table 3.

Table 2

Practical skills evaluated during hands-on assessment (n=19)

Evaluated parameter	Likert Scale*				Percentage Score†
	Poor	Fair	Good	Excellent
Patient positioning	0 (0.0)	0 (0.0)	0 (0.0)	19 (100)	100±0
Landmarks	0 (0.0)	0 (0.0)	1 (5.3)	18 (94.7)	98.6±2
Knobology	0 (0.0)	2 (10.5)	5 (26.3)	12 (63.2)	88.1±7
Image acquisition	0 (0.0)	0 (0.0)	9 (47.4)	10 (52.6)	88.1±5
Transducer placement	0 (0.0)	1 (5.3)	8 (42.1)	10 (52.6)	86.8±6
Quadriceps measurements	1 (5.3)	1 (5.3)	7 (36.8)	10 (52.6)	84.2±8

NOTE. Data are presented as n (%) for the Likert scale and as mean ± SD for percentage score.

Practical skills were evaluated through subjective direct observation of one instructor using the following Likert scale: 1=poor (the skill is not fulfilled or appears full of imperfections throughout the process); 2=fair (the skill is partially accomplished, with numerous imperfections limiting the process); 3=good (the skill is fulfilled almost entirely, with some imperfections that can be corrected); 4=excellent (the skill is accomplished in an outstanding and sustained way throughout the process).

Percentage score was calculated by dividing the score obtained by the total score, multiplied by 100 (percentage correct).

Table 3

Percentage scores of theoretical knowledge and practical skills per group and overall during the eMUSICS training program*

Variables	Group 1(n=4) (%)	Group 2(n=4) (%)	Group 3(n=4) (%)	Group 4(n=4) (%)	Group 5(n=3) (%)	Overall(n=19) (%)
eLearning	86.0	93.3	91.5	87.0	87.3	89.1
Theoretical knowledge
Pre-course questionnaire	72.5	73.0	72.3	63.5	62.0	69.0
Post-course questionnaire	82.0	86.0	96.8	88.3	92.7	88.9
Post-pre difference	9.5	12.8	24.8	24.5	31.0	19.9
Practical skills (Likert scale)	88.8	81.0	100.0	93.0	96.0	91.5
Final course score	85.8	85.0	97.3	90.3	93.0	90.1

Percentage score was calculated by dividing the score obtained by the total score, multiplied by 100 (percentage correct).

Reliability measurements

The on-site reliability measurement process during practical evaluation is presented schematically in figure 2. The highest inter- and intrarater reliability of instructors was for the QC thickness, which was 0.90 and 0.94, respectively. The lowest inter- and intrarater ICC values of instructors were for VL pennation angle, of 0.41 and 0.75, respectively. The reliability of participants ranged widely according to the assigned group. All inter- and intrarater reliability results of instructors and participants are presented in table 4. Interrater reliability was moderate to excellent (median [IQR]) for VI thickness (0.87 [0.73-0.91]), QC thickness (0.70 [0.59-0.79]), landmarks (0.47 [0.46-0.92]), and RF thickness (0.41 [0.13-0.66]); and was absent for VL pennation angle (0.00 [0.00-0.05]). There was good to excellent intrarater reliability (median [IQR]) for RF thickness (0.84 [0.69-0.87]), VI thickness (0.78 [0.73-0.85]), and QC thickness (0.76 [0.51-0.91]) and only weak intrarater reliability for VL pennation angle (0.35 [0.29-0.52]). Overall, moderate to excellent reliability was obtained for muscle thickness and moderate for landmarks. Similar to instructors, the lowest inter- and intrarater reliability ICC values of participants were obtained for VL pennation angle, which ranged from 0.00-0.10 and 0.23-0.69, respectively.

Fig 2

Reliability measurement process during hands-on assessment. (A) Landmarking of a left quadriceps. (B) Identification of anatomical structures in a transverse cross-sectional view using a curvilinear transducer. (C) Measurement of quadriceps muscle thickness using the ultrasound calliper. (D) Measurement of pennation angle of vastus lateralis. (E) Transcription of measurement values from ultrasound machine to the prespecified written record document.

Table 4

Inter- and intrarater reliability of instructors and participants in the eMUSICS training program

Group	Muscle Ultrasound Parameter	Mean ± SD	CV (%)	Interrater Reliability		Intrarater Reliability
				ICC (95% CI)*	SEM	ICC (95% CI)*	SEM
Instructors†	Landmarks (cm)	23.19±0.8	3.5	0.67 [0.48-0.87]	0.10	—	—
	RF thickness (cm)	2.21±0.3	14.5	0.83 [0.71-0.95]	0.06	0.93 [0.90-0.96]	0.02
	VI thickness (cm)	1.93±0.6	31.0	0.89 [0.81-0.97]	0.04	0.92 [0.88-0.95]	0.02
	QC thickness‡ (cm)	4.35±0.9	19.9	0.90 [0.83-0.98]	0.04	0.94 [0.92-0.97]	0.01
	VL pennation angle (degrees)	16.43±3.1	18.8	0.41 [0.13-0.70]	0.14	0.75 [0.66-0.85]	0.05
Group 1	Landmarks (cm)	25.12±4.2	16.6	0.12 [0.00-0.58]	0.23	—	—
	RF thickness (cm)	1.8±0.3	14.7	0.00 [0.00-0.00]	0.00	0.86 [0.78-0.93]	0.04
	VI thickness (cm)	1.94±0.3	16.8	0.41 [0.04-0.79]	0.19	0.75 [0.62-0.87]	0.06
	QC thickness‡ (cm)	3.94±0.6	14.6	0.39 [0.00-0.77]	0.20	0.89 [0.83-0.95]	0.03
	VL pennation angle (degrees)	15.28±4.6	30.3	0.00 [0.00-0.00]	0.00	0.69 [0.54-0.84]	0.08
Group 2	Landmarks (cm)	22.91±1.5	6.4	0.92 [0.84-0.99]	0.4	—	—
	RF thickness (cm)	1.98±0.2	8.8	0.17 [0.00-0.60]	0.2	0.34 [0.06-0.62]	0.14
	VI thickness (cm)	1.77±0.3	19.4	0.84 [0.70-0.98]	0.07	0.66 [0.49-0.82]	0.08
	QC thickness‡ (cm)	3.88±0.5	12.3	0.74 [0.53-0.95]	0.12	0.19 [0.00-0.52]	0.17
	VL pennation angle (degrees)	14.73±2.9	20.2	0.00 [0.00-0.00]	0.00	0.35 [0.07-0.63]	0.14
Group 3	Landmarks (cm)	23.66±1.4	5.8	0.96 [0.91-1.00]	0.02	—	—
	RF thickness (cm)	2.03±0.4	20.5	0.69 [0.45-0.94]	0.12	0.90 [0.84-0.95]	0.03
	VI thickness (cm)	1.67±0.6	36.1	0.95 [0.89-1.00]	0.03	0.96 [0.93-0.98]	0.01
	QC thickness‡ (cm)	3.83±1.0	26.4	0.95 [0.90-1.00]	0.02	0.96 [0.94-0.98]	0.01
Group 4	Landmarks (cm)	23.23±1.7	7.4	0.46 [0.09-0.82]	0.18	—	—
	RF thickness (cm)	2.12±0.3	14.9	0.65 [0.37-0.92]	0.14	0.81 [0.72-0.91]	0.05
	VI thickness (cm)	1.64±0.3	19.0	0.89 [0.79-0.99]	0.05	0.81 [0.71-0.90]	0.05
	QC thickness‡ (cm)	3.84±0.6	14.5	0.66 [0.40-0.92]	0.13	0.62 [0.46-0.78]	0.08
Group 5	Landmarks (cm)	23.77±1.1	4.8	0.47 [0.05-0.90]	0.21	—	—
	VL pennation angle (degrees)	19.55±2.2	11.2	0.10 [0.00-0.65]	0.28	0.23 [0.01-0.45]	0.11

Abbreviations: CI, confidence interval; SEM, standard error of measurement.

ICC values range from 0.00-1.00.

Four instructors evaluated their reliability 2 months before muscle ultrasound education program implementation.

Quadriceps complex thickness includes the rectus femoris and vastus intermedius thickness.

The mean percentage scores of the individual evaluated items in the satisfaction survey ranged from 90%±0.6% to 100%±0.0% on the Likert scale (table S4, available online only at http://www.archives-pmr.org/). The lowest evaluations were for the friendliness of the eLearning platform and for the prereading material. Additionally, when participants were asked whether they would recommend the course instructors, all 19 answered yes (100%). When the participants were asked whether they would recommend this course to other people, 16 answered yes (84%), 1 answered no (5.3%), and 2 left it blank (10.5%).

Discussion

This study reported the acquisition of theoretical knowledge and hands-on skills of 19 ICU physiotherapists with little to no prior experience in MUS who received training through a dedicated custom-made MUS education program. This is the first report of ICU physiotherapists evaluating the reliability of the quadriceps landmarks and pennation angle. Compared with the median reliability values of participants, the instructors had higher reliability in the measurement of all MUS parameters. Overall, the reliability of participating physiotherapists was moderate to excellent for muscle thickness, moderate for landmarks, and absent to poor for pennation angle. Instructors had excellent reliability for muscle thickness (interrater ICC ranged from 0.83-0.90; intrarater ICC ranged from 0.92-0.94), good for landmarks (interrater ICC: 0.67 [0.48-0.87]), and moderate for pennation angle (interrater ICC: 0.41; intrarater ICC: 0.75).

The eMUSICS training program in this study was developed based on available international recommendations,20, 21, 22, including online/digital media, hands-on sessions led by instructors, theoretical assessments, face-to-face practical assessments, and a logbook of scans performed by participants. The results of ultrasound learning for acute care physiotherapists have been published in only 2 articles., In a 1-day course dedicated to thoracic diagnostic ultrasound for physiotherapists there was a 13% increase on the post-pre training theoretical knowledge, and in other 1-day course dedicated to lung, diaphragm, and limb MUS there was no increase. The eMUSICS course included limb MUS exclusively, obtaining almost a 20% increase on the theoretical knowledge of participants. According to a systematic review of international ultrasound training competencies and programs, the first step in acquiring any new skill is assessing subsequent practical learning with theoretical knowledge as a foundation, but little practical assessment has been reported in published courses. Compared with a general critical care ultrasonography course, the eMUSICS program obtained a similar mean percentage practical score (86% vs 92%, respectively), as a result of supervised practical sessions. Although the number of measurements or scans in training courses is not yet standardized, critical care ultrasonography training of various national professional societies and organizations has reported that supervised and non-supervised numbers of scans ranged widely from 10-100. During the eMUSICS program, 25 supervised muscle scans, including all MUS parameters per participant, were achieved during training. By adding the muscle measurement performed during the whole practical assessment, at least 70 muscle scans per participant were performed during the course. This number of scans is consistent with non-physiotherapist ICU ultrasound training courses.

Ultrasound for the assessment of peripheral skeletal muscle architecture in critical illness has revealed excellent reliability levels, including parameters such as thickness, cross-sectional area, and echogenicity. In particular, the reliability of ultrasound measurement of quadriceps thickness in ICU patients has been widely reported to range from 0.76-1.00;15, 16, 17, 18,39, 40, 41, 42 notable, in the eMUSICS program it was consistently 0.70 for participants and 0.90 for instructors. Although reliability of pennation angle has not been reported for ICU patients, studies in non-ICU populations have reported weak to moderate reliability and high variability, ranging from 0.38-0.74.43, 44, 45 In the eMUSICS program, instructors and participants achieved an interrater reliability of VL pennation angle between 0.00 and 0.41, showing the lowest reliability values of this study. Pennation angle in this study had high interrater variability in healthy muscles, and thus we suggest it could be worse in patients in the ICU owing to loss of muscle echogenicity increasing the difficulty of visual interpretation of the muscle fibers.

Mourtzakis et al highlighted the importance of landmarking on training, standardization, reporting in articles, and reliability assessment. Not all ultrasound studies have reported the quadriceps landmarks used, which could modify the accuracy of MUS parameter depending on the measurement site. For example, Pardo et al reported an interrater reliability of quadriceps thickness of 0.76 using the midpoint site and 0.81 using the two-thirds site. In the eMUSICS program, the quadriceps landmarks were evaluated using the midpoint in centimeters and the interrater reliability was 0.47 and 0.67 for participants and instructors, respectively. Thus, ultrasound measurements could be modified by the selection of a specific landmark or by the accuracy of the evaluators to identify that landmark. More studies are needed to evaluate the reliability of landmarking because this could influence the consecutive measurements of thickness, cross-sectional area, echogenicity, and pennation angle.

Although inter- and intrarater reliability obtained by the instructors was consistent with previous reliability studies of muscle architecture,15, 16, 17, 18,39, 40, 41, 42 varied reliability was obtained between participants. Overall, the highest inter- and intrarater reliability was 0.96 and 0.96, respectively, and the lowest inter- and intrarater reliability was 0.00 and 0.19, respectively. The differences between groups on reliability results (see table 4) could be explained by the ultrasound devices used, the MUS parameters assessed, and the number of repeated measures used per group (see table S3, available online only at http://www.archives-pmr.org/). Another potential rationale for overall lower inter- and intrarater reliability may be the relative years of experience of our instructors, which is slightly lower than that of previously reported studies., Additionally, the learning performance was varied between groups. Although all participants received the same training and were randomized into 5 groups, the groups of participants with the lowest scores on the theoretical and practical assessments (groups 1 and 2) had the lowest reliability values, as presented in table 3 and table 4. Specifically, randomization concentrated the participants with the best scores in theoretical and practical assessments in group 3, which likely led to the best reliability values. Further studies should explore whether the level of knowledge acquisition during MUS training may modify the reliability of participants.

Study Limitations

This study has limitations that need to be mentioned. The reliability assessment was performed only on quadriceps muscles of healthy individuals, which does not guarantee that the participants would have the same reliability when evaluating patients in the ICU. This was done to facilitate the early learning using normal anatomy as a reference. The small sample sizes per group and the homogeneous sample limit the external validity of the reliability data. However, this study provides novel information on the reliability of landmarks and pennation angle. A longer washout period would strengthen the methodology on intrarater reliability; however, the timing in this study provides preliminary data supporting further analyses on intrarater reliability. Another limitation was the absence of certified expert instructors, which could have resulted in higher reliability measurement in the participants. To counter this, a meticulous training process of instructors was carried out to ensure accurate instruction during the in-person course. This study did not explore the durability of participant training and whether the acquired knowledge was translated into effective clinical practice to assess muscle mass in patients in the ICU during an ICU stay.

Conclusions

Although our sample was quite small and homogeneous, this novel MUS course focused on patients in the ICU demonstrated increasing on theoretical knowledge, high hands-on performance acquisition, and good satisfaction in a group of critical care physiotherapists with little to no prior experience in ultrasound. Despite the high theoretical results of the participants, this did not ensure high reliability in all ultrasound parameters. Overall, instructors and participants reached moderate to excellent inter- and intrarater reliability for quadriceps thickness, but reliability of landmarks and pennation angle remains challenging for physiotherapist training even for instructors. Further studies are needed to identify the variables that could modify the reliability results during MUS training programs.

Suppliers

Moodle; Moodle HQ.

PowerPoint; Microsoft Corp.

Philips Lumify, Philips InnoSight; Philips N.V.

Sonus SL-2C, Sonus DUO LCP; Sonus.

iPad; Apple Inc.

Intemodino RNG; Intemodino Group s.r.o.

STATA SE 15.0; StataCorp, LLC.