Reliability and Validity of the Modified Heckmatt Scale in Evaluating Muscle Changes With Ultrasound in Spasticity.

OBJECTIVES: To determine the reliability and validity of the Modified Heckmatt scale in assessing muscle echotexture in spasticity.
DESIGN: Prospective, observational, 2-center study. Two residents and 2 ultrasound experienced staff physicians each rated 100 ultrasound images that were also analyzed using quantitative gray-scale.
SETTING: Academic ambulatory spasticity clinic. PARTICIPANTS: Participants (N=50) included 45 patients with upper or lower extremity spasticity and 5 healthy references.
INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Modified Heckmatt scale ratings and quantitative gray-scale scores.
RESULTS: Inter- and intra-rater intraclass correlation coefficients were 0.76 and 0.81, respectively (P<.001), indicating good to excellent reliability. A significant relationship was found between Modified Heckmatt scores and quantitative gray-scale scores (r=0.829; P<.001).
CONCLUSIONS: The Modified Heckmatt scale demonstrated good reliability and validity to assess the pathologic muscle changes that occur in patients with spasticity.

Spasticity is characterized by intermittent or sustained involuntary muscle activation that can lead to severe disability. It is often present in both upper motor neuron (UMN) disorders and some neuromuscular conditions. The resulting spastic paresis from these conditions can cause impaired function and derangements of the muscle and tendon complex., Furthermore, the involuntary muscle hyperactivity caused by spasticity can lead to several harmful consequences such as pain, deformity, and impaired function., A common treatment for focal spasticity is the use of botulinum toxin (BoNT) injections, with or without the use of ultrasound (US) guidance. The use of US allows for more precise delivery of medication and keeps the injectant within the muscle fascial borders. There is data to suggest that US guidance may be associated with improved outcomes and decreased side effects.,

Muscle architecture can change in neuromuscular and UMN disorders, as evidenced by increased echointensity (EI) suggesting fatty infiltration or fibrosis., If spastic muscles with increased EI are injected with BoNT, therapeutic effects may be diminished and other treatments preferred., Conversely, injecting BoNT into muscles with more normal architecture showed improved outcomes in a cohort of patients with equinus spasticity., Thus, there is great interest amongst physiatrists and other physicians performing BoNT injections regarding the ability to differentiate the degree of EI using US in neuromuscular disorders and UMN patients with spasticity.

The most commonly used visual scale to assess muscle EI was developed by Heckmatt and Dubowitz, who proposed qualitative criteria in Duchenne muscular dystrophy.12, 13, 14 The original Heckmatt scale grades muscle echogenicity on a 4-point ordinal scale by visual assessment of the US image, with 1 being closest to normal muscle architecture and 4 showing the greatest increase in EI that is typically described as hyperechoic like bone echo. The more closely muscle echotexture resembles bone under US, the greater this pattern represents infiltration of fat and fibrosis. Differentiating grades 2 and 3 as described is a major drawback of the original Heckmatt scale because a muscle may not be homogenously affected throughout its length, hindering precise distinctions.

Although the original Heckmatt scale was designed to evaluate muscle echotexture in various myopathies, there is an increasing trend of its utilization in patients with spasticity, especially as it relates to muscle selection for BoNT injections.,,15, 16, 17 As far as we know, there has been no validation of this scale specifically in individuals with spasticity. The objective of this study was to determine the validity and reliability of a Modified Heckmatt scale in patients with spasticity and to compare data to a quantitative gray-scale analysis to establish concurrent validity.

Methods

Subjects

This is a prospective, observational 2-center study conducted at 1 academic American center and 1 academic Canadian center between 2018 and 2019. Fifty adult participants were recruited for this study, including 45 who were actively receiving BoNT treatment for upper or lower extremity muscle spasticity. In addition, 5 healthy adult reference subjects were also included to provide a normal comparison to the abnormal images obtained from patients. The reference patients were included to ensure the collection of a full range of scores, as it was anticipated that images from healthy participants would most likely be scored as Modified Heckmatt grade 1 (normal), whereas images collected from spasticity patients would be less likely to have this grade. Ethics approval was obtained from the University of Pittsburgh Medical Center (Pittsburgh, PA) and the University of British Columbia Division of Physical Medicine and Rehabilitation (Vancouver, British Columbia, Canada). Patients at both sites signed informed consent to obtain US images.

Inclusion criteria included adults (≥18y) with a UMN diagnosis and associated spasticity affecting the upper or lower extremity for which they were actively undergoing treatment with BoNT injections. Exclusion criteria included major trauma or surgery in the affected limb that would affect muscle architecture or inability to assume a body position for proper image acquisition. On the day that images were obtained, participants underwent study image collection before receiving BoNT injections. All patients who were included had received 1 or more sessions of BoNT injections at the time of study enrollment.

Modified Ashworth scale

Clinical spasticity was assessed using the Modified Ashworth scale (MAS). Joints evaluated include the elbow, wrist, finger, hip, knee, and ankle bilaterally. Spasticity was rated at each joint using a score of 0 for no increased tone to a maximum of 4 if the limb was rigid in flexion or extension.

Modified Heckmatt scale

The original Heckmatt scale was modified to obtain greater precision between grades, especially for muscles in the range of mild to moderately increased EI (grades 2 and 3) (table 1). Muscles defined as grade 1 are normal and can be easily identified. Similarly, muscles defined as grade 4 are extremely fibrosed and appear hyperechoic with image qualities like bone. Grade 2 can be distinguished from grade 3 when the examiner estimates that more than 50% of the visualized muscle demonstrates increased muscle EI (fig 1).

Table 1

Original Heckmatt scale versus the Modified Heckmatt scale

Grade	Original Heckmatt Scale	Modified Heckmatt Scale
1	Normal	Normal echogenicity in more than 90% of the muscle that is distinct from bone echo
2	Increased muscle echo intensity with distinct bone echo	Increased muscle echogenicity in 10%-50% of tissue, but with distinct bone echo and areas of normal muscle echo
3	Marked increased muscle echo intensity with a reduced bone echo	Marked increase in muscle echogenicity between 50% and 90% of tissue with reduced distinction of bone echo from muscle
4	Very strong muscle echo and complete loss of bone echo	Very strong muscle echogenicity with near complete loss of distinct bone echo from muscle in >90% of tissue.

Fig 1

Traced muscles demonstrating the Modified Heckmatt grades. (A) Grade 1: normal echogenicity in more than 90% of the muscle that is distinct from bone echo. (B) Grade 2: increased muscle echogenicity in 10% to 50% of tissue, but with distinct bone echo and some areas of normal muscle echo. (C) Grade 3: marked increase in muscle echogenicity between 50% and 90% of tissue with reduced distinction of bone echo from muscle. (D) Grade 4: very strong muscle echogenicity, with near complete loss of distinct bone echo from muscle in more than 90% of tissue.

US image acquisition

US machines and settings were kept consistent across both sites to reduce variability in image collection. A commercially available US machine was used at both locations using a 15-6 MHz linear array transducer.a The machines were calibrated similarly using the musculoskeletal preset and B-mode with autogain. The dynamic range setting was maintained at zero for all subjects at both sites. Static US images were taken from the subjects’ extremities using standardized locations and positioning for each muscle to allow for consistent analysis across sites. Transducer pressure was applied to obtain optimal images and, if possible, a bony window was captured to allow comparisons between muscle and bone echo (supplemental table S1, available online only at http://www.archives-pmr.org/). Images of muscle were taken from areas of maximal cross-sectional area where BoNT was to be administered. Muscles examined included flexor carpi radialis, flexor digitorum superficialis, flexor digitorum profundus, pronator teres, biceps brachii, brachialis, medial gastrocnemius, lateral gastrocnemius, soleus, and rectus femoris. We only used muscles that were clinically treated for spasticity. Thus, all muscles were not routinely sampled from each patient.

Image grading

Images used for grading had optimal clarity and were deidentified by the main study coordinator. Image grading was performed independently by 2 residents (postgraduate year 3 or 4) and 2 US-experienced staff physicians (15 and 27 years of experience, respectively) at each site. Reviewers were blinded to participant characteristics when reviewing the images, although the muscle was labeled in each image. Training sessions occurred with all study members before data acquisition to ensure that grading was standardized across sites. Training consisted of two 1-hour sessions in which we discussed various US images that were not used in the final study image group and came to a consensus via group discussion. The main study coordinator who was unblinded randomly arranged 100 deidentified US images collected equally between the sites. All images were graded by each rater over the course of 2 sessions to minimize grading fatigue. The 100 images graded by each physician were broken down into 50 images for session 1 and 50 images for session 2.

Quantitative gray-scale analysis

Modified Heckmatt scoring was compared with a software-generated quantitative gray-scale score, and correlations were calculated for each Modified Heckmatt grade. Images were imported into a postprocessing software program.b Regions of interest were placed around the muscle using a manual drawing tool. The region of interest was defined as the area on the inner aspect of the muscle fascia (ie, the area of muscle within the hyperechoic rim of the surrounding fascial border). The program calculated a relative gray-scale value based on a predefined reference standard. Test-retest reliability of the quantitative method was calculated from a single rater for 15 images. The median Modified Heckmatt score from all raters was used to determine the association with the quantitative gray-scale values. The study team member performing the quantitative tracing was blinded to the Modified Heckmatt scores.

Study data were collected and managed using Research Electronic Data Capture (REDCap),c an electronic data capture tool hosted at the University of Pittsburgh Medical Center and at the University of British Columbia Division of Physical Medicine and Rehabilitation, Canada. REDCap is a secure, web-based software platform designed to support data capture for research studies.

Data analysis

All statistical procedures were conducted using IBM SPSS Statistics 26.d Participant and image characteristics were examined with descriptive statistics. In addition, we created a nominal regression model to identify participant factors that may affect the variance in Modified Heckmatt scores.

We assessed reliability by calculating the intraclass correlation coefficient (ICC) and 95% confidence intervals (CI), which is a widely used reliability index in test-retest, inter-rater, and intra-rater reliability analyses., The ICC was used to determine the degree of agreement between and within raters as follows: based on the 95% confident interval of the ICC estimate, values less than 0.5, between 0.5 and 0.75, between 0.75 and 0.9, and greater than 0.90 were indicative of poor, moderate, good, and excellent reliability, respectively. ICC statistics were computed across all 4 raters and for expert and resident raters separately. We used the 2-way random ICC, which assumes that the variance of the raters is only adding noise to the estimate of the rates, and that mean rater error was equal to 0. Power calculations were performed estimating an ICC range of 0.7 to 0.9, and the minimum sample size required at 80% power α of 0.05 was 13 subjects. Our sample size of 50 subjects was very robust for this analysis, and similar in size to other studies that have examined the reliability of the original Heckmatt scale.,

In addition, Fleiss’ kappa (κ) was computed for score level to assess reliability by each Modified Heckmatt grade. Fleiss' kappa is a statistical measure for assessing the reliability of agreement between a fixed number of raters when assigning categorical ratings to several items. Interpretation of the kappa values was applied as follows: 0.01 to 0.20 indicated slight agreement, 0.21 to 0.40 indicated fair agreement, 0.41 to 0.60 indicated moderate agreement, 0.61 to 0.80 indicated substantial agreement, and 0.81 to 1.00 indicated almost perfect agreement.

Concurrent validity of the Modified Heckmatt scale was assessed by computing Spearman's rank correlation coefficient (rho) between the Modified Heckmatt scores and the quantitative gray-scale values. The level of significance was set at a P value less than .05.

Results

Patient characteristics

A total of 50 participants were included in this study. Forty-five were patients with upper and lower limb spasticity, with mean 14.6 years from the time of their neurologic diagnosis and an average age of 48.8 years. In addition, there were 5 healthy references with an average age of 27.2 years. For spasticity patients, the mean and standard deviation treatment time with BoNT was 5.3±4.5 years. Other subject demographics are listed in table 2.

Table 2

Demographics and clinical features of subjects

Variables	Data
Average age (y) ± SD	48.8±14.8
Average body mass index ± SD	26.6±6.6
Sex
Female, n (%)	18 (36)
Race
Black, n (%)	2 (4)
Asian, n (%)	6 (12)
White, n (%)	39 (78)
Hispanic, n (%)	3 (6)
Diagnosis
Cortical stroke, n (%)	5 (11)
Subcortical stroke, n (%)	1 (2)
Hemorrhagic stroke, n (%)	6 (13)
Traumatic brain injury, n (%)	5 (11)
Cerebral palsy, n (%)	3 (7)
Multiple sclerosis, n (%)	8 (18)
Other, n (%)	17 (38)
Overall years since diagnosis, mean ± SD	14.6±12.1
Years of treatment with BoNT, mean ± SD	5.3±4.6

The MAS was used to define clinical spasticity, defined as a MAS score of 2 or greater. Average scores from the affected limbs confirmed that subject images were taken from patients who had spasticity. For the upper extremity major joints (eg, elbow, wrist, fingers), the MAS ranged between 1.39 and 1.93. For the lower extremity major joints (eg, hip, knee, ankle), the MAS ranged between 1.71 and 2.20.

All 4 raters evaluated a minimum of 3 and a maximum of 27 distinct patient images for each muscle. Specifically, there were 5 images for flexor carpi radialis, 10 flexor digitorum superficialis, 7 flexor digitorum profundus, 3 pronator teres, 14 biceps brachii, 10 brachialis, 27 gastrocnemius, 12 soleus, and 12 rectus femoris images used for grading.

Inter- and intra-rater reliability

The overall inter-rater ICC among all 4 raters was 0.76 (95% CI, 0.70-0.82), with the expert inter-rater ICC being 0.76 (95% CI, 0.67-0.82) and resident (novice) inter-rater ICC being 0.77 (95% CI, 0.67-0.84), indicating good to excellent reliability. The intra-rater ICC was 0.81 for all raters, indicating good reliability when clinicians re-evaluated the images later.

Higher kappa values indicating moderate to substantial agreement were found for Modified Heckmatt grade 1 (κ=0.58; 95% CI, 0.58-0.58; P<.001) and Modified Heckmatt grade 4 (κ=0.67; 95% CI, 0.67-0.68; P<.001), respectively. Kappa values for Modified Heckmatt grades 2 and 3 indicated moderate agreement (κ=0.43; 95% CI, 0.428-0.433 and 0.430-0.435).

Concurrent validity

The overall correlation between the Modified Heckmatt scores and quantitative gray-scale scores was excellent (r=0.829, P<.001). As can be seen in figure 2, there was a high linear correlation, suggesting that the quantitative gray-scale and qualitative Modified Heckmatt scale were nearly interchangeable for evaluation of muscle echogenicity changes in patients with spasticity. The quantitative gray-scale data correctly classified Modified Heckmatt grades with the following correct percentages: grade 1 (50.0%), grade 2 (83.9%), grade 3 (77.1%), and grade 4 (58.3%) (supplemental table S2, available online only at http://www.archives-pmr.org/). Thus, quantitative analysis resulted in a better prediction of Modified Heckmatt grades 2 and 3.

Fig 2

Graph demonstrating the excellent correlation between quantitative gray-scale data and median Modified Heckmatt scores.

We also examined subject-level characteristics that might predict Modified Heckmatt scores, but the model was nonsignificant for the effects of age, body mass index, MAS, duration of disease, and duration of previous BoNT treatment (χ2(15)=18.15; P=.23).

Discussion

This prospective, 2-center study showed that the Modified Heckmatt scale can be used to assess changes in muscle echogenicity in patients with upper and lower limb spasticity. Our data established moderate to high inter- and intra-rater reliability for both expert and novice physicians when assessing the level of muscle echogenicity in spastic muscles. When compared with quantitative gray-scale measures, the Modified Heckmatt data correlated well, indicating a high degree of validity.

Previous literature has examined the reliability of the original Heckmatt scale in patients with inclusion body myositis. This study found that experienced clinicians had moderate intra-rater and substantial inter-rater reliability, but novice ultrasonographers achieved only slight to fair inter-rater reliability. Our data did not show differences in reliability based on physician experience, suggesting that the Modified Heckmatt scale may have advantages across experience levels. The current study also evaluated more muscles in both upper and lower limb locations, adding to the generalizability of the scale. In addition, because the Modified Heckmatt scale demonstrated good reliability among a variety of UMN conditions, future studies could be performed to test whether it improves grading EI in neuromuscular disorders.

Differences in rating agreement were noted between Modified Heckmatt grades. The inter-rater kappa results were higher with Modified Heckmatt grades 1 and 4 (κ=0.58 and 0.67, respectively) as compared with grades 2 and 3 (κ=0.43 and 0.43, respectively). Intuitively, this is understandable as muscles with grade 1 appear normal and muscles with grade 4 are very abnormal and more easily classified. Muscles with grades 2 and 3 are in the middle range of increased EI and can be more difficult to categorize and distinguish from one another. The modifications that were made to the original Heckmatt scale may be useful to assist in distinguishing these differences, although the study was not designed to compare the original Heckmatt scale with the current modified version. Interestingly, the quantitative gray-scale data best predicted Modified Heckmatt grades 2 and 3.

The Modified Heckmatt scale has good concurrent validity when compared with quantitative gray-scale analysis. Because there was a strong positive correlation between qualitative and quantitative scoring, clinicians can be confident that they would obtain a similar result using the Modified Heckmatt scale compared with a computerized quantitative gray-scale. This finding is clinically relevant because the Modified Heckmatt scale can be used in real time during BoNT injection sessions to identify muscles with the best likelihood of having beneficial outcomes. Additionally, this scale is easy to learn, and residents and attending physicians produced nearly identical ratings. If muscle architecture is severely altered, as indicated by Modified Heckmatt ratings of 3 or 4, the BoNT dose for that muscle may require reduction or elimination altogether because the presynaptic cleavage of SNAP-25 binding proteins may be of minimal clinical benefit in a fibrosed muscle. Correlating US muscle characteristics to functional outcomes after BoNT injections has been documented previously in 1 small prospective study.

We also examined subject level characteristics that might predict Modified Heckmatt scores, but the model was nonsignificant for the effects of age, body mass index, Modified Ashworth score, duration of disease, and duration of previous BoNT treatment. This finding suggests that the tool can be used among a diverse population of patients with spasticity, and that it is not necessary to control for age or other patient factors when implementing the Modified Heckmatt scale.

Study limitations

Limitations of this study include difficulty assessing deeper muscles with good beam penetration because most linear array transducers can only penetrate to 6 cm. However, this depth is representative of muscle targets in most patients with spasticity who are not severely obese. In addition, it was difficult at times to include a bone window for direct comparison to muscle during scanning. Although the study methods restricted ultrasonographers to obtain images at specific locations for consistency across sites, clinicians could easily scan other regions of muscle to compare to bone.

Future randomized, prospective studies using the Modified Heckmatt scale to guide BoNT injections are warranted to assess whether EI affects outcomes. These studies could also be designed to compare outcomes using US guidance to other guidance techniques such as surface palpation or electromyography. Prospective longitudinal studies are needed to document how the echotexture of spastic muscle evolves over time and whether this change is affected by chemodenervation, as not all patients with spasticity experience muscle changes.

Conclusions

The Modified Heckmatt scale can reliably evaluate echogenic changes in spastic muscle, indicating the level of fibrotic change. The scale showed excellent validity when compared with a quantitative gray-scale analysis. Future goals would assess the diagnostic value and effect of the Modified Heckmatt scale on patient outcomes for individuals undergoing treatment for spasticity.

Suppliers

Sonosite Xport; FujiFilm Sonosite Inc.

ImageJ; National Institutes of Health.

REDCap software; Vanderbilt University.

IBM SPSS Statistics; IBM Corp.