Reliability of the Shoulder Symptom Modification Procedure and association of within-session and between-session changes with functional outcomes.

BACKGROUND: Despite being a common problem, there is considerable diagnostic uncertainty with regard to shoulder pain. This uncertainty relates to the reliability and validity of current examination tests. The Shoulder Symptom Modification Procedure (SSMP) has been proposed as an alternative to existing approaches.
OBJECTIVE: To evaluate interclinician reliability of the SSMP and the association of within-session and between-session changes on clinical outcome at 1 week, and at 1 and 3 months.
DESIGN: A single-centre reliability study, with prospective follow-up.
METHODS: Twenty-six patients with shoulder pain were recruited. Following an initial SSMP-based examination, a second examination was performed by a second physiotherapist, blinded to the results of the first examination. Clinical outcome data were completed after 1 week, 1 month and 3 months via theNumeric Pain Rating Scale and the Shoulder PainandDisability Index. Reliability was evaluated using kappa and associations were evaluated using Spearman's r.
RESULTS: Inter-rater reliability of the SSMP was moderate (κ=0.47). Association of within-session changes ranged from fair to poor in the short term (r=0.24-0.01) to poor in the mid-term (r=-0.03). The association of between-session changes ranged from substantial to moderate in the short term (r=0.74-0.47) but slight in the mid-term (r=0.22).
CONCLUSIONS: Based on this study, we cannot recommend the SSMP as a reliable tool for physical examination of patients with shoulder pain. The importance of within-session and between-session changes remains uncertain.

The Shoulder Symptom Modification Procedure (SSMP) demonstrates moderate inter-rater reliability

The importance of creating within-session changes in pain on outcome remains uncertain.

Between-session changes in pain seem more strongly associated with outcome in the short term, but this diminishes in the mid-term.

There is insufficient evidence to recommend the SSMP as a reliable or validated evidence tool for physical examination of patients with shoulder problems.

Introduction

Musculoskeletal conditions are now ranked as the second highest cause of number of years lived with disability. After low back and neck pain, shoulder pain is the third most common cause for musculoskeletal consultations, with up to 20% of people reporting a shoulder-related issue at any one time. A large percentage of these shoulder complaints do not recover spontaneously, with approximately 50% reporting persistent problems 6 months after onset and 40% reporting incomplete recovery at 1 year.

Although there is considerable diagnostic uncertainty, the structures within the subacromial space such as the bursa and rotator cuff tendons are thought to be the most common causes of shoulder pain. However, despite such commonality and global burden, these disorders are poorly understood and poorly managed. The main reason for this could be the considerable diagnostic uncertainty, which is related to the limitations of the examination tests, in terms of reliability and sensitivity/specificity, used to diagnose pathology and inform treatment selection.

Some of the limitations associated with current approaches to the physical examination of the shoulder were highlighted by May et al in their systematic review, which concluded there is no consistent evidence that any physical examination test used in the assessment of the shoulder demonstrates acceptable levels of reliability. These findings were further supported by Hanchard et al,who systematically reviewed 33 studies involving 3852patients that investigated the reliability of orthopaedic tests that were designed to identify lesions of the subacromial tendons, bursa and labrum. They found extreme diversity in the interpretation and performance of all these tests and insufficient evidence that current clinical testing is able to identify these issues reliably.

In response to the limitations associated with the current orthopaedic physical examination, a potential enhancement to the physical examination of the shoulder, the Shoulder Symptom Modification Procedure (SSMP) (https://www.londonshoulderclinic.com/wp-content/uploads/2016/08/SSMPv6-2016.pdf), has been proposed with the aim of helping clinicians guide treatment without assumption about the underlying pathology. The SSMP is a series of standardised tests that attempt to reposition structures, facilitate movement or neuromodulate symptoms thought to arise from the shoulder. The aim of the SSMP is to identify whether such techniques can reduce pain associated with movement. It is hypothesised that these ‘within-session changes’ (changes that occur during the examination) can be useful to help guide therapists to choose appropriate treatments and might help to predict those who might respond to physiotherapy. The predictive validity of a within-session and a between-session change has been explored in a number of regions, including the neck, shoulder, back and hip.

The SSMP is performed in a sequential format through four key areas: thoracic repositioning, scapula facilitation, humeral head procedures and neuromodulatory techniques. Thoracic repositioning tests are conducted first to see if this reduces or resolves a painful arm or shoulder movement. If not the SSMP moves onto a number of scapula-based repositioning and facilitation tests. If these do not resolve or reduce the painful movement, then the SSMP moves onto a series of humeral head techniques done in standing, sitting or supine. Finally if none of these procedures have affected patients’ symptoms, a series of neuromodulatory techniques such as cervical distractions or mobilisation with movement, soft tissue massage, and taping techniques are attempted.

The SSMP has been widely adopted by many physiotherapists despite limited published evidence of its reliability or predictive value of the approach. Therefore, the purposes of this study were to (1) evaluate interclinician agreement of the SSMP (primary goal) and(2) evaluate the association of any initial within-session and between-session changes in pain on patient self-report of pain and disability at the first follow-up session and at 1 and 3 months (secondary goal).

Methods

Design

This is a single-centre reliability study, with prospective follow-up, based in a private hospital in Hertfordshire, UK. The manuscript is reported with reference to Quality Appraisal of Diagnostic Reliability checklist.

Recruitment

All patients over 18 years old with shoulder and/or upper arm pain who were referred for physiotherapy by their general practitioner and orthopaedic consultant or by self-referral were invited to participate in this study when they booked their appointment. Patients were informed of the study and its purpose, and all were given or posted a written information sheet outlining the study aims when making their first appointment. On average patients had between 1 and 7 days from making their booking to their first appointment, allowing them time to ask questions or withdraw from the study if they wished.

To be included all patients had to report and demonstrate, during initial examination by the lead author, shoulder and/or upper arm pain on elevation during flexion, scaption or abduction. They also had to report pain with or without weakness on isometric resisted testing into external rotation and/or abduction.

Patients with painful shoulder or upper arm pain that had significant signs of passive stiffness were excluded. We judged this to be anything more than a 25% loss of passive movement into external rotation and/or elevation when compared with the contralateral side. We also excluded any painful shoulder or upper arm that had signs of cervical spine involvement, such as pain reproduced in the shoulder or arm on cervical movement or axial compression, and/or if they had any positive distal neurological signs or symptoms on examination. Finally we excluded any patients with a history of shoulder surgery or gross orthopaedic trauma, such as dislocation or fracture, within the past 12 months.

Clinicians

All patients were first assessed by the lead author, a qualified physiotherapist with experience on treating patients with shoulder pain for over 15 years, who had received previous training in the use of the SSMP. The second assessors in this study comprised eight other qualified physiotherapists who worked with the lead author within the same physiotherapy department. Their clinical experience ranged from 3 to 9 years, and their understanding and exposure of the SSMP were variable. Two of the other assessors had completed SSMP workshops, three had read the SSMP papers, and three were unaware of the SSMP prior to participating in the study.

To ensure a comparable level of understanding and standardisation in the application of the SSMP, a 3-hourinservice training programme was attended by all the physiotherapists who would be involved prior to the start of the study. The lead author demonstrated the SSMP and ensured all other physiotherapists were consistent and confident in the procedures.

Procedure

Once informed written consent was gained, the participants completed the baseline Shoulder PainandDisability Index (SPADI). The SPADI is a self-report measure specifically developed to evaluate pain and function in patients with shoulder pathology. It is a commonly used measurethat has been validated for use in this patient population, and a minimally clinically important change of 10 points has been identified. The SPADI includes 13 items divided into twosubscales: pain (five items)and disability (eight items). The responses are indicated on a visual analogue scale, where 0=no pain/no difficulty and 10=worst imaginable pain/so difficult it requires help. The items are summed and converted to a total score out of 100.

Following completion of the SPADI, the patient was then asked to demonstrate their most painful movement or activity and report their level of pain on a 0–10Numeric Pain Rating Scale (NPRS). The SSMP was then performed as described. The patient’s response or lack of response to the SSMP was recorded on a standardised SSMP recording form (online supplementary appendix 1) and placed into a sealed envelope. A positive response to an SSMP test was classified as a complete resolution of symptoms or a reduction in the NPRS of 4 or more points to reflect prior guidance.

To reduce the risk of tissue irritation and an order effect, on completion of the SSMP examination by the first physiotherapist, the patient was asked to sit quietly in the assessment room for at least 10 min, prior to the subsequent application of the SSMP by the second physiotherapist. Selection of the second physiotherapist was opportunistic according to current availability. The second assessor was not informed of the first assessor’s results, only which side was to be assessed. The patients were also instructed not to discuss or inform the second therapist of what had or had not occurred previously with the first assessment. After 10 min had passed, the second therapist then performed the SSMP and recorded their findings on the same standardised form, and placed this into a second sealed envelope. Both sealed envelopes where then passed to a non-clinical administrator unaware of the SSMP and the data logged and recorded onto a secure spreadsheet to which the assessors had no access.

The patient’s treatment then commenced with the lead author as usual. All patients were followed up and assessed a week later, and second NPRS and SPADI scores were obtained. Follow-up treatments after this session were dependent on symptoms, availability and need. NPRS and SPADI scoreswere further obtained at 1 month and 3 months after initial assessment.

Data analysis

Demographic details of the participants, including ageand duration of symptoms, are presented descriptively. Inter-rater agreement was analysed via SPSS using the kappa statistic. Kappa is a measure of chance-corrected agreement with the following interpretations being suggested: ≤0= poor,0.01–0.20=slight, 0.21–0.40=fair, 0.41–0.60=moderate, 0.61–0.80=substantialand 0.81–1.0=almost perfect. A 95%CI for kappa was obtained through bootstrapping (1000 samples).

To reflect the data, the correlation between the degree of change (post-SSMP NPRS minus pre-SSMP NPRS) on the NPRS and the SPADI (SPADI at first follow-up session, 1 and 3 months minus SPADI at baseline) was analysed using the non-parametric Spearman’s rank correlation coefficient to evaluate the impact of within-session change, and repeated using the degree of change over 1 week on the NPRS (NPRS at 1 week minus pre-SSMP NPRS) to evaluate the impact of between-session change.

Sample size calculation

Sample size was determined based on the primary goal of the study, which was to evaluate inter-rater reliability (two raters) with regard to the categories of the SSMP. We assumed a proportion of positive findings of 50%, power of 80% and a (two-sided) significance level of 5%, and hence aimed to recruit 32 participants.

Results

A total of 26 patients were recruited (table 1).

Table 1

Demographic description of the participants (n=26)

Mean age in years (SD)	52.4 (10.5)
Gender (male/female)	13/13
Affected shoulder (right/left)	11/15
Dominant arm (right/left)	22/4
Duration of symptoms in months (SD)	4.6 (3.4)
Baseline NPRS (SD)	6.6 (1.4)
Post-SSMP NPRS (SD)	2.3 (1.5)
Baseline SPADI (SD)	46.5 (14.3)

NPRS, Numeric Pain Rating Scale; SPADI, Shoulder Pain and Disability Index; SSMP, Shoulder Symptom Modification Procedure.

Of 26 patients, all reported abduction as their most painful shoulder movement. Twenty-five patients reported complete or partial response to the SSMP, based on the assessment by therapist 1. Four patients reported complete response (NPRS=0 post-SSMP), with 21 reporting partial response. Of the 21 partial responders, two patients did not report change on the NPRS of greater than 4 points so were classified as overall non-responders. One patient reported no change in response to the SSMP. Hence, in total, 23 of 26 patients responded to initial SSMP procedures.

Primary goal

For therapist 1, response to SSMP was most commonly in relation to scapula repositioning (n=10), followed by humeral head procedures (n=8), thoracic repositioning (n=2), neuromodulation (n=2) and finally correction of scapula winging (n=1). For the second therapists, response to SSMP was similar and most commonly in relation to scapula repositioning (n=10), followed by humeral head procedures (n=8), thoracic repositioning (n=1), neuromodulation (n=2), and correction of scapula winging (n=1).

The physiotherapists agreed on 15/26 (57.7%) of cases. Inter-rater agreement for the SSMP was moderate (k=0.47; 95% CI0.20 to 0.71).

Secondary goal

Patient-reported outcomes were reported at 1 week, 1 month and 3 months (table 2). On average, patients reported improvement from baseline to 3 months. The reduction in NPRS (6.6(SD 1.4) to 2.3(SD 1.5)) reported in response to the SSMP on day 1 was not maintained at 1 week (6.2 (SD 1.7)), 1 month (4.3 (SD 2.0)) or 3 months (3.2 (SD 1.6)).

Table 2

Patient-reported outcome data

	Baseline	1week	1month	3months
Numeric Pain Rating Scale	6.6 (SD 1.4)	6.2 (SD 1.7)	4.3 (SD 2.0)	3.2 (SD 1.6)
Shoulder Pain and Disability Index	46.5 (SD 14.3)	39.6 (SD 14.9)	28.8 (SD 17.0)	21.4 (SD 14.7)

The association between the amount of change on the NPRS from pre-SSMP to post-SSMP testing and the amount of change from baseline to 1 week on the SPADI (r=0.24; 95% CI−0.25 to 0.63) was fair, from baseline to 1 month on the SPADI (r=0.01; 95% CI−0.43 to 0.49) was poor, and from baseline to 3 months on the SPADI (r=−0.03; 95% CI−0.54 to 0.63) was poor in a negative direction.

The association between the amount of change on the NPRS from pre-SSMP (baseline) testing to week 1 and the amount of change from baseline to 1 week on the SPADI (r=0.74; 95% CI0.42 to 0.91) was substantial, from baseline to 1 month on the SPADI (r=0.47; 95% CI0.01 to 0.82) was moderate, and from baseline to 3 months on the SPADI (r=0.22; 95% CI−0.37 to 0.76) was fair.

Discussion

The data from this study suggest that the SSMP demonstrates moderate interclinician agreement (κ=0.47). It has previously been suggested that kappa levels should be greater than 0.75 for a test to be of clinical utility. Hence, based on the findings of this study, there is insufficient evidence to recommend the SSMP as a reliable tool for physical examination of patients with shoulder problems.

With regard to the secondary goal, based on the findings of this study, within-session change in patients’ self-report of pain secondary to application of the SSMP is not well associated with patient self-report of pain and disability over 1 week, 1 month or 3 months. Between-session change, over 1 week, in patient self-report of pain secondary to application of the SSMP is substantially associated with patient self-report of pain and disability over 1 week, moderately associated at 1 month and only slightly associated by 3 months.

To date, two previous studies have evaluated the interclinician agreement of the SSMP; one study reported substantial levels of reliability, whereas the other study reported variable levels of reliability. The variability in findings might, in part, be explained by the different research designs. Bahat and Kernerevaluated reliability by asking two physiotherapists to examine the patient, whereas Lewis et alevaluated reliability by video recording one physiotherapist examining the patient while other physiotherapists recorded their judgement about the response. This current study adopted a similar method to Bahat and Kerner; this method has the advantage of mimicking real-world practice but with the risk that the examination of the patient by the first physiotherapist might change the presentation such that the second physiotherapist is examining a different clinical presentation, which hampers any judgement about reliability. In contrast, one of the advantages of the method used by Lewis et al is that the judgements of a larger number of physiotherapists can be more easily garnered but with the risk that the process artificially standardises the examination process, which might erroneously elevate levels of reliability. A number of approaches to evaluating reliability have previously been adopted and there is no definitive guidance regarding the optimal approach. However, it seems appropriate to be mindful that the different reliability designs adopted might explain some of the variance in reported results.

Similar to most other interventions for musculoskeletal disorders, the reasons why the SSMP might induce an immediate change in symptoms are unclear, but it is apparent that the majority of patients in this study experienced significant within-session change in pain. This response might be secondary to soft tissue or joint displacement, changing sensory motor control, neuromodulation, contextual effects or for reasons yet to be determined. Along with further research relating to the reliability and predictive value of the SSMP, development of the understanding of the possible mechanism of action could shed further light on the applicability of this procedure for people reporting shoulder pain.

The importance of generating symptom modification based on the SSMP requires further evaluation. It is widely suggested that creating a change in symptoms within the treatment session is desirable. This study cannot provide a definitive answer to this hypothesis, but it is interesting to note that the extent of the change in patient self-report of pain was not well associated with combined pain and function scores, according to the SPADI, in the short term (<1 month) or mid-term (<3 months). However, between-session changes were substantially associated with combined pain and function scores, according to the SPADI, over the first week, but beyond this short time frame the extent of the association diminished. This finding is similar to a previous shoulder study where between-session changes were not well associated with subsequent mid-term functional outcomes.

Limitations

This study was conducted in one centre with a limited number of patients. Most of the physiotherapists involved in the study had not been formally trained in the SSMP and had varied experience using the procedure. However, this reflects real-world practice and also extends an approach reported previously.

Implications

Based on this study, there is insufficient evidence to recommend the SSMP as a reliable or validated evidence tool for physical examination of patients with shoulder problems.

Conclusion

The SSMP has been proposed as an alternative to current approaches to examination. However, this study suggests moderate reliability of the SSMP, which might be regarded as insufficient to guide clinical decision-making. Furthermore, the importance of within-session changes as a basis for understanding future clinical outcome is uncertain, but between-session changes seem to be more strongly associated with clinical outcome in the short term but not the mid-term based on these data.