Psychometric testing of a short-form questionnaire for the measurement of health experiences among people with musculoskeletal disorders undergoing multimodal rehabilitation.

AIM: The aim of the present study was to assess if a previously suggested short-form questionnaire tested among women with non-specific neck-shoulder pain is suitable also for use among men and women with non-specific musculoskeletal disorders in any part of the body, by testing its construct validity by a confirmatory factor analysis. If not, the secondary aim was to investigate the evolving factor structure when performing an explorative factor analysis of data in the expanded sample.
METHODS: Questionnaire data were collected in three different contexts, in primary care via eight different multimodal rehabilitation teams and in specialised care via two different specialist care centres. The sample consisted of 116 men (n=29) and women (n=87) with non-specific musculoskeletal disorders.Data were analysed using confirmatory and exploratory factor analysis and a visual comparison between the result of the principal component analysis in the present study and the results attained in a previous study with a similar aim and design.
RESULTS: The confirmatory factor analyses did not end up in a model with acceptable measures for validity. Three models were tested, none of them met the criterion for an acceptable model, and the goodness-of-fit statistics were not fully acceptable. The exploratory factor analysis had only partly comparable result compared with previous study.
CONCLUSION: The results of the present study did not prove the suggested short-form questionnaire to be suitable for evaluation of symptoms among men and women with non-specific musculoskeletal disorders in any part of the body. Further studies including larger samples are recommended. TRIAL REGISTRATION NUMBER: 92199001. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This study included men and women with non-specific musculoskeletal disorders in any part of the body, participating in multimodal rehabilitation in primary care and at specialist centres.

Relevant biopsychosocial aspects that often are affected for a person with musculoskeletal disorders were considered when constructing a short-form questionnaire.

Internationally recommended outcome measures were chosen and psychometrically tested.

A rather small sample posed difficulties in the psychometric testing of the included questionnaires.

Introduction

Musculoskeletal disorders (MSDs) are common and often lead to disability and long-term absenteeism with negative consequences for both the individual and the society.1 2 MSDs are considered to have a multifactorial aetiology involving physical, psychosocial and personal risk factors,3 and accordingly multidimensional treatments based on the biopsychosocial model that takes biological, psychological and social factors into account are recommended.4–7 A multitude of instruments are available to evaluate treatments; however, there is a lack of consensus regarding which of them to use and how to combine them.8 This lack of consensus reduces the possibilities for comparisons of treatment results, which in turn may hinder the development of treatment methods. Another limitation is that a selection of outcome measures often is used, which do not capture the entire specific spectrum of symptoms the persons suffer from.9

We have previously performed a study on women with non-specific MSDs in neck-shoulders, where the intention was to initiate the development of a clinically useful short-form questionnaire.10 The study was based on measures recommended from The Initiative on Methods, Measurement and Pain Assessment in Clinical Trials (IMMPACT), which conducts ongoing international developmental work regarding treatments for pain and outcome evaluations in the field of clinical pain studies.11 Three questionnaires including 61 items were used and a factor analysis resulted in a 9-factor solution including 29 items. These items were derived from three of the four core domains recommended by IMMPACT: pain intensity, emotional functioning and physical functioning.11

Studying the usefulness of internationally recommended outcome measures in relation to MSDs can make a valuable contribution to increase the uniformity of outcome measures and thus increase comparability between studies.11 Access to specific outcome measures in a clinically useful format may increase the possibility of targeted interventions in terms of prevention and early treatment. In the present study, we want to test if the results from the previous study also are valid among a broader pain population including men and women with non-specific MSDs in any part of the body.

Accordingly, the aim of the present study was to assess if the short-form questionnaire is suitable for use among men and women with non-specific MSDs in any part of the body, by testing its construct validity by a confirmatory factor analysis (CFA). If not, the secondary aim was to investigate the evolving factor structure when performing an explorative factor analysis of data in the expanded sample.

Methods

Design

This is an instrument validation study using confirmatory and exploratory factor analysis of data from patients with non-specific MSDs in any part of the body, which were undergoing multidimensional rehabilitation in either primary or specialised care.

Sample and data collection

Data were collected in primary care via eight different multimodal rehabilitation teams and in specialised care via two different specialist care centres. The rehabilitation team distributed and collected questionnaires from those participants in the rehabilitation programme that were willing to participate in this study. All patients had been remitted to multimodal rehabilitation by their primary care physicians. Indications for multimodal rehabilitation were persistent or intermittent pain for at least 3 months; pain with a high degree affects the patients' daily life; a patient who have the potential for active change, despite pain; no comorbidity or other condition that will hinder participation in the rehabilitation programme.12

Measures

As in our previous study on the subject,10 we considered the recommended13 four core domains to include in clinical trials: (1) pain intensity; (2) physical functioning; (3) emotional functioning and (4) general improvement. However, general improvement can only be evaluated after treatment and was therefore excluded as data were obtained once. The outcome measures that are recommended for the remaining three domains are (1) a Numeric Rating Scale (NRS), (2) the Brief Pain Inventory (BPI)14 or the Multidimensional Pain Inventory (MPI)15 and (3) the Beck Depression Inventory (BDI)16 17 or the Profile of Mood States (POMS).18 For details on the selected outcome measures, see below.

Pain intensity and physical functioning

The MPI—Swedish version (MPI-S)19 that was selected includes two sections and a total of 34 items. Section 1 (22 items) consists of five subscales: pain severity, interference, life control, affective distress and support. Section 2 (12 items) consists of three subscales regarding responses from significant others: punishing, solicitous and distracting responses, where each included item has a 7-point response scale between 0 (no, not at all) and 6 (yes, very much). Question number 6 in part 1 has to be recoded in reverse to correspond to the scaling of all other questions. MPI-S was also chosen for the assessment of pain intensity, as it includes an NRS to measure pain intensity in two different temporal time perspectives.

Emotional functioning

Regarding negative emotional functioning, an alternative questionnaire compared with those recommended was chosen. Comparing the content of BDI and POMS, BDI was judged to be the most suitable for this sample. However, the final, alternative choice fell on the Montgomery-Åsberg Depression Scale (MADRS-S).20 It has the same qualities as the BDI with respect to its ability to differentiate between different diagnoses and sensitivity to change,21 and MADRS-S was considered a favourable choice since it can be used free of charge. When answering MADRS-S, people rate their last 3 days’ core symptoms of depression in nine items (mood, anxiety, sleep, appetite, ability to concentrate, ability of initiative, emotional commitment, pessimism and vitality). Items are scored from 0 (none at all) to 6 (maximum), with a maximum total score of 54. Reduction in score reflects symptom improvement.

Depression scales are normally used for the measurement of emotional functioning. However, positive aspects of psychological health have been actualised and also linked to biological processes and biomarkers.22–25 Enjoyment of life and emotional well-being have been appraised as the most important aspects of health among people with chronic pain, in addition to pain reduction.26 Psychological well-being, rather than specific somatic symptoms, has also been found to influence people’s care-seeking for MSDs.27 A balance between illness and wellness has been described as significant for the overall experience of health among people with MSDs.28 29 Including a well-being questionnaire may give information about health promoting aspects by capturing positive aspects of emotional functioning. Hence, in order to make the assessment of the emotional functioning domain more thorough, we decided to include the short form of the Ryff Psychological Well-being Scale (RPWS).22 23 30–33 RPWS consists of 18 items, divided into six subscales with three items each: self-acceptance, positive relations with others, autonomy, environmental mastery, purpose in life and personal growth.31 The items are posed as statements, as ‘I like most aspects of my personality’ and the answers are given on a response scale between 1 (very strongly disagree) and 6 (very strongly agree). Some questions (numbers 7, 9, 11, 32, 42, 55, 65 and 76) have to be recoded so that high scores correspond to high well-being for all items.

Statistical analyses

IBM SPSS Statistics V.22.0 was used for descriptive statistics and explorative factor analysis (principal component analysis (PCA)), and IBM AMOS V.22.0 was used for CFA.

Confirmatory factor analysis

A CFA was conducted to assess the construct validity of the questionnaire. In all, we tested three models (figures 1, 2 and 3A–C). First, we tested a model with a factor structure consisting of nine factors and 29 items1 (figure 1). We then tested a model with a factor structure originated from a biopsychosocial approach to chronic pain4 (figure 2). This model consists of four factors and the same 29 items as in the first model. Some of the four factors were allowed to correlate and even some of the error terms were allowed to correlate (figure 3B–C). In the sample of 116 participants, there were internal missing observations, that is, missing observations in parts of the questionnaire. Twenty-two participants had at least one missing observation of the included items. The models were estimated by the method of maximum likelihood (ML). The covariance matrix of the observed values of the items was the basis in the estimation process. ML estimation assumes items measured on a continuous scale and multivariate normality. Moreover, missing data must have a missing data pattern generated completely at random for ML estimates to be consistent and effective. When data are missing at random, ML estimates are unbiased and efficient as the sample size grows to be large. ML estimation (full information ML) in AMOS is considered to yield lesser biased estimates compared with other methods to handle missing data even if data are missing not at random.34 We also made the analysis of the models based on listwise deletion, that is, we deleted the 22 participants for which there were at least one missing observation on an item. The purpose of this was to be able to use modification indices and standardised residuals in AMOS to get indications of how to modify and refine the specification of the models. Moreover, we were also able to compare the estimates from listwise deletion with those based on the complete dataset. The main results are based on the complete dataset.

Figure 1

Previous model. MADRS-S, Montgomery Åsberg Depression Rating Scale; MPI-S, Multidimensional Pain Inventory; RPWS, Ryff’s Psychological Well-being Scale.

Figure 2

Model based on the biopsychosocial model. MADRS-S, Montgomery Åsberg Depression Rating Scale; MPI-S, Multidimensional Pain Inventory; RPWS, Ryff’s Psychological Well-being Scale.

Figure 3

Graphical illustration of tested models by confirmatory factor analysis. From left: (A) Model 1; (B) Model 2; (C) Model 3. MADRS-S, Montgomery Åsberg Depression Rating Scale; MPI-S, Multidimensional Pain Inventory; RPWS, Ryff’s Psychological Well-being Scale.

The measurements from the included items yield ordered categorical data. Therefore, we also made the estimation by Bayesian methods to compare the estimates and to evaluate estimation diagnostics. In AMOS, the approach to the analysis of ordered categorical data is by Bayesian method of estimation.34 Briefly, the Bayesian method of estimation produces a series of estimates of factor loadings, variances and covariances based on simulations by a Markov chain Monte Carlo (MCMC) algorithm. Thereby, we are able to study the distribution of the estimates and some summary measures of the simulation process to assess the statistical properties of the estimates.

In the specification of the models, we fixed one factor loading in each domain to one in order to set the scale for the unobserved factors. We allowed all the nine factors in the first model to correlate, and in the second and third model, we allowed some of the four factors to correlate according to the biopsychosocial model. Furthermore, we specified that some of the error terms correlated (see figure 3A–C).

For the evaluation of the models as a whole, we primarily used some of the statistics of goodness of fit. We used Model Chi-square (CMIN), the ratio (CMIN/df), where CFI is the comparative fit index, the normed fit index (NFI), parsimony-adjusted normed fit index (PNFI) and root mean square error of approximation (RMSEA) with 90% CIs. CMIN is a statistic of the closeness of the specified model in terms of estimated covariances and the observed covariances. A small value and a p value >0.05 indicate an acceptable model. The ratio of CMIN/df <3 is often used as a criterion for an acceptable model. Values for CFI and NFI range from 0 to 1, and values >0.9 are indicative of good models and values >0.95 of excellent models. PNFI is based on NFI and penalise model complexity by adjusting for the df. In conjunction with NFI values in the range 0.9 values as low as 0.5 could be accepted. Finally, acceptable values of RMSEA are <0.06. For discussion of which of the many goodness-of-fit indices to use and their cut-off values indicating well-fitted models, see, for example, studies by Byrne, Hu and Bentler, Hooper et al and Kääriäinen et al.34–37 Our model evaluation also includes inspection of estimated parameters where we looked for reasonable size and sign of the factor loadings, variances and covariances and if they were statistically significant.

Explorative factor analysis, PCA

Initially, Cronbach’s α was calculated to determine the internal consistency of the instruments’ total and subscale scores (table 1).

Table 1

Reliability of the questionnaires (n=116)

Questionnaire	Valid cases	Cronbach’s α	Item means	Range (max – min)
MPI-S—overall score	82	0.81	3.2	4.1
MPI-S—pain intensity	115	0.85	3.7	0.30
MPI-S—interference	88	0.92	3.8	2.5
MPI-S—life control	115	0.77	3.2	0.76
MPI-S—affective distress	115	0.86	2.9	0.28
MPI-S—support	115	0.77	4.1	0.31
MPI-S—responses from significant others: punishing	103	0.86	1.3	0.60
MPI-S—responses from significant others: solicitous	103	0.80	3.5	0.24
MPI-S—responses from significant others: distracting	103	0.69	2.9	0.98
MADRS-S—overall score	102	0.89	1.7	2.0
RPWS—overall score	114	0.84	4.2	1.7
RPWS—environmental mastery	114	0.76	4.1	0.33
RPWS—self-acceptance	114	0.71	4.1	1.2
RPWS—positive relations	114	0.59	4.4	0.89
RPWS—purpose in life	114	−0.10	4.4	1.3
RPWS—personal growth	115	0.59	4.4	0.86
RPWS—autonomy	115	0.61	4.3	0.42

MADRS-S, Montgomery Åsberg Depression Rating Scale; MPI-S, Multidimensional Pain Inventory; RPWS, Ryff’s Psychological Well-being Scale.

Thereafter, explorative factor analysis was performed, using PCA for extraction and orthogonal varimax rotation with Kaiser normalisation to clarify factor structure. A two-step procedure was chosen in the factor analysis to get an acceptable subject-to-variable ratio. Kaiser-Meyer-Olkin measure of sampling adequacy (KMO) and Bartlett’s test were used to determine the adequacy of performing a factor analysis on the selected data (table 2). The KMOs were acceptably above 0.70,35 and Bartlett’s test showed significant results for all three questionnaires (p<0.05). An eigenvalue >1 and scree plot were used to determine the number of factors to extract, as recommended for PCA.36 Two possible levels of factor loadings were considered throughout the analysis, ≥0.70 and ≥0.60. When selecting the level of factor loadings, coverage regarding included items in each factor and the logic of the final factor structure were judged.

Table 2

Sampling adequacy (n=116)

Questionnaire	Kaiser-Meyer-Olkin (KMO)	Bartlett*
MPI-S	0.785	0.000
MADRS-S	0.869	0.000
RPWS	0.893	0.000

*P<0.05.

MADRS-S, Montgomery Åsberg Depression Rating Scale; MPI-S, Multidimensional Pain Inventory- Swedish version; RPWS, Ryff’s Psychological Well-being Scale.

In the first step of the explorative factor analysis, separate analyses were performed on the data from different questionnaires. The satisfactory factor loading was initially decided to ≥0.7. As there was absence of items in one factor for the MPI-S, when choosing factor loading ≥0.7, we chose to set the factor loading to ≥0.6 instead. Second, we performed a joint factor analysis including all items with a factor loading ≥0.6 for MPI-S and RPWS. The MADRS-S was excluded as no rotated factor solution was attained in the separate factor analysis, not even after an analysis with exclusion of items with low values for communalities. The joint factor analysis of MPI-S and RPWS resulted in a factor solution with a KMO 0.699, but no rotated factor solution. Thus, we chose to do another joint factor analysis, this time including all items with a factor loading ≥0.7, which resulted in a rotated factor solution including eight factors. The final factor solution was then compared with the result from our previous study.10

Results

The sample consisted of 116 participants, male (n=29) and female (n=87), with non-specific MSDs in different parts of the body as indicated on pain drawings (mean age 46 years; range 20–69; SD 10.2). The participants reported pain durations of at least 3 months (mean duration 132 months; range 6–576; SD 116.4). Twenty-seven percent of the participants were on sick leave at the time of the study. Nineteen percent of the participants were living alone, 5% were living alone with one or several children, 36% were living with another adult and 40% were living with another adult and one or several children.

Confirmatory factor analysis

The three models that are the result of the CFA are presented in figure 3A–C. The goodness-of-fit statistics for Models 1 to 3 are shown in table 3. Neither of the values of Model 1 and Model 2 were satisfactory to meet the criterion for an acceptable model. By means of analysis based on listwise deletion and use of modification indices and standardised residuals, we tested some correlations between residuals. In Model 2, eight correlations were specified and in Model 3 eleven (see figure 3B,C). Although the values of the goodness-of-fit statistics are somewhat better, they are not fully acceptable.

Table 3

Goodness-of-fit statistics for confirmatory factor models estimated by ML (n=116)

Model	CMIN	df	CMIN/df	CFI	NFI	PNFI	RMSEA (90% CI)
1	706.60	350	2.019	0.763	0.636	0.512	0.094 (0.084 to 0.104)
2	770.50	371	2.077	0.735	0.603	0.514	0.097 (0.087 to 0.106)
3	670.96	368	1.823	0.799	0.654	0.554	0.085 (0.074 to 0.095)

CFI, comparative fit index; CMIN, Model chi-square; ML, maximum likelihood; NFI, normed fit index; PNFI, parsimony- adjusted normed fit index; RMSEA, root mean square error of approximation.

Table 4 shows the estimated factor loadings by ML and Bayesian method (see also figures 1 and 2 for item wordings and relation to factors). The factor loadings are all statistically significant, except ‘MADRS Q3’ in Model 1 and ‘RPWS Q 11’ in Models 2 and 3. Comparing the estimates of factor loadings from ML with the estimates from Bayesian method shows some discrepancies. Some of the distributions of the simulated estimates showed skewness and kurtosis. Further diagnostics from the Bayesian method implies problems with convergence and thus not stable or reliable parameter estimates. The MCMC sampling process started a thinning process to reduce the autocorrelation between successive samples (see the study by Arbuckle38 for a description).

Table 4

Parameter estimates of factor loadings from CFA Models 1–3 estimated by ML and Model 3 by Bayesian method

Parameters, items	ML 1*	ML 2*	ML 3*	Bayes ML 3†
MPI P1Q10	1.000	1.000	1.000	1.000
MPI P1Q8	1.124	1.124	1.145	1.193
MPI P1Q4	1.190	1.187	1.199	1.246
MPI P1Q9	0.788	0.793	0.810	0.843
MPI P1Q17	0.953	0.958	0.973	1.015
MPI P1Q19	1.071	1.059	1.075	1.126
MPI P1Q21	1.319	1.307	1.274	1.327
MPI P1Q12	1.214	1.202	1.145	1.192
MPI P1Q2	0.909	0.903	0.927	0.965
MPI P1Q3	0.756	0.754	0.779	0.813
MPI P2Q1	1.000	1.000	1.000	1.000
MPI P2Q11	1.129	1.080	1.138	1.183
MPI P2Q12	0.938	0.900	0.926	0.967
MPI P2Q6	1.138	1.115	1.143	1.189
MPI P2Q10	1.537	1.563	1.523	1.581
MPI P2Q4	1.231	1.170	1.206	1.256
MPI P2Q8	1.000	−0.492	−0.370	−0.378
MPI P2Q5	1.449	−0.743	−0.662	−0.676
MPI P1Q6	1.000	1.000	1.000	1.000
RPWS Q55	−1.103	−0.664	−0.664	−0.641
RPWS Q7	−1.016	−0.490	−0.489	−0.464
MADRS Q8	0.914	1.116	1.112	1.097
RPWS Q28	1.000	−0.503	−0.503	−0.496
MPI P1Q16	1.000	−0.598	−0.602	−0.623
RPWS Q11	1.000	−0.062	−0.053	−0.057
MADRS Q3	−0.122	1.016	0.955	0.990
MADRS Q4	1.000	0.346	0.382	0.394
MPI P1Q1	1.000	1.000	1.000	1.000
MPI P1Q7	1.063	0.993	0.927	0.944

Bold indicate non statistically significant loadings.

CFA, confirmatory factor analysis; MADRS-S, Montgomery Åsberg Depression Rating Scale; ML, maximum likelihood; MPI-S, Multidimensional Pain Inventory; RPWS, Ryff’s Psychological Well-being Scale.

The conclusion was that the models were not totally acceptable based on goodness-of-fit statistics and not feasible parameter estimates. That some of the problem may be caused from the ordered categorical data structure may not be ruled out based on the results from the Bayesian method.

In the separate explorative factor analysis (PCA) of MPI-S, eight factors were extracted that explained 72.6% of the variance, while the PCA of RPWS extracted five factors that explained 62.6% of the variance. In the PCA of MADRS-S, no rotated component matrix was attained. As the values for communalities were low for the items sleep and appetite (slightly above 200), these were excluded and a new PCA was run. Unfortunately, no rotated component was attained despite the removal of items.

In the joint PCA, all items ≥0.6 for MPI-S and RPWS were chosen, as a level of ≥0.7 left factor seven in MPI-S without any item. MADRS-S was not included in the joint PCA as no rotated component matrix was attained in the separate PCA. The joint PCA of MPI-S and RPWS including items with a factor loading ≥0.6 did not produce any rotated component matrix; thus, another PCA including items with a factor loading ≥0.7 of MPI-S and RPWS was performed. The result is presented in tables 5 and 6. This final PCA extracted an eight-factor solution, which explained 72.7% of the variance. Items and factor structure of the eight-factor solution including 16 items loading ≥0.70 are shown in table 5.

Table 5

Rotated component matrix based on MPI and RPWS

Rotated component matrix*
	Component
	1	2	3	4	5	6	7	8
MPI P1Q10. How much has your pain changed the amount of satisfaction or enjoyment you get from family-related activities?	0.880	-0.070	–0.116	0.177	0.050	–0.071	0.002	0.042
MPI P1Q8. How much has your pain changed your ability to participate in recreational and other social activities?	0.876	0.014	–0.094	0.213	–0.007	–0.041	–0.102	–0.006
MPI P1Q4. How much has your pain changed the amount of satisfaction or enjoyment you get from taking part in social and recreational activities?	0.840	–0.079	–0.157	0.173	0.007	0.113	–0.057	0.030
MPI P1Q21. How much has your pain changed or interfered your friendships with people other than your family?	0.804	–0.188	–0.094	0.024	0.089	0.010	0.015	0.180
MPI P1Q5 How much support or help do you get from your family or others related when you are in pain?	-0.037	0.787	0.289	0.181	0.143	–0.018	0.051	0.101
MPI P2Q10. Encourages me to work on a hobby.	–0.053	0.765	0.142	0.011	0.185	–0.131	0.040	0.285
MPI P2Q8. Gets angry with me.	0.180	–0.704	–0.001	0.079	–0.069	0.094	0.109	0.283
MPI P2Q1. Asks me what he/she can do to help.	0.122	0.702	0.154	0.203	0.096	–0.137	0.008	0.257
RPWS Q3 I am quite good at managing the many responsibilities of my daily life	–0.212	0.197	0.762	–0.015	–0.054	0.172	0.078	0.033
MPI P1Q1. Rate the level of your pain at the present moment.	0.260	0.128	–0.102	0.851	0.063	0.046	0.075	–0.014
MPI P1Q7. On average, how severe has your pain been during the last week?	0.239	0.036	0.025	0.868	0.092	–0.060	–0.124	–0.003
MPI P2Q9. Gets me pain medication.	0.023	0.122	–0.094	0.061	0.776	0.202	–0.047	0.184
MPI P2Q12. Turns on the TV to take my mind off my pain.	0.095	0.257	–0.116	0.111	0.755	–0.040	0.190	0.018
RPWS Q 11. I live life 1 day at a time and don’t really think about the future.	–0.002	–0.214	0.171	0.001	0.127	0.811	–0.074	–0.056
RPWS Q 59. Some people walk through life without plans, but I am not one of them.	–0.014	0.006	0.236	–0.067	0.174	–0.160	0.826	–0.078
MPI P2Q7. Tries to involve me in some activity.	0.081	0.205	–0.029	–0.023	0.119	–0.031	–0.052	0.809

Extraction method: principal component analysis. Rotation method: varimax with Kaiser normalisation.

Bold figures are selected items with factor loading above 0.7. *Rotation converged in 12 iterations.

MPI-S, Multidimensional Pain Inventory; RPWS, Ryff’s Psychological Well-being Scale.

Factor 1 included four items from MPI-S concerning consequences in daily life and relations due to the pain (see table 5). Factor 2 included four items from MPI-S regarding solicitous and distracting and punishing responses from significant others. Factor 3 included one item on life control from RPWS. Factor 4 included two items regarding pain intensity. Factor 5 included two items from MPI-S about solicitous and distracting responses from significant others. Factor 6 included one item from RPWS on life control, as did also factor 7. Factor 8 included one item about solicitous and distracting responses from significant others.

A visual comparison was performed between the factor structure attained in this last joint PCA (see table 5) and the results attained in our previous study with a similar aim and design10 (see figure 1). The results show that 11 items were similar between the two models, while some items only were included in either the new or the old model. Comparing the new model (table 5) with previous (figure 1), all four items included in factor 1, and the four in factor 2, were included in previous study. The single item in factor 3 was not included in previous model. The two items in factor 4 were also included in previous model, while one of the two items in factor 5 was included. The three single items in factors 6, 7 and 8 were not included in previous model. Starting comparison in the previous model (figure 1), six items from MPI-S were not included in factor 1 in the present model (table 5). Three items from MPI-S were not included in factor 2. None of the four items from MPI-S, RPWS and MADRS-S were included in the present model, while both the two items from MPI-S in factor 4 and the two in factor 5 were included. None of the items from factors 6 to 9 was included in the present model.

Discussion

The main findings of the present study are that the CFA not ended up in a model with acceptable validity, neither when the analysis was based on a model from a previous study nor when it was based on a model inspired by the biopsychosocial model. The second step in the analysis, the exploratory factor analysis, resulted in a factor solution only partly corresponding to the previously suggested short-form questionnaire.10 The results of the present study do not support the suitability of the suggested short-form questionnaire for evaluation of symptoms among men and women with non-specific MSDs in any part of the body.

Looking at the items that corresponded to the exploratory factor analysis, all items concerning pain intensity, interference of pain and solicitous/punishing responses from significant others included in the present model were also included in the previously suggested short-form questionnaire.10 For one factor (5), one of the two items was similar. For the rest of the factors, none of the items was included in the results of the previous exploratory factor analysis. One explanation to the differences between the final exploratory factor analysis in the present study and previous is to find in the results of the separate factor analyses. In the present study, no rotated factor structure was attained for MADRS-S, why no items from this questionnaire are included in the final factor structure.

What could be the cause to such a different result of the explorative factor analysis compared with our previous study10? One explanation can be differences between the two samples. The sample in previous study10 included women with non-specific MSDs in the neck/shoulder, while the present included women and men, and non-specific MSDs in any part of the body. The previous study also had a strict schedule of criteria for exclusion, as, for example, low-back pain, previous trauma, rheumatic, inflammatory or neurological disease or fibromyalgia (for details on study protocol, see ISRCTN trial registration number 92199001 at https://www.isrctn.com). The sampling procedure also differed, in that the participants in previous study were recruited via advertising in local papers. A weakness of the present study is the rather small sample, and in addition to this some internal missing data. Several rules of thumb are presented regarding sufficient sample sizes for factor analyses. Subject-to-variable ratios of 4:1 or 5:1 or a sample size of 100 or 200 people are among those.39 Another approach is to determine the stability of a factor solution by determining component saturation (primarily size of factor loadings; second number of items) in combination to total sample size.40 Stable factor solutions can, according to this second approach, be attained using samples as small as 50 with factor loadings at 0.80.40 Sample sizes of 150 and factor loading of 0.60 may also produce stable factor solutions and if four or more items have a factor loading of 0.60 on each component the sample size may even be smaller.40

Yet, another circumstance that may lead to inconsistent results was analysing ordinal data as they were normally distributed. To circumvent this potential problem, we applied Bayesian estimation to compare those results with the ML results in the CFA. There were some discrepancies and indications of lack of convergence. This may be interpreted as that part of the problems with unacceptable goodness-of-fit values may be caused from the ordered categorical data structure in the estimation procedure.

Small sample sizes are a common problem when collecting questionnaire data. Collecting data in this group probably adds to the problem, as difficulty concentrating seem to be part of the symptoms in this group,9 especially as our study required answering several questionnaires. An effort we made to increase the sample size was to ask the personnel to inform the participants regarding the study and the possible benefits for future patients and offer time during the rehabilitation sessions to fill out the questionnaires.

The amount of missing cases in the inclusion of participants is not possible to establish, as participation built on voluntary inclusion, not distribution of a certain number of questionnaires to a specific, selected group. However, we also had missing values for some of the participants. Under the assumption that the missing data process was missing at random, the estimation methods produce efficient and consistent estimates of factor loadings in the CFA. There is, although, a possibility that the missing data process is not at random yielding biased estimates.

IMMPACT recommendations seem sound in including pain intensity, physical function, emotional function and general improvement in assessment of MSDs.13 Including relevant aspects, representing the biopsychosocial aspects that often are affected for a person with MSDs41 when constructing a short-form questionnaire is considered a strength of the present study. However, the recommendation of questionnaires to use might need reconsidering. IMMPACT have recently published recommendations regarding the assessment of physical function and participation in chronic pain, taking the WHO’s International Classification of Function and Health (ICF) in to consideration.42 Unfortunately, these IMMPACT recommendations does not include a thorough analysis of the content and quality of the recommended questionnaires. In the last decade, a development regarding evaluation of questionnaires has taken place. For example, several reviews regarding the content and/or quality of questionnaires for assessment of physical function have been published.43–45 Starting by choosing high-quality questionnaires for the recommended dimensions based on reviews of content and quality could be a possible way to finally end up with a high-quality short-form questionnaire. Another vital aspect is also to attain a larger sample to get a more reliable result of the psychometric tests. One question still rendering answer is whether one short-form questionnaire is possible for measurement of symptoms in all sorts of MSDs or if there is a need to develop specific questionnaires for different MSDs such as neck pain and low back pain.

Conclusion

There is a multitude of different questionnaires that are used in different combinations. Our intention was to create a short questionnaire that combines relevant aspects to measure in this group of patients. By selecting these and combining them into one short questionnaire, the intention was to contribute to a reduction of the use of a multitude of questionnaires and combinations of them and to hopefully increase the comparability between the results of different studies. However, neither the CFA nor the exploratory ended up in an acceptable model; thereby, the results of the present study do not support the suitability of the suggested short-form questionnaire for evaluation of symptoms among men and women with non-specific MSDs in any part of the body. To draw any final conclusions regarding the usefulness of this questionnaire in different samples, as well as among women with neck pain, further studies are required.

Patient and public involvement

Patient and public were not involved in the design or analysis of the study. Patients and staff were involved in data collection, as the staff presented the study and provided questionnaires, while the patients answered the questionnaires.

Table 6

Total variance explained

Total variance explained
Component	Initial eigenvalues			Extraction sums of squared loadings			Rotation sums of squared loadings
	Total	% of variance	Cumulative %	Total	% of variance	Cumulative %	Total	% of variance	Cumulative %
1	5.402	23.487	23.487	5.402	23.487	23.487	4.035	17.544	17.544
2	3.462	15.053	38.540	3.462	15.053	38.540	3.052	13.269	30.813
3	1.662	7.225	45.765	1.662	7.225	45.765	2.092	9.097	39.910
4	1.475	6.415	52.179	1.475	6.415	52.179	1.950	8.480	48.390
5	1.325	5.762	57.941	1.325	5.762	57.941	1.744	7.580	55.971
6	1.218	5.297	63.238	1.218	5.297	63.238	1.370	5.956	61.927
7	1.141	4.962	68.200	1.141	4.962	68.200	1.269	5.519	67.446
8	1.029	4.475	72.674	1.029	4.475	72.674	1.203	5.229	72.674
9	0.925	4.023	76.697
10	0.788	3.426	80.123
11	0.628	2.731	82.854
12	0.604	2.627	85.482
13	0.559	2.432	87.914
14	0.502	2.181	90.095
15	0.448	1.949	92.044
16	0.353	1.533	93.577
17	0.299	1.300	94.878
18	0.291	1.267	96.144
19	0.247	1.073	97.217
20	0.205	0.890	98.107
21	0.171	0.744	98.851
22	0.142	0.616	99.466
23	0.123	0.534	100.000

Extraction method: principal component analysis.