Factor structure and convergent validity of the Derriford Appearance Scale-24 using standard scoring versus treating 'not applicable' responses as missing data: a Scleroderma Patient-centered Intervention Network (SPIN) cohort study.

OBJECTIVE: Valid measures of appearance concern are needed in systemic sclerosis (SSc), a rare, disfiguring autoimmune disease. The Derriford Appearance Scale-24 (DAS-24) assesses appearance-related distress related to visible differences. There is uncertainty regarding its factor structure, possibly due to its scoring method.
DESIGN: Cross-sectional survey.
SETTING: Participants with SSc were recruited from 27 centres in Canada, the USA and the UK. Participants who self-identified as having visible differences were recruited from community and clinical settings in the UK. PARTICIPANTS: Two samples were analysed (n=950 participants with SSc; n=1265 participants with visible differences). PRIMARY AND SECONDARY OUTCOME MEASURES: The DAS-24 factor structure was evaluated using two scoring methods. Convergent validity was evaluated with measures of social interaction anxiety, depression, fear of negative evaluation, social discomfort and dissatisfaction with appearance.
RESULTS: When items marked by respondents as 'not applicable' were scored as 0, per standard DAS-24 scoring, a one-factor model fit poorly; when treated as missing data, the one-factor model fit well. Convergent validity analyses revealed strong correlations that were similar across scoring methods.
CONCLUSIONS: Treating 'not applicable' responses as missing improved the measurement model, but did not substantively influence practical inferences that can be drawn from DAS-24 scores. Indications of item redundancy and poorly performing items suggest that the DAS-24 could be improved and potentially shortened. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

This is the first study to evaluate concerns regarding the scoring methodology of items with a ‘not applicable’ response option in the Derriford Appearance Scale-24 (DAS-24).

Two large samples of individuals with physical disfigurement were drawn from clinical and community settings.

This study used confirmatory factor analysis, a contemporary and theory-driven approach to testing factor structure and goodness of fit, with a weighted least squares means and variance adjusted estimator which performs well with observed variables on an ordinal scale.

The two samples did not include the convergent validity measures used in the original DAS-24 study.

Convergent validity could not be compared between the two samples due to differences in the measures taken for each sample.

Introduction

Systemic sclerosis (SSc or scleroderma) is an autoimmune disease characterised by skin thickening, internal organ fibrosis, disability, significant appearance changes and disfigurement in highly visible parts of the body.1 Common physical appearance concerns include hypopigmentation and hyperpigmentation of the skin, changes to skin texture, telangiectasia, hand contractures, sclerodactyly, calcinosis and altered facial features, including a pinched appearance to the nose and eyes, loss of lip contour with thinning of the lips and a decreased ability to fully open the mouth. Treatments may also lead to appearance changes: certain immunosuppressant drugs can cause hair loss and long-term steroid use may contribute to a Cushingoid facial appearance.2

Visible differences in appearance are associated with distress, poorer self-esteem, body image problems and diminished quality of life.3 In SSc, visible differences are common4 and associated with greater body image dissatisfaction,5–7 poorer psychosocial functioning8 and increased anxiety and depression.9 Thus, there is a significant need to study physical appearance concerns in SSc and to validate measures related to appearance in SSc.10 11

The Derriford Appearance Scale-24 (DAS-24),12 13 the short form of the DAS-59,14 measures distress and dysfunction related to physical appearance concerns in populations with visible differences such as SSc. The DAS-24 is hypothesised to yield a single factor representing appearance-related distress and dysfunction.13 In an adult sample of 535 patients with appearance concerns, the DAS-24 total score demonstrated internal consistency reliability (α=0.92), concurrent validity with the DAS-59 criterion measure (r=0.88) and convergent validity (rs=0.45–0.66 with measures of anxiety, depression, social avoidance and distress, fear of negative evaluation, internalised shame).12

However, recent results from 1265 community and clinical participants with visible difference suggested that the DAS-24 may not be unidimensional.15 Exploratory factor analysis with principal axis factoring and oblique rotation revealed two factors: general self-consciousness (18 items) and sexual and bodily self-consciousness (six items).15 The interfactor correlation was not reported; it is presumed to be large given that internal consistency for the total score was high (α=0.93). The authors concluded that the DAS-24 may be conceptualised as either the original unidimensional total score or the two subscale scores.15

One possible explanation for the uncertain factor structure may relate to the DAS-24 scoring procedure. To complete the measure, respondents first identify and describe a part of their appearance that is concerning to them (referred to as their ‘feature’). The respondent then rates each item on a 1–4 scale, with higher values indicating greater appearance-related distress and dysfunction.13 Fourteen items include a ‘not applicable’ (NA) option (scored as 0) to accommodate situations that are not relevant to a respondent. For example, an unemployed respondent might choose NA for the work item. Total scores are calculated by summing the items.13 Thus, respondents who marked many items as NA could receive a low total score, even if they report significant appearance-related distress and dysfunction on the other items.

At the broadest level, NA responses in surveys are best considered within the context of missing data.16 Treating these responses as very low scores, as is the procedure for the DAS-24, could lead to factorial solutions that represent an artefact of the NA procedure itself. In the two-factor model reported by Moss et al,15 the second factor (sexual and bodily self-consciousness) was composed of six items, which all had the NA option (distressed at beach, avoid communal changing, avoid undressing with partner, distressed playing sports/games, distressed by clothing limitations, affects sex life). Among these items,15 the NA response was chosen an average of 26% of the time (range: 19%–36%). NA was only chosen an average of 13% of the time (for the items with the NA option) that loaded on the first factor of general self-consciousness (range: 6%–32%). It is also worth noting that several items without the NA option (eg, distressed at reflection) loaded on the first factor (general self-consciousness) but conceptually match the second factor (sexual and bodily self-consciousness). Thus, it is possible that the two-factor solution may at least partly reflect shared method variance, rather than the conceptual coherence of two factors.

Given these concerns, the aim of this study was to evaluate two different scoring methods to determine whether the standard scoring procedure affects the factor structure and convergent validity of DAS-24 scores. The first aim was to evaluate the one-factor model in a sample of patients with SSc and Moss et al’s visible difference sample15 using the two scoring methods. In the standard method, as described in the DAS-24 manual13 and existing studies, NA responses were scored as 0. In the missing method, NA responses were modelled as missing data. It was hypothesised that model fit for the one-factor model scored according to the missing method would be superior in both samples. The second aim was to evaluate the convergent validity of DAS-24 scores calculated via both scoring methods in the SSc sample. It was hypothesised that DAS-24 scores would have a strong association with satisfaction with appearance (rs≥0.5), moderate to strong associations with fear of negative evaluation and social interaction anxiety (r≈0.4–0.5) and a moderate association with depressive symptomatology (r≈0.3). It was also hypothesised that correlations would be significantly stronger when the missing method was used.

Methods

Patients and procedure

SSc sample

Patients enrolled in the Scleroderma Patient-centred Intervention Network (SPIN) Cohort completed study questionnaires from April 2014 through April 2016. Research Ethics Committee approval of the Jewish General Hospital and by the Institutional Review Boards of each participating center was obtained. Patients were enrolled at 27 centres from Canada, the USA and the UK. Eligible participants were1 classified as having SSc according to the 2013 American College of Rheumatology/European League Against Rheumatism classification criteria17 as confirmed by a SPIN physician,2 at least 18 years of age,3 able to give informed consent,4 fluent in English or French and5 able to respond to questionnaires via the Internet. Physicians or supervised nurse coordinators from SPIN recruiting sites invited eligible patients to participate, obtained written informed consent and completed a medical data form to initiate patient registration in the study. After registration, an automated welcome email was sent to patients with instructions to activate their SPIN account and complete the online questionnaires. SPIN participants complete outcome measures online on enrolment and subsequently every 3 months; the present study includes only the baseline assessment.

Visible difference sample

Data were obtained from the study by Moss et al.15 Respondents were community and clinical participants who self-identified as being ‘visibly different’ recruited through advertisements, general practice physicians and outpatient medical clinics including prosthetics, dermatology, ophthalmology, general plastics, burn, ear/nose/throat clinics (including cleft lip and palate), oncology and laser treatment. Data were downloaded from https://peerj.com/articles/1070/#supplemental-information.

Main study measure (SSc sample and visible difference sample)

DAS-2412 13 contains 24 items with a response scale from 1 to 4; 10 items have only this scale; 14 items also include a NA option (scored as 0). Total scores range from 10 to 96, with higher scores indicating a greater level of appearance-related distress and dysfunction. Good internal consistency reliability and validity has been reported.13 In both of the present samples, internal consistency reliability calculated using the standard method was: SSc α=0.92; visible difference α=0.96.

Convergent validity measures (SSc sample)

Social Interaction Anxiety Scale-6 (SIAS-6)18 is the six-item short form of the SIAS19 and measures anxiety resulting from social interactions. Respondents rate statements (eg, I feel tense if I am alone with just one person) on a scale from 0 to 4. Total scores range from 0 to 24, with higher scores indicating greater social anxiety. Internal consistency reliability in the SSc sample was α=0.89.

Patient Health Questionnaire-8 (PHQ-8)20 is an adapted version of the PHQ-9,21 a measure of depression that has been validated for use in SSc.22 Frequency of symptoms (eg, Little interest or pleasure in doing things) over the past 2 weeks is rated on a scale from 0 to 3. Total scores range from 0 to 24, with higher scores indicating more symptoms of depression. Internal consistency reliability in the SSc sample was α=0.89.

Brief Fear of Negative Evaluation Scale-Revised (BFNE-II)23 is a 12-item short form of the BFNE24 that assesses apprehension and distress of being negatively evaluated. Respondents rate statements (eg, I worry about what kind of impression I make on people) on a scale from 1 to 5. Total scores range from 12 to 60, with higher scores indicating greater fear of negative evaluation. Internal consistency reliability in the SSc sample was α=0.98.

Satisfaction with Appearance Scale (SWAP)25 is a 14-item measure of satisfaction with appearance developed for burn survivors,25 then adapted for SSc and validated.6 9 Respondents rate statements about their physical appearance or social discomfort on a scale from 0 to 6. The SWAP yields two subscale scores: social discomfort (eg, My appearance makes other people uncomfortable) which reflects social discomfort related to body dissatisfaction (six items; range 0–36 with higher scores indicting greater discomfort) and dissatisfaction with appearance (eg, I am satisfied with the appearance of my face) which reflects dissatisfaction with various body parts (eight items; range 0–48 with higher scores indicating greater dissatisfaction). The SWAP has demonstrated good measurement properties in SSc.7 9 Internal consistency reliability in the SSc sample was: social discomfort α=0.93, dissatisfaction with appearance α=0.92.

SSc measures (SSc sample)

Years since the first non-Raynaud’s phenomenon symptoms was calculated by taking the deviation between the date that forms were completed online and the date of a patient’s first non-Raynaud’s symptom. This method is recommended as an approximation of the onset of SSc.26

The Modified Rodnan Skin Score (mRSS),27 a clinician-administered indicator of skin disease severity, is calculated by measuring the extent and severity of skin thickening on 17 body surfaces by palpation on scale ranging from 0 to 3. Total scores range from 0 to 51 with higher scores indicating greater severity.

SSc disease subtype is classified by the pattern and severity of skin fibrosis.28 The limited subtype is characterised by skin involvement distal to the elbows and knees, slower fibrosis and milder internal organ involvement. Limited patients without cutaneous involvement are classified as having sine SSc. The diffuse subtype is characterised by more extensive skin thickening involving proximal extremities and/or the trunk in addition to proximal involvement, rapidly progressing fibrosis and more significant internal organ involvement.

Appearance-related variables documented by study physicians included telangiectasia (facial or any), skin pigmentation changes (facial or any) and hand contractures. Telangiectasia involves dilated superficial blood vessels that appear as red lines or patches on the skin that collapse on pressure and fill slowly when pressure is released. Skin pigmentation changes were defined as hyperpigmentation or hypopigmentation. Disfigurement due to hand contractures (tightening around the proximal interphalangeal joints, metacarpals and/or wrists that limits range of motion) was defined as the percentage of limitation in range of motion, categorised as none/mild (0%–25%), moderate (25%–50%), or severe (>50%).

Data analysis

Scoring

DAS-24 scores were calculated using two methods. The standard method scored the 14 items with the NA option using the 0–4 metric described in the manual.13 That is, 0 was used when a person chose NA for an item and 1–4 indicated that a person had chosen that response option. The second method, the missing method, scored the 14 items with the NA option such that NA responses were treated as missing data (described below in the section Analytic Strategy) and 1–4 indicated that a person had chosen that response.

Analytic strategy

Confirmatory factor analysis (CFA) was conducted in Mplus V.7.229 to examine the one-factor model in the SSc and visible difference samples using both scoring methods. Weighted least squares means and variance adjusted (WLSMV) estimation, which is robust to non-normal and non-independent data, was used to handle the ordinal rating scale of the DAS-24. Missing data were handled using a variation of the full information maximum procedure employed for WLSMV estimation in Mplus which is analogous to pairwise present analysis.29 Models were evaluated according to statistical and descriptive indices, given that χ2 tests have significant limitations, including a high degree of dependence on sample size.30 Following Bentler’s31 recommendations, the root mean square error of approximation (RMSEA)32 and Comparative Fit Index (CFI)33 were evaluated. Cut-off thresholds were based on commonly used recommendations.34 For the RMSEA, values. 0.06–0.08 were interpreted as acceptable model fit, with values <0.06 indicating close fit. For the CFI, values. 0.90–0.95 were interpreted as acceptable model fit, with values >0.95 indicating close fit. Standardised factor loadings were evaluated and compared across the two scoring methods.

Convergent validity analyses of the DAS-24 were conducted using the SSc sample. In order to capture differences between scoring methods, two DAS-24 averages were computed for each respondent. To represent the standard method, values for all 24 items were totaled (NA responses were scored as 0) and divided by 24 to yield a standard method average score (possible range 0.42–4.00). To represent the missing method, values for all items for which a respondent chose a numeric value were totaled (10–24 items) and divided by the total number of items that were answered to yield a missing method average score (possible range 1.00–4.00). Bivariate correlations between both average scores (standard and missing) and each convergent validity measure (SIAS-6, PHQ-8, BFNE-II, SWAP) were evaluated in SPSS V.24.0.35 Correlation coefficients for each scoring procedure were compared using methods described by Steiger36 via an online calculator.37

Results

Description of SSc and visible difference samples

Characteristics of the SSc sample (n=950) are reported in table 1. The majority of participants were female (87%), married (85%) and white (73%). Age ranged from 18 to 84 years (M=55.5). The average skin thickening (mRSS score) was 8.0 (range 0–48); this differed by disease subtype (diffuse M=13.6 (SD=10.4), limited/sine M=4.2 (SD=4.2)). Nearly three-quarters of the sample reported telangiectasia, one-third reported skin pigmentation changes and one-third reported hand contractures.

Table 1

Systemic sclerosis sample description (n=950)

	M±SD or n (%)
Demographic characteristics
Age in years (n=947)	55.48±12.05
Language
English	882 (92.8)
French	68 (7.2)
Sex
Women	827 (87.1)
Men	123 (12.9)
Race/ethnicity
White	807 (84.9)
Black	54 (5.7)
Other	88 (9.3)
Missing	1 (0.1)
Marital status
Married/partnered	692 (72.8)
Never married	111 (11.7)
Divorced/separated	110 (11.6)
Widowed	37 (3.9)
Employment status
Full-time employment	282 (29.7)
Part-time employment	103 (10.8)
Unemployed	123 (12.9)
Homemaker, student, other	439 (46.5)
Missing	1 (0.1)
Disease characteristics
Years since first non-Raynaud’s symptoms (n= 876)	11.75±8.90
Modified Rodnan Skin Score (n=759)	7.98±8.71
Disease subtype
Diffuse	383 (40.3)
Limited/sine	558 (58.8)
Not available	9 (0.9)
Telangiectasia (any; n=950)
Yes	677 (71.3)
No	257 (27.1)
Not available	16 (1.7)
Telangiectasia (face; n=677)
Yes	452 (66.8)
No	75 (11.1)
Not available	150 (22.1)
Pigmentation changes (any; n=950)
Yes	285 (30.0)
No	600 (63.2)
Not available	65 (6.8)
Pigmentation changes (face; n=285)
Yes	147 (51.6)
No	94 (33.0)
Not available	44 (15.4)
Hand contractures (n=950)
No/mild (0%–25%)	674 (70.9)
Moderate (25%–50%)	172 (18.1)
Severe (>50%)	50 (5.3)
Not available	54 (5.7)

Detailed characteristics of the visible difference sample (n=1265; community n=614, clinic n=651) are available elsewhere.15 The majority of participants were female (69%), married (62%) and white (81%). Age ranged from 18 to 91 years (M=47.3).

Description of DAS-24 scores: SSc sample and visible difference sample

DAS-24 scores were lower for the SSc (M=35.7, SD=13.7) than visible difference (M=41.7, SD=16.3) sample (Hedges’ g=0.39). A description of the patterns of NA responses is available in table 2. In the SSc sample, the proportion of NA responses ranged from 11% to 49% per item; in the visible difference sample, the proportion of NA responses ranged from 6% to 36% per item. Item 5 (self-consciousness affects work; SSc=44%, visible difference=32%) and item 16 (distressed playing sports/games; SSc=49%, visible difference=36%) had the highest percentage of NA responses.

Table 2

Data missing due to ‘not applicable’ and standardised factor loadings for the one-factor model of the Derriford Appearance Scale-24 using the SM and MM for the SSc and visible difference samples

Item	SSc			Visible difference
	Not applicable, n (%)	SM λ	MM λ	Not applicable, n (%)	SM λ	MM λ
1. Feeling confident	–	0.65	0.68	–	0.68	0.71
2. Distressed at reflection	–	0.77	0.77	–	0.77	0.79
3. Irritable at home*	207 (21.8)	0.76	0.76	204 (16.1)	0.75	0.80
4. Feel hurt†	–	0.78	0.79	77 (6.1)	0.78	0.81
5. Self-consciousness affects work*	415 (43.7)	0.45	0.80	401 (31.7)	0.58	0.77
6. Distressed at beach*	289 (30.4)	0.48	0.77	241 (19.1)	0.64	0.77
7. Misjudged due to appearance*	260 (27.4)	0.65	0.69	237 (18.7)	0.60	0.65
8. Feel feminine/masculine	–	0.57	0.62	–	0.57	0.62
9. Self-conscious of appearance*	128 (13.5)	0.73	0.71	79 (6.2)	0.75	0.77
10. Feel irritable	–	0.73	0.73	–	0.72	0.74
11. Adopt concealing gestures	–	0.68	0.65	–	0.68	0.66
12. Avoid communal changing*	377 (39.7)	0.36	0.64	248 (19.6)	0.55	0.67
13. Distressed in supermarkets/department stores*	113 (11.9)	0.65	0.73	101 (8.0)	0.66	0.74
14. Feel rejected	–	0.79	0.79	–	0.82	0.82
15. Avoid undressing with partner*	249 (26.2)	0.44	0.72	351 (27.7)	0.41	0.72
16. Distressed playing sports/games*	462 (48.6)	0.18	0.71	460 (36.4)	0.43	0.74
17. Close into shell	–	0.77	0.79	–	0.74	0.75
18. Distressed by clothing limitations*	285 (30.0)	0.67	0.73	343 (27.1)	0.62	0.74
19. Distressed at social events*	104 (10.9)	0.76	0.84	85 (6.7)	0.75	0.84
20. Feel normal	–	0.73	0.76	–	0.73	0.76
21. Affects sex life*	266 (28.0)	0.50	0.76	321 (25.9)	0.56	0.77
22. Avoid leaving house	–	0.73	0.76	–	0.75	0.78
23. Distressed at others remarks about appearance*	312 (32.8)	0.65	0.71	212 (16.8)	0.67	0.75
24. Avoid pubs/restaurants*	137 (14.4)	0.62	0.78	98 (7.7)	0.59	0.78

*Items with the ‘not applicable’ option.

†This item contained ‘0/not applicable’ responses in the visible difference data but was scored using the 1–4 scale only in the SSc sample; ps for all factor loadings <0.001.

MM, missing method (‘not applicable’ scored as missing); SM, standard method (‘not applicable’ scored as 0); SSc, systemic sclerosis.

CFA: SSc sample

Overall fit for the one-factor model scored according to the standard method was not adequate (χ2(252)=2415.6, P<0.001; CFI=0.896, RMSEA=0.095; table 3). Standardised factor loadings (table 2) were statistically significant and ranged from small to large (λs=0.18–0.79, P<0.001). When the missing method was used, overall fit for the one-factor model was improved and good (χ2(252)=1188.0, P<0.001; CFI=0.958, RMSEA=0.063; table 3). Standardised factor loadings (table 2) were statistically significant and large (λs=0.62–0.84, P<0.001).

Table 3

Goodness of fit statistics for one-factor models of the DAS-24

Data	Scoring method	χ2	df	P value	CFI	RMSEA (90% CI)
SSc	Standard	2415.63	252	<0.001	0.896	0.095 (0.092–0. 099)
	Missing	1188.01	252	<0.001	0.958	0.063 (0.059–0. 066)
Visible difference	Standard	4593.25	252	<0.001	0.870	0.117 (0.114–0. 120)
	Missing	2061.49	252	<0.001	0.952	0.075 (0.072–0. 078)

Note. Standard method: ‘not applicable’ scored as 0; missing method: ‘not applicable’ scored as missing.

CFI, Comparative Fit Index; DAS-24, Derriford Appearance Scale-24; RMSEA, root mean square error of approximation.

The factor loadings for several items with a high number of NA responses were larger when scored according to the missing method. These differences can be seen for item 5 (% missing=44%; Standard λ=0.45, Missing λ=0.80), item 6 (% missing=30%; Standard λ=0.48, Missing λ=0.77), item 12 (% missing=40%; Standard λ=0.36, Missing λ=0.64), item 15 (% missing=26%; Standard λ=0.44, Missing λ=0.72) and item 16 (% missing=49%; Standard λ=0.18, Missing λ=0.71).

CFA: visible difference sample

Overall fit for the one-factor model scored according to the standard method was not adequate (χ2(252)=4593.3, P<0.001; CFI=0.870, RMSEA=0.117; table 3). Standardised factor loadings (table 2) were statistically significant and ranged from moderate to large (λs=0.41–0.82, P<0.001). When the missing method was used, overall fit for the one-factor model was improved and good (χ2(252)=2061.5, P<0.001; CFI=0.952, RMSEA=0.075; table 3). Standardised factor loadings (table 2) were statistically significant and large (λs=0.62–0.84, P<0.001).

The factor loadings for several items with high a high number of NA responses were larger when scored according to the missing method. These differences can be seen for item 5 (% missing=32%; Standard λ=0.58, Missing λ=0.77), item 15 (% missing=28%; Standard λ=0.41, Missing λ=0.72) and item 16 (% missing=36%; Standard λ=0.43, Missing λ=0.74).

Convergent validity: SSc sample

Correlations describing the relationship between average DAS-24 scores (standard and missing) and scores of social interaction anxiety, depression, fear of negative evaluation, social discomfort and dissatisfaction with appearance are in table 4. Correlations for the standard method average score were large and statistically significant (rs=0.44–0.69). Correlations for the missing method average score were large and statistically significant (rs=0.47–0.72). In all but one case, the missing method correlations were significantly larger than the standard method correlations, but the magnitude of differences were very small (0.02–0.07).

Table 4

Correlations of standard and missing method average Derriford Appearance Scale-24 (DAS-24) scores with convergent validity measures and evaluation of significant differences between correlations

Validity measure	r		P value
	Standard	Missing
Social Interaction Anxiety score (total score)	0.51	0.53	0.012
Depressive symptoms (Patient Health Questionnaire-8 total score)	0.65	0.72	<0.001
Fear of negative evaluation (Brief Fear of Negative Evaluation-Revised total score)	0.62	0.62	0.916
Social discomfort (Satisfaction with Appearance Scale (SWAP) subscale score)	0.69	0.71	0.001
Dissatisfaction with appearance (SWAP subscale score)	0.44	0.47	0.027

Note. ps<0.001 for all rs.

Discussion

Confirmatory factor analyses revealed that a one-factor DAS-24 model fit poorly when using the standard method, but that the one-factor model fit well when using the missing method in both the SSc and visible difference samples. Correlations with measures of convergent validity calculated using the missing method were significantly larger than those calculated using the standard method, but the implications for inferences drawn from DAS-24 scores are likely negligible given the small magnitude of differences.

The finding that a one-factor model fit well when items were scored in a manner more logically consistent with their intent (missing method) supports the original conceptualisation of the DAS-24 as representing a single latent construct of distress and dysfunction related to appearance.13 This also adds to a growing literature suggesting that model-based missingness methods are the most suitable solution for dealing with NA responses.16 On one hand, this would suggest that parameter estimates for models based on the standard method are misleading and that use of the DAS-24, which includes items that are not relevant to some respondents, should be discontinued. On the other hand, convergent validity coefficients were similar across scoring methods, suggesting that use of the existing version of the DAS-24, despite its less optimal structure and scoring method, may not substantively impair its utility. Although factor structure is an important component of construct validity, it should be considered in the context of all validity evidence and, in particular, the extent to which a measure’s internal structure affects criterion validity and its practical application.38 Thus, it appears that scoring NA items as 0 yields statistical problems when models are judged by stringent criteria, but that the implications for the overall validity of DAS-24 scores using the standard method may be of less consequence.

The DAS-24’s reasonably strong functional performance, as assessed via convergent validity analyses suggests that there is likely excessive redundancy within the measure. When items rated as NA were modelled using the standard method, factor loadings were substantially impaired, and, for many items, clearly unacceptable. Nonetheless, convergent validity was only minimally reduced, and internal consistency reliability was very high (SSc α=0.92; visible difference α=0.96). It is well-established that very high Cronbach’s alphas (eg, α >0.90) tend to reflect overly redundant items.39–42 Thus, it seems that there is a core group of DAS-24 items that function well, and the inclusion of unnecessary items and even problematically scored items does not have a substantive negative impact on the practical performance of the measure or clinical inferences that can be drawn from it.

Nonetheless, the problems of unnecessary item redundancy and problematically scored items are concerns in research and clinical contexts because they reduce measurement precision and increase survey length and time burden on in populations with physical appearance concerns, like SSc. These concerns are particularly salient in collaborative research contexts where medical and patient-reported outcome data are collected from large numbers of patients across span countries, languages and clinical settings, as is the case with the SPIN cohort. Research on the DAS-24 should focus on reducing the number of items and addressing existing scoring issues with the goal of maximising item parsimony and retaining the full construct domain of distress and dysfunction related to concerns with physical appearance as previously defined.12 13 Recently, optimal test assembly methods have been used to shorten measures based on objective, prespecified criteria,43 and these methods could be used to identify a shorter version of the DAS-24 that performs similarly and eliminates its problematic scoring.

This study contributes to the growing literature on the DAS-24 using two large samples composed of respondents with diverse physical appearance concerns that are generalisable to other visible difference populations. There are also limitations that should be considered in the context of the current results. First, the SSc data did not contain the convergent validity measures used in the original DAS-24 validation study, and thus measures of closely related constructs were used as proxies. Second, convergent validity analyses could not be conducted in the visible difference sample as these measures were not available.

In sum, these findings suggest that scoring the DAS-24 according to the standard method leads to a suboptimal measurement model. The missing method yields a superior one-factor structure but does not lead to substantively different relationships with theoretically related constructs. Thus, while it is preferable to treat NA responses as missing, particularly in research settings, it is unlikely that using the standard method greatly impacts the practical interpretation of DAS-24 scores. The development of a short form that removes item redundancy while retaining adequate psychometric properties is needed and would particularly benefit research in populations with visible differences in appearance such as patients with SSc.