Minimum clinically important difference of the Social Functioning in Dementia Scale (SF-DEM): cross-sectional study and Delphi survey.

OBJECTIVES: Good social functioning is important for people living with dementia and their families. The Social Functioning in Dementia Scale (SF-DEM) is a valid and reliable instrument measuring social functioning in dementia. However the minimum clinically important difference (MCID) has not yet been derived for SF-DEM. This study aims to define the MCID for the SF-DEM.
DESIGN: We used triangulation, incorporating data from a cross-sectional study to calculate the MCID using distribution-based and anchor-based methods, and a Delphi survey. SETTING AND PARTICIPANTS: The cross-sectional survey comprised 299 family carers of people with dementia. Twenty dementia experts (researchers, clinicians, family carers) rated whether changes on clinical vignettes represented a meaningful change in the Delphi survey. PRIMARY OUTCOME MEASURES: We calculated the distribution-based MCID as 0.5 of an SD for each of the three SF-DEM domains (1-spending time with others, 2-communicating with others, 3-sensitivity to others). We used the carers' rating of social functioning to calculate the anchor-based MCID. For the Delphi survey, we defined consensus as ≥75% agreement. Where there was lack of consensus, experts were asked to complete a further survey round.
RESULTS: We found that 0.5 SD of SF-DEM was 1.9 points, 2.2 and 1.4 points in domains 1, 2 and 3, respectively. Using the anchoring analysis, the MCIDs were 1.7 points, 1.7 points, and 0.9 points in domains 1, 2 and 3, respectively. The Delphi method required two rounds. In the second round, a consensus was reached that a 2-point change was considered significant in all three domains, but no consensus was reached on a 1-point change.
CONCLUSIONS: By triangulating all three methods, the SF-DEM's MCIDs were 1.9, 2.0 and 1.4 points for domains 1, 2 and 3, respectively. For individuals, these values should be rounded to a 2-point change for each domain. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is the first study which aims to derive the minimum clinically important difference (MCID) for the Social Functioning in Dementia Scale; this has useful implications for future research aiming to improve social functioning in dementia.

This study triangulates the MCID from three established methods, which are the distribution-based, anchor-based and Delphi-based methods.

The cross-sectional study comprised a large research sample, which was diverse in terms of gender, ethnicity, background and severity of dementia.

A diverse range of dementia experts were recruited onto the Delphi survey, including family carers, doctors, researchers and therapists.

The study took place in the UK so it is not clear if the findings are applicable to populations outside of the UK.

Introduction

Social functioning is important to human experience including for those with dementia. Decline in social functioning—how individuals interact in society and their own personal environment1—is a diagnostic criterion for dementia2 and social functioning impairments are distressing to people with dementia and their families.3 As well as increasing the risk of dementia onset, loneliness and social isolation have lasting psychosocial effects on those living with dementia and their caregivers.4–7 Conversely, increase in social interaction has been associated with improvement in cognition,8 9 and improved quality of life for individuals with dementia.10

Accurate measurement of social functioning in dementia is essential for research aiming to understand the causes of decline in social functioning in dementia, effects of different lifestyles and to assess whether interventions are effective in maintaining or improving social functioning. There are few instruments designed to assess social functioning in dementia.11 We, therefore, developed and psychometrically tested the Social Functioning in Dementia scale (SF-DEM) scale showing it to be acceptable, reliable and valid12 and establishing its factor structure.13 SF-DEM has, therefore, been recommended for research into social functioning in dementia.11

A statistically significant difference between or within groups on an instrument may not equate to a clinically important difference.14 15 An important metric of a scale is therefore the minimum clinically important difference (MCID), defined as ‘the smallest change or difference in an outcome measure that is perceived as beneficial and would lead to a change in the patient’s medical management, assuming an absence of excessive side effects and costs’.16 To our knowledge, no MCID has been reported for the SF-DEM or any other instrument that measures social functioning in dementia. The MCID is subjective and there are several ways to calculate it, including statistically based approaches based on distribution of data or anchoring score changes to another measure, and seeking expert opinion via a Delphi survey.17 In this study, we aimed to derive the MCID of the SF-DEM using all of these approaches. By utilising three established methods to calculate the MCID and subsequently triangulating the results, we aim to mitigate any potential biases which may arise from the different approaches.

Methods

We conducted a study using data from a cross-sectional survey of family carers of people with dementia and a Delphi survey of dementia professionals and experts by experience.

Cross-sectional survey

Setting and participants

This study used data collected from a previous research project13 to calculate the MCID for the SF-DEM. Two hundred and ninety-nine family carers of people with mild, moderate and severe dementia were recruited across three UK National Health Service mental health trusts in Sussex and North London and gave informed consent to participate.

Procedures

Trained researchers conducted a single interview with the family carer particpants to obtain demographic information about the person with dementia and scores for the SF-DEM. The carers also provided information for other validated carer-rated measures used to determine dementia severity, using the Clinical Dementia Rating Scale (CDR).18

Social functioning was assessed using the SF-DEM, a 20-item questionnaire administered by an interviewer. There are seventeen items covering different aspects of social functioning which are divided into three domains (‘spending time with other people’ (dDomain 1), ‘communicating with other people’ (domain 2) and ‘sensitivity to other people’ (domain 3)). These items are scored using a Likert scale (0–4 indicating frequency: ‘never’ to ‘very often’). A score is calculated for each domain; the scale scores range for each domain from 0 to 21, 0–18 and 0–12, respectively. High scores indicate better social functioning. There are also three unscored summary questions which assess overall impression of social functioning, recent change and willingness to make future changes.

Analysis

Distribution-based methods are based on the statistical characteristics of obtained samples. They determine what magnitude of change in an outcome measure, here the SF-DEM score, is greater than what would be expected from chance alone.19 Previous studies have suggested that 0.5 SD may be clinically significant.20–22 We, therefore, calculated the SD of the SF-DEM score for each of the three domains, and 0.5 of the SD

Anchor-based approach determines the MCID by associating the change in the numerical scale to a subjective and independent assessment of improvement.19 This allows a numerical measurement to become ‘anchored’ to a qualitative assessment which is likely meaningful to patients. In this study we used the carers’ overall impression of social functioning (four points on a Likert scale: excellent, good, fair, poor) to anchor to the SF-DEM score. We first calculated whether there was correlation between the score in each of the three SF-DEM domains and the overall impression of social functioning using Spearman’s correlation coefficient. We then calculated, for each domain, the mean difference in SF-DEM domain score, per different overall rating of impression in social functioning (excellent, good, fair, poor), as the anchor-based MCID.

All analyses were conducted using IBM SPSS V.25.

Delphi survey

Twenty dementia experts were recruited as participants, providing informed consent to participate in the survey. We defined experts as researchers who specialise in dementia care, healthcare professionals with experience working with people with dementia, or family members, close friends, or carers of a person with dementia with at least weekly contact.

We created eight brief fictionalised anonymous vignettes for each of the three SF-DEM domains based on previous studies using the SF-DEM scale (see online supplemental appendix 1).12 13 These changes corresponded to improvement and decline in SF-DEM score of 1, 2, 3 and 4 points. The study participants were asked for each vignette ‘do you consider the change described as important to the health or quality of life of the person’, and asked to answer ‘yes’, ‘no’ or ‘not Sure’. They were invited to explain more about their answer to help move to consensus by clarifying and specifying. The survey was delivered using Opinio, a web-based survey tool.

Participants were also asked to define their gender (male, female, other, prefer not to say), ethnicity (using UK census categories), expertise (dementia carer, psychiatrist, neurologist, geriatrician, clinical psychologist, nurse, social worker, occupational therapist, researcher, other), country of residence (UK, other European country, North America, other) and years of experience in caring for or working with people with dementia (less than 5, 5–10, more than 10). The survey was piloted by two researchers prior to being circulated to the participants.

The Delphi method aims to obtain a consensus regarding what would constitute a meaningful change from a panel of experts in the field, using a questionnaire.23 There is a range of recommendations for Delphi study sample sizes,24 including that 10–15 participants would be sufficient to reach consensus.25 Several rounds may be required before the process ends, in order to reach overall agreement.26 We, therefore, aimed for 15 respondents21 and anticipated an attrition rate of approximately 20%–30% over two or three rounds27; thus we recruited 20 experts for the Delphi study. There are no existing guidelines for establishing consensus within a Delphi study, however, many Delphi healthcare studies define consensus as a 75%–80% agreement.28 We, therefore, defined a consensus as an agreement of ≥75% (75% agreement that the change is either considered meaningful or not meaningful).

We analysed the Delphi results by calculating the percentage consensus for each vignette, which corresponded to a change (improvement or decline) of 1, 2, 3 or 4 points in each SF-DEM domain. We then calculated the overall consensus for each point of change by calculating the proportion of study participants who judged that the vignettes reflected a meaningful difference (averaged across the vignettes which described improvement and decline). We judged any level of SF-DEM change as having reached consensus if 75% of participants judged it as being either clinically meaningful or not clinically meaningful. For levels of SF-DEM change where consensus was not reached in the first Delphi round, we wrote new vignettes using feedback and comments provided by the participants in the first round and presented these to study participants for round 2. We determined the MCID using the Delphi method as the minimum agreed meaningful points of change, as decided by a consensus among the Delphi participants. For example, if an overall consensus is reached that a 1-point change is not meaningful or no consensus can be reached, and a 2-point change is meaningful, the MCID would be calculated as 2 points.

Triangulation

We present MCIDs derived from the three methods separately and, as it is common practice to triangulate values from the methods used in order to determine an overall MCID,20 21 29 30 we present a triangulated MCID for each SF-DEM domain by calculating the simple mean (the values summed and divided by three) from the three methods.

Patient and public involvement

No patients involved.

Results

Demographics

Of the 299 carers, the mean age was 63 (SD 14, min 21, max 90) years and 218 (73%) were female. The mean age of the people with dementia was 81 (SD 8, min 55, max 98) years and 179 (60%) were female. Half (148) of the family carers were spouses or long-term partners of the person with dementia and the majority of those remaining 128 (43%) were children of the person with dementia. There was a range of dementia severity as scored on the CDR (very mild=31 (10%), mild=108 (36%), moderate=99 (33%), severe=61 (20%)). Alzheimer’s disease was the the most common dementia subtype (159, 53%).

Distribution method

The mean SF-DEM score was 6.8, 5.9 and 8.3 for domains 1, 2 and 3, respectively. Table 1 details the values of the mean, SD and 0.5 SD for each of the three domains. The MCIDs, defined as 0.5 SD, were calculated as 1.9, 2.2 and 1.4 points for domains 1, 2 and 3, respectively.

Table 1

Distribution-based minimum clinically important difference on SF-DEM scale

Domain	N	Mean SF-DEM score	SD	0.5 SD
1 (Spending time with other people)	296	6.8	3.9	1.9
2 (Communicating with other people)	291	5.9	4.4	2.2
3 (Sensitivity to other people)	293	8.3	2.9	1.4

SF-DEM, Social Functioning in Dementia Scale.

Anchor method

As is reported in table 2, we found a moderate correlation of SF-DEM score vs overall impression in social functioning in domain 1, and a weak correlation in domains 2 and 3. We then calculated the MCID as the mean difference in score per different level of overall impression of social functioning. The MCIDs were 1.7, 1.7 and 0.9 points for domains 1, 2 and 3, respectively.

Table 2

Anchor-based minimum clinically important difference on SF-DEM scale

Domain	N	Impression of social functioning*	N per impression of social functioning	Mean SF-DEM score	Spearman’s Correlation: SF-DEM score vs impression of social functioning	P value	Mean SF-DEM points difference per impression of social functioning (Anchor-based MCID)
1 (Spending time with other people)	296	Excellent	7	9.4	−0.58	<0.001	1.7
		Good	77	9.8
		Fair	93	7.3
		Poor	119	4.4
2 (Communicating with other people)	291	Excellent	7	9.7	−0.29	<0.001	1.7
		Good	78	7.7
		Fair	92	5.5
		Poor	114	4.8
3 (Sensitivity to other people)	293	Excellent	7	10.4	−0.13	0.025	0.9
		Good	80	8.5
		Fair	93	8.8
		Poor	113	7.7

*Response to question ‘Thinking about their social life as a whole, how is it now?’ on Likert scale (1=excellent, 2=good, 3=fair, 4=poor).

MCID, minimum clinically important difference; SF-DEM, Social Functioning in Dementia Scale.

In the first round of the Delphi study, there were twenty participants. Table 3 details the demographic information of the participants in rounds one and two. Approximately two-thirds of the participants were female. The majority of the participants were either psychiatrists (50%) or researchers (30%). They had a varied level of experience, and the majority (95%) were based in the UK. There were 14 participants in the second round of the Delphi survey. Seventy-one per cent of the participants in round 2 were female, and the majority were psychiatrists (43%) or researchers (36%), as in round 1. 93% of participants were based in the UK.

Table 3

Delphi study participants’ demographic information

Category	Demographic information	Delphi round 1	Delphi round 2
		N (%)	N (%)
Gender	Female	13 (65)	10 (71.4)
	Male	7 (35)	4 (28.6)
Ethnicity	White	15 (75)	10 (71.4)
	Asian or Asian British	4 (20)	3 (21.4)
	Other	1 (5)	1 (7.1)
Primary role	Psychiatrist	10 (50)	6 (42.9)
	Academic researcher	6 (30)	5 (35.7)
	Social worker	1 (5)	0 (0)
	Family carer	1 (5)	1 (7.1)
	Clinical psychologist	2 (10)	2 (14.3)
Years of experience	Less than 5	4 (20)	3 (21.4)
	5–10	7 (35)	4 (28.6)
	More than 10	9 (45)	7 (50)
Country of residence	UK	19 (95)	13 (92.9)
	Other	1 (5)	1 (7.1)

Round 1

Table 4 reports the results of both Delphi rounds. In the first round of the Delphi process, a consensus was reached for eight out of the twelve levels of change represented by the vignettes. For domains 1 and 3, there was consensus that 2-point, 3-point and 4-point changes were considered significant but no consensus with regard to a 1-point change. For domain 2, a consensus was reached that 2-point and 4-point changes were considered significant, but no consensus for 1- or 3-point changes.

Table 4

Delphi survey results

	No of points of change	Delphi round 1 (n=20)								Delphi round 2 (n=14)
		Improvement			Decline			Total (combining improvement and decline)		Improvement			Decline			Total (combining improvement and decline)
		Y	N	Not sure	Y	N	Not sure	% agreed meaningful	Consensus reached?	Y	N	Not sure	Y	N	Not sure	% agreed meaningful	Consensus reached?
Domain 1	1	18	2	0	11	5	4	72.5	N	12	2	0	8	4	2	71.4	N
	2	17	2	1	20	0	0	92.5	Y
	3	18	0	2	20	0	0	95	Y
	4	19	0	1	19	0	1	95	Y
Domain 2	1	11	4	5	5	13	2	40	N	11	2	1	1	11	2	42.9	N
	2	18	0	2	18	0	2	90	Y
	3	19	0	1	7	11	2	65	N	14	0	0	14	0	0	100	Y
	4	18	1	1	20	0	0	95	Y
Domain 3	1	9	9	2	7	10	3	40	N	7	5	2	4	9	1	39.3	N
	2	14	4	2	17	0	3	77.5	Y
	3	18	0	2	13	5	2	77.5	Y
	4	17	1	2	18	0	2	87.5	Y

Round 2

We presented amended vignettes for the levels of SF-DEM change which did not reach consensus in round 1. The second round led to a consensus that a 3-point change in domain 2 was clinically significant. However, no consensus was reached with regards to a 1-point change in all three domains with 71%, 43% and 39% viewing 1-point change as meaningful for the three domains (see table 4). Therefore, the Delphi consensus MCIDs are defined as 2 points in all three domains.

Triangulation results

The mean MCIDs from the three methods are 1.9 points for domain 1 (range 1.7–2), 2.0 points for domain 2 (range 1.7–2.2) and 1.4 points for domain 3 (range 0.9–2).

Discussion

We used three different methods, the distribution, anchor and Delphi methods, in order to establish the MCID for the SF-DEM, which measures social functioning in people living with dementia.

The mean MCIDs from the three methods are 1.9 points for domain 1, 2.0 points for domain 2 and 1.4 points for domain 3. If using the MCID for an individual patient, it would be appropriate to consider 2 points as the MCID in all three domains. However, in a research study such as a clinical trial, the triangulated values may be more useful. The MCIDs calculated from the three methods have also been reported separately in this study, and the most applicable value could be used with the researcher’s judgement and prespecified in the study protocol.

For the distribution method, we chose to use the value of 0.5 SD to define the MCID. An influential systematic review demonstrated a consistency of 0.5 SD among the MCIDs reported in health-related quality of life measures in chronic diseases.22 Several studies have since used 0.5 SD as the value to calculate the MCID using the distribution method.20 31–33 The data in the cross-sectional survey13 find a significant correlation between SF-DEM scores and carers’ impression of social functioning. This allowed us to successfully anchor the SF-DEM to the subjective opinions of the family carers and calculate MCIDs using this method. The results of the Delphi survey led to a consensus on 2, 3 and 4-point changes, but to no consensus with regards to a 1-point change in all three domains with some experts judging it as important and others not. Interestingly, in round 2, there was a 71.4% agreement (close to the required 75% consensus) that a 1-point change in domain 1 was significant, in contrast to domains 2 and 3, which had 42.9% and 39.3% agreement for a 1-point change respectively. Nonetheless, we conclude that it would be appropriate to consider a 2-point change in domain 1 as clinically important, particularly when using the MCID to compare the scores of individual patients, given it is the most conservative estimate and so most appropriate in this context.

In other long-term conditions, similar methods have been used to determine an MCID to measure social functioning.34 35 One study used the distribution and anchor method to establish the MCID for a scale measures quality of life (including social functioning) in children with cerebral palsy. Another used the anchor method to determine the MCID for a scale which measures quality of life (including social functioning) after total knee replacement.

Strengths and limitations

Stengths include the fact that no previous study has established an MCID for the SF-DEM. This innovation is therefore useful for future research which uses the SF-DEM as an outcome measure. We also used three different methods in order to calculate the MCID, each of which have their specific benefits and limitations. The distribution and anchor methods were determined using data from the cross-sectional survey which took place in 2019.13 This study used a large research sample, which was diverse in terms of gender, ethnicity, background and severity of dementia. The distribution method is a standardised method of statistical analysis which has been demonstrated to be consistent.22 However, this method is not recommended as a first line means for determining MCID due to the lack of an anchor value which links the scores to a value that is meaningful to patients.19 36 We have mitigated this potential drawback by also using the anchor method, which anchors the score to the subjective views of the family carers, and the Delphi method, thus taking expert opinion into account (in this study encompassing the views of clinicians, researchers, social workers, psychologists and family carers).

A limitation of the anchor-based method is that the results will differ depending on the choice of the anchor.15 The anchor used in this study enabled us to factor in the subjective views and experiences of family carers for people with dementia. These are the individuals who spend the most time with people with dementia, and arguably may have the most insight into a clinically important change. There were 20 participants in first round of the Delphi survey, and 14 participants in the second round. We had a diverse research sample with an appropriate sample size for a Delphi survey. However, the participants were mostly UK-based which renders the results less applicable to populations outside of the UK.

Conclusions

The results of this study specify that the MCID values for the SF-DEM are 1.9 points for domain 1, 2.0 points for domain 2 and 1.4 points for domain 3. These values are derived from the distribution-based, anchor-based and Delphi methods. Given the lack of consensus regarding a 1-point change for all three domains in the Delphi survey, it would be appropriate to round the MCID to a 2-point change as a more conservative value. As the SF-DEM is the only validated scale for measuring social functioning in dementia,13 these results are of potential value for future research in this field. The calculation of the MCID will allow future researchers to identify a change which is of clinical benefit to patients, when using the SF-DEM as an outcome measure and therefore enable research into the important person-centred domain of social functioning for people with dementia.