Development and validation of an instrument to measure collaborative goal setting in the care of patients with diabetes.

OBJECTIVE: Despite known benefits of patient-perceived collaborative goal setting, we have a limited ability to monitor this process in practice. We developed the Patient Measure of Collaborative Goal Setting (PM-CGS) to evaluate the use of collaborative goal setting from the patient's perspective. RESEARCH DESIGN AND METHODS: A random sample of 400 patients aged 40 years or older, receiving diabetes care from the Virginia Commonwealth University Health System between 8/2012 and 8/2013, were mailed a survey containing potential PM-CGS items (n=44) as well as measures of patient demographics, perceived self-management competence, trust in their physician, and self-management behaviors. Confirmatory factor analysis was used to evaluate construct validity. External validity was evaluated via a structural equation model (SEM) that tested the association of the PM-CGS with self-management behaviors. The direct and two mediated (via trust and self-efficacy) pathways were tested.
RESULTS: A total of 259 patients responded to the survey (64% response rate), of which 192 were eligible for inclusion. Results from the factor analysis supported a 37-item measure of patient-perceived CGS spanning five domains: listen and learn; share ideas; caring relationship; measurable objective; and goal achievement support (χ=4366.13, p<0.001; RMSEA=0.08). Results from the SEM supported the external validity of the PM-CGS. The relationship between CGS and self-management was partially mediated by perceived competence (p<0.05). The direct effect between the PM-CGS and self-management was significant (p<0.001).
CONCLUSIONS: CGS can be validly measured by the 37-item PM-CGS. Use of the PM-CGS can help illustrate actionable deficits in goal-setting discussions.

Patient reports of collaborative goal setting have been linked to increased self-management and trust in the physician.

This study produced a valid measure of collaborative goal setting.

This measure can help highlight actionable deficits in goal-setting discussions.

Introduction

Previous studies have found that patient reports of collaborative goal setting with healthcare providers are associated with increased perceived self-management competence1–3 and trust in their physician.1 4 Patient reports of engaging in collaborative goal setting have also been found to be associated with improved self-management behaviors and health outcomes, including improvements in Hemoglobin (Hb) A1c and blood pressure levels among patients with diabetes and hypertension.1 5–7 However, despite recommendations for its use8–9 and the growing evidence of its benefits, we continue to have a limited understanding of how to measure and foster this process in practice.

Heisler et al5 hypothesized that collaborative goal setting is a process that involves five domains: (1) sharing responsibility for making decisions, (2) mutually agreeing on the goal, (3) discussing self-care management options, with the patient, (4) sharing beliefs about illness treatment, and (5) information about their life and values.5 To the best of our knowledge, no study has empirically tested the conceptualization put forth by Heisler.5 Moreover, studies that have shown associations between patient-reported engagement in collaborative goal setting and improved outcomes have not enabled an understanding of the specific processes necessary for patients to acknowledge their participation in a collaborative goal-setting process. Instead, these prior studies have each relied on the Patient Assessment of Chronic Illness Care (PACIC), an instrument that leaves the definition of collaborative goal setting open to the respondents' individual interpretations.1 6 10

We have previously used qualitative research methods to identify what needs to occur for a goal-setting process to be considered collaborative in the eyes of patients with diabetes.11 Findings from that study indicated that patients conceptualize collaborative goal setting as a multidimensional process that unfolds over time within the context of a caring relationship with their clinician.11 Similar to how Heisler et al5 conceptualized the process, patients identified multiple domains of collaborative goal-setting discussions: (1) the patient and physician listen and learn from each other, (2) the patient and physician share ideas, (3) the patient and physician agree on a measurable objective, and (4) the physician provides support for goal achievement.

The primary objective of this study was to use results from our prior formative study to develop a patient-reported measure of collaborative goal setting that could be used for monitoring and evaluating the occurrence of collaborative goal setting among patients with diabetes. We report findings from analyses testing the internal and external validity of the Patient Measure of Collaborative Goal Setting (PM-CGS) measure. The latter was done by using the previously hypothesized pathways through which engagement in collaborative goal setting is thought to impact self-management behaviors.1–4

Research design and methods

Sample

Study eligible participants were those patients receiving care for their diabetes from the Virginia Commonwealth University Health System (VCU-HS). Eligible patients were aged 40 years or older with an outpatient visit to a primary care or endocrinology for diabetes (ie, ICD-9=250 or 366.41, 357.2, 362.0) between August 2012 and August 2013. Patients <40 years of age and those with gestational diabetes were excluded. Individuals who had previously participated in a focus group in support of our formative work11 were also excluded. Patients were identified using the structured data contained within the electronic health record at VCU-HS. From among the over 18 000 patients identified, we used random sampling methods to select N=400 patients for study inclusion. This was achieved using the SPSS V.21 random number generator.

Initial measure development

Initial measure development was divided into two consecutive phases. In the first phase, we developed survey items for potential inclusion in the PM-CGS based on results from our formative research. Thus, while Heisler's conceptualization of collaborative goal setting was considered, items developed were based primarily on findings from a previously conducted qualitative study in which we used focus groups to explore patient perceptions of collaborative goal setting in diabetes care.11 That study resulted in a conceptualization of collaborative goal setting as including five domains: (1) listen and learn from each other (ie, patients and physicians listen and learn from each other), (2) share ideas (ie, the physician shares his or her ideas and gives the patient the opportunity to share), (3) the context of a caring relationship (ie, physicians have a good bedside manner, are compassionate and sensitive to patient needs), (4) agree on a measurable objective (ie, patients agree on a measureable objective with their physician), and (5) support for goal achievement (ie, the provision of support by the physician in a number of forms: emotional, tangible, or instrumental).11 For each of these domains we developed 6–11 survey items with a 5-point Likert-type response format ranging from 1 (strongly disagree) to 5 (strongly agree). In total, 77 items were originally developed.

In the second phase, the initial pool of 77 items was revised and refined by incorporating input from an expert panel and by conducting cognitive interviews. The expert panel included a psychometrician, a primary care physician, a health communication specialist, and a health psychologist. Input from the expert panel was used to revise question wording and eliminate redundant items (n=10). The remaining 67 items were tested via patient cognitive interviews. Cognitive interviews used the ‘concurrent think aloud’ method12 13 and were conducted among a convenient subsample of original focus group participants from our qualitative study of two males (one black and one white) and two females (one black and one white). These interviews were used to pilot test the items for comprehensibility and relevance, and to ensure items reflected themes identified from the focus groups. Participants were asked to provide their thoughts as they completed the questionnaire and responses were probed for further insight.13 Cognitive interviews took on average 30 mins to complete (range: 25–40). Based on feedback from the cognitive interviews, the initial pool of items was further reduced resulting in a total of 44 items for further consideration in the PM-CGS.

Survey administration

A letter of study introduction and the survey were mailed to patients in October 2013. The letter of study introduction described the study in general terms, asked that the patient complete the enclosed questionnaire, and included a $2.00 bill. Survey administration followed a Dillman approach:14 two weeks after the initial mailing, non-respondents received a reminder postcard; after an additional 2 weeks, non-respondents were sent another survey packet. Correspondence was mailed using first class postage, personalized communication, and stamped return envelopes, each of which has been shown to improve response rates.15 16 Patients who returned a survey received a $20 gift card to a local retail store. Data entry for returned surveys was achieved with optical mark recognition using Remark OMR. To be included in the current analyzes, the survey respondent had to report engaging in a health-related goal discussion in the previous 6 months with a healthcare provider.

Measures

The questionnaire included the PM-CGS items, as well as the following previously validated measures: the Perceived Competence Scale,17 the Trust in Physician Scale,18 and the Summary of Diabetes Self-Care Activities (SDSCA) measure.19 The survey also included items specific to the patient's socio-demographic characteristics including age, race, gender, marital status, employment status, income, and level of education achieved. The collaborative goal setting and self-management measures were treated as latent variables, each comprising five domains.

Patient perceptions of collaborative goal setting

The PM-CGS items were designed to measure the extent to which patients report engaging in collaborative goal setting with their healthcare provider/provider team over the past 6 months. Measure responses depicted participation in collaborative goal setting within five domains: (1) listen and learn from each other, (2) share ideas, (3) caring relationship, (4) agree on a measurable objective, and (5) support for goal achievement. For each domain, there were between five and nine survey items for which respondents were asked to express the degree to which they agree with the statement using a 5-point scale ranging from strongly disagree (1) to strongly agree (5) with higher scores representing patient reports of relatively more collaborative goal setting. A summary score across all domains provided an overall assessment of the PM-CGS where higher scores also represented patient reports of more collaborative goal setting.

Statistical methods

Confirmatory factor analysis with full information maximum likelihood estimation20 was used to test the internal validity (ie, measurement model) of the instrument. Prior to conducting the confirmatory factor analysis, survey data were evaluated for missing data. No item was found to be missing more than 5%.

Consistent with our conceptual model of collaborative goal setting, we tested a second-order factor model. This model used a multidimensional representation of collaborative goal setting where the five first-order factors were represented by the five conceptual domains of collaborative goal setting as previously identified: (1) listen and learn from each other, (2) share ideas, (3) caring relationship, (4) agree on a measurable objective, and (5) support goal achievement. Collaborative goal setting was represented as a second-order factor. Mplus V.6 was used to estimate the model. Hu and Bentler21 criteria were used to evaluate model fit.21 Items with standardized factor loadings <0.40 were removed prior to final model estimation.

Structural equation modeling (SEM)was used to test the external validity of the final measure. This was done by testing the pathways through which collaborative goal setting impacts self-management behaviors. Collaborative goal setting and self-management variables are specified as latent variables in the SEM (figure 1) where each latent variable is comprised of five domains and the items that collectively represent the respective domains. As depicted in figure 1, perceived competence and trust were used as observed variables mediating the relationship between collaborative goal setting and self-management. The model was tested with and without the presence of covariates.

Figure 1

Unadjusted and adjusted structure equation model parameter estimates.

Results

Sample characteristics

A total of 259 individuals returned the survey (64% response rate). Of those, 20 individuals were excluded (n=19 did not report having diabetes and n=1 reported having gestational diabetes). Among the remaining 239 individuals, n=192 reported discussing a health-related goal with their clinician in the past 6 months and were included in the current analyses. Respondents were able to designate their goal topic from a list of five common goals. Of the 192 individuals, goal topics included: weight loss, exercise, healthier eating, smoking cessation, or better control of blood pressure, lipid levels, or HbA1c. The mean age of eligible survey respondents was 60.1 years (SD=9.36; range 41–89); 71% were women and 67% were black (table 1). The majority had graduated from high school (69%) and was insured (92%). A total of 32% were married, and 45% had an annual income below $15 000.

Table 1

Sample characteristics (n=192)

Age	60.1 (SD=9.36)
Gender
Male	29%
Female	71%
Education
Grades 1–8	9%
Grade 9–11	22%
High school graduate	25%
College 1–3 years	25%
College graduate	19%
Marital status
Currently married	32%
Never married	24%
Separated	9%
Divorced	22%
Widowed	13%
Race
White	27%
Black	67%
White and American Indian	6%
Hispanic or latino	3%
Employed	25%
Insured	92%
Income
$0–$14 999	45%
$15 000–$74 999	42%
$75 000 or more	12%
Self-reported health status
Excellent	2%
Very good	17%
Good	38%
Fair	33%
Poor	10%
Current smoker	18%
Depressive symptoms	47%

On average, patients endorsed that the goal-setting process used with their clinician over the past 6 months was collaborative (mean=4.47, range 1 (strongly disagree) to 5 (strongly agree)). Participants were found to report high levels of trust in their physician (mean=85.02, SD=14.14, range 33–100), as well as high levels of perceived self-management competence (mean=5.80, SD=1.51). Patients reported engaging in each of the self-management behaviors most days of the week. On average, they reported engaging in general diet self-management behaviors 4.91 days (SD=1.74), specific diet 4.22 days (SD=1.51), exercise 3.72 days (SD=2.23), blood sugar testing 5.04 days (SD=2.42), and foot care 4.37 days (SD=1.43) all with ranges from zero to seven.

Internal validity

Table 2 lists the 37 items included in the instrument. The overall fit of the model was good (χ2=4366.13, df=666; p<0.001; RMSEA=0.07). Each of the items had significant (p<0.001) loadings onto their respective domains. These first-order factors had loadings that ranged from 0.57 to 0.93 (mean=0.78; median=0.78) (table 2). Within the second-order factor model, where the construct of collaborative goal setting is represented by the five first-order factors, each factor is significantly (p<0.001) and strongly (range: 0.82–0.95; mean=0.90; median=0.90) associated with the construct of collaborative goal setting (table 2). The internal consistency of each of the five domains was high (range: 0.894–0.940), as was the level of consistency for the second-order factor collaborative goal setting (α=0.927).

Table 2

Second-order factoral model: collaborative goal setting confirmatory factor analysis (n=192)

First-order factors	Factor loadings	Mean (SD)
Factor 1: Listen and learn from each other	0.82	4.6 (0.60)
Item 1: I asked my doctor any questions I had	0.77	4.6 (0.77)
Item 2: My doctor asked me if I had any concerns	0.73	4.7 (0.80)
Item 3: My doctor explained the reasons for the goal	0.77	4.7 (0.63)
Item 4: I learnt important things from my doctor	0.70	4.6 (0.79)
Item 5: My doctor and I discussed the reasons for the goal	0.80	4.6 (0.79)
Item 6: I listened to what my doctor had to say	0.64	4.6 (0.82)
Item 7: I told my doctor important things about me	0.61	4.6 (0.76)
Item 8: I told my doctor about any concerns I had	0.63	4.5 (0.95)
Item 9: My doctor gave me the opportunity to ask any questions I had	0.72	4.7 (0.82)
Factor 2: Share ideas	0.91	4.4 (0.80)
Item 10: I made sure my doctor knew about things that were important to me	0.66	4.4 (1.03)
Item 11: I told my doctor about important things in my life	0.66	4.2 (1.13)
Item 12: My doctor shared his/her ideas with me	0.80	4.4 (1.02)
Item 13: I was interested in my doctor's ideas	0.73	4.5 (0.92)
Item 14: My doctor provided important medical information to me	0.77	4.5 (0.87)
Item 15: I shared my ideas with my doctor	0.77	4.4 (0.90)
Item 16: I felt confident my doctor understood what was important to me	0.90	4.5 (1.01)
Factor 3: Caring relationship	0.90	4.6 (0.60)
Item 17: My doctor treated me as a person	0.78	4.8 (0.72)
Item 18: I respected my doctor's opinions	0.93	4.7 (0.69)
Item 19: My doctor showed he/she cared about me as a person	0.93	4.6 (0.74)
Item 20: My doctor respected my opinion	0.83	4.6 (0.79)
Item 21: My doctor was honest with me	0.90	4.7 (0.68)
Item 22: My doctor spent enough time with me	0.81	4.5 (0.87)
Item 23: I showed my doctor that I cared about achieving the goal	0.74	4.5 (0.86)
Item 24: I was honest with my doctor	0.72	4.7 (0.71)
Factor 4: Agree on a measurable objective	0.95	4.4 (0.79)
Item 25: I felt good about the goal	0.79	4.3 (0.94)
Item 26: My doctor helped me understand what the specific goal is	0.84	4.5 (0.82)
Item 27: I had confidence that I could achieve the goal	0.72	4.2 (1.05)
Item 28: My doctor and I agreed on the specific goal that was set	0.82	4.4 (0.94)
Item 29: My doctor and I discussed the potential specifics of the goal	0.79	4.4 (0.93)
Factor 5: Support for goal achievement	0.90	4.3 (0.87)
Item 30: My doctor gave me information I could take home about the goal	0.57	4.3 (1.14)
Item 31: I told my doctor I felt like I could achieve the goal	0.78	4.3 (0.93)
Item 32: My doctor and I discussed strategies for achieving the goal	0.89	4.2 (1.06)
Item 33: I was comfortable discussing any challenges I might have achieving the goal	0.74	4.4 (0.93)
Item 34: My doctor made me feel like I could achieve the goal	0.88	4.4 (0.94)
Item 35: My doctor and I came up with a strategy for how to achieve the goal	0.90	4.2 (1.15)
Item 36: My doctor checked to make sure I understood the goal	0.88	4.3 (1.08)
Item 37: My doctor described how to achieve the goal	0.90	4.3 (1.03)

Note: All factor loadings are significant (p<0.001). Standardized parameter estimates are shown. All second-order factor loadings are in bold. The second-order factor has Cronbach's α of 0.927.

External validity

The overall fit of the unadjusted model was good (χ2=4827.38, df=820; p<0.001; RMSEA=0.07). Three of the pathways tested within the model were found to be statistically significant. Collaborative goal setting was significantly associated with increased perceived competence (p<0.03) as well as with self-management behaviors (p<0.001). Furthermore, perceived competence was significantly associated with self-management behaviors (p<0.001). Patient reports of physician trust were not related to either collaborative goal setting or self-management behaviors (p>0.10). Standardized parameter estimates can be found in figure 1.

In addition to testing the pathways of significance, a mediation model was also tested as increased reports of collaborative goal setting were shown to be significantly associated with increased perceived self-management competence, which was significantly associated with increased self-management behaviors. The mediation model that was tested, therefore, was whether the relationship between collaborative goal setting and self-management was mediated by the patient's perceived competence. Results supported that the relationship between collaborative goal setting and self-management was partially mediated by perceived competence (p<0.05).

After controlling for patient socio-demographic characteristics, the partial mediation model with perceived competence was no longer statistically significant (p=0.055). However, the direct effects remained significant: collaborative goal setting and perceived competence were significantly associated with improved self-management (p<0.001), and collaborative goal setting remained associated with improved perceived competence (p<0.05). In addition, a positive relationship between collaborative goal setting and a patient's trust in their physician also became statistically significant (p<0.05).

Covariate effects are shown in table 3. Older patients were more likely to report engaging in collaborative goal setting, have a high level of perceived competence, and were more likely to report trust in the physician. Black patients were less likely than whites to report engaging in collaborative goal setting. Compared with patients with a college degree, patients with a high school degree were less likely to report engaging in collaborative goal setting. Finally, patients with lower levels of education were less likely to have a high level of trust in their physician and to report engaging in self-management behaviors.

Table 3

Regression coefficients of constructs from the adjusted structural equation model

	Collaborative goal setting outcome	Perceived competence outcome	Trust in physician outcome	Self-management outcome
Socio-demographics as predictors
Age	0.06	0.18**	0.01	0.00
Race
Black	−0.01	−0.14	0.14	0.14
Other	0.08	−0.15*	−0.12	0.05
Education
Less than HS	0.12	0.08	−0.11	−0.03
HS degree	−0.03	−0.11	−0.09	−0.20**

*p<0.05

**p<0.01.

HS, high school.

Conclusions

Among a sample of patients receiving outpatient care for their diabetes, a multidimensional measure of collaborative goal setting was created. Consistent with an a priori conceptualization of collaborative goal setting,11 as well as other previous depictions,5 the measure consisted of items that span five domains: (1) listen and learn from each other, (2) share ideas, (3) caring relationship, (4) agree on a measureable objective, and (5) support for goal achievement. The PM-CGS measure is reliable and its external validity was supported among patients with diabetes. Collaborative goal setting, as measured by the PM-CGS, was positively and substantively associated with improved self-management behaviors. In addition, we found collaborative goal setting, as measured by the PM-CGS, might be associated with improved self-management behaviors by improving patients' perceived competence for those behaviors. Furthermore, once patient socio-demographic characteristics were controlled, patient reports of collaborative goal setting as reported by the PM-CGS were also associated with improved physician trust, a patient–provider relationship characteristic known to be associated with improved health outcomes.

The PM-CGS was designed to ascertain the extent to which a collaborative goal-setting process occurred between a patient and their clinician team. As such, the PM-CGS builds on the initial conceptualization of collaborative goal setting put forth by Heisler,5 and brings forth the patients' voice to the conceptualization by focusing on those domains previously identified in a qualitative study as important to patients' consideration of collaborative goal setting. Consistent with Heisler,5 the conceptualization of collaborative goal-setting process tested here depicts collaborative goal setting as a communication process in which patients and their clinicians share ideas and information, and reach agreement on a goal or target level. However, our prior qualitative findings11 led us to test the appropriateness of adding that these interactions needed to occur within the context of a caring relationship with the clinician, and that they required ongoing support for a goal to be achieved. Results support the appropriateness of including both these domains, as they were associated with the overall collaborative goal-setting construct. The PM-CGS assesses the components of collaborative goal setting, including the patient's and the healthcare providers' responsibilities that patients endorse as important to collaborative goal setting. As the focus of this measure is the patient's perception of the extent to which the interaction that transpires between them and clinicians reflected a collaborative goal- setting process, factors such as goal quality or other goal characteristics is not considered.

As a result of this research, a tool now exists that can appropriately and accurately measure collaborative goal setting. With such a measure in hand, we can use patient reports to identify when collaborative goal setting has occurred in practice as well as gain a better understanding of which of the specific components of collaborative goal setting may be falling short. It is the ability to elucidate these missing components that is critical to our ability to facilitate the use of CGS in practice.

There are a number of strengths and limitations associated with this study. Our primary aim with this study was to identify a set of patient-reported items that could be used to accurately measure the construct of collaborative goal setting. Having done so, this now provides an opportunity to investigate how to reduce the number of items included and thus, the feasibility of using such a measure to routinely assess the quality of patient engagement in diabetes care. Previous studies have illustrated the utility and practicality of using shorter instruments to reduce participant burden.22

One of the limitations is that the sample was limited to survey respondents receiving care from VCU-HS. As such, study participants may not be reflective of populations in other healthcare settings or those not responding to the survey. Another limitation is the relatively small sample size for the development of a measure. While there was a high response rate, only 192 respondents (74%) reported engaging in a goal-oriented discussion. A priori power estimates suggested a minimum of 200 respondents. Thus, non-significant findings may be due to insufficient power. Even with a small sample size, the effect sizes indicated that the PM-CGS was a highly reliable measure of collaborative goal setting. However, while we were able to uncover robust findings regarding the external validity of the collaborative goal- setting measure, power may have precluded the detection of other important relationships. Finally, while the conceptual framework on which survey items were developed highlights the importance of collaborative goal setting occurring over time, the cross-sectional survey design precluded exploration of this dimension of collaborative goal setting. On the other hand, strengths of the study include the use of a diverse and random patient sample for which we were able to achieve a relatively high response rate to a mailed survey.

Maximum likelihood is often a preferred estimator for items with a 5-ordered category response format especially when item response distribution is symmetrical. Ideally, an asymptotically distribution-free estimator (eg, diagonally weighted least squares) is appropriate when N is large and item distributions are non-symmetrical. In this study, we presented a relatively small N and non-symmetrical item distributions and opted for maximum likelihood estimator. Studies with large N will be needed to replicate our results using an asymptotically distribution-free estimator in future studies.

We have shown that a patient-reported instrument can be used to measure collaborative goal setting between patients and their clinicians. The PM-CGS enables us to determine if collaborative goal setting has occurred, and allows us to identify the specific components that are lacking when it does not occur. The ability to do so is important as patient reports of collaborative goal setting have been repeatedly shown to be associated with improved health outcomes.