Protocol for development, calibration and validation of the Patient-Reported Inventory of Self-Management of Chronic Conditions (PRISM-CC).

INTRODUCTION: Assessing and measuring patients' chronic condition self-management needs are critical to quality health care and to related research. One in three adults around the world live with multiple chronic conditions. While many patient-reported measures of self-management have been developed, none has emerged as the gold standard, and all have one or more of the following limitations: (1) they fail to measure the different domains of self-management important to patients, (2) they lack sufficient specificity to support patient-centred care or identify the specific components of self-management interventions that work and/or (3) they lack suitability for patients with multiple chronic conditions. METHODS AND ANALYSIS: The Patient-Reported Inventory of Self-Management of Chronic Conditions (PRISM-CC) is being developed to overcome these shortcomings. It will measure respondents' perceived success (or difficulty) in self-managing seven domains important to patients. The protocol has three phases. Phase 1 is conceptual model development and item generation. Phase 2 is assessment of the relevance and understanding of items by people with chronic conditions. Phase 3 is item analysis, dimensionality assessment, scaling and preliminary validation of the PRISM-CC using an online survey of people with chronic conditions (n~750). The expected completion date is early 2021. ETHICS AND DISSEMINATION: This study will adhere to the Canadian Tri-Council Policy Statement on Ethical Conduct for Research Involving Humans. Ethics approval for all phases has been obtained from the Nova Scotia Health Authority Research Ethics Board. Once completed, the PRISM-CC will be made available for research and healthcare at minimal to no cost. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Patient-Reported Inventory of Self-Management of Chronic Conditions (PRISM-CC) development will be based on a validated conceptual framework that includes seven domains of self-management identified by patients as relevant to their experiences.

PRISM-CC will be designed and validated for patients with one or more chronic conditions, facilitating broad application in clinical and research settings.

Items will be calibrated using item response theory models, enabling their use in computer-adaptive testing.

Initial calibration will be based on data collected online from a convenience sample, which may limit diversity and generalisability.

Additional studies will be required to assess applied construct validity and scale invariance across different populations.

Introduction

Globally, approximately one in three adults live with multiple chronic conditions.1 Chronic condition management and patient ‘self-management’, in particular, are a major focus of health service policy, redesign and research.2 3 Self-management is defined by the Institute of Medicine as the ‘tasks that individuals must undertake to live well with one or more chronic conditions’,4 and includes the daily actions people do and do not take to successfully manage their chronic conditions.5 It includes making decisions, taking action and altering behaviours in the context of living every day.6–8 Learning to self-manage is not a one-time event. It is a lifelong process because chronic conditions, and life itself, fluctuate and change over time.8–12

Assessing and measuring patients’ difficulty self-managing their condition(s) are thus critical to quality healthcare and related research. While many patient-reported measures of self-management have been developed, none has emerged as the gold standard, and all have major limitations: (1) failure to measure the different domains of self-management important to patients, (2) lack of sufficient specificity to support patient-centred care or identify the specific components of self-management interventions that work, and/or (3) lack of suitability for patients with multiple chronic conditions.

Failure to measure the different domains of self-management important to patients

Extensive qualitative research on patient self-management, including a number of rigorous reviews, demonstrates that self-management consists of different domains, or aspects of self-management, comprising different skills, abilities and attitudes patients use to address the challenges of living everyday life with a chronic condition.8 9 13–18 For example, the Taxonomy of Everyday Self-Management Strategies (TEDSS) has identified seven different domains of self-management important to patients (see table 1).19 Yet, a scoping review conducted to understand the different instruments used to measure self-management found that 15 of 28 included self-management measures are unidimensional, providing only a single score.20 While statistical evidence for the unidimensionality of some measures is documented,21–23 this may result from limiting the focus to one domain of self-management24 or from the use of insufficient statistical criteria such as principal components analysis or exploratory factor analysis.25 Other measures, for example, the Partners in Health Scale, provide a single composite score but analyses provide strong evidence of underlying multidimensionality.24 26 27 While the scoping review found that 13 of the 28 measures were multidimensional, none of them captured the range of domains important to patients, as identified by the TEDSS.19

Table 1

Domains in the TEDSS and associated working definitions used by PRISM-CC

Domain	TEDSS definition	PRISM-CC working definition
Process strategies	Strategies used to be well informed and to make good decisions. Often used to support use of other, non-process strategies.	Self-perceived success at seeking information, being aware of choices and making good decisions.
Resource strategies	Proactively seeking, pursuing and/or managing needed formal or informal supports and resources.	Self-perceived success at seeking, pursuing and/or managing needed formal or informal supports and resources.
Activity strategies	Finding ways to participate in everyday activities (leisure activities, work activities, household chores) despite problems such as fatigue, pain, memory loss or disability.	Self-perceived success in participating in everyday activities (leisure activities, work activities, household chores).
Internal strategies	Preventing and managing stress, negative emotions and internal distress; creating inner calm.	Self-perceived success at creating inner calm by preventing and managing stress, negative emotions and internal distress.
Social interaction strategies	Managing social interactions and relationships to be able to participate without exposure to negative reactions.	Self-perceived success at disclosing health issues, managing social interactions and relationships.
Health behaviour strategies	Maintaining a healthy lifestyle in order to enhance health and limit the risk of lifestyle-related illness.	Self-perceived success at maintaining a healthy lifestyle.
Disease controlling strategies	Preventing, controlling and limiting symptoms, complications and/or disease progression.	Self-perceived success at managing health conditions including managing medications and treatments, monitoring symptoms and limiting complications.

A more detailed description of TEDSS domains with examples of patient strategies used in each is provided in online supplementary table 1.

PRISM-CC, Patient-Reported Inventory of Self-Management of Chronic Conditions; TEDSS, Taxonomy of Everyday Self-management Strategies.

Lack of sufficient specificity for individualised patient care and self-management research

Single-score measures can be brief, reliable and valid instruments for screening, risk prediction and for use in outcome studies28 but they have limited utility in differentiating individual patients’ self-management support needs or in personalising or tailoring care. A Cochrane review (19 studies n=10 856 participants) concluded that when compared with usual care, personalised care planning leads to improvements in ‘physical and psychological health status, and peoples’ capacity to self-manage their condition’.29 Measures that differentiate self-management into separate domains are needed to move beyond ‘one size fits all’ interventions that are responsive to patients’ diverse and changing needs over their disease trajectories. Research to isolate the critical ingredients of self-management interventions also depends on the ability to differentiate and measure separate domains of self-management.

Suitability for patients with multiple chronic conditions

A growing percentage of patients experience multiple chronic conditions (ie, multimorbidity),1 yet condition-specific measures of self-management are most common. Of the 28 measures identified in the scoping review, 20 were specific to a particular condition.20 While disease-specific measures clearly play an important role in care and research, there is a growing need for measures of self-management appropriate for patients with multiple chronic conditions and complex needs. Non-medical aspects of living with chronic conditions, such as social isolation combined with social determinants of health such as low socioeconomic status, have large effects on patients’ ability to self-manage.30 Having multiple chronic conditions also makes self-management more difficult for patients.11 13 31 32

In addition to addressing the above limitations, measures of self-management are needed that can be routinely used, in diverse settings, with minimal to no cost. Licencing costs pose practical barriers to routine use. For use in clinical care, measures must be efficient to administer, yet provide sufficient detail to support individualised care. There is a practical trade-off between multidimensional measures that have higher response burden and are more time-consuming to administer, and shorter, unidimensional scales that lack meaningful detail. Innovations in measurement, such as computer-adaptive testing, using well-calibrated items, can minimise response burden for patients while providing information on multiple domains.33

This paper describes the protocol for development of a new measure of self-management: the Patient-Reported Inventory of Self-Management of Chronic Conditions (PRISM-CC). Specifically, the PRISM-CC will be designed to measure patient-reported success (or difficulty) self-managing in each of the domains of the TEDSS conceptual framework (see below).19 The PRISM-CC aims to overcome the limitations of current self-management measures. A conceptual framework that includes domains of relevance to patient experiences forms its foundation, enabling patients and health providers to identify areas of difficulty and design individualised care. PRISM-CC will also facilitate research to isolate and test critical ingredients of self-management programmes and related social and environmental determinants of the different domains. It will be designed and validated for patients with single and multiple chronic conditions.

Methods and analysis

Development of the PRISM-CC will follow the Patient-Reported Outcomes Measurement Information System (PROMIS) Instrument Development and Validation Scientific Standard, V.2.0.34 This paper first describes the preliminary development of the PRISM-CC (phases 1 and 2) then outlines the protocol for the final study (phase 3). Phase 1 was completed in the fall of 2018 by generating a bank of potential items from existing measures and available qualitative data material (figure 1). During phase 2, completed in February, 2020, items were tested in an online survey (n=40 persons with multiple chronic conditions), followed by cognitive interviews to select potential items for further testing. In phase 3, the bank of potential items will be administered to a large and diverse sample of persons with chronic conditions (n~750) between March and November 2020. Item analysis, dimensionality assessment, scaling and preliminary validation of the PRISM-CC will then be conducted. The expected completion date is early 2021.

Figure 1

Flow diagram for the development of the Patient-Reported Inventory of Self-Management for patients with Chronic Conditions. PRISM-CC, Patient-Reported Inventory of Self-Management of Chronic Conditions; TEDSS, Taxonomy of Everyday Self-management Strategies.

Patient and public involvement

In developing a patient-reported outcome measure, inclusion of ‘patients’, both as participants and research partners, is important. Patient perspectives and involvement have and will continue to be an integral part of development of the PRISM-CC. Two members of the research team are ‘patient–partners’ (community members, each living with multimorbidity and/or complex chronic health conditions) and other members of the research team live with multimorbidity. These team members have and will continue to be involved in project planning and key decision-making meetings. For example, during phases 1 and 2, patient–partner team members aided in the development and refinement of the items, provided feedback on the design and usability of the first online survey and facilitated participant recruitment. Their continued input will be sought throughout the development of PRISM-CC, including the calibration study, format of the final PRISM-CC outcome measure, and project dissemination. Additionally, patient participants with chronic conditions provided original data to inform PRISM-CC item development through survey and cognitive interviewing processes. Further data collection from patient participants will play an important role during phase 3.

Phase 1: generate potential items for each domain in our conceptual framework (completed)

Due to its comprehensiveness, recency and applicability to patients with multiple conditions, the TEDSS, a conceptual framework that describes the everyday strategies people use to live with and manage their conditions was chosen to guide development of the PRISM-CC. It incorporates a broad and inclusive definition of self-management.8 15 The construction and validation of the TEDSS conceptual framework are described by Audulv et al.19 Briefly, TEDSS was developed using concept mapping methodology,35 then refined and validated using original data from individuals with one or more neurological conditions. First, data from 77 qualitative and quantitative research papers were analysed in order to clarify the overlapping concepts of coping, adapting and self-managing a neurological condition.15 Framework domains (n=7) and subdomains (n=26) were then refined and validated using original data from repeated, structured interviews with 117 individuals with neurological conditions over a 11-month period. Individuals with neurological conditions vary widely by age of onset, trajectory and impact; motor, perceptual, cognitive and psychological symptoms exist in variable combinations, providing a strong foundation for understanding self-management in individuals with multiple conditions. Additionally, many of these individuals also had non-neurological conditions such as heart disease, diabetes and hypertension. To confirm applicability beyond neurological conditions, the framework was compared with three additional conceptual frameworks, identified in the literature, which included patients with prevalent and diverse types of chronic conditions.7 13 36 Since PRISM-CC will measure respondents’ perceived success (or difficulty) in self-managing each TEDSS domain, TEDSS domain definitions were reframed into PRISM-CC measurement definitions, as shown in table 1.

Potential items for each domain in the PRISM-CC were first drawn from outcome measures identified in the scoping review of self-management outcome measures.20 The context (instructions associated with answering the question), item, any stem (initial phrase of the item, repeated across items), response options, recall period and origin of items were tabulated. Additional items were generated using qualitative data previously collected in Canada to develop and validate the TEDSS framework37 and by members of the research team. Grounding items in real-life examples is expected to increase face validity.

The research team reviewed 250 potential items using an iterative process that included assessment of face validity, coherence, relevance and patient-centred wording. No fewer than three research team members participated in the assessment of items in each domain. Based on consensus, items were eliminated if (1) they did not conceptually fit a domain, (2) were semantically identical/redundant, (3) the content was too narrow or disease specific to be applicable to patients with multiple chronic conditions or (4) the item did not assess patient perception of success or difficulty in self-managing. Item stems and response scales were developed to meet PROMIS standards and recommendations (eg, preferred response set options, time reference, context).34 A total of 30–35 items per domain (n=231) were selected for further assessment for relevance and understanding to persons with multiple chronic conditions.

Phase 2: assess potential items for relevance and understanding to people with multiple chronic conditions; generate preliminary item bank (completed)

Online survey number 1

People (n=40) with two or more chronic conditions were recruited to complete an online survey of potential items (n=231). The purposive sample was recruited via posters displayed in public areas and healthcare settings; newsletters and distribution lists and social media. Participants had to be able to read/speak English and have self-reported diagnosis of two or more chronic conditions. Interested participants first underwent a screening telephone interview to assess inclusion criteria and gather data needed to achieve maximum variation: age (younger or older than age 50); gender identity (man, woman or other); high school completion (yes or no) and number of diagnoses and time since diagnoses. The first 20 eligible participants were invited to participate. Subsequent recruitment, using the same methods, was purposive, filling gaps to achieve maximum variation in overall participant profiles.

Surveys were administered using an anonymous online survey platform housed on a Canadian University server. To reduce participant burden, the item pool (total n=231) was divided in half, with each participant rating only 50% of the items. Participants were asked to complete each potential item using a 5-point response scale. They also rated each item for difficulty and importance using a 10-point rating scale.

Item analysis was conducted based on item responses and answers to questions assessing difficulty and importance. Given the small sample size, analyses were considered indicative only and used to flag extremely poorly fitting items and those requiring cognitive interviewing. Response distributions provided evidence of potential floor and ceiling affects, while poor item-rest correlations identified items likely to have poor fit to the domain or that appeared to be interpreted differently than expected. Exploratory factor analysis was used, with caution given the small sample size, to provide insight into item groupings within domains where many item-rest correlations were weak. Low mean scores on importance and/or high scores on difficulty provided insight into respondents’ views on importance and ease of understanding.

Items that performed well on all aspects of the item analysis, and were consistently evaluated as easy to understand and important to respondents, were carried forward to phase 3. Items which showed inconsistency in understandability or importance, or which performed moderately well in the item analysis, were carried forward to the cognitive interviews. Those with overall poor performance were removed from the item bank.

Face-to-face cognitive interviews

A subsample of survey participants (n=10) were invited, based on availability and maximum variation, to participate in individual cognitive interviews. Participants met individually with trained research staff to assess a version of the survey with 30–40 items. This resulted in two to five cognitive interviews per item. One team member conducted the interview, while a second-team member took field notes. Interviews were audio-recorded and transcribed verbatim.

Each item was first scored by the participant. This was followed by an interview using ‘think-aloud’ methodology that invited participants to describe how they interpreted it and selected their response.38 39 Participants were also asked how alternative instructions or response scales might change their interpretation and/or answer. Finally, participants were asked about the importance and difficulty of items using questions such as: ‘You circled “x” as your response to the question asking about the importance of this item. Please tell me what things you were thinking about when answering this question’.

Text was tabulated by item and analysed to provide insight into interpretation and acceptability of item stems, response scales and patient-oriented language. Specific words or terms found to be unclear, easily misunderstood or judgemental and were flagged for application across all items.

Generation of preliminary item bank

Using findings from the online survey and cognitive interviewing data, each item was assessed for domain coherence, clarity and understandability. Whether the item measured perceived success or difficulty in self-management and its potential scalability were also considered. To ensure consistent assessment, the first 20 items were reviewed by the full research team and all other items were assessed by a minimum of two team members. All potential items were assessed for semantic similarity with only the most promising one or two items per group retained. Finally, the overall face and content validity of each domain was assessed by ensuring coherence between items and domain definitions. These items (n=105) form the preliminary PRISM-CC item bank for testing in phase 3.

Phase 3: protocol to select and calibrate the final PRISM-CC items in a sample of people with chronic conditions and to conduct preliminary validation (to be completed)

Participants

Participants will be recruited using the following inclusion criteria: 18 years of age or older, able to read/speak English and self-reported diagnoses of one or more chronic conditions. Multiple strategies will be used to recruit participants:

Posters displayed in public areas including healthcare settings.

Posters and information distributed to patients attending group/individual sessions.

Invitations to participate placed in newsletters and/or via relevant distribution lists.

Social media and online advertising sites.

All recruitment materials will include the internet link to the survey; participants will not be required to contact the research team nor to enter information such as name, address or health card numbers.

Online survey

When potential participants access the public internet link to the online survey, the landing page will include the purpose, inclusion criteria, what participants will be asked to do, and information regarding confidentiality and anonymity, compensation and who to contact in case of questions or concerns. Participants will consent by answering ‘yes’ to the question ‘Do you wish to participate in this survey?’ The first questions of the survey will be used to confirm eligibility, and those not eligible will receive a thank you message with no further questions asked. The remainder of the survey will consist of 105 potential PRISM-CC items (15–17 items per domain), 10 sociodemographic questions (age, gender identity, country of residence, current living situation, first language, education, types of chronic conditions, status of general health, status of general mental health and impact of chronic condition(s) on life) and a six-item validated self-management tool - the Self-Efficacy for Managing Chronic Disease Scale (SEMCD).40 Based on the phase 2 survey, 10–20 seconds per item is a reasonable expectation resulting in a total time of 20–30 min to complete the survey. Participants will have the option to leave the survey at any time, returning to complete it later.

At the end of the survey, participants will have the opportunity to enter their name and email address for a chance to win one of four CAN$100.00 gift cards. This will be collected in a second, unconnected survey and will be used only to select winners of the gift cards.

Dimensionality and calibration analysis

Stages of analysis will include item analysis, assessment of dimensionality and calibration.34 41 Analysis will be done in Stata and R (packages ltm and MultiLCIRT). Item analysis will be used to identify and exclude items that are highly skewed, lack variance or are weakly correlated with other items in their domain.

The structural validity of the items against our conceptual model will be empirically tested. Given the strong conceptual model and qualitative validation, confirmatory factor analysis (CFA) will be the primary analytic method to evaluate whether the data fit our seven-domain conceptual framework based on TEDSS.42 43 Fit will be assessed using multiple indices (eg, Comparative Fit Index, Tucker Lewis Index, Root Mean Square Error of Approximation (RMSEA) and Standardised Root Mean Residual (SRMR)).42 A joint model will estimate the correlation between domains. CFA will also be used to help determine if any domains should be collapsed (augmented by results from the qualitative dimensionality assessment described above), and to assess measurement invariance by gender identification, age group, education and health status groups. Modification indices will identify items with high cross-loadings across domains. If the CFA for any of the domains shows poor fit, exploratory factor analysis will be conducted to explore structural patterns of the items.

Items within each domain will be calibrated with IRT.34 Well-calibrated IRT items can be incorporated into computerised adaptive testing to measure constructs using as few as four to five items, thus reducing response burden.41 Assumptions for IRT scaling (local independence and monotonicity of items) will be assessed. Items violating assumptions will be excluded, or further examined in the IRT analysis to determine their influence on parameter estimates. Samejima’s graded response model will be the preferred modelling approach, but other polytomous IRT models will also be estimated for comparison.34 44 45 The performance of each item/response category will be reviewed using category response curves and estimates of item information. Differential item functioning (DIF) analysis will be conducted by age, gender, self-reported health status, number of chronic conditions and education. For each domain, we will seek a set of best items that provide reliable estimates across levels of the latent construct, and which do not have DIF.44 45

Sample size

Simulations on PROMIS data, based on similar item development methods, show that as few as 250 subjects are sufficient for this analysis.46 However, sample size calculations for CFA were also performed to inform the needed sample size.47 48 Assuming 80% power to differentiate good versus moderate fit (RMSEA of 0.04 vs 0.06) in the most complex model, a fully stratified (two to five groups) non-orthogonal model including mean structures (substantially similar to IRT), with seven domains and a minimum of four to seven items per domain, a sample size of 587 was determined. We will recruit at least 750 participants to compensate for loss of power due to non-normality and unequal numbers between groups.

Preliminary validation analysis

Preliminary validity will be assessed by examining known relationships between each TEDSS domain and the generic SEMCD, a well-known and highly used outcome measure of confidence to manage a chronic condition.40 The SEMCD is used in research, clinical and practice settings to measure self-management. Evidence suggests that this tool is a reliable generic measure of self-management with evidence of construct validity.24

In addition to the SEMCD, individual TEDSS domains will be validated by examining associations with the known correlates of education, general and mental health (including number of chronic conditions) and/or a measure of impact on everyday life (see table 2). Level of education (categorical) and number of chronic conditions will be extracted from collected demographic information. Self-reported general and mental health will be assessed using common single item measures from the Canadian Community Health Survey: ‘In general, would you say your health/mental health is excellent, very good, good, fair or poor?’.49 The impact of the chronic condition will be measured using a single Likert response item: ‘Overall, how much do you feel that your chronic condition(s) affect(s) your life (not at all—extremely)’.37

Table 2

Known relationships by TEDSS domain

TEDSS domain	Known relationship
Process	Positive relationship with education.51 52
Resources	Positive relationship with education.53–55
Disease management	Inverse relationship with number of conditions.56 57 Inverse relationship with general health.58
Health behaviours	Positive relationship with education.59 60
Activities	Inverse relationship with participation in everyday life.53
Internal	Positive relationship with mental health.53 60
Social interaction	Positive relationship with mental health.53

TEDSS, Taxonomy of Everyday Self-management Strategies.

Bivariate and multivariate analyses will be conducted to examine relationships between the TEDSS domains and relevant categorical variables (education, general health, mental health and impact on participation).

Ethics and dissemination

All procedures will adhere to the Canadian Tri-Council Policy Statement on Ethical Conduct for Research Involving Humans. Phase 1 did not include collection of original data; therefore, ethics approval was not required. Ethics approval for phases 2 and 3 has been obtained from the Nova Scotia Health Authority Research Ethics Board.

This study has and will continue to use an integrated knowledge translation approach.50 Team members include patient-partners, policymakers and managers in primary care. All have and will shape the research process. A summary of our results will be posted on our website, accessible to the public. The Primary Health and Chronic Disease Portfolio in the Nova Scotia is actively using the conceptual framework employed for the PRISM-CC to guide assessment and planning in primary care and chronic disease management and has provided extensive consultations to our team on the attributes of PRISM-CC that will be required to facilitate its integration into care. To facilitate uptake and use, the final PRISM-CC will be made available for research and clinical care at minimal or no cost.