Enhancing self-resources in patients with chronic diseases: development and initial validation of the Disease and Care Management Score.

INTRODUCTION: Despite the importance of the assessment in the primary care of the self-resources among patients with chronic diseases, there is not available a measurement that allows this kind of comprehensive assessment. For this reason, the aim of this study was to develop a multi-dimensional score to determine the level of self-resources in chronic patients, describing its initial validation through face and content validity. The developed score was labelled as Disease and Care Management Score.
METHODS: We performed a methodological study, encompassing two main phases. The first phase was aimed to develop the Disease and Care Management score, choosing the most suitable measurement to assess each pre-identified determinant of wellbeing in chronic patients. The second phase was aimed to determine the Disease and Care Management score face and content validity through the views of 20 experts.
RESULTS: Disease and Care Management score shows evidence of face and content validity. All the obtained quantitative content validity indices (i.e. Content Validity Ratio, Content Validity Indices) were higher than 0,70, showing the pertinence and the adequacy of each pre-identified measure to compute Disease and Care Management score.
CONCLUSION: Disease and Care Management score has the potential of addressing the health coaching interventions in primary care for chronic patients. Future research should show its predictive performance, as well as the cut-off to discriminate patients.

Introduction

Chronic diseases require continuous treatments over time, which could last from years to decades (1). Currently, caring for people with chronic diseases involves 70-80% of the economic resources of healthcare with slight differences among Countries (2). The increased epidemiology of chronic diseases is strictly linked to the population ageing, as well as the improvements in the healthcare delivery, and the enhancements of the economic and social conditions (3). Accordingly, the population over 65 among the European Countries has raised from roughly the 10% in 1960s to the 19% in 2015, being expected to increase towards the 30% by 2060 (4). This scenario comes as a result of considerable progress in life expectancy, and opens new areas, which are worthy of investigation. Thus, research should address the issues underpinning the improvement of quality of life in elderly as well as the life expectancy (5). Accordingly, it is estimated that roughly 50 million of Europeans have two or more chronic diseases, where these diseases tend to correspond with an increased aging (4). As per Italy, roughly 39% of the population is affected by a chronic disease, where almost the 45% of those patients are aged over sixty-five (5, 6). More precisely, diabetes, cardiovascular diseases, stroke, COPD, cancers and dementias affect the 18% of the Italian population with a chronic disease (6).

The traditional hospital-based approaches of healthcare delivery seem to be unsuitable for addressing the needs of patients with chronic diseases (2). In fact, the treatment of chronic diseases does not purpose to healing patients, but it is mainly aimed to enhance patients’ self-resources in managing their condition to achieve an improvement of their functional status, adherence to treatments, and an increased quality of life (7). Accordingly, the appropriate management of care pathways is pivotal, and it should be proactive, multi-dimensional, patient-centered, and globally aimed to empower the patients (7, 8). Further, chronic disease management interventions should address the possible inequalities in the accesses for primary care, such as in the case of some minority groups (9).

So far, healthcare systems are mainly organized to treat acute and episodic diseases (10). However, the Chronic Care Model (CCM), developed in the late 1990s, provides a framework to face with the current burden of chronic diseases (11). Properly, CCM was designed to engage patients and improve their health outcomes by changing the routine delivery of ambulatory care, shifting the paradigm of care from being reactive towards the characteristics of proactivity, planning population-based interventions where possible (12). Some authors have also identified in the CCM an opportunity to re-organize care, emphasizing the need of a coordination of the care path to allow a continuum between the various healthcare systems consulted by chronic patients (11). Many authors emphasized the importance of the patients’ stratification to allow the correct identification of specific pathways (7, 13).

However, the current possibilities to helpfully stratify the chronic population led to a high heterogeneous scenario. For instance, patients could be stratified using the diagnosis taxonomy, their behaviors or lifestyles, their adherence to treatments and follow-ups (14). The choice to select patients for specific educational paths is currently undermined by the unavailability of a measure that encompasses the main different characteristics (i.e. determinants of wellbeing) described in chronic patients, which are dietary habits, physical resources, adherence, psychological status, ability to perform daily activities, self-care, and the overall perceived quality of life (15). A comprehensive score of the above-mentioned determinants could be useful to select patients for specific educational or support interventions, using an evidence-based approach. Further, we can argue that, theoretically, a comprehensive score could also present some important predictive characteristics towards the decline of clinical conditions, re-hospitalizations, and even mortality. For all these reasons, this study was aimed to develop a multi-dimensional score to identify the comprehensive health status of patients with chronic diseases, providing its initial validation through face and content validity. The developed measure was labelled as “Disease & Care Management Score” (D&CM).

Methods

This was a multi-phase and methodological study, encompassing two phases: phase one referred to the development of the D&CM score, while phase two referred to its face and content validity.

Development of D&CM score

The development of D&CM score was mainly based on a previous description of individual and social determinants of health in population with chronic diseases (15). Then, these determinants were categorized into dietary habits, physical activity levels, adherence to treatments, psychological burden, and overall functional status. A panel of three expert authors (AP, RC, AC) in chronic diseases operationalized the measures needed to assess each health determinants. Accordingly, authors identified the following valid and reliable measures for each determinant, that subsequently need to be scored in a unique measure to identify a comprehensive health status score (D&CM score).

Dietary habits and physical activities were detected using a scale developed by the research center of the Mario Negri Institute (16). This self-report scale encompassed 12 items to explore dietary habits, and three items to explore physical activities. Conversely, we selected the beliefs about medicines questionnaire (BMQ) as a proxy assessment for patients’ adherence (17). In fact, some authors used BMQ to explore patients’ adherence, because there is a strong relationship between beliefs about medicines and the actual adherence to pharmacological treatments (18). Psychological burden was explored for one of its major problematic aspect, i.e. depression. For this reason, we chose the Hamilton depression rating scale (HAMD) to intercept patients’ depression (19). Finally, Karnofsky’s scale correlates with physical functioning, such as walking and stair climbing, and it has predictive validity for poor prognosis (20). All the above mentioned measures of health determinants were kept into account to develop the initial version of D&CM score. In other words, D&CM score embodied all the measurements required to assess dietary habits and physical activities, BMQ, HAMD and Karnofsky into a single score.

D&CM scoring procedure

All the scores coming from the different used scales (raw scores) were standardized in a comprehensive score ranging from one to four. The standardization of the different scores coming from the original scales was aimed to adjust each values measured on different metrics to a common measure. We used formula of Z-score through ‘standardization of normal distribution’, as described by several authors (21). Overall, D&CM scoring procedure is available using an ad hoc software developed by GPI Group (AC).

Face and content validity

Once terminated the initial choice for selecting scales aimed to assess each single health determinant, we performed the face and content validity study for the overall D&CM score. Content validity refers to the methodology developed in the 1970s by Lawshe (22), being aimed to detect the level of the agreement among expert raters in defining the pertinence of each measure in relation to the objective of the overall measurement (identifying a comprehensive score to explore health status in patients with chronic conditions). More precisely, content validity encompasses the quantitative assess of raters using the Content Validity Ratio (CVR), and the Content Validity Index for item and scale level (I-CVIs and S-CVI). CVR could potentially range between -1 (perfect disagreement among panellists) and +1 (perfect agreement among panellists), while I-CVIs and S-CVI range between 0 (no content defined as valid) and +1 (content totally judged as valid). As per the CVR critical values (i.e. the lowest level of CVR such that the level of agreement was greater than 50%), the recent literature proposed a revisiting of the originally-developed critical values, considering that the CVR critical values have been originally determined by the normal approximation to the binomial distribution applied to the panel sizes encompassing less than 13 panellists (23). More precisely, the normal approximation to binomial has been caused concerns on CVR critical values (only when panel size is lower to 13 participants due to for larger sizing no approximation was used), as the critical values determined with this approach seem to be inferior to the ones determined using the exact binomial probabilities (24). For this reason, the recent proposed CVR critical values seems to be more prudent in defining the values expressing a level of agreement among panellists higher than 50% for a given α (i.e. type I error probability, which is 0,05 using a one-tailed test), even when panel size encompasses less than 13 panellists (24). Conversely, face validity explored experts’ understanding of each measure, and their views about eventual amendments to improve the overall content of D&CM score (25). Both face/content validity could have brought some amendments to the initial authors’ choice in operationalizing the individual and social determinants of health in chronic population, using the above-described scales.

Further, face/content validity implicitly requires the selection of a panel of experts to provide their judgments on the proposed items aimed to measures the content area it is expected to measure with the D&CM score (i.e. a multi-dimensional score to identify the comprehensive health status of patients with chronic diseases). Overall, the panelists’ answering closely determines this kind of validity, and for this reason the selection of the panelists is pivotal to ensure rigor. Despite the determination of the number of panelists is generally partly arbitrary, when the number of panelists increases, the probability of chance agreement decreases (26). Accordingly, higher is the number of panelists, higher is the difficulty in find agreement. The first recommendations to determine the choice of the experts were proposed in the 1980s (27), being criticized and discussed during the last two decades (28, 29). Overall, the literature suggests to invite the panelists using a declared rationale of selections, which represents the pros and cons of the overall content validity process.

Having said that, we decided to invite a relatively large number of panelists (n=20), decreasing the implicit possibility of achieving high rates of agreement, but increasing the caution in determining the content validity in relation to the aim of D&CM score. In this study, the evaluated items were related to previously validated scales with the purpose to embody these different measures into a single score. Further, the selection of panelists was performed inviting them from a list of educators involved in post-graduate courses on chronic management for both the continuing medical education program (CME) and university-level courses on chronic management in nursing (bachelor) and/or medicine (MD program) courses. More precisely, the selection of the panelists coming from the list of educators was guided by the following inclusion criteria: (a) medical degree (MD) or Master of Science (MSc) degree; (b) five years of minimum working experience with chronic patients (excluding internships or similar educational trainings); (c) active involvement as educators on topics related to the caring for chronic patients (e.g. professors, tutor, mentor). Overall, content validity is generally considered as the initial step of a complex validation process that often requires more inferentially robust analysis to determine construct validity, such as multivariate latent variable modeling (e.g. exploratory factor analysis, exploratory structural equation modelling).

Ethical considerations

This study did not involve patients. The authors planned the designing, conducting, recording and reporting of the study in a consistent way with the international ethical and scientific quality standards, indicated by Good Clinical Practice (GCP) and standard operating procedures (SOPs). All the involved experts were informed on the study aim.

Statistical analysis

Socio-demographics of the involved experts were represented using descriptive statistics. CVR was computed as follow: CVR=(Ne - N/2)/(N/2), in which the Ne is the number of raters indicating “essential” and N is the total number of raters. It could varies between +1 and -1, where higher score indicates higher agreement among raters. The interpretation of CVR was performed comparing the observed CVR coming from the panelists’ answering and the critical CVR recently proposed using the discrete binomial calculations, given the discrete nature of the variables used to compute CVR (24). To obtain I-CVIs, we calculated the number of those judging the measurement as relevant (i.e. ratings ≥ 3) divided by the number of content experts. Thus, I-CVIs expressed the proportion of agreement on the relevancy of each measure, where the index could range between zero and one (23). Furthermore, S-CVI was defined as the proportion of total items judged content validity (23), computing the mean of each obtained I-CVIs.

Results

The enrolled experts involved for the content and face validity were 20, being selected considering the pre-defined inclusion criteria aimed to guide their selection. They were mainly males (n=11; 55%), physicians (n=11; 55%), their median of age was equal to 43,7 years (IQR=8,3 years) with a median of 19,6 years of experience (IQR=7,9 years) (Table 1).

Table 1.

Characteristics of the experts (n = 20)

		N	%
Gender	Male	11	55
	Female	9	45
Profession	Physician	11	55
	Nurse	9	45
		Median	IQR
Age		43,7	8,3
Years of experience	19,6	7,9

Legend: IQR = interquartile range

As Table 2 shows, the obtained CVRs were higher than 0.70, as well as the I-CVIs and each S-CVI. Thus, no need of amendments occurred by this step of validation. Further, the narrative analysis on the free-text comments for each scale selected to compute D&CM score showed the ‘usefulness’ of a unique measure to frame the peculiarities of chronic patients.

Table 2.

Content validity scores

Expert panellists (n = 20)	Ne	CVR	Interpretation	I-CVIs	Interpretation	S-CVI	Total Score S-CVI
Scale on dietary habits
Item 1	19	0,9	Relevant	0,9
Item 2	18	0,8	Relevant	0,85
Item 3	18	0,8	Relevant	0,9
Item 4	19	0,9	Relevant	0,9
Item 5	20	1	Relevant	0,95
Item 6	18	0,8	Relevant	0,9	Pertinent	0,9
Item 7	18	0,8	Relevant	0,85
Item 8	19	0,9	Relevant	0,95
Item 9	20	1	Relevant	1
Item 10	17	0,7	Relevant	0,8
Item 11	20	1	Relevant	0,95
Item 12	19	0,9	Relevant	0,85
Scale on physical activities							0,90
item 1	19	0,9	Relevant	0,88
item 2	20	1	Relevant	0,95	Pertinent	0,91
item 3	18	0,8	Relevant	0,85
Beliefs about medicines questionnaire
item 1	18	0,8	Relevant	0,9
item 2	18	0,8	Relevant	0,85
item 3	19	0,9	Relevant	0,95
item 4	20	1	Relevant	1
item 5	17	0,7	Relevant	0,8
item 6	20	1	Relevant	0,95
item 7	18	0,8	Relevant	0,85	Pertinent	0,91
item 8	19	0,9	Relevant	0,88
item 9	20	1	Relevant	0,95
item 10	18	0,8	Relevant	0,95
item 11	18	0,8	Relevant	0,88
Hamilton depression rating scale
Item 1	20	1	Relevant	0,9
Item 2	17	0,7	Relevant	0,85
Item 3	20	1	Relevant	0,95
Item 4	18	0,8	Relevant	0,95
Item 5	19	0,9	Relevant	0,85
Item 6	20	1	Relevant	0,9
Item 7	18	0,8	Relevant	0,95
Item 8	18	0,8	Relevant	0,95
Item 9	19	0,9	Relevant	0,88	Pertinent	0,90
Item 10	20	1	Relevant	1
Item 11	18	0,8	Relevant	0,77
Item 12	18	0,8	Relevant	0,85
Item 13	19	0,9	Relevant	0,9
Item 14	20	1	Relevant	0,95
Item 15	17	0,7	Relevant	0,95
Item 16	20	1	Relevant	0,88
Item 17	19	0,9	Relevant	0,88
Physical functioning (Karnofsky)
Item 1	17	0,7	Relevant	0,95
Item 2	20	1	Relevant	0,9
Item 3	18	0,8	Relevant	0,8	Pertinent	0,91
Item 4	19	0,9	Relevant	0,9
Item 5	20	1	Relevant	1

Legend: Ne = the number of panel members indicating an item “essential”; CVR = Content Validity Ratio

I-CVIs = Content Validity Indices calculated at the item-level: S-CVI = Content Validity Indices calculated at each scale-level

Discussion

This study was mainly aimed to assess the content and face validity of a new proposal of comprehensive measurement of the level of available self-resources in patients with chronic diseases. This study provides solid basis for future explorations of the predictive characteristics of the D&CM score, as well as the assessment of its construct validity and reliability. Overall, D&CM score is functional in closing the current gap given by the unavailability of a comprehensive measure of determinants of wellbeing in chronic patients to determining an assessment of their dietary habits, physical resources, adherence, psychological status, ability to perform daily activities.

All the content validity scores showed the high level of relevance and pertinence of the selected pre-existing measures, thus no modifications were needed in relation to the proposed scales embodied into D&CM score. Accordingly, it could be argued that D&CM score has the potentiality to address the educational interventions to enhance the modifiable self-resources of chronic patients. The idea underpinning the development of D&CM score is consistent with the literature on educational interventions for chronic patients, such as the using of health coaching (30). Precisely, health coaching can be defined as an approach to helping patients gain their knowledge, skills, tools and confidence in becoming active and reaching their self-identified health goals (30). Consistently, the areas encompassed in D&CM score are those needed to plan a health coaching for patients with chronic diseases (31). In this regard, the high multi-dimensional nature of D&CM score could be useful to overcome the main limits of the tools used to assess the effectiveness of the coaching interventions (31, 32).

Further, D&CM score encompasses the main common areas of health determinants in chronic patients, being potentially useful for a wide range of chronic conditions, and overcoming some constraints of tools available only for specific clinical conditions, such as for chronic obstructive pulmonary disease (COPD) (33) or for patients with diabetes (34). Precisely, D&CM score is not intended to be a surrogate of specific scores (e.g. self-care scores), but it is developed to provide a comprehensive orientation for evaluating the general self-resources, especially in the setting of primary care (35).

Further tests of D&CM are needed to assess the cut-off to address the interventions. Particularly, it will be useful to choose an external validation parameter for describing the sensitivity and the specificity of D&CM score in relation to the selected external parameter (36). Further, it will be necessary to test the predictive propriety of D&CM score on some identified outcomes, such as re-hospitalizations. Overall, D&CM is promising, even if it is still under testing for more robust considerations.

Considering this study, the main limitation is related to the pilot nature of the aim. Thus, the results of this study have to be intended as the synthesis of the views of 20 experts on the content validity of D&CM score, with a poor possibility of inferential considerations. This limit is in line with the methodologies of content validity studies (25). However, this study has the worth to give solid basis for the further development of D&CM score, being a methodological description related to the score development and its initial validation process.

Conclusions

D&CM score shows evidence of content and face validity. It could be useful to assess the main common areas of self-resources in patients with chronic diseases, addressing educational intervention of health coaching. Future research has to provide more evidence of validity of D&CM score, describing its predictive performance in the different cohorts of chronic patients. Further, it is needed support with an external validation parameter for the study of D&CM score sensitivity and specificity. Clinicians working in the primary care settings might benefit of a comprehensive measure of assessment of patients’ self-resources.