Development and validation of the Person-Centeredness of Research Scale.

AIM: Person-centeredness shifts the focus of healthcare and research to the needs and priorities of patients and communities, and may improve health outcomes. There are no instruments available, however, with which we can assess the degree to which research is indeed person-centered. Our aim was to develop and validate a quantitative instrument to rate person-centeredness of research. MATERIALS &
METHODS: Scale development and validation entailed a multistep approach that led to the seven-item Person Centeredness of Research Scale (PCoR Scale) that uses a 5-point Likert rating scale. The scale was validated using ratings of the Patient-Centered Outcomes Research Institute-funded research abstracts or abstracts submitted to a translational science meeting.
RESULTS: Psychometric properties of the PCoR Scale showed high internal consistency (α = 0.96). All seven items were highly correlated with the total score (rs range from 0.63 to 0.90). An exploratory factor analysis demonstrated that all of the items loaded on a single factor, explaining 80% of the variance. The Patient-Centered Outcomes Research Institute-funded research abstracts had a mean PCoR Scale score of 6.52 (±8.01) that was significantly higher than the abstracts submitted to the translational science meeting (-2.56 (±9.18); t = 8.09; p < 0.0001). Inter-rater reliability in the validation of the revised instrument was high (Intraclass Correlation Coefficient [ICC](group1) = 0.89; ICC(group2) = 0.95).
CONCLUSION: This brief, quantitative rating scale is the first to assess the main constructs that describe person-centeredness of research products. The PCoR Scale can be used to assess person-centeredness in research products; for example, by funders evaluating proposals, data networks evaluating data requests or researchers evaluating their research designs.

The term ‘patient-centered’ was introduced in the mid-1980s [1] and gained increasing recognition in the 1990s in an effort to define healthcare delivery organized around patients rather than providers and health systems [2]. The number of publications focused on ‘patient-centered care’ has increased exponentially since then and the term has evolved to emphasize patient preferences, needs and values [3-7]. Person-centered is often used interchangeably with patient-centered, but it adds nuances to its meaning [8,9]. Person-centered defines approaches to delivering care centered on the individual independent of the clinical setting [10] and it also signifies the person is an active partner in healthcare processes [9], as well as considering the individual as a whole (person) rather than as a temporary role (e.g., client, participant, resident, patient) [11].

Recently, patient-centeredness has been described as one of the primary goals the US healthcare system should address to improve its quality [12-14]. While this is a relevant goal, advances in healthcare delivery build on discoveries in health research. For healthcare delivery to include patient-centered principles, those principles must be applied across the continuum of healthcare delivery, including effective application in research.

Initiatives aimed to promote and facilitate the implementation of patient-centered principles in research [15] have been twofold: allow patients and other stakeholders to assess the value of healthcare options to accelerate the translation of research into practice [16-19]; and allow the public to fund research that matters to persons and their caregivers. While different tools have been developed to assess the extent to which a healthcare service or intervention is patient-centered [21-23], no such tools exist to assess patient-centered principles in research. A survey assessing research participants’ perceptions of how to improve future research is available [24], but no instruments have been developed to measure the extent of patient-centeredness (or person-centeredness) of proposed research studies or research study products, such as reports, abstracts or publications.

This study fills the need for a quantitative tool to assess the extent of person-centeredness of research products. The PCoR Scale, will allow researchers, health professionals, funders, healthcare consumers, policy makers and other stakeholders to compare and assess person-centeredness in a variety of research products.

Materials & methods

This methodological study used a multistep process to develop and validate the Person-Centeredness of Research Scale (PCoR Scale). These steps included: generation of possible items for the scale; a pilot study of the initial item pool; refinement of the initial version of the scale and establishing the initial psychometric properties of the instrument. The original focus of the instrument was on patient-centeredness; however, after the pilot study, the focus changed to person-centeredness to encompass the needs, preferences and priorities of persons outside the clinical setting.

Step 1: item generation

The first step in the development of an instrument aimed at assessing patient-centeredness of research products was to create a set of questions relevant to the construct to be assessed based upon a conceptual framework.

A review of the literature for instruments and tools to assess patient-centeredness of research (conducted in June 2014) produced no results. The literature review found several conceptual frameworks focused on patient-centered care [25], which informed the generation of items; however, the instruments measuring patient-centered care were of limited value because they focused on care delivery, not research. Item generation was guided by the Patient-Centered Outcomes Research Institute (PCORI) Engagement Rubric [25] that establishes principles for engagement in patient-centered research and provides guidance on how to engage patients, families and other stakeholders in research. Key concepts from the literature review were a focus on the values, needs and preferences of patients and families. The initial set of items included 17 questions, some of which followed definitions of key terms. A panel of academic researchers and community stakeholders (nonacademics) experienced in research refined the items and narrowed the pool to 11 items, each of which were to be rated using a 4-point ordinal rating scale ranging from ‘No, completely’ (1 point) to ‘Yes, completely’ (4 points). (See Table 1 for this initial version of the scale.)

 

Counts of abstracts of the Patient-Centered Outcomes Research Institute-funded projects.

PCoR scale items	Δ = 0	Δ = 1 same	Δ = 1 different	Δ = 2
1. Beliefs are defined as the state of mind in which a person thinks something to be the case, with or without there being empirical evidence to prove that something is the case with factual certainty. Beliefs can be cultural or faith-based. Does the information reflect beliefs you think are relevant to the population of interest (or to patients/community members in general)?	10	8	5	7
2. Attitudes are defined as ‘settled/ingrained ways of thinking or feeling about someone or something, typically one that is reflected in a person's behavior, whether consciously or unconsciously’. Attitudes can be positive or negative, mixed or unsure, and are particularly relevant in reference to research/healthcare. Does the information reflect attitudes you think are relevant to the population of interest (or to patients/community members in general)?	10	10	3	7
3. Concerns are defined as ‘matters of interest or importance to someone’. Concerns can be related to feelings about how the research is developed and carried out. Does the information reflect concerns you think are relevant to the population of interest (or to patients/community members in general)?	11	11	5	3
4.Values are defined as ‘a person's principles or standards of behavior; one's judgments of what is important in life’. Values can relate to new medication, treatment opportunities and/or cost–effectiveness. Does the information reflect values you think are relevant to the population of interest (or to patients/community members in general)?	7	12	1	10
5.Are patient and/or community needs taken into consideration?	12	3	3	12
6.For any suggestions given, are the suggestions relevant to the population of interest (or to patient and/or community members in general)?	8	8	2	10
7. Does the information address patient-centered and/or community-centered outcomes?	10	12	3	5
8. Does the information address research priorities of the population of interest (or patient and/or community members in general)?	8	11	3	8
9. Does the information address healthcare needs of the population of interest (or patient and/or community members in general)?	10	11	4	5
10.Does the information address the best methods to communicate with the population of interest (or patient and/or community members in general)?	11	7	2	10
11. Does the information address opportunities to engage the population of interest (or patients and/or community members in general) in decision-making around the research planning, implementation or results dissemination?	10	10	3	7

Note: Questions in italics are those that were eliminated from the final version of the PCoR Scale.

Table shows the number of questions given the exact same rating by each of two raters (Δ = 0), as well as how many questions had scores that differed by only one point but remained in the same category (either both rated as nonpatient-centered [a 1 and a 2] or both rated as patient-centered [a 3 and a 4]; Δ = 1 - same), and those that differed by one point but changed the category (e.g., a rating of 2 by one rater and 3 by the other rater; Δ = 1 - different) or those that differed by two or more points (indicating extreme unreliability; Δ = 2).

PCoR Scale: Person Centeredness of Research Scale.

Step 2: pilot study

This initial version of the scale was then used to rate 60 research abstracts (see Supplementary Table 1 for abstract titles) including 30 from PCORI-funded research and 30 abstracts submitted to the Association for Clinical and Translational Science 2015 Annual Conference. The hypothesis was that abstracts of PCORI-funded research would have higher ratings on patient-centeredness than the abstracts submitted to the translational science meeting. A total of 17 raters who were blinded to the origin of the abstracts used the initial version of the rating scale to assess the extent of patient-centeredness of the submitted abstracts. Of these raters, nine were patient and/or community stakeholders while the other eight raters were researchers based in academic institutions. Two raters assessed each abstract and each rater assessed either seven or eight abstracts, some from each conference. Raters were also asked for qualitative feedback on the items and the overall rating instrument.

Analysis of the pilot study data

The average rating across the two independent raters for each question for each of the 60 abstracts was computed, and the average item scores were summed over the 11 questions to get a total score for each abstract. Higher item and total scores indicated higher patient-centeredness. These average ratings for each question were used to conduct an exploratory principal component factor analysis to determine how many latent dimensions could explain the 11 questions. The average total score for each abstract was also used to compute a t-test comparing the mean patient-centeredness of the PCORI-funded abstracts to those abstracts submitted to the translational science meeting. Finally, an item analysis was computed to determine the internal consistency of the rating scale using Cronbach's alpha. All analyses were performed with SPSS v.25.

Because each abstract was rated by two raters and raters were not paired on sets of abstracts, conventional statistics to determine inter-rater reliability could not be calculated. Instead, we counted how many questions were given the exact same rating by each of two raters, as well as how many questions had scores that differed by only one point but remained in the same category (either both rated as nonpatient-centered [a 1 and a 2] or both rated as patient-centered [a 3 and a 4]), and those that differed by one point but changed the category (e.g., a rating of 2 by one rater and 3 by the other rater) or those that differed by two or more points (indicating extreme unreliability). We performed this count on only the PCORI-funded research abstracts as the aim of the scale was to assess patient-centeredness and abstracts submitted to the Association for Clinical and Translational Science meeting were included for comparison purposes.

Pilot study: results & discussion

A principal components factor analysis showed that all 11 questions loaded onto a single factor with an eigenvalue of 6.54 that explained a total of 59.41% of the common variance. An item analysis of those 11 items resulted in a high degree of internal consistency (Cronbach's α = 0.93), which remained high (alphas ranging from 0.92 to 0.93) if we deleted any single item from the scale. Pearson correlation coefficients of each item with the total score ranged from 0.60 to 0.86. There were significant differences (t = 5.32; p < 0.0001) between the average item score for the PCORI-funded research abstracts (M = 2.91; SD: 0.71) compared with those submitted to the translational science meeting (M = 2.13; SD: 0.89).

Table 1 shows the items of the scale and the abstracts’ counts by consistency of item ratings by the pairs of raters who judged the PCORI-funded abstracts. Although there was a fair amount of inter-rater inconsistency, four questions (#s 4, 5, 6 and 10) stand out as having the greatest amount of inter-rater unreliability (see the last column in Table 1).

Qualitative feedback from the raters recommended different wording for some of the of the questions and changing the rating scale from four response options to five options, thus allowing raters to indicate that an abstract was neither patient-centered nor not patient-centered on a given attribute. It was also recommended that the definitions of terms such as beliefs, attitudes, concerns and values be removed from the questions themselves and placed elsewhere, perhaps in the instructions for raters. It was also suggested that the scale be renamed to assess person-centeredness rather than patient-centeredness, emphasizing the need to consider preferences and values of individuals outside of the clinical setting.

The results of the pilot study suggest that the initial 11-item version of the PCoR Scale was a reliable, unidimensional scale. Yet, because we found that all 11 questions loaded into a single, highly internally consistent factor we recognized that it might be possible to reduce the scale to fewer items (see Step 3 section). We revised the scale's title (to Person-Centeredness of Research Scale), changed the item wording to declarative statements instead of questions and conducted a validation study of this new, shorter version of the PCoR Scale.

Step 3: refinement & validation of the seven-item version of the PCoR Scale

To address feedback from the raters, each item in the revised PCoR Scale was changed to a 5-point Likert rating scale (from ‘Strongly disagree’ [-2] to ‘Strongly agree’ [+2], with ‘0’ indicating ‘Neither Agree nor Disagree.’ In addition, the total number of items was reduced from 11 to 7 by eliminating four items that showed the least consistency between raters in the pilot study. The total summed score for the new PCoR Scale could range from -14 to +14, with the higher scores indicating greater person-centeredness of the research product. This new PCoR Scale was then used to rate 40 of the same abstracts that had been rated in the pilot study. The number of abstracts to be rated was reduced by discarding those 20 abstracts that had consistently high or low patient-centeredness scores in the pilot study with a stringent goal of demonstrating that the revised scale could discriminate in the middle ground between extremes of high and low person-centeredness.

A total of 12 raters, all of whom had participated in the previous pilot study, used the new PCoR Scale to assess the extent of person-centeredness of the abstracts. Of these raters, seven were community stakeholders while the other five raters had academic affiliations. Two teams of six raters were randomly created and balanced based on their affiliation. Each rater assessed 20 abstracts and was blind to each abstract's origin. Two different packets of abstracts were created (each packet containing a total of 20 abstracts from both sources), thus enabling us to increase the total number of abstracts assessed by the same team of raters. Each rater in each team received one packet of abstracts in a random order to control for order effects in the ratings. This design allowed assessment of inter-rater reliability within each team. Qualitative feedback on this version of the scale was also gathered. A principal components factor analysis was conducted to explore the number of latent factors underlying the PCoR Scale's items. This was followed by an item analysis to compute the scale's internal consistency (i.e., Cronbach's α). Intraclass correlation coefficients were computed to establish the inter-rater reliabilities of the judges in the validation study. A discriminant analysis was performed to confirm that the revised scale could successfully assign the abstracts to the appropriate group. We also compared the mean PCoR ratings of academic researchers versus community stakeholders, as well as between abstracts submitted to the two different types of research conferences.

Validation of the new PCoR Scale: results & discussion

Table 2 shows the mean scores of each item, their factor loadings and the correlation of each item with the total scale score. The items loaded on a single factor with an eigenvalue of 5.29, explaining a total of 75.63% of the variance. An item analysis showed high internal consistency (Cronbach's α = 0.95) for the seven items. Mean ratings (SD) for those abstracts funded by PCORI were 6.52 (8.01), compared with -2.56 (9.18) for those submitted to the translational science conference. The difference in mean PCoR ratings of the abstracts was highly significant (t = 8.09; p < 0.0001), larger than four-times its standard error. Discriminant analysis showed that the new version of the scale successfully classified 71.4% of the abstracts as belonging to the appropriate group. Inter-rater reliability was high, with an average intraclass correlation coefficient of 0.89 (p < 0.001) for one team of raters and 0.95 (p < 0.001) for the other team of raters. No differences were found (t = 0.247; p = 0.805) in the ratings of the PCORI-funded research abstracts made by community stakeholders (mean [SD] = 6.67 [8.36]) and raters with an academic affiliation (mean [SD] = 6.32 [7.63]). Differences were also not found (t = 0.517; p = 0.606) in the ratings of abstracts submitted to the translational science meeting between community stakeholders (mean [SD] = -2.18 [9.59]) and raters with an academic affiliation (mean [SD] = -3.11 [8.61]). All raters expressed satisfaction with this version of the PCoR Scale and had no more suggestions for changes.

 

Descriptive statistics for the final Person Centeredness of Research Scale items along with factor loadings and Pearson Correlations with total scale score.

PCoR scale items	Factor loadings	Mean (SD)	Pearson r
1. There is evidence that beliefs relevant to the population of interest or to patients/community members in general are included or addressed in the research.	0.85	0.22 (1.62)	0.85†
2. There is evidence that attitudes relevant to the population of interest or to patients/community members in general are included or addressed in the research.	0.92	0.26 (1.61)	0.92†
3. There is evidence that concerns relevant to the population of interest or to patients/community members in general are included or addressed in the research.	0.90	0.59 (1.55)	0.89†
4. Person/community-centered goals and/or outcomes are included or addressed in the research.	0.89	0.61 (1.56)	0.89†
5. Research priorities of interest to the patient/community are included or addressed in the research.	0.86	0.50 (1.52)	0.86†
6. The needs of the patient/community are included or addressed in the research.	0.87	0.40 (1.56)	0.87†
7. Individuals representing patients and/or communities are engaged in the research as stakeholders, advisors, consultants or team members (beyond serving as research participants or volunteers).	0.78	-0.18 (1.72)	0.80†
Total PCoR Score	N/A	2.41 (9.67)	N/A

†p < 0.001.

N/A: Non applicable; PCoR: Person Centeredness of Research; SD: Standard deviation.

Discussion

The results of this validation of the new PCoR Scale confirm that it is unidimensional, possess high levels of internal consistency and inter-rater reliability and discriminate well between research products that are hypothesized to differ on their person-centeredness. The PCoR Scale is the first validated quantitative scale to assess person-centeredness of research products. This 7-item scale allows an objective and structured assessment of the extent to which research reflects the needs, priorities and values of patients, families, individuals and communities that are the focus of the research. This instrument will help funding organizations, policy makers, researchers, patients, community organizations and other stakeholders objectively assess research products and will inform the healthcare community on the extent of person-centered approaches in research and the evolution of those methods over time. Importantly, the scale was validated using both academic-based researchers and community stakeholders as raters and there was no difference in ratings between these two groups. This result suggests that raters do not have to be credentialed to use this instrument.

In comparison to previous scales or questionnaires on patient- or person-centeredness, the PCoR Scale is explicitly designed to assess research, not healthcare service delivery. For instance, the Person-Centered Climate Questionnaire [20] explicitly assesses aspects of person-centeredness related to the setting of a clinical unit, and the Patient-Centered Inpatient Scale [21] assesses the extent of patient-centeredness of health delivery processes in acute care settings. Although the scales assessing patient-centeredness in healthcare are extremely valuable to improve healthcare delivery, none of the scales can be reliably used to assess person-centeredness of research.

One of the limitations of this study is the lack of concurrent validity assessment. The absence of other rating scales assessing the same (or similar) construct on research products did not allow us to assess the extent to which the PCoR Scale ratings correlate with other measures. Further research should include the assessment of other research products such as proposals, reports, presentations and publications to assess whether the psychometric properties observed when rating abstracts are similar to those other types of products. A second limitation is that the only type of research product examined in this article is abstracts submitted to research conferences. Another limitation is that both the pilot study of the initial version of the PCoR Scale and the validation study of the final version of the scale made use of the same set of research abstracts and also, to some extent, the same set of researchers.

Conclusion

Overall, the PCoR Scale provides a quantifiable measurement of the extent of person-centeredness of research products and has the potential to be used to assess person-centeredness in a wide variety of research products. Training in the use of the PCoR Scale can be accomplished in one short session, and despite being brief it allows the rating of seven aspects of person-centered research for a given product. The PCoR Scale fills an important gap in instruments to measure the person-centeredness of research and may help identify opportunities to make research, and ultimately healthcare, more person-centered.

Person-centered principles in research allow patients to assess the value of healthcare options to accelerate the translation of research into practice.

Person-centered principles in research allow the public to fund research that matters to persons and their caregivers.

There are no instruments available to assess the degree to which research is person-centered.

The Person-Centeredness of Research (PCoR) Scale is brief.

The PCoR Scale is the first to assess the main constructs that describe person-centeredness of research products.

The PCoR Scale can be used to assess person-centeredness in research products.

Click here for additional data file.