Inter-Rater Reliability of a Pressure Injury Risk Assessment Scale for Home Care: A Multicenter Cross-Sectional Study.

PURPOSE: The aim of the current study was to assess the inter-rater reliability and agreement of the Pressure Injury Primary Risk Assessment Scale for Home Care (PPRA-Home), a risk assessment scale recently developed for Japan-specific social welfare professionals called care managers, to predict pressure injury risk in geriatric individuals who require long-term home care needs.
METHODS: A multicenter cross-sectional study was conducted at 30 home-based geriatric support services facilities located at four local districts in Japan. Eligible participants were individuals who needed partial or full assistance for daily living under Japan's long-term care insurance system (care levels 1-5). The degree of agreement and kappa coefficient were calculated for each item and the total score, after which inter-rater reliability was determined. The effect of the participant's care level on reliability was also evaluated as secondary analysis.
RESULTS: A total of 96 participants were assessed by 83 care managers (two assessors scored each participant). The degree of agreement and calculated kappa coefficient of the PPRA-Home total score were 59% and 0.72, respectively, with the inter-rater reliability for the total score determined to be "Substantial". Our subgroup analysis showed that the inter-rater reliability differed according to the participant's care level. Accordingly, the kappa coefficient for the total score was lower in subgroup "care level 1-3" than in subgroup "care level 4-5" (0.51 and 0.76, respectively).
CONCLUSION: Our result showed that the PPRA-Home has substantial inter-rater reliability for evaluation of risks of pressure injury development at home care. However, some research focusing on intra-later reliability and validity of the PPRA-Home with adequate sample sizes are required to provide categorical conclusions on whether it can be used for the risk assessment scale in actual clinical settings.

Introduction

The focus of healthcare, including nursing care and specialized medical services, is expected to shift from hospital-based to home- and community-based care as more individuals are expected to avail of home healthcare services in the aging society.1–3 In Japan, the number of individuals requiring long-term care benefits at home under Japan’s long-term care insurance (LTCI) system established by Japanese government has continued to increase annually, reaching 6 million in 2016.4 Considering that most individuals have single or multiple diseases that are associated with pressure injury development, risk of pressure injury development during home care appears to be increasing.5 Several studies have reported a 3.6–10% prevalence of pressure injury among the Japanese population.6–8 Moreover, elderly individuals at home or within communities suffered from more serious pressure injury problems than those admitted to acute care hospitals.

Prevention of pressure injuries is always better than its treatment. As such, preventive measures are need to be developed for the successful management of pressure injuries during home care. The use of risk assessment tools after classifying single or multiple risk factors of development of immobility-related injuries can be vital first step for evaluating individuals in pressure injury prevention.9 The Braden scale has been the most widely used risk assessment scale for predicting the risk of hospital- or facility-acquired pressure injuries.10 Several reports have shown that the Braden scale had moderate to high levels of reliability and validity.11,12 Thus, acute care and university hospitals have recommended the use of pressure redistribution mattresses based on risk assessment using Braden Scale.13 The Ohura-Hotta (OH) scale, which consists of four subscales (ie, spontaneous body turning, sacral bony prominence, edema, and articular contracture) has also been widely accepted as a risk assessment scale for predicting pressure injury risk among elderly individuals specifically within Japan.14 One study showed that the clinical application of a support surface selection algorithm based on the OH scale resulted in reduced incidence rates of pressure injuries within a Japanese general hospital.15

Despite the benefits offered by the Braden and OH scales, they have seen limited use in current home care settings wherein Japanese social welfare professionals work. Our previous questionnaire survey targeting care managers, one type of professional occupation under the social welfare services of the LTCI system, revealed that less than half of the participants found it difficult to use the Braden or OH scales in daily actual clinical practice.16 Reports have shown that the Braden scale did not have a strong enough impact for use in long-term care facilities.17,18 The Japanese government had originally introduced a medical-social welfare networking model into the LTCI system with care managers as gatekeepers.19,20 Accordingly, they are entirely responsible for providing home care management services, including assessing the physical conditions and living circumstances of individuals, planning all care, and assisting with procedures necessary to avail such services. In addition, the care service plan should be adjusted based on changes to each individual’s condition. To obtain a care manager license, 5 years or more of experience in geriatric care as a medical and/or social welfare professional is necessary, as well as passing a national examination. Care managers visit the house of each individual to interview them and/or their cohabitants at least once a month. Thus, risk assessment by care managers is imperative for a comprehensive understanding on the necessary support needed by individuals receiving long-term care under the LTCI system.

Recently, a risk assessment scale, called “Pressure Injury Primary Risk Assessment Scale for Home Care (PPRA-Home)”, has been developed for use by care managers under the LTCI system of Japan.21 Moreover, the application of the PPRA-Home could be one solution for achieving successful individual-centered multidisciplinary collaboration between medical professionals and social welfare professionals with care managers acting as a bridge. Notably, although one study has been conducted on the reliability and validity of the PPRA-Home in a Japanese geriatric health services facility, more research regarding its reliability is needed before being introduced into practical use.21

The current study therefore sought to evaluate inter-rater reliability of the PPRA-Home among care managers in facilities located across several local areas wherein the use of medical and nursing care resources tends to be limited.

Materials and Methods

Study Design

This multicenter cross-sectional study was conducted at 30 different home-based geriatric support services facilities in Takaoka city (Toyama prefecture), Funabashi city (Chiba prefecture), Nara city (Nara prefecture), and Sendai city (Miyagi prefecture), Japan from November 2019 to June 2020.

Samples and Setting

Eligible participants were individuals classified as requiring care levels 1–5 in the four aforementioned districts. Two types of Japanese public insurance programs allow elderly individuals access to medical and nursing care: 1) the medical insurance program and 2) the LTCI system. The former is available for all people (regardless of age) upon needing appropriate treatment or examination at a hospital, while the latter is available for individuals aged 40–64 years with specific diseases or those aged 65 years or over. Individuals that use the LTCI are categorized into seven care needs levels: support levels 1 and 2 and care levels 1 to 5.22 The definition of each care level is presented in the Data Collection section of this paper.

Individuals with existing pressure injuries at the start of the study were excluded. The present study needed 66 participants to obtain a power of 0.08, an alpha of 0.05, and a desired reliability coefficient of 0.8, as demonstrated in a previous research.21 After accounting for attrition, the target number of 75 participants was required.

Instruments

Since its creation in 2019, the PPRA-Home has yet to be translated into other languages from its original language of Japanese.21 Briefly, the concepts underlying the designing of PPRA-Home included: 1) identifying risk factors for pressure injury development among Japanese elderly individuals; 2) simple question and answer; and 3) utilizing descriptions that Japanese care managers can easily understand. The developer extracted risk factors in a home-based pressure injury development by reviewing the existing risk factors from well-established pressure injury risk assessment scales such as Braden and OH scales. Next, the words of the extracted risk factors were rephrased to become more understandable for Japanese care managers. Finally, the PPRA-Home was completed after receiving advice to be revised in regard with the words and their meaning of each item on the PPRA-Home by an associate professor specialized in home health care in a certain university. The scale includes the following eight items: spontaneous body turning, body type, mobility, decreased food intake, skin moisture, using diapers, edema, and using head-of-bed elevation. Responses to each item of the PPRA-Home can be indicated by two categories, “yes” or “no”, with a score of “1” being given when the answers were “yes” and “0” when “no”. Scores for each item were then summed up, resulting in a total score ranging from 0– 8. A total score of 8, the highest score possible, indicates the highest risk for developing pressure injuries. The definitions of each item on the PPRA-Home and their scoring rules are shown in Table 1. A fair agreement with Fleiss’s kappa statistics of 0.29 was obtained in a previous reliability study conducted by the developer.21 In the same literature, a cutoff score of 2 on the PPRA-Home had a sensitivity of 77% and specificity of 99% in a cutoff validity testing.

Table 1

Definition and Scoring Rule of Pressure Injury Primary Risk Assessment Scale for Home Care

Descriptions	Definitions	Scoring Rules
Spontaneous body turning	Unable to spontaneously move or turn over in bed.	1=yes. 0=no.
Body type	Body shape is identified as skinny or boney.	1=yes. 0=no.
Mobility	Reduced joint mobility, and bending/stretching joints of knee or elbow is difficult.	1=yes. 0=no.
Food intake	Daily food intake is reduced (total amount of eating or eating frequency can be considered).	1=yes. 0=no.
Skin moisture	Skin is occasionally moist with sweat (degree of moisture is not considered).	1=yes. 0=no.
Using diapers	Constantly has usage of diapers for elderly people.	1=yes. 0=no.
Edema	Swelling in the lower legs (degree of swelling is not considered).	1=yes. 0=no.
Using head-of-bed elevation	Usually using Head-of-bed elevation when raising upper body from a spine position (Degree of elevation is not considered).	1=yes. 0=no.

Study Procedure

All assessors had a care manager license and several years of experience in the field of home-based care. Moreover, they worked at the home-based geriatric support services facilities studied herein. The investigators orally explained the outline of this study to the candidates of the assessors who then received a written agreement for cooperation upon deciding to participate in this study.

Two assessors performed the data collection during regular visits to the participant’s home or day-care service center. A day-care service center where individuals could receive any care support services, such as meals and baths, without staying the night was also included during data collection. Among the two assessors for each participant, one supervised the participant in actual practice, while the other worked at the same facility where the first one worked. The PPRA-Home in this study was performed by a pair of assessors who visually assessed the physical condition of the participants. The assessors were also allowed to ask the participants, cohabitants, or their caregiver regarding anything related to the circumstances of the participant’s daily life. Two assessors visited the participants at their home or day-service center throughout the study period. However, both assessors performed observations independently, with no communication between them. Neither of the assessors received training regarding the PPRA-Home prior to start of this study considering the importance of clarifying the inter-rater reliability and agreement of the PPRA-Home under the care manager’s current level of knowledge and experience without any specific training. The assessment lasted for an average of 2 minutes per assessor.

Data Collection

All data were obtained from the study records and prepared using written data collection forms. The records were reviewed for the assessor’s demographic data (age, gender, and years of experience as a care manager) and professional background (previous role) and the participant’s demographic data (age, gender, living arrangement), care needs level, level of functional disability (bedridden level), and level of cognitive impairment (dementia rating). Each item and total score of the PPRA-Home were also recorded on the data collection forms. The forms were placed in an envelope and returned to the authors by assessor pairs.

Data on care level, bedridden level, and rating of dementia were obtained from official data of the Care Needs Level Certification Committee of the city’s administrative office. Individuals with care level 1 required any partial support for daily living, whereas those with care level 5 had difficulty living without excessive assistance by medical and/or social welfare professionals. The bedridden level was ranked from Rank J (independent but required partial support) to Rank C (completely bedridden). The rating of dementia was also ranked from Independent (very low cognitive disorder) to Rank M (severe cognitive disorder).

Statistical Analysis

Continuous variables are expressed as means and standard deviation, while categorical variables are expressed as frequencies and percentages.

Inter-rater reliability was addressed using both degree of agreement and kappa coefficient for assessor pairs considering that these were the most prevalent reliability measures in this context.21,23 Degree of agreement was defined as the number of agreed cases divided by the sum of the cases with agreements and disagreements. Unweighted kappa was performed to calculate the inter-rater reliability of each item, while kappa with quadratic weighting was employed to calculate sum scores. An unweighted and weighted kappa of <0.00 were identified as poor, 0.00–0.20 as slight, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1.00 as almost perfect based on previous literature.21 Finally, kappa values above 0.70 were considered as adequate acceptance criteria in this inter-rater reliability study.11

Post-hoc subgroup analysis was also performed to explore possible differences in the inter-rater reliability of the PPRA-Home total score according to subgroups of participants.24 This analysis focused on care needs level, bedridden level, and rating of dementia given their importance for confirming the participant’s current physical/cognitive status.

All statistical analyses were conducted using the Statistical Package for the Social Sciences version 20.0 (IBM Corporation, Tokyo, Japan).

Results

Characteristics of Assessors and Participants

Data from a total of 83 assessors were collected in this study. The characteristics of the assessors are summarized in Table 2. Most of the assessors were 40–49 years old (41%) and female (82%). Approximately half of the assessors (50%) had a certificated care worker license, whereas only 6% had a nursing license (registered nurse and/or licensed practical nurse).

Table 2

Assessor Characteristics

Characteristics	Overall (n=83)
Age in years, n (%)
<40	10 (12)
40–49	34 (41)
50–59	18 (22)
≥60	21 (25)
Gender, n (%)
Male	15 (18)
Female	68 (82)
Years of experience as a care manager, n (%)
<5	27 (33)
≥5 and <10	29 (35)
≥10	27 (33)
Professional background, n (%)a
Certified care worker	67 (50)
Certified social worker	15 (11)
Home attendant	26 (20)
RN or LPN	8 (6)
Others	17 (13)

Note: aThe 83 assessors had 133 professional licenses.

Abbreviations: RN, registered nurse; LPN, licensed practical nurse.

Data from a total of 96 participants with a mean age of 84 (9.7) located at Takaoka city (n=45), Funabashi city (n=30), Nara city (n=20), and Sendai city (n=1) were collected in this study (Table 3). Approximately 71% were female, and 80% were living in the same house as others (spouse and/or children). Care level was almost equally distributed (level 1, 20%; level 2, 22%; level 3, 19%; level 4, 25%; and level 5, 13%).

Table 3

Participant Characteristics

Characteristics	Overall (n=96)
Age in years, mean (SD)	84 (9.7)
Gender, n (%)
Male	28 (29)
Female	68 (71)
Living arrangement, n (%)a
Alone	19 (20)
With others	76 (80)
Care level, n (%)a
Level 1	19 (20)
Level 2	21 (22)
Level 3	18 (19)
Level 4	24 (25)
Level 5	12 (13)
Functional disability, n (%)a
Rank J	14 (15)
Rank A	43 (45)
Rank B	28 (29)
Rank C	10 (11)
Cognitive impairment, n (%)a
Independent/Rank 1	29 (31)
Rank 2	44 (46)
Rank 3	15 (16)
Rank 4/M	7 (7)

Note: aMissing data were excluded from this table.

Abbreviation: SD, standard deviation.

Inter-Rater Reliability

The degree of agreement on each item and total score for the two assessors are presented in Table 4. The degree of agreement was considered good, ranging from 80–93% for each item and 59% for the total score. Kappa coefficients for each item and total score are also detailed in Table 3. The item “using head-of-bed elevation” yielded the highest kappa coefficient (unweighted kappa=0.82; 95% CI=0.69–0.95), whereas the item “skin moisture” yielded the lowest kappa coefficient (unweighted kappa=0.58; 95% CI=0.42–0.75). The calculated weighted kappa for total score was 0.72 (95% CI=0.44–0.98), while findings determined that the PPRA-Home score had “Substantial” inter-rater reliability.

Table 4

Inter-Rater Reliability and Agreement of the Pressure Injury Primary Risk Assessment Scale for Home Care

Descriptions	Observed Agreement (95% CI)	Kappa Statistic (95% CI)	Inter-Rater Reliability
Spontaneous body turning	88% (79–93%)	0.68 (0.51–0.85)	Substantial agreement
Body type	90% (81–95%)	0.77 (0.63–0.91)	Substantial agreement
Mobility	89% (80–94%)	0.74 (0.60–0.88)	Substantial agreement
Food intake	84% (75–91%)	0.66 (0.50–0.82)	Substantial agreement
Skin moisture	80% (71–87%)	0.58 (0.42–0.75)	Moderate agreement
Using diapers	85% (76–91%)	0.65 (0.48–0.82)	Substantial agreement
Edema	83% (74–90%)	0.66 (0.51–0.81)	Substantial agreement
Using head-of-bed elevation	93% (85–97%)	0.82 (0.69–0.95)	Almost perfect agreement
Total score	59% (49–69%)	0.72 (0.44–0.98)	Substantial agreement

Abbreviation: CI, confidence interval.

Subgroup Analysis

The degree of agreement, kappa statistics, and inter-rater reliability of each item and total score for each subgroup are outlined in Table 5. Notably, inter-rater reliability differed according to the participant’s physical characteristics. In particular, subgroup “care level 1–3” had a lower kappa statistic for the total score compared to subgroup “care level 4–5” (weighted kappa=0.51; 95% CI=0.03–0.98 and 0.76; 95% CI=0.38–1.00, respectively). Similar results are shown in the functional disability subgroup (bedridden level). In contrast, almost no difference was observed in the cognitive impairment subgroup (dementia rating).

Table 5

Subgroup Analysis for Inter-Rater Reliability and Agreement of Each Item and Total Score of the Pressure Injury Primary Risk Assessment Scale for Home Care

Subgroup	Observed Agreement (95% CI)	Kappa Statistic (95% CI)	Inter-Rater Reliability
Care needs level
Care level 1–3 (n=58)
Spontaneous body turning	88% (76–95%)	0.40 (0.00–0.82)	Fair agreement
Body type	91% (79–97%)	0.76 (0.55–0.96)	Substantial agreement
Mobility	93% (82–98%)	0.80 (0.61–0.99)	Substantial agreement
Food intake	86% (74–93%)	0.70 (0.51–0.89)	Substantial agreement
Skin moisture	82% (70–91%)	0.62 (0.40–0.83)	Substantial agreement
Using diapers,	82% (70–91%)	0.64 (0.44–0.84)	Substantial agreement
Edema	85% (72–92%)	0.68 (0.48–0.87)	Substantial agreement
Using head-of-bed elevation	93% (83–98%)	0.63 (0.28–0.98)	Substantial agreement
Total score	60% (47–73%)	0.51 (0.03–0.98)	Moderate agreement
Care level 4–5 (n=36)
Spontaneous body turning	86% (70–95%)	0.72 (0.50–0.95)	Substantial agreement
Body type	87% (70–96%)	0.75 (0.52–0.98)	Substantial agreement
Mobility	80% (63–91%)	0.61 (0.32–0.89)	Substantial agreement
Food intake	83% (67–93%)	0.61 (0.36–0.87)	Substantial agreement
Skin moisture	75% (57–87%)	0.49 (0.21–0.78)	Moderate agreement
Using diapers,	91% (76–98%)	0.62 (0.21–1.00)	Substantial agreement
Edema	81% (63–91%)	0.61 (0.35–0.87)	Substantial agreement
Using head-of-bed elevation	92% (76–98%)	0.83 (0.65–1.00)	Almost perfect agreement
Total score	56% (38–73%)	0.76 (0.38–1.00)	Substantial agreement
Functional disability
Rank J, A (n=57)
Spontaneous body turning	88% (75–95%)	0.31 (0.00–0.79)	Fair agreement
Body type	93% (82–98%)	0.81 (0.63–0.99)	Almost perfect agreement
Mobility	91% (80–97%)	0.76 (0.55–0.96)	Substantial agreement
Food intake	85% (74–93%)	0.71 (0.52–0.89)	Substantial agreement
Skin moisture	82% (70–91%)	0.64 (0.44–0.84)	Substantial agreement
Using diapers,	84% (72–93%)	0.67 (0.47–0.87)	Substantial agreement
Edema	86% (74–93%)	0.70 (0.51–0.89)	Substantial agreement
Using head-of-bed elevation	95% (84–99%)	0.81 (0.60–1.00)	Almost perfect agreement
Total score	63% (49–75%)	0.62 (0.26–1.00)	Substantial agreement
Rank B, C (n=38)
Spontaneous body turning	87% (72–95%)	0.75 (0.54–0.95)	Substantial agreement
Body type	85% (68–94%)	0.71 (0.47–0.94)	Substantial agreement
Mobility	84% (69–94%)	0.69 (0.46–0.92)	Substantial agreement
Food intake	82% (65–91%)	0.57 (0.28–0.86)	Moderate agreement
Skin moisture	77% (60–88%)	0.49 (0.20–0.78)	Moderate agreement
Using diapers,	87% (72–95%)	0.54 (0.16–0.92)	Moderate agreement
Edema	79% (63–90%)	0.59 (0.34–0.84)	Moderate agreement
Using head-of-bed elevation	92% (78–98%)	0.84 (0.67–1.00)	Almost perfect agreement
Total score	51% (35–67%)	0.72 (0.38–1.00)	Substantial agreement
Cognitive impairment
Independent/Rank 1, 2 (n=73)
Spontaneous body turning	85% (74–92%)	0.53 (0.28–0.79)	Moderate agreement
Body type	91% (81–96%)	0.79 (0.64–0.95)	Substantial agreement
Mobility	92% (83–97%)	0.81 (0.67–0.96)	Almost perfect agreement
Food intake	85% (75–92%)	0.67 (0.49–0.85)	Substantial agreement
Skin moisture	86% (76–93%)	0.71 (0.55–0.88)	Substantial agreement
Using diapers,	88% (78–99%)	0.72 (0.55–0.89)	Substantial agreement
Edema	85% (75–92%)	0.70 (0.54–0.86)	Substantial agreement
Using head-of-bed elevation	92% (83–97%)	0.77 (0.60–0.95)	Substantial agreement
Total score	62% (50–73%)	0.69 (0.32–1.00)	Substantial agreement
Rank 3, 4, M (n=22)
Spontaneous body turning	91% (69–98%)	0.82 (0.58–1.00)	Almost perfect agreement
Body type	86% (64–96%)	0.68 (0.34–0.99)	Substantial agreement
Mobility	77% (54–91%)	0.50 (0.12–0.89)	Moderate agreement
Food intake	77% (54–91%)	0.50 (0.12–0.89)	Moderate agreement
Skin moisture	60% (37–78%)	0.14 (0.00–0.57)	Slight agreement
Using diapers,	77% (54–91%)	0.35 (0.01–0.65)	Fair agreement
Edema	77% (54–91%)	0.52 (0.15–0.89)	Moderate agreement
Using head-of-bed elevation	95% (75–99%)	0.90 (0.72–1.00)	Almost perfect agreement
Total score	45% (25–67%)	0.71 (0.36–1.00)	Substantial agreement

Abbreviation: CI, confidence interval.

Discussion

Given the importance of reliability studies of risk assessment scales, several previous studies investigating the inter-rater reliability of well-established pressure injury risk assessment scales have helped establish clinical guidelines for the prevention and treatment of pressure injuries.25–27 Currently, the PPRA-Home is a unique risk assessment scale that targets Japanese care managers for pressure injury prevention. However, the inter-rater reliability of the PPRA-Home has not been adequately addressed in contemporary literature.21 Accordingly, our findings revealed the PPRA-Home had adequate reliability, suggesting that care managers should consider utilizing this scale when providing social welfare geriatric services in aging societies.

The current study had been based on a previous study by Kottner et al,28 which proposed a guideline for reporting reliability and agreement studies across a wide range of disciplines. Moreover, our multicenter approach can be even more advantageous given that it allowed for adequate sample size, which ensured that the current study achieved adequate participant enrolment and reached its target sample size. Furthermore, considering that 86 assessors were included herein, data obtained from this study should comprise important information for all specialists in the management of pressure injuries that would facilitate the use of the PPRA-Home.

The most important finding of the current study was that the PPRA-Home total score had substantial inter-rater reliability, with a weighted kappa of 0.72 (Table 4), indicating that the PPRA-Home meets the generally acceptable criteria for inter-rater reliability. A previous report showed that each item on Braden scale had a Cohen’s kappa ranging from 0.72–0.86,25 while another showed that the total Braden scale score measured by incentive care unit nurses had an intra-class correlation coefficient ranging from 0.72–0.84.26 A similar result was also reported by another study conducted in a home care setting.27 To the best of our knowledge, an official kappa value acceptance criterion with sufficient reliability has yet to be established in the field of medical research. Nonetheless, values above 0.70 have generally been considered adequate.29 Therefore, our findings indicate that the obtained kappa coefficient for the PPRA-Home total score is currently acceptable with sufficient reliability.

The inter-rater reliability for each item of the PPRA-Home ranged from moderate to almost perfect (Table 4). Moreover, items “body type (skinny or not)”, “mobility”, and “using head-of-bed elevation” complied with the acceptance criteria of this study (Kappa value>0.70), whereas the remaining items did not. The item “skin moisture” yielded the lowest kappa value (0.58; 95% CI=0.42–0.75), which may have been due to the different interpretations of the term among the assessors. The item was included in the PPRA-Home given that exposure to the excessive moisture from urine and/or sweat can increase the possibility for skin surface and superficial tissue damage due to external stimuli, such as friction.30 However, the PPRA-Home employs simple and concise words with no supplementary explanation for each item.21 Therefore, the assessors participating in this study were considered to have had varying interpretations of the term “skin moisture”.

Another possible reason for the low kappa (below 0.7) in some items was the lack of education regarding pressure injury prevention among care managers. Although all assessors included herein were licensed care managers, their background profession (previous occupation) varied considerably, ranging from nursing specialists to home attendants. The difference in their background may have had an effect on their knowledge and practice related to pressure injury prevention. All assessors need to understand not only the words themselves but also the background in which those words were employed in the risk assessment scale. The authors suggest that more education among care managers would increase inter-rater reliability and agreement among assessors.

PPRA-Home was originally developed for easy to use in home care without the need for specific training and/or education aimed at assessors (care managers) in the context of pressure injury prevention.21 Considering the effect of training and education is also important when investigating risk assessment scale for ulcer management. However, the authors did not consider the training and education for the assessors in this study, because it should be respected as a philosophy of the developer of PPRA-home. A problem of this study that the assessors did not receive any training was that their current level of knowledge in pressure injury prevention was not recognized. It would have been better if knowledge of the assessors was evaluated with some validated scales prior to the start of this study.

The inter-rater reliability and kappa values of PPRA-Home obtained herein were better that those reported in Morita et al,21 which were conducted in one Japanese geriatric health services facility (Kappa in total score=0.29). However, strictly comparing the kappa coefficients of both studies is difficult given the differences in statistical approach used for calculations. Notably, the current study utilized weighted kappa for the PPRA-Home total score, whereas the previous report employed unweighted Fleiss kappa.21

The present study hypothesized that the results of the PPRA-Home would be sufficient among all subgroups. However, participants in the “care level 4–5” subgroup had a higher kappa coefficient than those in the “care level 1–3” subgroup (Table 5). Although the reason for this is unclear, care managers may often check the physical condition to provide comprehensive care to individuals with higher care levels. Moreover, the possible influence of the assessors’ individual interests on kappa and agreement cannot be excluded and will require further study. As such, the competence of care managers with respect to preventing pressure injury may differ based on the care level of the individuals.

From a clinical point of view, the use of the PPRA-Home is a chance to promote a multidisciplinary approach for achieving pressure injury prevention. A previous study suggested that care managers rarely communicate with physicians and nurses regarding the physical condition of individuals needing care support for their daily living.31 Most care managers currently believe that physician and/or nurse interventions can be initiated after the development of pressure injury, which is not ideal in real-world settings.16 We therefore believe that pressure injury risk assessment scales, including the PPRA-Home, can serve as a bridge between medical and social welfare professionals to achieve a multidisciplinary approach in preventing pressure injuries during home care.

The current study has some limitations worth addressing. The subgroup analysis was underpowered given that this trial focused on determining the overall effects of assessment. Our subgroup analysis showed that the upper limit of the 95% CI in several subgroups reached 1.0, suggesting an excessively wide CI due to small numbers during each analysis.32,33 Considering that subgroup analysis may address practical concerns, such as identifying which type of individuals would benefit the most, more observational or prospective clinical studies with larger sample sizes, including participants with higher care levels, will be needed to determine the effectiveness of each subgroup. The second limitation is intra-observer variability was not evaluated, although the outcome of this study was to measure inter-rater reliability. The busy schedule made it difficult for assessors to conduct intra-observer variability testing in-parallel with this inter-rater reliability study. Separate study needs to be required to clarify whether the assessors themselves valued the same with a temporal distance.

Despite such limitations, the PPRA-Home is currently considered to be a useful pressure injury risk assessment scale among care managers in Japanese home care settings. Based on a previous validity research, this was performed on the premise that the PPRA-Home would be a valid scale for measuring pressure injury risk.21 However, the authors consider that the previous study was preliminary and scarce. Future research shall be conducted to investigate the validity of the PPRA-Home in order to investigate all aspects of the scale with regard to pressure injury prevention. An estimation of cutoff value with sensitivity and specificity will allow care managers to identify risk levels for pressure injury development. Also, an evaluation for clinicometric properties of PPRA-Home with assessors and individuals who have a risk of pressure injury development will be expected to include as a validity study.

Conclusions

The current multicenter cross-sectional study assessed the inter-rater reliability and agreement of the PPRA-Home for pressure injury prevention among individuals who needed partial or full assistance for daily living under the Japanese LTCI system. Our result showed that the PPRA-Home has substantial inter-rater reliability for evaluation of risks of pressure injury development at home care. It is expected that the use of the PPRA-Home potentially contribute to the improvement of the quality of nursing care by early detection of the individuals who have a risk of developing pressure injuries in home care. However, some research focusing on intra-later reliability and validity of the PPRA-Home with adequate sample sizes are required to provide categorical conclusions on whether it can be used for the risk assessment scale in actual clinical settings.