Concurrent Validity of Pictorial Fit-Frail Scale (PFFS) in Older Adult Male Veterans with Different Levels of Health Literacy.

Introduction: Frailty is a state of vulnerability characterized by multisystemic physiological decline. The Pictorial Fit Frail Scale (PFFS) is a practical, image-based assessment that may facilitate the assessment of frailty in individuals with inadequate health literacy (HL). Objective: Determine the concurrent validity and feasibility of the PFFS in older Veterans with different levels of HL and cognition.
Methods: Cross-sectional study in a geriatric clinic at a Veteran Health Administration (VHA) medical center. Veterans ≥65 years old completed a HL evaluation, PFFS, FRAIL scale and cognitive screening. We assessed the associations between PFFS, FRAIL scale, and VA-Frailty Index (VA-FI), and compared PFFS and FRAIL scale accuracy with a Receiver Operating Characteristic curve, Area Under the Curve (AUC) analysis, using the VA-FI as reference.
Results: Eighty-three Veterans, mean age 76.20 (SD = 6.02) years, 65.1% Caucasian, 69.9% had inadequate HL, 57.8% were frail and 20.5% had cognitive impairment. All participants completed the 43 PFFS items. There were positive correlations between PFFS and VA-FI, r = .55 (95% CI: 0.365-0.735, p < .001), and FRAIL scale, r = .673 (95% CI: 0.509-0.836, p < .001). Compared to the VA-FI, the PFFS (AUC = 0.737; 95% CI: 0.629-0.844) and FRAIL scale (AUC = 0.724;95% CI: 0.615-0.824; p < .001) showed satisfactory diagnostic accuracy. Conclusions: The PFFS is valid and feasible in evaluating frailty in older Veterans with different levels of HL and cognition.

Patients with inadequate health literacy may benefit from the use of the PFFS.

The PFFS is a feasible instrument in the assessment of frailty.

The PFFS diagnostic accuracy is comparable to other frailty assessment tools.

Introduction

Frailty, a common condition in older adults, is a clinical syndrome characterized by vulnerability to stressors resulting from a loss of physiological reserve across multiple systems. (Fried et al., 2001). Frailty is associated with adverse health care outcomes including disability (Covinsky et al., 2003; Topinkova, 2008), morbidity (Gobbens et al., 2010; Vetrano et al., 2019), mortality (Mitnitski et al., 2002; Romero-Ortuno & Kenny, 2012), and increased healthcare utilization (McNallan et al., 2013; Zylberglait Lisigurski et al., 2017). Identification of frailty then becomes an important step to design and implement strategies aimed at optimizing the care of these patients. However, most frailty assessments are based on physical and mental tests that may be difficult to conduct in clinical practices. Recently claim-based assessments from data in electronic health records systems may also generate determinations of frailty but are not widely available in most settings (Clegg et al., 2018; Kim and Schneeweiss, 2014). The other assessments are based on questionnaires, which may be difficult to accurately complete by individuals with frailty who may have higher rates of mood disorders, sensory loss, cognitive impairment, or limited health literacy (Baker et al., 2000; Federman et al., 2009).

Health literacy is the capacity to obtain, process, and use basic health information and services needed to make health care decisions (Nielsen-Bohlman et al., 2004). Patients with inadequate health literacy may benefit from the use of pictures to improve comprehension of medical information (Entwistle & Williams, 2008). Pictorial aids may help improve adherence to medication regimens (Katz et al., 2006; Negarandeh et al., 2013) and medical decision making (Garcia-Retamero & Cokely, 2017). On the other hand, contrary evidence reveals that pictures may not always be effective at improving the comprehension of medical information (Ruiz et al., 2013). The Pictorial Fit-Frail Scale (PFFS) is a recently developed, image-based questionnaire for frailty assessment that patients themselves, caregivers and healthcare providers may use to assess frailty in clinical settings (McGarrigle et al., 2019; Theou et al., 2019). The aim of this study was to evaluate the concurrent validity of PFFS in assessing frailty status in a group of racially and ethnically diverse older Veterans with different levels of health literacy and cognition. A secondary aim was to evaluate the feasibility of implementing this instrument in geriatric primary care practice.

Methods

Design and Procedure

A cross sectional validation study was conducted at a government-run, US Veterans Health Administration (VHA) Medical Center in the Southeast of the US. Participants included a convenience sample of community-dwelling Veterans 65 years and older, enrolled in an outpatient geriatrics clinic. Veterans with unstable illnesses or sensory impairment, and those who did not speak English were excluded. This study was reviewed and approved by the Miami VA Healthcare System Institutional Review Board (IRB), and a written informed consent was obtained from all participants before the administration of the assessments.

Outcomes and Measures

Socio-demographics

The research team collected baseline socio-demographic data from the VA electronic health record (CPRS Vista) and the VA Corporate Data Warehouse (CDW). The Veterans Healthcare Administration (VHA) uses the International Classification of Diseases, Ninth (ICD-9) and Tenth Revision (ICD-10) classification systems to code procedures and diagnosis. All data can be linked to socio-demographic information including age, gender, marital status, race, ethnic group, tobacco use and education.

Frailty

We assessed frailty using three instruments:

The VA-frailty index

For each patient, a frailty index was automatically generated from electronic health record data on the date of the geriatric primary care visit between the period of September 1, 2019 and May 31, 2020. In this study, we used the 31-item VA-FI (see Table 1) (Orkaby et al., 2019) as the reference assessment for frailty, which is based on a deficit accumulation conceptual framework. It assumes that frailty is the result of interacting physical, functional, psychological, and social factors (Orkaby et al., 2019; Rockwood & Mitnitski, 2007) that belong to five major categories: morbidity, function, sensory loss, cognition, and mood. The VA-FI was then calculated by adding up the number of deficits obtained from the patient and divided by 31 items (total number of health deficits) (Searle et al., 2008). The individual variable calculations resulted in scores between 0 and 1, where the higher score represented the presence of a deficit. Total scores were categorized as non-frail (<0.21) and frail (≥0.21) based on previously published cut-points (Orkaby et al., 2019).

Table 1.

Components of the VA-FI.

Morbidity:		17	Fall or Fall related diagnoses
1	Anemia	18	Fatigue
2	Atrial fibrillation	19	Gait abnormality
3	Cancer (except basal cell skin cancer)	20	Parkinson’s disease or tremor disorders
4	Cerebrovascular disease	21	Peripheral vascular disease or intermittent claudication
5	Coronary artery disease	22	Muscular wasting
6	Diabetes	Sensory loss:
7	Heart failure	23	Hearing impairment/aid
8	Hypertension	24	Peripheral neuropathy
9	Kidney disease	25	Vision impairment
10	Liver disease or cirrhosis	Cognition and mood:
11	Lung disease	26	Dementias
12	Thyroid disease	27	Anxiety
13	Osteoporosis or pathological fracture	28	Mood disorders (Depression, Bipolar Disorder)
14	Incontinence	Other:
Function:		29	Chronic pain
15	Arthritis	30	Failure to thrive
16	Use of durable medical equipment	31	Weight loss in the past year

The pictorial fit-frail scale (PFFS)

This is a recently developed simple and comprehensive visual image-based questionnaire designed to assess frailty. Patients, caregivers, or healthcare providers can complete it in less than 5 minutes. The scale includes 14 domains (mood, number of medications, mobility, function, balance, social connections, daytime tiredness, memory and thinking, vision, hearing, pain, unintentional weight loss, aggression, and bladder control), which are represented by three to six images that correspond to the patient’s level of ability. Participants select the image that best represents their day-to-day life, which is assigned to a score that ranges from 0 to 5 on each domain. The final score is calculated by adding up the individual scores on all domains, which falls within a scale ranging from 0 (no frailty; very fit) to 43 (severely frail) (Theou et al., 2019). The PFFS has good test–retest reliability among patients, and the inter-rater reliability between health care professionals (HCPs) was satisfactory (McGarrigle et al., 2019). There was a significant moderately high correlation between the PFFS and a frailty index derived from a comprehensive geriatric assessment when the PFFS questionnaire was completed by caregivers, nurses, and geriatricians in a memory clinic, but not significant when self-administered by the patients (Wallace et al., 2020). In this study, all participants completed the PPFS on their own after brief instructions were provided by the research coordinators. We did not perform additional testing for those participants screening positive for cognitive impairment.

FRAIL scale

This is a validated instrument based on the frailty phenotype and deficit accumulation models that consists of five questions assessing fatigue, resistance, ambulation, illnesses, and loss of weight. The frail score ranges from 0 to 5 (i.e., 1 point for each component; 0 = best to 5 = worst) and patients are assigned to three categories: robust (0 points), prefrail (1–2 points), and frail (3–5 points) (Morley et al., 2012). The participants completed the FRAIL scale following directions provided by the research coordinators.

Health literacy

The Newest Vital Scale (NVS) is a valid and reliable screening test for the evaluation of health literacy. It consists of a nutritional label from a container of ice cream and six associated questions that require the ability to recognize, explain, and complete basic mathematical calculations. The total score is calculated by adding up the questions answered correctly, and the results are then classified as inadequate (<4) or adequate (≥4–6) health literacy (Weiss, 2005).

Cognitive status

The Mini-Cog served as the screening tool for cognitive impairment. It relies on the clock-drawing test and a three-item recall. A positive cognitive impairment screen requires either an incorrect clock drawing test and at least one incorrect item on recall or, three incorrect items on recall (Borson et al., 2003).

Depression

Participants were screened for depression using the patient health questionnaire 2 and 9 (PHQ-2 and PHQ-9). All items were scored on a 4-point response scale (0 = not at all, 1 = several days; 2 = more than half of the days; 3 = nearly all days). The PHQ-2 contains only the first two items of the PHQ-9, the sum score ranges from 0 to 6, with values ≥ 3 indicating clinically relevant depressive symptomatology (Kroenke et al., 2003; Spitzer et al., 1999).

Functional status

Trained research associates assessed functional status by interviewing participants using the Katz ADL Scale and the Lawton IADL Scale. The Katz ADL Scale includes six categories related to ambulating, feeding, dressing, bathing, continence, and toileting. The Lawton IADL Scale requires complex thinking and organizational skills and includes nine categories related to phone use, transportation, meal preparation, shopping, housekeeping, laundry, handyman work, managing finances, and medications. Both scales measure the patient’s ability to perform any of the activities on their own and classifies the answers into “without help” (independent), “with some help” (partially dependent) and “unable to perform” (dependent). By answering ≥2 items as “dependent” on any of these scales, an individual will be classified as dependent on ADL or IADL, respectively.

Statistical Analysis

Descriptive sociodemographic characteristics, including age, gender, marital status, race, ethnic group, tobacco use, and education were obtained. Feasibility of the PFFS was based on percent completion rate. Categorical variables were presented as frequency (percent), and continuous variables as mean ± SD. Normality of the distribution was checked using the Kolmogorov–Smirnov test. All variables were non-normally distributed. For categorical and continuous variables, Pearson’s chi-square test and Kruskal–Wallis test were used, respectively. A Pearson’s correlation coefficient analysis was performed to determine the level of correlation between continuous variables (PFFS, FRAIL scale, and VA-FI). To ascertain the diagnostic accuracy of the PFFS and FRAIL scale, we used a Receiver Operating Characteristic (ROC) analysis to compare their tradeoff sensitivity and specificity using the VA-FI categories (non-frail and frail) as our reference for frailty assessment. All analyses were performed using SPSS 26.0 for Windows (SPSS, Inc., Chicago, Illinois). All statistical tests were two-tailed and statistical significance was assumed for a p-value <.05.

Results

Patient Characteristics

Participants in this study were 83 male Veterans, had a mean age of 76.2 years (SD = 6.02), 65.1% were Caucasians, 74.7% non-Hispanic and 50.6% were married (see Table 2). The proportion of non-frail and frail Veterans according to the VA-FI (reference standard) was 42.2% (n = 35) and 57.8% (n = 48) respectively. Inadequate health literacy, as measured with the NVS, was present in 69.9% (n = 58); while 20.5% (n = 17) screened positive for cognitive impairment and 8.4% (n = 7) for depression. The mean VA-FI scores among patients with adequate and inadequate levels of health literacy were 0.22 (SD = 0.13) and .26 (SD = 0.23), respectively.

Table 2.

Participant Characteristics.

	Non-Frail (VA-FI < 0.21) n = 35 (42.17%)	Frail (VA-FI ≥ 0.21) n = 48 (57.83%)	All n = 83 (100%)	p-value
Age, mean (SD)	75.74 (5.36)	76.54 (6.49)	76.20 (6.02)	.708
Race
Caucasian, n (%)	22 (62.86)	32 (66.67)	54 (65.06)	.719
African American, n (%)	10 (28.6)	13 (27.1)	23 (27.7)	.881
Native Hawaiian/PI, n (%)	0 (0)	2 (4.17)	2 (2.41)	.222
Other, n (%)	3 (8.57)	1 (2.08)	4 (4.82)	.173
Non-hispanic, n (%)	25 (71.43)	37 (77.08)	62 (74.70)	.558
Married, n (%)	20 (57.14)	22 (45.83)	42 (50.60)	.309
Smoking history, n (%)	0 (0)	9 (18.75)	9 (10.84)	.009
10 or more years of education (%)	34 (97.14)	48 (100)	82 (98.80)	.422
Dependent on ≥2 ADL, n (%)	0 (0)	6 (12.50)	6 (7.23)	.037
PFFS, mean (SD)	7.77 (5.35)	13.60 (7.42)	7.42 (7.20)	<.001
FRAIL scale, mean (SD)	1.17 (1.04)	2.25 (1.37)	1.80 (1.35)	<.001
Inadequate health literacy, n (%)	24 (68.57)	34 (70.83)	58 (69.87)	.820
Cognitive Impairment, n (%)	5 (14.29)	12 (25.00)	17 (20.49)	.232
Depression Positive Screen, n (%)	1 (2.86)	6 (12.50)	7 (8.43)	.230

Note. SD = standard deviation; n = number of participants; ADL = activities of daily living; PFFS = Pictorial Fit Frail Scale; VA-FI = VA frailty index; PI = pacific islander.

Significant differences are in bold (p < .05).

The PFFS was feasible to administer based on the percent completion rate. Each one of the 83 participants in the study was able to complete all the PFFS items without difficulty.

Concurrent Validation of the PFFS

Among 83 Veterans, the mean PFFS score was (13.60 ± 7.42) in the frail group and (7.77 ± 5.35) in the non-frail group, p < .001; whereas the mean FRAIL scale score was (2.25 ± 1.37) in the frail group and (1.17 ± 1.04) in the non-frail group, p < .001. The sensitivity and specificity of the PFFS were compared against VA-FI and FRAIL scale using ROC. Compared with the VA-FI, the PFFS (0.737 [95% CI: 0.629–0.844]) and the FRAIL scale (0.724 [95% CI: 0.615–0.824]) showed similar satisfactory accuracy, p < .001 (see Figure 1). Using the FRAIL scale as the reference standard, the accuracy of the PFFS was higher (0.907, [95% CI: 0.833–0.981]); p < .001. There were moderate positive correlations between the PFFS and VA-FI (r = 0.55 [95% CI: 0.365–0.735]), p < .001, and between FRAIL scale and VA-FI (r = .50 [95% CI: 0.308–0.691]), p < .001. The PFFS and FRAIL scale also showed strong positive correlations (r = 0.673 [95% CI: 0.509–0.836]), p < .001.

Figure 1.

ROC curve of FRAIL scale and PFFS in the determination of the frailty status.

Note. (1) The area under the curve (AUC) for PFFS was 0.737 [95% CI: 0.629–0.844].

(2) The area under the curve (AUC) for the FRAIL scale was 0.724 [95% CI: 0.615–0.824]. The curve was constructed using data of 83 veterans, where 48 (57.83%) were frail and 35 (42.17%) were non-frail. Frailty status of all subjects was confirmed by VA-FI.

Discussion

In this cross-sectional validation study, we investigated the concurrent validity and feasibility of the PFFS as an instrument to evaluate frailty status in a racially and ethnically diverse sample of Veterans with different levels of health literacy and cognitive status. Previous studies with VHA samples including Veterans from different age groups have shown levels of inadequate health literacy that range between 28% and 55% (Artinian et al., 2001; Dolan et al., 2004; Ferreira et al., 2005; Mosher et al., 2012; Rodríguez et al., 2013). The higher levels of inadequate health literacy in our sample may be explained by the comparatively higher age of our geriatric clinic participants as shown by others (Paasche-Orlow et al., 2005). Another explanation is the use of the NVS instrument. As compared with the REALM and TOHFLA, two of the most widely used instruments in previous research, the NVS has demonstrated higher sensitivity for the detection of inadequate health literacy (Osborn et al., 2007). Results from the ROC curve analysis using the VA-FI as the reference standard showed a good validity and diagnostic accuracy for the PFFS. The PFFS was also highly comparable with the FRAIL scale. The PFFS represents a feasible and practical tool for the assessment of frailty in geriatric primary care.

The PFFS has already demonstrated feasibility and good diagnostic performance in patients with different levels of cognitive impairment at a memory disorders clinic (Wallace et al., 2020). Our study is important as it shows for the first time the feasibility and potential diagnostic utility of the PFFS in older primary care patients with different levels of health literacy. Even though, more than half of the participants in this study had inadequate levels of health literacy, the PFFS was still able to discriminate between levels of frailty with comparable accuracy to two of the most widely used frailty assessment instruments, the VA-FI and the FRAIL scale. Furthermore, one third of the patients were not Caucasian and a quarter was Hispanic, an important advantage when assessing populations with diverse racial and ethnic compositions. The high degree of correlation between the PFFS and the FRAIL scale is not surprising considering that both scales were self-administered, whereas the VA-FI was generated from electronic health record data. The comparability of the PFFS and FRAIL scale provides clinicians with alternative diagnostic instruments in populations with high levels of inadequate health literacy. A decision on which instrument to employ to assess patients at risk for frailty was beyond the scope of this study, and further research is required.

The strengths of our study are the administration of validated scales yielding a more comprehensive assessment for frailty status, health literacy, cognition, depressive mood, and functional status. However, our study has some limitations, including a small sample size consisting of only male Veterans. On a similar note, the study was limited to Veterans at one medical center, and ethnic, racial, educational, and socio-economic composition may be different from other Veterans’ facilities in the US.

In summary, the PFFS was feasible to use in a sample of patients with different levels of mood, cognition, and health literacy. The PFFS may assist clinicians in the earlier identification of frailty in high-risk community dwelling older Veterans, and foster interventions to minimize the healthcare burden. Larger studies are needed to further validate the PFFS use in older adults with inadequate health literacy using other clinical settings, databases and diverse populations including a larger representation of women.

Conclusion

Our study has shown that the concurrent validity of the PFFS exhibits a comparable diagnostic accuracy with the VA-FI and FRAIL scale. The PFFS is a feasible instrument in the assessment of frailty in racially and ethnically diverse older Veterans with different levels of health literacy and cognitive status in geriatric primary care.