Practicality and Reliability of Self Vs Administered Rapid Geriatric Assessment Mobile App.

OBJECTIVES: To develop and cross-validate self-administered Rapid Geriatric Assessment (SA-RGA) app against administered Rapid Geriatric Assessment (A-RGA) to identify seniors with geriatric syndromes such as frailty, sarcopenia, and anorexia of ageing who may benefit from targeted intervention.
DESIGN: Prospective observational study.
SETTING: Primary Care and Community. PARTICIPANTS: A-RGA and SA-RGA app were administered to older adults ≥ 60 years old from December 2020 to April 2021. MEASUREMENTS: The RGA app screens for frailty (FRAIL), sarcopenia (SARC-F), anorexia of aging (SNAQ) and cognition (Rapid Cognitive Screen) with assisted management pathway. Patient Health Questionnaire 9 is administered for those who score positive for fatigue. The diagnostic performance of SA-RGA was compared against A-RGA as a reference by calculating the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) and positive likelihood ratio (+LR).
RESULTS: 123 participants with a mean age of 71 ± 5.9 years completed both the SA-RGA and A-RGA. Questions on fatigue, 5 or more illnesses, loss of weight and falls in the past year performed better with high sensitivity, specificity, NPV and +LR than self-functional assessment where SA-RGA participants reported lower prevalence on the FRAIL scale aerobic and resistance components, and higher prevalence on the SARC-F strength and rising from a chair components.
CONCLUSION: The SA-RGA app performed well in certain domains such as assessment for weight loss, falls, number of chronic illness and fatigue. Self-functional assessment can be improved further by removing ambiguity in wordings such as "some" or "a lot" and replacing it with functional difficulty scale. SA-RGA has the potential to be incorporated in the eHEALTH platforms worldwide for early identifications of older adults at risk and to reduce health inequalities, at the same time building community resilience in the era of Covid-19 pandemic.

Introduction

Between 2015 and 2030, the number of older adults ≥ 60 years is projected to grow by 56% from 901 million to 1.4 billion with increased prevalence of disability (1). The aim of healthcare in the 21st century is adding life to years and moving from disease specific to population focussed (2). The World Report on Ageing and Health by World Health Organisation (WHO) defines healthy ageing as the “process of developing and maintaining functional ability that enables wellbeing” (3). There is an urgent need to redesign care for older adults through early screening and assessment to recommending appropriate interventions such as nutrition and physical exercise and referral for comprehensive geriatric assessment (4).

The early identification and management of geriatric syndromes such as frailty, sarcopenia, falls, cognitive impairment, anorexia/weight loss etc has been shown to reduce disability and improve quality of life in older persons (5). Comprehensive geriatric assessment by geriatricians which takes up to 45 minutes have been shown to reduce hospitalisation, mortality and institutionalisation (6). The Rapid Geriatric Assessment (RGA) screens for frailty, sarcopenia, anorexia of aging and cognition, takes less than five minutes to complete and does not require a geriatrician to administer (1, 5). It was developed by St. Louis University and has been validated worldwide (7, 8). An iPad mobile application (app) for RGA Clinic with an assisted management pathway is available in English and Mandarin, and has been tested in the primary care setting(9).

RGA app is a rapid and practical tool which can be used by any healthcare professional to screen at-risk populations but the reliability of the self-administered version of the app has not yet been explored. Studies have shown beneficial impact on health behaviour and healthcare cost of self-assessment with personal recommendations and management(10, 11). However, self-screening tests are well-known for being ambiguous and prone to reporting bias. In addition, screening tests administered digitally such as through apps may be affected by the user’s computer literacy, vision and hearing impairment. It is, therefore, important to determine the diagnostic performance of self-administered RGA (SA-RGA) against administered RGA (A-RGA) by trained healthcare professionals (reference standard) to identify the likely presence or absence of a condition. The aim of this study was to develop and cross-validate SA-RGA against A-RGA to identify seniors with geriatric syndromes such as frailty, sarcopenia, and anorexia of ageing who may benefit from targeted intervention.

Methodology

The RGA app for administration by healthcare professionals has been developed and tested in primary care setting. The app is available in English and Chinese language. The RGA app contains administrative information (name, identifier number, ethnicity, employment, living arrangement, smoking), FRAIL (fatigue, resistance, aerobic, ≥ 5 chronic illness and loss of weight) scale for frailty screening, SARC-F (strength, assistance with walking, rise from a chair, climbing stairs, and falls in the past year) for sarcopenia screening, RCS (rapid cognitive screen) for cognition, and SNAQ (appetite, taste of food, portion consumed, early satiety, and number of meals consumed daily) for anorexia of ageing which predicts 5% or more weight loss within 6 months (9). As RCS has not been validated locally, only FRAIL, SARC-F and SNAQ were cross-validated. The self-assessment RGA app was developed in English and Chinese, with a user-friendly interface of large fonts with up to three questions per page with clear answer categories (Figure 1), and randomly administered to older adults ≥ 60 years old in primary care and community setting from December 2020 to April 2021.

Figure 1

Screenshot of administered (left) and self-administered (right) Rapid Geriatric Assessment

The screening questions were followed with assisted management pathway (5). For instance, if participants scored ‘yes’ to fatigue, this will prompt the Patient Health Questionnaire 9 (PHQ-9) to further screen for depression. Trained coordinators administered the RGA questionnaire, followed with participants completing the self-administered version.

Using the A-RGA as reference, positive predictive value (PPV) and negative predictive value (NPV) were calculated to explore the predictive utility of the self-administered version on the relevant scales and scale items; and sensitivity and specificity testing, and positive likelihood ratio (+LR) were analysed to determine its predictive capability. The analysis was performed using IBM SPSS Statistics Version 23 (IBM Corporation, Armonk, New York, USA). Ethics approval for the study was obtained from the National Healthcare Group Institutional Review Board.

Results

A total of 123 participants completed both the SA-RGA and A-RGA. Mean age of participants was 71 ± 5.9 years. Amongst them, 82.1% were from the Chinese ethnic group, 12.2% from the Malay and 4.9% from the Indian ethnic group. Slightly more than two thirds completed the SA-RGA in English. Amongst them, 62.6% had retired and 13.0% were homemakers and 8.9% unemployed. More than half (58.5%) lived with their spouse and 17.1% lived alone.

The prevalence of FRAIL, SARC-F, SNAQ, and PHQ-9 for the A-RGA and SA-RGA are shown in Table 1. There was a higher prevalence of pre-frailty (32.5% compared to 24.4%) and much lower prevalence of frailty (1.6% compared to 6.5%) in the SA-RGA group. For the aerobic and resistance domain, the prevalence was much lower in the SA-RGA group compared with A-RGA group, 0.8% compared with 12.2% for aerobic and 5.7% compared with 13.8% for resistance domain. For SARC-F, the SA-RGA compared with A-RGA group reported higher prevalence of “some difficulty lifting 10lbs” (23.6% compared with 14.6%), “some difficulty walking across the room” (4.9% compared with 4.1%), “some difficulty transferring from chair to bed” (13.0% compared with 4.9%), “some difficulty climbing a flight of stairs” (23.6% compared with 22.0%) and a higher summed prevalence of 1 or more falls (18.7% compared with 13.8%).

Table 1

Administered and self-administered prevalence of geriatric syndrome, sensitivity, specificity, positive predictive value, negative predictive value and positive likelihood ratio

	Prevalence admin RGA (A-RGA) n (%)	Prevalence self-admin RGA (SA-RGA) n (%)	Sensitivity (%)	Specificity (%)	Positive Predictive Value (PPV) (%)	Negative Predictive Value (NPV) (%)	Positive Likelihood Ratio (+LR)
FRAIL (frailty)
Robust	85 (69.1)	81 (65.9)	83.5	73.7	87.7	66.7	3.17
Pre-frail	30 (24.4)	40 (32.5)	70.0	79.6	52.5	89.2	3.43
Frail	8 (6.5)	2 (1.6)	12.5	99.1	50.0	94.2	13.89
Fatigue	21 (17.1)	23 (18.7)	85.7	95.1	78.3	97.0	17.49
Resistance	17 (13.8)	7 (5.7)	17.6	96.2	42.9	87.9	4.63
Aerobic	15 (12.2)	1 (.8)	0.0	99.1	0.0	87.7	0.0
Illnesses	6 (4.9)	12 (9.8)	83.3	94.0	41.7	99.1	13.88
Loss of Weight	4 (3.3)	11 (8.9)	100.0	94.1	36.4	100.0	16.95
SARC-F (sarcopenia)	7 (5.7)	7 (5.7)	57.1	97.4	57.1	97.4	21.96
Strength (difficulty lifting 10lbs)
none	98 (79.7)	89 (72.4)	86.7	84.0	95.5	61.8	5.42
some difficulty	18 (14.6)	29 (23.6)	72.2	84.8	44.8	94.7	4.75
a lot or unable to	7 (5.7)	5 (4.1)	57.1	99.1	80.0	97.5	63.44
Assistance in Walking (difficulty walking across the room)
none	117 (95.1)	115 (93.5)	94.9	33.3	96.5	25.0	1.42
some difficulty	5 (4.1)	6 (4.9)	0.0	94.9	0.0	95.7	0.0
a lot or unable to	1 (.8)	2 (1.6)	100.0	99.2	50.0	100.0	125.0
Rise from a chair (difficulty transferring from chair to bed)
none	114 (92.7)	107 (87.0)	90.4	55.6	96.3	31.3	2.04
some difficulty	6 (4.9)	16 (13.0)	50.0	88.9	18.8	97.2	4.5
a lot or unable to	3 (2.4)	0 (0.0)	-	100.0	-	97.6	-
Climb stairs (difficulty climbing a flight of 10 stairs)
none	92 (74.8)	90 (73.2)	87.0	67.7	88.9	63.6	2.69
some difficulty	27 (22.0)	29 (23.6)	59.3	86.5	55.2	88.3	4.39
a lot or unable to	4 (3.3)	4 (3.3)	75.0	99.2	75.0	99.2	93.75
Falls (times have you fallen in the past year)
none	106 (86.2)	100 (81.3)	94.3	100.0	100.0	73.9	-
1–3 falls	16 (13.0)	21 (17.1)	100.0	95.3	76.2	100.0	21.28
4 or more falls	1 (.8)	2 (1.6)	100.0	99.2	50.0	100.0	125.0
SNAQ (anorexia of ageing)	27 (22.0)	46 (37.4)	77.8	74.0	45.7	92.2	2.99
PHQ-9	(n = 19)	(n = 26)
Normal	2 (11.1)	10 (38.5)	100.0	87.5	50.0	100.0	8.0
Minimal depression	9 (50.0)	5 (19.2)	37.5	100.0	100.0	66.7	-
Mild depression	5 (27.8)	9 (34.6)	100.0	69.2	55.6	100.0	3.25
Moderate depression	3 (16.7)	2 (7.7)	66.7	100.0	100.0	93.8	-

In Table 1, the sensitivity and specificity for different categories of frailty screening were frail 12.5% and 99.1%, pre-frail 70.0% and 79.6%, and robust 83.5% and 73.7% respectively. Similarly, the PPV and NPV for frail were 50.0% and 94.2%, pre-frail 52.5% and 89.2%, and robust 87.7% and 66.7% respectively. Screening for frailty had the highest +LR of 13.89, followed by pre-frail 3.43 and robust 3.17. As for the individual component of FRAIL scale, screening for loss of weight, fatigue and five or more illnesses had the highest sensitivity of 100.0%, 85.7% and 83.3%, with specificity of 94.1%, 95.1% and 94.0% respectively. The PPV and NPV for loss of weight were 36.4% and 100.0%, fatigue 78.3% and 97.0%, and five or more chronic illness were 41.7% and 99.1% respectively. Loss of weight, fatigue and five of more illnesses similarly had high +LR, reflecting large usefulness in the self-administered app in affirming positive scoring of administrators. Both resistance (climbing 1 flight of stairs) and aerobic (walking 50 metres or 1 bus stop) had high specificity of 96.2% and 99.1%, and NPV of 87.9% and 87.7% respectively.

SARC-F had a sensitivity of 57.1% and specificity of 97.4% with PPV of 57.1%, NPV of 97.4% with +LR of 21.96. The reporting of falls had the highest sensitivity and specificity, with 94.3% and 100% for no falls, 100% and 95.3% for 1–3 falls and 100% and 99.2% for 4 or more falls. The PPV and NPV were 100% and 73.9% for no falls, 76.2% and 100.0% for 1–3 falls and 50.0% and 100.0% for 4 or more falls. Highest +LR was reported for 4 or more falls of 125.0 followed by 1–3 falls 21.28. For the strength question (how much difficulty do you have lifting 10lbs?), the sensitivity and specificity for some problem was 72.2% and 84.8%, a lot or unable to 57.1% and 99.1%, and PPV and NPV for some problem were 44.8% and 94.7% and a lot or unable to 80.0% and 97.5% respectively. For climbing 1 flight of stairs, the sensitivity and specificity for some difficulty was 59.3% and 86.5%, and a lot or unable was 75.0% and 99.2%, with PPV and NPV for some difficulty was 55.2% and 88.3%, and a lot or unable to 75.0% and 99.2% respectively. For ambulation (walking across the room), the sensitivity and specificity for a lot of difficulties or unable to was 100.0% and 99.2% respectively, PPV 50.0%, NPV 100.0% and positive likelihood ratio of 125.0. For transferring from chair to bed, sensitivity and specificity for some difficulty was 50.0%, and 88.9% respectively with PPV of 18.8% and NPV of 97.2%.

SNAQ was used to assess for anorexia of ageing, with sensitivity of 77.8%, specificity 74.0%, PPV of 45.7% and NPV of 92.2%. Only those who scored positive for fatigue were required to complete PHQ-9. The sensitivity and specificity for mild depression was 100.0% and 69.2% with a PPV of 55.6% and NPV of 100.0%. For moderate depression, the sensitivity and specificity were 66.7% and 100% respectively with a PPV of 100.0% and NPV of 93.8%.

Discussion

Our results show older participants were able to complete self-administered RGA with minimal to no assistance with relatively high specificity and +LR. The participants performed relatively well with high sensitivity, specificity, NPV, and positive likelihood ratio on questions pertaining to fatigue, 5 or more illnesses, loss of weight and falls in the past year. Interestingly, there were discrepancies for self-functional assessment between the FRAIL (aerobic and resistance) where the participants reported lower prevalence on the questions “cannot walk up 1 flight of stairs’ and “cannot walk one block (50m)” and SARC-F where the participants reported higher prevalence of “some difficulties” lifting 10lbs and transferring from chair to bed. For PHQ-9 self-assessment, the prevalence of no depression was higher in the self-assessment group, and as the sensitivity of minimal depression was low with high specificity and PPV, it is possible that a few of those with minimal depression may be missed.

RGA is a screening app, with the aim of identifying those at risk of functional decline and disability. It is not invasive, not dangerous, cost-effective, practical, fast with no physical discomfort, and meant to reduce long term healthcare system demands. A typical example includes that of early identification of fatigue with assisted management pathway can help identify anaemia, hypothyroidism, low vitamin B12, obstructive sleep apnoea and other undiagnosed chronic diseases with downstream consequences if not managed early. Similarly, sarcopenia which is a precursor for physical frailty and defined as age-related loss in muscle mass, quality and strength affecting physical performance is reversible with protein enriched diet and physical exercise (12). Frailty results from impairment in multiple systems, and disability is caused by declining functional reserve. Similar to sarcopenia, frailty too is reversible through multicomponent interventions if identified early (13).

Screening tests are well known for not being perfect, and the questions can be ambiguous. The screening tools themselves have different sensitivity and specificity depending on setting and patient group e.g. FRAIL scale (sensitivity 88%, specificity 85.71%, PPV 44.90%, NPV 98.18%)(14) and SARC-F (sensitivity 29.5%, specificity 98.1%)(15) and hence, are referred to as reference standard. Screening test administered through app or digital means can also be affected by computer literacy. Sensitivity and specificity have its own constraints in screening tests, and does not help with probability of the condition in the participants when they screen to be positive. While high PPV is desirable to reduce false positives, especially in situations when there is risk of harm from follow up assessments and overtreatment, this may not necessarily be true for the RGA screening. The advice provided through the assisted management pathway are standard advices which would be useful for older adults to age well regardless of the presence or absence of geriatric syndrome such as resistance exercise, consult your doctor on high protein diet, consult your pharmacist on advice about your medications amongst others.

With an ageing population and shrinking active workforce, many countries are moving towards informal support, productive ageing, training of peer leaders, and/or empowering seniors to take charge of their own health through microjobs, befriending services and volunteering (16). Covid-19 has revealed gaps in health and social services in many countries, which may lead to a cascade of disability, increased intermediate and long-term care, rehabilitative needs, and healthcare costs (17). Covid-19 has also increased technology adoption amongst seniors, and many countries encourage use of HEALTH eSERVICES (e.g. Healthhub in Singapore) which includes medication, appointments, and tips on how to remain healthy and safe(18). Currently, there are no self-reported screening app for geriatric syndromes, and most prior self-assessment questionnaire were in the form of postal questionnaires or lay interviewers(19). The benefits of a screening app include being able to reach greater numbers of older adults, cost savings and empowering older adults to take charge of their own health. This is particularly crucial in the current time where many older adults avoid healthcare institutions, are socially isolated and vulnerable to functional decline, falls and sarcopenia, and can benefit from personalised intervention before the onset of disability (20).

Our study generated interesting findings with certain domains in RGA having high sensitivity, specificity, NPV and +LR, and can be administered digitally. Ambiguous options such as “some”, “a lot or unable to” may need to be modified to include specific measurements or activities to improve sensitivity. Moderate or low PPV may not lead to excessive harm, but in fact may increase awareness and encourage behaviour change to promote healthy ageing. However, there are some limitations which warrants mention. We did not collect information on baseline education, cognitive status, hearing and vision impairment. There was no objective evaluation of ease of comprehending the questions, difficulties of technology use, ownership of personal device or assistance required to complete the self-administered version of RGA. Neither was information collected on the benefits of assessments and assisted management pathway on behaviour change. The A-RGA and SA-RGA were not correlated with any functional measures such as gait speed, 5 times sit-to-stand test or handgrip strength. In addition, we did not assess the responsiveness of the RGA app’s ability to detect change over time. However, incorporating measures from Integrated Care for Older People (ICOPE) Handbook which screens for vision, hearing, cognition, and measures such as 5 times sit-to-stand test may complement SA-RGA done in home setting (21). Finally, the self-administered RGA was completed within few minutes of administered RGA. This could have affected the overall validity but the results were consistent in different settings.

A major criterion for self-administered questionnaire (SAQ) is proper wording, clarity and replace ambiguous questions e.g. “some difficulties” with difficulty scales such as the OMNI-Walk Scale of Perceived Exertion (22). The SAQ must be self-explanatory and incorporate closed-ended versus open ended questions. Another critical aspect is the functionality of design and additional aids (e.g. size of font, spacing, audio computerassisted self-interviewing, navigational aids and use of colour) in facilitating and including people with additional needs, such as those with vision impairment and literacy problems (23). Appropriate graphics with immediate feedback may also help maintain respondent’s attention.

Conclusion

With an ageing population and declining active workforce, the RGA app can be adopted by many countries for early identification of geriatric syndromes before the onset of disability. The RGA self-assessment app performed well in certain domains such as weight loss, falls, number of chronic illness and fatigue, and can be improved further with the inclusion of functional difficulty scale for quantifying functional difficulties, appropriate graphics, and audio computer-assisted self-interviewing. RGA self-assessment app has the potential to be incorporated in eHEALTH platforms worldwide for early identifications of older adults at risk and to reduce health inequalities, at the same time building community resilience in the era of the Covid-19 pandemic.