Development and Testing of the interRAI Acute Care: A Standardized Assessment Administered by Nurses for Patients Admitted to Acute Care.

BACKGROUND: Underpinning standards for developing comprehensive care in hospital is the need to identify, early in the admission process, functional and psychosocial issues which affect patient outcomes. Despite the value of comprehensive assessment of patients on admission, the process is often sub-optimal due to a lack of standardized assessment practices. This project aimed to develop a concise, integrated assessment for patients admitted to acute care and test its psychometric properties.
METHODS: Two international expert panels of clinicians and health scientists collaborated to establish design parameters. Using clinical observations and a variety of derivative applications sourced from the interRAI research collaborative repository, the panels constructed a draft instrument to examine feasibility, resource requirements, and inter-rater reliability. Field testing was conducted in Australia and Canada. Next, the system was revised to its final form, the interRAI Acute Care, after feedback and review from international interRAI members.
RESULTS: Constructed using 56 items, the interRAI Acute Care required a median of 15 minutes to complete. Inter-rater reliability tested on 130 paired assessments was substantial to almost perfect for 78% of the clinical items and moderate for the remaining 22% of items. A subset of 30 items from the admission assessment comprised the discharge assessment. DISCUSSION: The interRAI Acute Care has been shown to be an efficient nursing assessment instrument with good psychometric properties. Implementation in a digital environment will enable documentation and care planning to comply with standards for quality of care in the general adult hospital population.

Introduction

Timely, expedient, and high-quality assessment of patients is critical to the optimal planning and management of patients during their stay in the acute care hospital setting.[1] In addition to the presenting illness or injury, evidence suggests that, particularly for vulnerable patients, functional and psychosocial problems such as inability to self-care, mobility, and cognitive impairments are a common cause of hospital complications.[2,3] In many cases, these problems pre-date the acute illness or are aggravated by it[4,5]; for some, these issues are new, developing during the course of the hospital stay and further complicate recovery.

Functional and psychosocial impairments often result from an accumulation of deficiencies in multiple domains.[6] If undetected or left untreated, many of these problems can lead to undesirable outcomes including delirium, falls, pressure injury, functional decline, low morale and depression, institutionalization, prolonged hospital stay, and death.[3] Not only does this severely affect the quality of life of the patient, the costs associated with this also become a significant economic burden on a health care system.[7] Evidence indicates that early detection of ‘at-risk’ patients on the acute care unit decreases adverse events.[8] In addition, conducting a comprehensive assessment of functional and psychosocial problems on admission and at intervals during the hospital stay has been shown to improve patient outcomes.[9] Although the high prevalence of such problems in older people in acute care is well recognized,[4,5] they can occur in all age groups,[10] hence the need for an inclusive assessment system that applies to all inpatients.

Background

The nursing assessment process provides an ideal opportunity to assess and record patient needs, problems, and risks.[1,11] As a core component of nursing practice, assessment is designed to guide clinical decisions in the delivery of safe patient care.[12] At admission to an acute care hospital, all patients undergo some form of routine nursing assessment. This generally includes the collection of important administrative data, physiological measures, and functional and psychological evaluation and risk appraisal to inform the nursing care plan.

The effective, timely collection and documentation of this information is essential for the development of a targeted and useful care plan. This plan assists important communication between the patient, family, and all members of the health care team, as well as providing an opportunity for the evaluation of the nursing care being offered.[13] It is this documentation that sets the standard for patient care; therefore, the use of a comprehensive and validated assessment instrument with established quality guidelines to support this task is recommended to promote best practice.[14,15] High-quality clinical information also has benefits beyond immediate clinical care. It can inform service planning, workload assessment, and appraisal of quality. To fully secure these benefits, information should be dynamic across an episode of care including at the point of discharge.

Despite the value of a comprehensive assessment of the patient on admission to an acute care setting, completion of documentation of patient assessments is often sub-optimal.[16] Nursing time constraints, a lack of standardized assessment practices, and an abundance of unstructured and incoherent assessment and documentation forms have been suggested as reasons for poor compliance or delay in completing this essential nursing task.[13,16-18] Over time, such practices result in important patient issues being missed or attended to in a fragmented manner, incomplete care, poor health outcomes, and preventable and reversible patient challenges.[19,20] Duplication of collected information across forms is common.[16] In addition, routine documentation appears to have increased significantly over the years, without the review and removal of pre-existing forms that may no longer have relevance[21] and also contributing to nurse perception of high workload burden allowing less time to care for patients.[16] The phenomena of missed nursing care appear to exist worldwide and are clearly limiting efficient use of resources and quality care planning.[22-24]

Standardized and integrated data gathering approaches, processes, and documentation have enormous potential to improve the effectiveness and efficiency of any type of assessment, to reduce administration burden and the risk of adverse events, and to enable quality interdisciplinary care and discharge planning.[2] In addition, many health care systems are operating in, or planning to migrate to, a digital documentation structure. This movement in itself ought to reduce the documentation burden and consequent potential for missed care but requires a structured assessment protocol. This generally implies that all clinical observations and assessments must have robust psychometric properties, can be scored unambiguously, and that information can be entered into a well-organized and protected computer system that is accessible to all involved in the patient’s care. Ideally, the time taken to complete this process should be broadly commensurate with, or be less, than the current time allocation for the nursing assessment procedures.

The interRAI research collaborative

As a not-for-profit international research organization, interRAI (www.interrai.org) is a collaborative of researchers and practitioners in more than 35 countries seeking to improve the quality of care of vulnerable persons within and across health settings, through the assembly of accurate clinical information in a standardized format. The principles behind the development of instruments in the interRAI suite are that each instrument is designed for a particular population or health care setting, but with sharing of common measures across settings to form an integrated health information system.[25] In response to the above issues and the need to address standards to support clinicians (particularly nurses) to deliver comprehensive care,[14,15] members of the Acute Care Network within the interRAI research collaborative set out to develop a concise, robust electronic nurse-administered system to support assessment of adult patients within the acute care setting.

This study reports on the development and psychometric testing of a standardized assessment administered by nurses for patients admitted to acute care – the interRAI Acute Care.

Design

The process to develop the assessment used a modified nominal group technique directed at generation of ideas and setting priories to achieve consensus of opinion in expert panels.[26]

Method

Establishment of expert panels

Two expert panels of clinicians and health scientists were established. The development of the concept and design parameters was undertaken by a working group comprising representatives of the Acute Care Network within the interRAI research collaborative at the Centre for Health Services Research (CHSR) at the University of Queensland in Brisbane, Australia, in partnership with senior staff of the Canterbury District Health Board (CDHB) in Christchurch, New Zealand. In parallel, an international working sub-group of the interRAI Acute Care Network provided technical advice and reviewed the system as it evolved.

The 2 groups met in person on 10 occasions, with monthly teleconferences throughout the development period from 2013 to 2016. Through consensus, the groups set broad design parameters for the assessment, including the target population, desired content, system outputs, resource allocation, and potential integration with other interRAI hospital assessments (emergency department, acute care comprehensive geriatric assessment, and post-acute care and rehabilitation).

Key design parameters

Together, the 2 working groups identified the key clinical domains that are related to functional and psychosocial problems. Medical diagnoses, medications, physiological measures, and administrative information were not included, as these are expected to be collected in other components of an admission assessment. An important design requirement was that embedded applications, such as screeners for delirium, or risk assessments for falls and pressure injury should be at least as valid as current ‘stand-alone’ tools. The following were identified as the key design drivers for the assessment system:

Determine the immediate patient issues that require a nursing care plan response (eg, the patient is incontinent);

Indicate risk of potential future adverse events or outcomes where nursing care has an important preventive function (eg, the patient is at high risk of pressure injury);

Reduce time taken to complete the admission assessment by having sufficiently few observations to enable completion within 20 minutes for most patients;

Identify clinical problems that require further assessment by the nurse or through referral to specialist clinicians or services;

Suggest the need to engage other care providers in the care delivery process as part of discharge planning if it is likely that the patient will require continuing care (eg, the person is at risk of requiring long term care at discharge);

Provide data to enable construction of a discharge profile for presentation to providers who offer continuing care after discharge and to enable assessment of outcomes of care;

Be suitable for application to all adult inpatients aged 18 years and above, including those admitted to general and specialist medical and surgical units and both elective and emergency admissions;

Enable assessment to be completed in a computerized environment with software to support applications for screening, scaling, and quality measures.

Selection of items

Clinical items considered relevant to the specification were selected from the interRAI inventory, comprising highly reliable clinical observation items and questions developed over 20 years across multiple clinical settings. These items have been subjected to extensive multi-national field-testing of their psychometric properties.[27-30] Where a suitable item was not available, an existing item was modified or a new one was created. Some items were selected as they contributed to risk screeners (eg, falls risk), scales (eg, pain scale), or quality indicators that had previously been developed by interRAI for application in the acute hospital setting.[31] As a result of this process, a 60-item nursing admission assessment, suitable for use with all adult patients in acute care, was created.

Instrument testing

Psychometric properties of the instrument which were tested included feasibility, resource requirements (time taken), and inter-rater reliability.

Feasibility was tested in a small pilot study involving 8 nurses from 3 Australian hospitals. Each nurse performed assessments on 2 patients and then completed a semi-structured questionnaire (with Likert responses) for feedback on item content, degree of difficulty completing the assessment, adequacy of the summary output reports for planning patient care and comparison with usual assessment.

For inter-rater reliability, trained nurse assessors who were not directly involved in the care of patients were recruited at each of 4 hospitals (3 in Australia and 1 in Canada). Patients aged 70 and older (expected to be the most complex patients) were identified from admission lists. Those who gave informed consent to participate were assessed by 2 of the trained nurses within a 2-hour time frame and within 12 hours of admission to the ward. The function as either first or second assessor was determined at random each time a new patient was included. Assessors were blinded to the other’s results and not permitted to discuss the case with each other nor to exchange information.

It was planned to collect a minimum of 25 paired assessments at each of the 4 hospitals (total 100 paired assessments). Assessments were completed using the draft interRAI Acute Care, according to standard interRAI convention, based on semi-structured clinical interview, clinical observation, and chart review. If present during the assessment, informal caregivers were interviewed to obtain collateral information. Time to complete the assessment was automatically recorded by the data entry software.

Individual items were compared between the 2 assessors using observed agreement in tandem with kappa coefficients. Unweighted Cohen kappa was used for nominal items, with weighted Cohen kappa used for ordinal items. The strength of agreement for the kappa coefficient is considered as poor for kappa values below 0.40, moderate from 0.41 to 0.60, substantial from 0.61 to 0.80, and above 0.81 almost perfect.[32]

As a last step, the interRAI Acute Care assessment tool was reviewed by the Instrument and Systems Development Committee of the interRAI research collaborative. This committee, which comprises multi-disciplinary clinicians and scientists, reviews systems to ensure clinical appropriateness, high scientific integrity, and consistency with interRAI standards.

Ethical considerations

Ethics approval was obtained for the studies from the participating institutional review committees (The University of Queensland Institutional Human Research Ethics 2015000995; Metro South Human Research Ethics Committee HREC/15/QPAH/313 for Queensland hospital sites; Northern Health HREC/15/NH/27 for the Victorian hospital site; Conestoga College Research Ethics Board for the Canadian hospital site). Informed written consent was given by all participants.

Results

Face validity, feasibility, and acceptability

Qualitative feedback from the semi-structured questionnaire indicated that the information collected using the interRAI Acute Care was sufficient to plan patient care and that the summary report of the patient profile was easy to understand. Most of the nurses (7 of 8 nurses) reported a low level of difficulty completing the assessment and rated it an improvement on their usual practice.

Inter-rater reliability

In total, 130 paired assessments were completed. Patients had a mean (SD) age of 78.2 (7.6) years and 70 (53.8%) were women. Of the 41 clinical items evaluated, 16 were dichotomous (usually yes/no) and the remainder had ordinal responses. In all, 6 items (14.6%) had almost perfect agreement (kappa > 0.8), 26 (63.4%) had substantial agreement (kappa 0.61-0.80), and 9 (22.0%) had moderate agreement (kappa 0.41-0.60). Supplementary Table S1 lists the clinical items, levels of agreement, and kappa values with 95% confidence intervals.

Resource requirements

The 260 assessments completed as part of the inter-rater reliability assessments showed that the median time for completion of the admission assessment, including data entry, was 15 minutes (inter-quartile range 11-20 minutes).

interRAI Acute Care

Following item testing, the expert panels refined the instrument, resulting in an interRAI Acute Care admission assessment of 56 items (4 redundant items were discarded) and a discharge assessment of 30 items. The admission clinical observations were assessed across a wide array of domains (Table 1), and a further 9 items were triggered for completion in certain cases. For example, if a patient reports pain, then pain intensity and frequency are recorded. Pre-morbid functional status is only scored if the person is not independent in hospital. The pre-morbid assessment period is defined as the 3 days prior to the onset of the acute illness that resulted in the admission. A subset of 30 items from the admission assessment comprises the discharge assessment completed on the day of discharge. This permits construction of a patient profile to support transition to ongoing care (eg, in a community programme, step-down programme or, long-term care) and record outcomes, enabling quality indicators to be scored.

Table 1.

Clinical domains in the interRAI Acute Care.

Cognition

Communication, hearing, and vision

Mood

Behaviour

Functional status, activities of daily living

Continence

Health conditions

Health behaviours

Nutrition

Skin conditions

Applications derived from the interRAI Acute Care include a variety of diagnostic and risk screeners, scales to measure and monitor severity, and quality indicators (Table 2). To establish baseline functional performance, the assessment is ‘locked’ at 24 hours after arrival. However, there is provision to update the assessment with changes in condition progressively, or at intervals, across the hospital stay.

Table 2.

Applications derived from the interRAI Acute Care.

Applications	Description
Scales (severity)
Cognitive performance scale	This scale describes the cognitive status of a person. Validated against the Mini-Mental State Examination (MMSE)[33-36]
Activities of daily living (ADL) hierarchy scale	This scale reflects the disablement process by grouping ADL performance levels into discrete stages of loss (early loss: personal hygiene; middle loss: toileting and locomotion; late loss: eating)[37]
Activities of daily living (ADL) short form	This scale provides a measure of the person’s ability to perform basic ADLs[37-39]
Communication scale	This scale provides a summary of communication measures (making self-understood and ability to understand others)[40]
Pressure ulcer risk scale	This scale identifies persons at various levels of risk for developing a pressure injury, validated against the Braden Scale[41,42]
Pain scale	This scale summarizes the presence and intensity of pain and validates well against the visual analogue scale[43]
Body mass index (BMI)	Calculated as weight (kg)/height (m)2 the BMI is used as a measure of nutritional status[44-46]
Diagnostic screening
Delirium	The delirium screener assists in identifying the presence of delirium at the time of assessment[47]
Dementia	The dementia screener assists in identifying the presence of cognitive impairment/dementia at the time of assessment[35]
Depression	The depression screener assists in identifying the presence of depression at the time of assessment[48]
Undernutrition	The undernutrition screener assists in identifying the presence of undernutrition at the time of assessment[44,49]
Risk screening
Delirium	The delirium risk screener seeks to identify those at risk of developing delirium[50]
Falls	The falls risk screener seeks to identify those at risk of falling[51]
Pressure injury	The pressure injury risk screener seeks to identify those at risk of developing a pressure injury[42]
Frailty Index	A Frailty Index can be derived based on the accumulation of deficits across domains[52]
Quality indicators[a]	Description
Mobility	The proportion of patients discharged with worse levels of locomotion/walking compared with pre-morbid levels
Falls	The proportion of patients who fall (at least once) during the hospital episode
Pain	The proportion of patients with no pre-morbid pain who reported both pain at admission and unimproved pain at discharge
Bladder catheter	The proportion of female patients with a new urinary catheter at admission
Self-care	The proportion of patients with pre-hospital decline who failed to return to pre-admission function (or better) by discharge
Skin integrity/pressure injury	The proportion of patients with a new or worsening pressure injury at discharge compared with admission
Cognitive health	The proportion of patients with delirium indicating behaviours at discharge
Institutional placement	The proportion of community-dwelling patients discharged to long-term care
Prolonged stay	The proportion of patients with prolonged length of stay (greater than the 90th percentile)

Only validated for use in people aged more than 70 years.

The assessment is designed to be completed in a computerized environment with software support to generate outputs in the forms of problem lists, diagnostic and risk screeners, scales, and quality indicators.

Discussion

The interRAI Acute Care was developed with a process of extensive consultation and psychometric testing. The instrument achieved all the desired objectives delineated in the ‘key design parameters’ listed above and is capable of integration into a digital environment.

It supports a set of applications that matches, and mostly exceeds, that found in most of the nursing assessment systems that have been examined, and further, it achieves this with vastly fewer clinical observations.[16] For example, an analysis of 52 assessment forms collected from hospitals in Victoria, Australia, showed that 150 to 586 data items (median 345) were collected per patient.[16] Using standardized items for multiple applications and limiting the scope of assessment to that which is able to be completed by nurses in a busy acute care setting, achieves the aims of ease and speed of completion with high acceptability by staff.

Implementation recommendations

Designed to be used when the patient is admitted to an inpatient unit in an acute care setting, the assessment ensures that critical clinical care is promptly administered. Some information about function and psychosocial problems may not be readily available on admission. In this case, a ’best’ estimate of scoring is made, but with provision for adjustment as new information comes to hand. As clinical observations are updated, all applications may be rescored and the care plan adjusted. The applications generate outputs in the form of problem lists, diagnostic and risk screeners, scales, and quality indicators. To best use these outputs, software developers need to work with clinicians and administrators to configure reports that match workflows and administrative requirements. This may include automated or semi-automated triggers for a care plan (eg, high-pressure injury risk generates a relevant care planning action such as the use of pressure redistribution devices) and referrals (eg, a new mobility problem generates a referral to a physiotherapist).

Although the assessment is intended to be performed by nurses, supporting nursing practice and informing patient care planning, it is anticipated that the data will be of considerable value to other members of the clinical team. The assessment contains information that should alert other staff or members of the interdisciplinary team such as doctors, physiotherapists, and dieticians to those patients who are likely to experience problems and the plan of care to address those risks. The information has the potential to inform resource allocation. If patient information is recorded in a robust manner on a large scale, it may complement traditional systems based on medical diagnoses and procedures to enhance case mix and work assignment systems. Similarly, if discharge assessments are conducted appropriately, it will likely inform appraisal of service quality (eg, through quality indicators).

Related clinical assessments developed by interRAI

The interRAI Acute Care forms part of the interRAI Hospital Systems designed to support care across the hospital continuum, allowing for seamless transition from admission, through the hospital stay to discharge. The Hospital Systems include assessments in the emergency department (the interRAI Emergency Department Contact Assessment), for comprehensive geriatric assessment in acute care (the interRAI Acute Care for Comprehensive Geriatric Assessment) and for rehabilitation or other form of post-acute care (the interRAI Post-Acute Care and Rehabilitation). It is also compatible with other interRAI assessment systems including community and palliative care, and long-term residential care, sharing many clinical observations, screeners, and scales.

Limitations

Although implementation trials of the interRAI Acute Care are planned in Australia and New Zealand, this system is yet to be applied in usual day-to-day clinical practice. This will require careful appraisal of, and alignment with, care delivery systems and with well-designed software that facilitates the execution of these processes.

Development of the instrument remains incomplete. Our group is presently exploring the possibilities of developing further scales and measures to enhance interpretation of the assessment. Examples of work in progress are screeners to support targeting of patients suitable for rehabilitation or post-acute care or to identify those patients who are at risk of requiring long term residential care at discharge. The quality indicators were developed specifically for older patients[31] and have yet to be tested in cohorts of younger patients. Their use is thus recommended only for patients aged more than 70 years at this time.

Although the development of this system engaged clinicians from almost 20 nations, until international field testing is conducted on representative samples of acute care patients, particularly in cohorts younger than 70 years, we cannot fully guarantee applicability. In nations or hospitals with current minimal (and in our view inadequate) assessment protocols, this system will represent an increased workload.

Conclusions

The interRAI Acute Care has been designed as a comprehensive and efficient system to assess functional and psychosocial needs of adult inpatients in acute care, thus addressing standards for quality inpatient care. This approach meets a set of challenging design criteria. The clinical observations and derivative applications have excellent psychometric properties. It is compatible with other interRAI systems designed for use across the hospital continuum of care and into the community. To our knowledge, there are no similar published systems designed for systematic planning and documentation of care in the general adult hospital population.

Click here for additional data file.

Supplemental material, Supplemental_PDF for Development and Testing of the interRAI Acute Care: A Standardized Assessment Administered by Nurses for Patients Admitted to Acute Care by Leonard C Gray, Elizabeth Beattie, Veronique M Boscart, Amanda Henderson, Yvonne C Hornby-Turner, Ruth E Hubbard, Susan Wood and Nancye M Peel in Health Services Insights