Validation of the detection of elder abuse through emergency care technicians (DETECT) screening tool: a study protocol.

INTRODUCTION: Elder mistreatment (EM) is a high prevalence threat to the health and well-being of older adults in the USA. Medics are well-positioned to help with identification of older adults at risk for EM, however, field robust screening tools appropriate for efficient, observation-based screening are lacking. Prior work by this team focused on the development and initial pilot testing of an observation-based EM screening tool named detection of elder abuse through emergency care technicians (DETECT), designed to be implemented by medics during the course of an emergency response (911) call. The objective of the present work is to validate and further refine this tool in preparation for clinical dissemination. METHODS AND ANALYSIS: Approximately 59 400 community-dwelling older adults who place 911 calls during the 36-month study observation period will be screened by medics responding to the call using the DETECT tool. Next, a random subsample of 2520 of the 59 400 older adults screened will be selected to participate in a follow-up interview approximately 2 weeks following the completion of the screening. Follow-up interviews will consist of a medic-led semistructured interview designed to assess the older adult's likelihood of abuse exposure, physical/mental health status, cognitive functioning, and to systematically evaluate the quality and condition of their physical and social living environment. The data from 25% (n=648) of these follow-up interviews will be presented to a longitudinal, experts and all data panel for a final determination of EM exposure status, representing the closest proxy to a 'gold standard' measure available. ETHICS AND DISSEMINATION: This study has been reviewed and approved by the Committee for the Protection of Human Subjects at the University of Texas School of Public Health. The results will be disseminated through formal presentations at local, national and international conferences and through publication in peer-reviewed scientific journals. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The size and scope of this study (ie, screening data on more than 50 000 emergency response calls among community-dwelling older adults and follow-up data on 2500 community-dwelling older adults) will represent one of the largest studies of elder mistreatment (EM) screening ever conducted.

The use of a planned missingness study design and rigorously documented longitudinal, experts and all data panel ‘gold standard’ will provide valuable information for the design and conduct of future EM studies.

Practical considerations require us to screen out participants with probable dementia, which will limit the generalisability of our results to older adults who are not living with dementia.

There are many salient aspects of the physical and social environment that act as indicators of EM, and the current study cannot capture them all perfectly.

The current study does not attempt to address or prevent the occurrence of EM—including patient-important outcomes—beyond reporting potential mistreatment to adult protective services.

Background

Elder mistreatment (EM) is commonly defined as an intentional act, or failure to act, by a caregiver or another person in a relationship involving an expectation of trust that causes harm or creates a risk of harm to an older adult.1 2 Population-based studies suggest that more than one in ten cognitively intact, community-dwelling older adults experience EM annually;3 older adults with disabilities face an even greater risk.4 5 EM may take many forms, including physical, emotional/psychological and sexual abuse, neglect and financial exploitation. EM exposure is often chronic, and polyvictimisation (ie, exposure to multiple forms of EM concurrently) is common.1 2 6

The public health impact of EM is considerable. EM is associated with depression,7 functional decline,8 9 emergency room visits,10 hospital admissions11 and all-cause mortality compared with non-maltreated older adults.12–15 The costs associated with lost income, recovery from financial exploitation and the medical, legal and social services interventions needed by maltreated older adults is estimated to be in the billions of dollars annually.2 Nevertheless, studies consistently find that as many as 80%–90% of cases are never reported.3 16 17 Therefore, effective and efficient EM screening tools are urgently needed to improve detection.

Emergency medical technicians and paramedics, collectively referred to as medics, constitute an important and largely untapped EM surveillance force.18 19 Medics’ access to older adults’ residences allows them to observe the older adult’s physical and social environment—access which is shared by few others.18 20 This access facilitates unique opportunities to identify indicators of EM that may otherwise go undetected.20 However, until recently existing EM screening tools were inappropriate for use in emergency medical settings because of their length and/or because of their reliance on direct questioning of the older adult or caregiver.20–31 The detection of elder abuse through emergency care technicians (DETECT) tool was developed in collaboration with medics specifically to address this gap, and to increase systematic surveillance and reporting of potential EM in the community.

Beginning in 2014, our research team partnered with MedStar Mobile Healthcare—the exclusive ambulance service provider to 15 Tarrant County (Texas) cities—and Texas adult protective services (APS) to develop and pilot test the DETECT screening tool.18 20 32 The DETECT tool was designed to (1) be brief, (2) based on the medic’s direct observations of the older adult and his/her physical and social environment, (3) provide reporting guidance and (4) be integrated into existing procedures and medical charting software.18 The pilot test of the 26-item screening tool produced positive results. During the 5-week pilot test, the DETECT screening tool was used 1247 times by 251 medics—resulting in 209 positive screens (16.8%). Immediately following the introduction of the DETECT screening tool, there was an increase of 5.4 (p=0.0056) validated reports of EM per month—a 226% improvement.32 Results from these preliminary studies provide evidence of the feasibility of implementing the DETECT screening tool to enhance the detection of EM with emergency medical services (EMS) providers.

While this preliminary work provides a strong foundation, further research is needed to examine concordance between DETECT screening results and validated EM. In the pilot study, positive DETECT results were compared with APS investigations, but this work did not permit validation of negative DETECT results. It was not possible to calculate any measure of diagnostic performance that required information about true exposure to EM in cases that screened negative (eg, sensitivity and specificity) with the DETECT tool. Additionally, validation of the DETECT tool against a proper ‘gold standard’ EM assessment is warranted.33 Finally, the pilot study was not designed to gather contextual information about EM cases, eclipsing the opportunity to fully understand the social, psychological, health, behavioural and environmental risk factors that contribute to EM, are observable in the older adult’s environment, and may serve as early EM warning signs.

Study aims

The overarching goal of this study is to evaluate the validity and reliability of the DETECT screening tool. Specifically, this study will examine three specific aims:

To validate DETECT for the screening and detection of EM. We will match DETECT screening results with an expert panel determination ‘gold standard’ to calculate the tool’s diagnostic performance.

To develop an abbreviated version of the DETECT screening tool. We will use confirmatory factor analysis to determine the relative predictive value of each DETECT screening item. Results will inform systematic item reduction efforts—streamlining the tool for optimally efficient administration.

To identify potentially modifiable risk and protective factors for EM using follow-up in-person interviews which will provide rich contextual data that highlight modifiable personal and environmental factors.3 34 35 In this aim, we will mine that data for novel relationships and potential targets for future intervention.

Methods

Study design

The DETECT validation study employs a prospective cohort design that includes three distinct data collection activities. These activities, and their relationship to each other, are shown in figure 1 and described in detail below. Briefly, they include:

Figure 1

Data collection activities and timeline. DETECT, detection of elder abuse through emergency care technicians; LEAD, longitudinal, experts and all data.

Initial DETECT screenings

MedStar medics have been using the DETECT screening tool in the context of all emergency responses (ie, 911) for community-dwelling older adults since February 2017 (n=approximately 1650 per month). The medics will continue to use the tool throughout the 36-month study observation period.

Follow-up interviews

Each month, a random subset of the 1650 screenings completed in the previous month (n=approximately 70/month) will be selected for a more in-depth EM assessment. Older adults who consent to participate will receive an in-home interview conducted by a trained community paramedic. The medic will complete a 1-hour assessment that includes a structured clinical interview and survey instruments designed to characterise the older adult’s physical and mental health, current/recent EM exposure and functional status/disability. Additionally, the assessment will include a systematic (with photographs) evaluation of the older adult’s home environment (interior and exterior).

Longitudinal, experts and all data (LEAD) panel case reviews

Each month a randomly selected subset of the 70 follow-up interviews completed in the previous month (n=approximately 18/month) will receive a LEAD panel case review—a method of determining whether EM is occurring when no true ‘gold standard’ exists. Data from the follow-up interview will be synthesised into a report that will be discussed by all LEAD panel members. The determination of these reviews (EM vs no EM) will serve as the ‘gold standard’ measure of true EM occurrence used to calculate the sensitivity and specificity of the DETECT screening tool.

Sample and setting

All data collection activities will be carried out in Tarrant County, Texas (population 218 000 adults aged 65+)36 in partnership with MedStar Mobile Healthcare. All older adults treated by MedStar medics at their place of residence, and who reside in the community (eg, private home, unlicensed adult foster homes, unlicensed board and care homes and so on) during the 36-month study observation period will receive an initial DETECT screening (n=59 400). Other residences (eg, licensed skilled nursing facilities) will be excluded because reports of EM in these settings are generally not investigated by Texas APS.37

A simple random sample of the older adults who received an initial screening and meet inclusion criteria will receive a follow-up interview during the 36-month study observation period (n=2520). We chose to sample 2520 older adults for follow-up based on a balance between a desire to maximise study power and available resources (see below for full details of the power analysis). Eligibility criteria for follow-up interview selection include: (1) screened with the DETECT tool by MedStar medics during the study observation period; (2) reachable by telephone; (3) fluent in English; (4) able to communicate by telephone; (5) cognitively able to consent and (6) provides written informed consent.

As part of a planned missingness design, a simple random sample of completed follow-up interviews will be selected for LEAD panel review (n=648). Based on our previous study, the anticipated average age of the older adults screened will be 77 and will approximate the demographic composition of Tarrant County.32

Procedures

Initial DETECT screening data will be gathered as part of the routine clinical care (emergency response) protocol delivered by MedStar. Screenings are based on the total number of response calls made during the study observation period and the study design allows for the same individual to be screened on multiple occasions; for instance, if an older adult places multiple calls during the study observation period. As EM risk is dynamic, over time, re-administration of the screening is appropriate in the context of repeated or sequential calls placed on different dates (ie, multiple screenings will not be administered within the same 24 hours period) by the same individual.

When a call comes into MedStar’s dispatch centre, ambulances are routed to the scene of the call as usual. After arriving on scene and assessing the situation, medics will begin the process of creating a medical record for each patient in MedStar’s electronic patient care reporting system (ePCR) —ImageTrend Elite. EMS nationwide, and 36 statewide EMS systems, uses the ImageTrend ePCR.38 The DETECT screening tool is built as an ImageTrend Elite module, which is incorporated directly into the ePCR, and could easily be incorporated into the ePCR of every EMS system that uses ImageTrend. The ePCR is programmed to automatically prompt medics to complete the DETECT tool while at a qualified 911 response. DETECT screens are automatically scored within the ePCR, and positive screens will prompt the medic to file an APS report.

Recruitment for follow-up interviews

On the first day of each month, MedStar will generate a list of all older adults screened with the DETECT tool in the previous month. That list will be uploaded to Filemaker Pro, a programme for designing and implementing data collection and data storage applications. Filemaker Pro will randomise the patient list using a built-in pseudo-random number generator. Beginning at the top of the randomised patient list, a trained MedStar employee will attempt to schedule a 2-week post initial screening interview follow-up. Two weeks was selected to give patients time to be discharged from the hospital and/or to give APS time to conduct an investigation, where applicable.

The scheduler will attempt to reach each patient up to five times, occurring on five different calendar days. The scheduler will note the date and time of each unsuccessful contact attempt in the database. If the patient refuses to participate, the scheduler will note the date, time and refusal reason in the database. If the patient agrees to participate, the scheduler will administer a version of the Montreal cognitive assessment that has been adapted for use over the telephone (T-MoCA).39 The T-MoCA demonstrates excellent psychometric properties and is accurate in detecting dementia and significant cognitive impairment.39 Patients who fall below a cut-off score of 17 on the T-MoCA will be considered ineligible to consent to participate. Finally, an in-home follow-up visit from a specially trained community paramedic (CP) will be scheduled for patients with a T-MoCA score of 18 or higher who consent to participate.

Follow-up interviews involve a community paramedic going to the older adult’s home, obtaining written informed consent from the older adult, and administration of a structured clinical interview and validated survey instruments designed to characterise salient demographic characteristics, physical and mental health status, recent and lifelong EM exposure, the older adult’s disability and functional status, and assessing and documenting (ie, with photographs) the quality of the older adult’s home environment (box 1). Participants who complete the entire in-person interview will receive a $25 gift card.

Section/measure/questions

Sociodemographic information

Household size

Marital status

Age

Ethnicity

Race

Educational attainment

Household income

Military service history

General health

Pain intensity and interference44

Medical Outcomes Study45

Geriatric Depression Scale46

Alcohol use/misuse

National Epidemiological Survey on Alcohol and Related Conditions-III (NESARC-III)†47

Alcohol use (AUDIT-C)48

Self-report measures of current and past abuse, neglect and exploitation

National Elder Mistreatment Study3

Self-Neglect Severity Scale49

Military sexual violence history (veterans only)

‘When you were in the military, did you ever receive unwanted, threatening or repeated sexual attention (for example, touching, cornering, pressure for sexual favors or inappropriate verbal remarks, etc.)?’

‘When you were in the military, did you have contact against your will or when you were unable to say no (for example, after being forced or threatened or to avoid other consequences)?’

Self-reported adult protective services investigations history

‘Has anyone from adult protective services (APS) ever attempted to investigate whether or not you were living with elder abuse or neglect?’

‘How many times has this happened in your life?’

‘When was the first time APS attempted to do an investigation?’

‘When was the most recent time APS attempted to do an investigation?’

Observational measures of older adult and environment

Clutter Image Rating Scale50

Elder Assessment Instrument51

†We will use two questions from the NESARC-III as screen-in questions for the AUDIT-C. ‘In your entire life, have you had at least 1 drink of any kind of alcohol, not counting small tastes or sips?’, and ‘During the last 12 months, did you have at least 1 drink of any kind of alcohol?’

DETECT, detection of elder abuse through emergency care technicians.

In addition to assessing capacity prior to obtaining formal consent to participate in this study, older adults will be informed of all applicable local, state and federal laws regarding mandated reporting of suspected or confirmed EM. We will inform older adults of our responsibility to report any suspicions of EM to social services and/or law enforcement agencies. The community paramedic will be instructed to report any suspicion of EM to APS immediately following any follow-up interview where a suspicion arises. Further, if the community paramedic feels as though they, or the older adult, are at risk of serious immediate harm, they will be instructed to communicate that risk to MedStar dispatch via their two-way radio, and contact police if necessary. We anticipate that informing participants about mandatory reporting laws, and our intention to comply with them, will have minimal impact on participation. A recent study that conducted similar interviews in the community with older adults and their caregivers experienced only 1 out of 130 (<1%) potential participants refused to participate based on the mandated reporting requirement.5

All community paramedics will undergo training that equips them to properly administer study measures, and properly obtain written informed consent from participants. Training for the administration of the DETECT tool and the follow-up interview will involve a multi-hour in-person training prior to data collection. A web-based training module on the DETECT tool and the follow-up interview instruments will be available to medics throughout the study, and in-person refresher training will be offered quarterly. New medics that join the team after the onset of the study will undergo the training prior to administering any measures.

LEAD panel case reviews

Using a LEAD panel framework similar to that used by Wiglesworth and colleagues,5 40 approximately 25% of all follow-up interviews will be randomly selected (n=18 per month, 648 total) each month for expert review. The LEAD panel includes a Texas based: (a) board-certified geriatrician, (b) geriatric nurse practitioners, (c) board-certified geriatric psychiatrist, (d) geriatric social worker and (e) special victims’ prosecutor. This is consistent with the composition of LEAD panels used in the EM literature. 1 5 40

Prior to the first case review LEAD panel session, the principal investigator (PI) will convene a meeting of the LEAD panel members to establish a systematic process for considering the data presented in each case review, operational definitions of each type of EM and a priori thresholds for making a determination of any type of EM. For example, kicking an older adult once may be considered EM, but perhaps, given no other evidence of psychological abuse, insulting and swearing at an older adult must occur 6–10 times over a year to be considered EM.5 The established process, definitions and criteria will be included in a manual that all LEAD panel members will review each month.

On the first day of each month, we will create a summary report of all of the information gathered by the CP during the follow-up interview for each of the 18 randomly selected cases. We will securely transmit that report to all LEAD panel members who will then review each case prior to the monthly meeting. This summary report will not contain the results of the initial DETECT screening. Qualtrics survey software will be used to gather an initial independent determination of EM for each case reviewed from each LEAD panel member. For each case, the panel member will select ‘yes’ or ‘no’ for each of the following EM categories: physical abuse, emotional/psychological abuse, sexual abuse, financial exploitation and neglect.

Finally, the LEAD panel will meet in-person for approximately 3 hours on 1 day per month to discuss each case and make a final consensus-based EM determination after considering all the evidence presented (ie, follow-up interview responses and medical information collected by MedStar Mobile Healthcare). Aggregate deidentified results from this initial survey will be presented with each case at the monthly LEAD panel meeting. After up to 10 min of discussion, each LEAD panel member will vote for a second time in Qualtrics. If voting is not unanimous for each form of EM, then the determination will be made by a simple majority. Other LEAD panels used to determine EM found that more than 96% of cases reviewed resulted in a unanimous decision. In the five cases that were not unanimous, only one LEAD panel member disagreed with the majority.5 Any case that the LEAD panel determines to be positive for any single form of EM will be considered a positive instance of EM. Conversely, any case receiving a negative vote for all forms of EM will be considered a negative instance of EM. The LEAD assessments will be treated as the gold standard in subsequent analyses.

Planned analyses

Aim 1: validation of DETECT for the screening and detection of EM

The sensitivity and specificity of the DETECT tool will be estimated relative to the LEAD panel standard. Ideally, the LEAD panel review would be performed on all subjects. However, the LEAD panel reviews are time intensive and therefore not feasible for all participants. Therefore, we will adopt a planned missingness strategy for our Aim 1 analysis. Specifically, we will randomly sample 25% of follow-up investigations to receive LEAD assessment. The results of this sample will be used to estimate the sensitivity and specificity of the DETECT tool with high statistical efficiency and without bias by treating the LEAD sample as a validation study and employing modern missing data techniques. We will use multiple imputation for measurement error correction (MIME) to impute the expected LEAD panel review result from the same measures the LEAD panel will use to make their determinations.41 This imputed gold standard measure will then be compared against DETECT to estimate the sensitivity and specificity of the screening tool. All imputations will be done using a fully conditional specification in PROC MI in SAS V.9.4. This approach has been successfully used with validation subsamples in chronic disease studies including studies of older adults entering hospice care.42 Sensitivity and specificity will be calculated directly from the collected data using PROC FREQ in SAS V.9.4. Exact CIs will be estimated based on a binomial test using PROC FREQ. Multiple imputations will be combined using PROC MIANALYZE in SAS V.9.4. The use of a validation sample will allow for efficient estimates of the sensitivity and specificity of DETECT, while still allowing a large sample size for etiological analyses not using the LEAD outcome.

To characterise the expected precision of our validation analysis accounting for the sampling error introduced by the MIME procedure, we performed Monte Carlo simulations estimating the marginal error of our sensitivity estimate—varying the baseline EM prevalence and sensitivity. Prevalence was varied between 11% (estimated population baseline rate) and 16% (the estimated rate in the DETECT pilot studies). Sensitivity was varied from 0.7 to 0.9. Type 1 error was fixed at 0.05. Based on expected monthly screenings from the pilot study and budgetary constraints, sample size was fixed at 2500 follow-up interviews. The marginal error of our estimated sensitivity decreased with increased sensitivity and increased prevalence (figure 2).

Figure 2

Marginal error around estimates of sensitivity at varying levels of sensitivity and baseline EM prevalence. EM, elder mistreatment.

Aim 2: DETECT item reduction

DETECT is a priori hypothesised to assess a single underlying latent construct, EM. As such, confirmatory factor analysis (CFA) will be used to reduce the number of DETECT screening tool items. The CFA model will estimate factor loadings of all DETECT screening items on a single latent construct while allowing for covariance between the items. Any items with negative factor loadings will be trimmed from the model. Further items will be considered for removal based on their factor loadings (loading <0.3), positive covariance with other items and overall model fit. Appropriate model fit will be assessed using the comparative fit index (>0.9) and the root mean square error of approximation (<0.08). Internal consistency will be estimated for the abbreviated tool using Cronbach's alpha.

To verify that item reduction has not negatively impacted the validity of the DETECT tool, we will reassess the validity of the shortened tool using the methods described in Aim 1 analysis. If substantive reductions in sensitivity or specificity are seen in comparison to the full tool, we will iteratively replace removed items until sensitivity and specificity are restored.

Aim 3: explore potentially modifiable risk and protective factors

The association of modifiable risk and protective factors with EM will be estimated using logistic regression with PROC GENMOD in SAS V.9.4. Information from our follow-up measures and the LEAD panel’s determination will be pooled using the multiple imputation techniques previously discussed.

Patient and public involvement

Medics at MedStar and Texas APS caseworkers previously identified barriers to the detection and reporting of EM experienced by medics providing EMS in the field.20 MedStar medics were also involved in creating and pilot testing the DETECT tool.18 32 In the current study, our data collection software includes a link that community paramedics can use to provide the PI with feedback at any time. There is also a plan to elicit feedback from medics during all training sessions, and the PI will elicit feedback from older adults in the community during ‘ride-alongs’.

Discussion

In our experience, hospital environments are highly complex, and changing screening practices can be a slow process with many barriers. By comparison, EMS organisations are nimble, adaptive and eager to find new ways to contribute to the public’s health. The primary purpose of the current study protocol is to test the validity and reliability of the DETECT screening tool using a gold standard LEAD panel. This will be the first study conducted, to our knowledge, to validate and psychometrically test an EM screening tool that uses systematic observation of the older adult and their environment.

Best practices for LEAD panel case review methodology—including their composition—are incompletely understood. We modelled our LEAD panel after the LEAD panels described by Wiglesworth and colleagues,5 40 which were the best-documented LEAD panels at the time this protocol was written. Although we did not specifically design the current study to test hypotheses related to LEAD panel best practices, we are currently planning exploratory studies with other research groups who are also using LEAD panel methodology that will investigate the impact of panel composition on case adjudication. Indeed, documenting the LEAD panel composition and procedures we are using in this manuscript, along with future studies documenting the potential impacts of the composition and procedures, may prove to be of great value to the field.

Given that there are more than 800 000 medics providing services in every county nationwide,43 a valid and reliable screening tool that is easy for EMS providers to use could dramatically increase sentinel surveillance of EM in a very short time. Therefore, successful completion of this project has the potential to make a significant, immediate public health impact.

Ethics and dissemination

Ethical and safety considerations for this work include consideration of matters of capacity (ie, to consent), personal safety (ie, safe to participate in an interview about maltreatment), the reality that disclosure of maltreatment may be unpleasant and uncomfortable for older adults even in circumstances where it is not physically unsafe, and participants may experience embarrassment about EM, the condition of their health, home environment and so on. However, the study protocol ensures that all participants are well informed about the scope of the study and the topics included in the interview prior to obtaining consent. Participants are informed that their participation is voluntary, they may choose to skip any question they like and may withdraw from the study at any time without consequence. In addition, older adults will be informed of all applicable local, state and federal laws regarding mandated reporting of suspected or confirmed EM prior to obtaining consent. We will inform older adults, and other informants/guardians where applicable, of our responsibility to report suspected EM to the appropriate social services or law enforcement agencies.

Ethical and safety considerations regarding breach of privacy and the social, economic and safety consequences such a breech may introduce also warrant comment. As with any research, the risk of breach of confidentiality, particularly the inadvertent transmission of health information, personal identifiers, contextual factors associated with the quality of environment in one’s home, the status of one’s physical and/or mental health and the health of one’s relationship with a primary caregiver, is a serious ethical consideration. Our study protocol ensures that data are collected, stored, analysed and ultimately discarded in a manner consistent with the highest ethical standards.

It is our intention to disseminate study findings to the scientific community through formal presentations at local, national and international conferences and through publication in peer-reviewed scientific journals. Given the large number of Tarrant County older adults who will participate in this study, we will also work with local agencies that serve this population, local churches and community centres to hold town hall meetings where our findings are discussed. Finally, if successful, we intend to develop and implement continuing medical education and professional credentialing education about the use of the DETECT tool among medics.