RECAPDOC - a questionnaire for the documentation of rehabilitation care utilization in individuals with disorders of consciousness in long-term care in Germany: development and pretesting.

BACKGROUND: A multitude of different rehabilitation interventions and other specific health care services are offered for individuals with disorders of consciousness in long-term care settings. To investigate the association of those services and patient-relevant outcomes, a specific instrument to document the utilization of those services is needed. The purpose of this study was to develop such a questionnaire administered to caregivers in epidemiological studies or patient registries in Germany.
METHODS: The development process of the RECAPDOC questionnaire was carried out in three steps. Step 1 consisted of a systematic literature review and an online-based expert survey to define the general content. Step 2 was an expert interview to evaluate the preliminary content of the questionnaire. Step 3 was a pretest including cognitive interviews with caregivers. After each step, the results were combined into a new version of the questionnaire.
RESULTS: The first version of the questionnaire included items on utilization of medical care, medical aids, nursing and therapeutic care. The results of the expert interview led to the integration of five new items and the modification of six other items. The pretest led to some minor modifications of the questionnaire since it was rated as feasible and acceptable. The final questionnaire consisted of 29 items covering the domains "living situation", "social insurance status", "utilisation of home health care", "domestic services", "outpatient health care", "specific diagnostic measures", "adaptive technologies", "medical aids" and "utilization of therapies". Also the experience of family support and multidisciplinary collaboration of health professionals is covered.
CONCLUSIONS: The developed questionnaire is a first step to make the situation of patients with disorders of consciousness in the long-term care setting accessible for evaluation in epidemiological studies and in the context of patient registries. However, further reliability and validity studies are needed.

Background

More and more individuals survive severe brain injuries due to optimized treatment in the acute and the post-acute situation. Consequently, the number of those with chronic disorders of consciousness (DOC) increases [1]. DOC is characterized as coma (complete unawareness, eyes-closed state), vegetative state (VS; complete unresponsive, eyes-open state), or unresponsive wakefulness syndrome (UWS, as a more neutral and descriptive term) and minimally conscious state (MCS; limited conscious interaction with the environment) [2, 3].

Despite the presumably increasing number of individuals with DOC, current knowledge on diagnosis, optimal management and treatment is scarce. Furthermore, reliable data on prevalence and incidence is limited and shows great variation [4, 5]. For examestimation of VS ranges from 2.8 to 3.4 and for MCS from 1.5 to 2.8 per 100,000 [6, 7].

A major challenge in management of individuals with DOC is the prediction of long-term outcomes. For the last two decades, VS was considered as permanent when lasting longer than 3 month in individuals with non-traumatic brain injuries or 12 months in those with traumatic brain injuries [8, 9]. However, recent studies with long-term follow-up showed that recovery of consciousness and functional progress is possible, even 2 to 5 years post injury [10-15]. A systematic review from 2015 revealed that modern diagnostic methods are still not satisfactory regarding reliability of diagnoses and prognosis [16] indicating that decisions regarding treatment and rehabilitation withdrawal are far from evidence-based. Also, findings from a German prospective multicentre neurologic rehabilitation registry support these findings and indicate that clinical improvement may take place even several months after brain injury [17-19], despite poor initial prognosis. Considering this, it seems advisable, that a proper level of rehabilitation care has to be maintained even after discharge from initial inpatient rehabilitation and the effectiveness of offered rehabilitation and other health care interventions has to be evaluated carefully. A systematic review of rehabilitation interventions in the long term care setting has shown that implemented interventions are very diverse, their quality of evidence is low and high quality research is needed [20]. Epidemiological research and research in patient registries needs instruments to collect data on potential determinants of favourable patient-relevant outcomes. However, instruments that are able to reflect the specific pattern of health care utilization in individuals with DOC and that can be used alongside more general instruments on health care utilization are missing so far. Therefore, the objective of this study was to develop a specific questionnaire to document provision of health care services in individuals with DOC living in an inpatient or outpatient long-term care setting for the use in epidemiological studies or patient registries.

Methods

The development process of the RECAPDOC questionnaire was carried out in three subsequent steps and aimed to take into account all relevant perspectives, i.e. those of researchers in the field, health professionals, physicians, and lay caregivers (see Fig. 1). First, a systematic review of the literature, a review of existing guidelines and a review of existing questionnaires for health resource utilization as well as online Delphi-survey with clinical experts were carried out to determine the content of the questionnaire (Step 1). These results were summarized in a first version of the questionnaire. Second, we carried out an expert interview with physicians specialized in the treatment and management of individuals with DOC to define specific content relevant from the physician’s perspective (Step2). These results were summarized in a second version of the questionnaire. Finally, a pretest with cognitive interviews was carried out to evaluate feasibility and comprehensibility of the questionnaire (Step 3). This step led to a final version of the questionnaire.

Fig. 1

Development process of the questionnaire to investigate rehabilitation care of individuals with disorders of consciousness (DOC) in long-term care

Researchers’ perspective: literature review

Systematic review

The full description of the methods of the systematic review can be found elsewhere [20]. In brief, the objective of the systematic review was to identify rehabilitation interventions for individuals with DOC in long term care.

Existing guidelines

To design an instrument that is in line with current health care practice, a comprehensive online and hand search was carried out to identify relevant practice guidelines or recommendations from professional organisations.

Existing questionnaires

To build our questionnaire upon the experience of related work and make is useable together with instruments that are more generic, existing questionnaires for health resource utilization in Germany were reviewed.

Health professionals’ perspective: delphi-survey

The objective of the expert Delphi-survey was to substantiate the content of the questionnaire in terms of describing the current practice of rehabilitation care of health professionals in the long-term care setting in individuals with DOC in Germany. Experts were defined as health care professionals (nurses, physical therapists, occupational therapists or speech and language therapists) with at least 5 years of professional experience in rehabilitation of individuals with DOC, advanced specialized education in the field of neurological rehabilitation and experience in the long-term care setting.

Experts were recruited via snowball sampling. We contacted national long-term care facilities, authors of relevant publications, scholars from nursing and rehabilitation sciences, and national professional organisations and asked to nominate experts according the definition (see Fig. 2).

Fig. 2

Flow of participants of expert Delphi-survey

All recruited experts were asked to participate in two rounds of an internet-based Delphi survey [21].

In round one (July 2013), the participants were approached with questions on the use of special treatment concepts, adaptive technologies and medical aids, measures to support the family, measures to support social participation and measures to support interdisciplinary collaborations using open-ended questions. In round two (September 2013), each answer that was named more than once in round one was presented to the same group of experts in closed-ended questions and the experts were asked to rate the relevance of each item. As an example, experts were asked to rate the relevance of a certain therapeutic approach for patients with DOC in the targeted setting on a four-point Likert scale (“very important”, “important”, “slightly important”, “not important at all”). Every category that was rated as “very important” or “important” by a majority of the experts (> 50%) was included in the questionnaire. To describe the participating experts, each round contained questions about age, sex and profession and inclusion criteria (see Additional file 1).

Physicians’ perspective: expert interview

According to the guidelines of the German Federal Rehabilitation Council and the funding agencies of the German Federal Health Insurance [22] rehabilitation in long-term care setting has to be coordinated by a specialized physician. To define content relevant from the physician’s perspective, the questionnaire was evaluated via semi-structured telephone interviews with physicians specialized in neurology and expertise in rehabilitation in the long-term care context.

The physicians were recruited from cooperating clinics in different federals states of Germany and following personal suggestions. Sample size was determined by saturation, i.e. the point where new data collection is unlikely to provide new insights [23]. The current version of the questionnaire was sent to the participants 5 days before the interview. During the telephone interview, the participants were asked to comment of the relevance and comprehensibility of items related to medical care, i.e. items on consultation of medical specialists, use of specialized treatment concepts, adaptive technologies and medical aids, or therapeutic services (see Additional file 2).

The interviews were documented in written form and analysed via investigator triangulation involving two independent researchers. This should minimize the influence of an individual researcher and lead to a higher level of reliability of the analysis [24].

Caregivers’ perspective: cognitive interviews

To assess feasibility and comprehensibility of the RECAPDOC questionnaire, semi-structured cognitive interviews [25-27] with caregivers were carried out. Sample size was determined by saturation, i.e. participants were included up to the point were inclusion of new participants is unlikely to provide new insights [23]. The caregivers were recruited via the German prospective multicentre neurologic rehabilitation registry [18]. This registry was set up in five facilities across the state of Bavaria/Germany with a special expertise in the rehabilitation of acquired brain injury and was started in August 2011. A staff member of the registry contacted the caregivers and asked for consent to be contacted for the study. If the caregivers consented, members of the research team scheduled an interview. Inclusion criteria were: the individuals cared for receive long-term care (in Phase F, according to the classification of neurological rehabilitation by the German Statutory Pension Insurance Scheme), and diagnosis was either coma, VS or MCS. When necessary, a declaration of consent of the legal guardian of the patient had been obtained prior to the start of the interviews. To meet the participants’ needs and to minimize their burden, all interviews took place in the participants’ homes.

During the interview process, the techniques ‘think-aloud’, ‘probing’ and ‘observation of respondents behaviour’ were used [26, 27]. This means that we asked the participants to verbally express their thoughts during filling each item of the questionnaire. We also requested to mention any ambiguities or other comprehension problems. In the case that items were answered without any verbal statement, the participants were reminded of the initial instructions and the interviewers asked further questions to clarify the thoughts of the participant. In addition, the participants’ behaviour was observed and interesting observations like flipping a page back, entry at the wrong position in the questionnaire or facial expressions were documented and addressed immediately. Additional questions (‘probes’) were asked to investigate whether the questionnaire items were understood correctly.

After completing the questionnaire, the method of ‘confidence rating’ was used to evaluate the reliability of the answers (e.g. on duration of therapies or expenses). The participants were asked to rate their confidence in their own answers using a numerical scale ranging from 1 to 10, where 1 means absolutely no confidence in the answer and 10 means a maximum confidence. The same scale was used to assess the general feasibility of the questionnaire and to rate the usefulness of the instructions.

All interviews were conducted and recorded after the participant had been properly informed and given their written consent. The interviews were transcribed verbatim and complemented with field notes. The results were evaluated using investigator triangulation [24] involving two independent researchers.

After each step, potential changes in the questionnaire were discussed among all involved members of the research team until consent was reached.

Results

The detailed results of the systematic review on rehabilitation interventions for patients with DOC in long-term care are described elsewhere [20]. In brief, this systematic review could not identify any effective rehabilitation intervention, but gave an overview of current clinical practice. A list of categories of rehabilitation interventions was developed from the included studies and defined the first content of the questionnaire.

The search identified one relevant guideline, the “Recommendations for inpatient long-term care and treatment of people with severe brain damage in the phase F” provided by the major federal funding bodies of rehabilitation care, the German Federal Rehabilitation Council [22]. This guideline describes interventions provided by nurses, physical, occupational, and speech and language therapists, the use of special treatment concepts, interdisciplinary collaborations, environmental adaptions e.g. adaptive technologies and medical aids and measures to support the family and participation in community as essential components of rehabilitation care for individuals with severe brain injuries. Since the guideline emphasizes the involvement of the personal caregivers in all therapies as well as intensive counselling, we developed a scale to document the implementation of this issue (see Table 1).

Table 1

Scale to document the involvement of personal caregivers in therapies and care (to be documented for each profession separately)

	Fully applies	Partly applies	Does rather not apply	Does not apply at all
I feel adequately informed.
I feel consulted by the therapist/nurse in a competent way.
I feel well guided to continuously implement therapeutic/ nursing measures.
I feel fully involved into the therapeutic process.
I feel physical and psychological relief through the therapy.
I feel supported to realize an appropriate extent of social participation for the patient.

The search identified the “German questionnaire for health-related resource use in the elderly population – FIMA” [28] as the only relevant instrument. In order to use this experience and to facilitate a joint use, we decided to use its time frames (3 months retrospectively), question formats, response options and general layout. Permission and further consultation was obtained from the first author of the questionnaire.

The synthesis of the result of the systematic review, the guideline search and the search for existing questionnaires resulted in an initial version of the questionnaire.

Health professionals’ perspective: delphi survey

Fifty-two national experts (from 194 initially contacted, see Fig. 2) agreed to participate in the Delphi-survey. A total of 43 experts participated in round one and 42 in round two (see Fig. 2). Characteristics of experts participated in round two are shown in Table 2.

Table 2

Characteristics of the 42 participants from the expert survey

	Percent	Number
Sexa
Female	66	(27)
Male	34	(14)
Profession
Nurse	36	(15)
Physical therapist	19	(8)
Occupational therapist	12	(5)
Speech and language therapist	26	(11)
Other therapeutic	7	(3)
Experience in outpatient long-term care (years)	62	(26)
Experience in neurologic day-care centre (years)	17	(7)
Experience in nursing home (years)	45	(19)
Experience in special neurologic nursing home(years)	64	(27)
	M (SD)	Min-Max
Age (years)b	47 (8)	30–62
Professional experience in neurologic rehabilitation (years)	18 (9)	5–43
Professional experience in neurologic long-term care (years)	13 (7)	0.5–35

aInformation on sex was not provided by 1 participant

bInformation on age was not provided by 5 participants

In round one, experts named 23 items that could be classified into six topics (special treatment concepts, adaptive technologies and medical aids, measures to support the family, measures to support social participation and measures to support interdisciplinary collaborations). In round two, 17 items from round one were judged to be relevant (i.e. was rated as “very important” or “important” by at least 50%). In the topic “special treatment concepts”, four out of the 10 suggested concepts were not judged as relevant and where therefore excluded from the questionnaire (see Table 3).

Table 3

Results of the expert Delphi survey

Topic	Topic-related items judged as relevant by the expert participants
Special treatment concepts	Facio-Oral Tract Therapy (FOTT), Stimulation programs, Neuro-developmental treatment (Bobath therapy), Affolter therapy, Kinaestetics
Adaptive technologies and medical aids	Wheel chairs, positioning materials, suction units, hospital beds, tracheostomy equipment, bathroom hoists and seats, standing boards, communication devices, mobility devices, orthotics/ splints
Measures to support the family	Providing information, consulting and counselling, teaching, integration into nursing/therapy process, physical help and emotional relief
Goals of successful rehabilitation	Communication, perception, mobility, interaction and social relationships, self- sufficiency and social participation
Measures to support interdisciplinary collaborations	Interdisciplinary care plans and documentation, regular team meetings and case conferences, conceptual alignment, inter-professional practice

The results of the Delphi survey and the literature review were combined into the version 1 of the RECAPDOC questionnaire (see Fig. 1).

Three medical directors of neurological rehabilitation facilities from different German regions experienced in long-term care treatment of individuals with DOC were recruited for the interviews. Those experts suggested the integration of five new items into the questionnaire: Evaluation by electroencephalogram, main diagnosis, date of injury, discharge from acute rehabilitation and payments from the German social accident insurance. In addition, they proposed small adaptations of another six items on living conditions, level of care, number of physician visits, medical aids, treatment concepts and medical treatments.

The resulting changes led to version 2 of the RECAPDOC questionnaire.

Caregivers’ perspective: cognitive interview

We contacted the legal guardians of 25 affected individuals. Among those, eight persons could not be reached. Finally, four personal caregivers – which were also legal guardians - agreed to participate in the interviews. Three interviews took place in the participants’ private homes and one was conducted in a nursing home.

On average, it took the participants 17 min to complete the questionnaire. All participants rated the questionnaire as easy to manage and comprehensible, except the items referring to the special treatment concepts. Participants rated their responses on time intervals and financial resources as being very accurate.

The findings from the cognitive interviews led to removing the items regarding special treatment concepts. In addition, some minor adaptions were necessary. First, the item regarding the level of care was amended by a detailed explanation of the term “hardship provision”. Second, the item on involvement of a multidisciplinary rehabilitation team was rephrased with less abstract terms. Third, the item on utilization of adaptive technologies and medical aids was amended by an explanation of the ownership status, e.g. owning a wheelchair does not necessarily require its purchase. Moreover, “commode chair” was added to the list of adaptive technologies and medical aids.

The inclusion of these findings led to the final version of the RECAPDOC questionnaire. The content is presented in Table 4. In brief, the final questionnaire contains 29 items and should be used alongside other, more generic instruments to collect sociodemographic and condition-specific data or the use of drugs. The final German questionnaire and an English translation are inluded as Additional files 3 and 4.

Table 4

Content of the final questionnaire

Item no.	Area of assessment	Items	Response options	Time interval
1–4	Living situation	Private household; nursing or residential home; special nursing facility for people with severe DOC; assisted living community; others	Yes/no	Current situation
5–8	Utilisation of home health care and domestic services	Home health care service; paid domestic assistant; informal care by family caregivers, friends or neighbours; other forms of inpatient care e.g. day-care centre	Yes/no, Amount in days/hours/minutes per week Starting date	3 months
9–11	Social insurance benefits	Benefits of statutory long-term care insurance	Yes/noCare levelAttendance allowance: € per monthHardship provision: Yes/no/unknown	Current situation
		Benefits of statutory accident insurance	Yes/no	Current situation
12	Utilisation of outpatient health care	General practitioner; internist; gynaecologist; surgeon; orthopaedist; neurologist or psychiatrist; dermatologist; ophthalmologist; urologist; dentist; psychotherapist; outpatient treatment in hospital; special outpatient clinic for people with severe DOC; others	Yes/noNumber of visits	Last 3 months
13	Utilization of diagnostic measures	Electroencephalography (EEG)	Yes/no	Last 3 months
14	Utilization of adaptive technologies and medical aids	Wheelchair or multifunctional wheelchair; mobility aids e.g. hoist; standing boards; walking aids e.g. wheeled walker; bathing aids e.g. shower couch; toileting aids e.g. commode chair; hospital bed (height adjustable); positioning material e.g. bed wedges; tracheostomy equipment; suction units; ventilator; inhalation devices; feeding tube e.g. PEG-tube; feeding pump; communication devices; orthosis or splints; continence products; others	Owner: Yes/noUser: Yes/no	Current situation and last 3 months
15–27	Utilization of therapies	Physical therapy; occupational therapy; speech and language therapy; rehabilitation nursing	Yes/noStarting dateAmount in days/hours/minutes per week	Last 3 months
		Others	Yes/no;Amount in days/hours/minutes per week	Last 3 months
15–27	Experience of family support (see Table 1)	Information provision; competent consulting; instructions for therapeutic und nursing measures; inclusion in therapeutic process; physical and psychological relief; support for social participation (separately for physical therapy; occupational therapy; speech therapy; rehabilitation nursing)	Strongly agree/agree/disagree/strongly disagree	Current situation
28	Multidisciplinary collaboration of health professionals	Support by a multidisciplinary team	Yes/no	Last 3 months
29		Collective treatment and care planning; collective documentation; joint team and case discussions; working across disciplines; collective conceptual orientation; no multidisciplinary team work	Strongly agree/agree/disagree/strongly disagree/unknown	Current situation

Discussion

To our knowledge, the RECAPDOC questionnaire is the first specific instrument to document the utilization the multitude of rehabilitation and other health care services provided for individuals with DOC in long-term care. This questionnaire is a first step to make the care of this highly vulnerable population amenable for evaluation.

When comparing the content of the initial version of the RECAPDOC questionnaire with the final tested version, minor adaptions and amendments had to be made, except in the section on special treatment concepts. Specialized treatment concepts like Facio-Oral Tract Therapy or Neurodevelopmental Treatment were identified as an important component in the rehabilitation of patients with DOC in long term both in our systematic review [20] and the expert Delphi survey. However, our pre-test revealed that documenting the utilization of those therapies is not possible by approaching the personal caregivers. Although therapeutic alliance, i.e. established and trustful communication on collaboration, task and treatment goals among patients, their families and therapists, is usually well established [29], the communication among therapists and family caregivers did not sufficiently cover specific treatment concepts in our study. To acknowledge the relevance of special treatment concepts in the treatment and management of individuals with DOC, further studies need to explore reliable ways to collect data on that from other sources, presumably from the therapists in charge.

We decided to use a 3-months retrospective approach to document health care utilization for two reasons. One important generic questionnaire for health care utilization in Germany, the FIMA [28], used the same format and a common time frame makes it easier to administer both instruments jointly. In addition, even if a shorter interval might lead to more precise information, the burden for the caregivers with filling the questionnaire should not be multiplied.

The major strength of this study is the use of an iterative and comprehensive development approach based on a systematic review of the literature, guidelines and existing questionnaires and with the involvement of all relevant user groups, including caregivers, professional nurses, therapists and physicians using different methods. Since rehabilitation care provision for individuals with DOC is considered to be multidisciplinary [22], the involvement of all relevant professionals as well as the personal caregivers aspect is indispensable. The pre-test of our questionnaire by means of cognitive interviews allowed us to evaluate comprehensibility and manageability from the perspective of the relevant users. Cognitive interviews are most valuable for questions that are complex, sensitive, and intrusive for specific groups [26, 27].

Some limitations of the study need to be acknowledged. The main challenge of the study was the recruitment of caregivers as participants for the pre-test. To revise items and eliminate problems 5 to 15 interviews are recommended [25]. Even though we had access to an established registry for DOC patients in Southern Germany [18], only four persons consented to participate, presumably due to a high personal burden [30-33]. Therefore, the results of our pre-test must be interpreted with care. However, despite the small sample size, our results were consistent among the participants and our sample was heterogeneous and covered a variety of different relevant circumstances the what is also a precondition for meaningful results from cognitive interviewing [25].

A further fact needs to be acknowledged. In 2013, a UK-based working group published a paper on challenges in developing resource use measures [34] including a best-practice guideline. Even though we were not able to include this guideline in the planning of our study because we started the project in 2013, our development process is largely in line with this paper.

Since resource utilization depends on the reimbursement principles of the respective heath care system, the use of this questionnaire is only meaningful in Germany and adaptions are necessary to use the questionnaire in other health care systems.

Further studies, e.g. addressing the economical perspective or investigating resource utilization as predictor for patient outcomes need to be done.

Conclusions

The developed RECAPDOC questionnaire makes the situation of patients with disorders of consciousness in the long-term care setting accessible for evaluation in epidemiological studies and patient registries. The documentation of special therapeutic treatment concepts needs to be appraised in an alternative way. The questionnaire can now be used in future studies that may want to explore the association of rehabilitation interventions and patient-related outcomes.

Questionnaires of the Delphi survey (round 1 & 2). (DOCX 15 kb)

Interview guide for the expert interviews with physicians. (DOCX 13 kb)

German version of the REACPDOC questionnaire. (PDF 684 kb)

English translation of the RECAPDOC questionnaire. (PDF 779 kb)