A team approach to applying the International Classification of Functioning, Disability and Health Rehabilitation set in clinical evaluation.

OBJECTIVE: To develop a team approach to applying the International Classification of Functioning, Disability and Health Rehabilitation Set (ICF-RS) in clinical evaluation.
DESIGN: A Delphi study.
SUBJECTS: Experts from rehabilitation institutions in China including physicians, nurses, physiotherapists and occupational therapists.
METHODS: A 2-round Delphi survey and expert panel discussion were used to generate the team approach. Firstly, the candidate types of professionals for team rating were chosen through expert panel discussion. A carefully selected group of participants was then asked to score the suitability of physicians, nurses, or other candidate therapists for each category's rat-ing, applying the International Classification of Functioning, Disability and Health Rehabilitation Set in clinical evaluation. After initial assignment of cate-gory to types of professionals, a second round Delphi survey was conducted to quantify the professionals' agreement with the category assignments and generate a final team evaluation approach.
RESULTS: Thirty of the category assignments achiev-ed consensus. The final team evaluation approach assigned 6 categories to physicians to evaluate, 7 categories to nurses, 9 categories to physiotherapists, and 8 to occupational therapists.
CONCLUSION: Such a team evaluation approach could facilitate implementation of the ICF-RS in clinical settings and provide a more convenient assessment tool for professionals.

The International Classification of Functioning, Disability and Health (ICF) is a standardized framework for describing and understanding health, functioning and disability, which was endorsed by the World Health Assembly in 2001 (1). An individual’s functioning and level of disability may be recorded by selecting the appropriate ICF category and using its corresponding code and qualifiers (2). With the increasing number of people with disability, there have been calls to strengthen rehabilitation services in the world’s health systems (3). The ICF can be used as a unifying model for the conceptual description of rehabilitation strategies and as a useful tool for quantifying the effectiveness of rehabilitation services (4, 5). In order to facilitate clinical practice, the ICF Core Sets specify shortlists of ICF categories relevant to a particular health condition or setting. They have been rigorously developed to enhance the utility of the ICF in clinical practice and research (6, 7).

The ICF Rehabilitation Set (ICF-RS), with 30 categories, is a minimal set for reporting and assessing functioning and disability in clinical rehabilitation and in research (8–10). Its domains cover body functions, activities and participation. As the ICF Rehabilitation Set is essentially a list of categories, it needs to be considered from a measurement perspective (11, 12). A Chinese version of the ICF-RS has been developed, describing how to use it to measure aspects of functioning (13). The Chinese psychometric properties have shown good reliability and validity in multi-centre research, and they have been widely used as a functional evaluation tool in China (14).

However, as the Chinese ICFRS became widely used, some problems were observed in its application. Using it involved clinical examination and interviews. That was time-consuming. A single rater would have difficulty finishing the entire evaluation in a single sitting, especially with a patient with complex complaints or poor language expression. Some raters reported that even 30 min to complete one evaluation was too long in a busy rehabilitation department. That undoubtedly limited the instrument’s application. In addition, the categories of the ICF-RS cover a wide range of functioning, and some categories were not closely related to the job responsibilities of some of the rehabilitation professionals using it. This may have resulted in a further decrease in overall attention and enthusiasm in its use.

A team approach may be a way to increase the evaluation efficiency and facilitate the process. In such an approach the assessment would be completed by a team of professionals, with each rater handling some categories consistent with his or her routine job responsibilities in the clinic. Ottiger has reported a new multidisciplinary observation scale for stroke patients, in which neurologists, nurses, physiotherapists, occupational therapists and speech therapists each rate assigned domains (15). A group led by Catz has reported a revised version of the Spinal Cord Independence Measure evaluated by a team including an occupational therapist, a nurse and a physiotherapist (16). Both of those scales were found easier and more convenient to implement, as each professional in the team rated their own part of the scale. As yet, however, there have been no reports about a team version of the ICF Rehabilitation Core Set.

This study aimed to develop a team evaluation approach to using the Chinese version of the ICF-RS. It was designed to be applied in rehabilitation institutions in China, with the professionals in the team rating categories closely related to their daily work. The team approach was supported by a mobile app, which the team members could use to record and share their evaluation results (17).

METHODS

Overview

The objective of the team approach was to divide the original 30 categories into sections, and to have each section evaluated by a different rehabilitation professional in the team: one professional for each category. The study was conducted using a modified Delphi technique and an expert panel (18, 19). The process had 3 steps, as follows. (i) To establish a panel of ICF experts, which would screen candidate professionals for participating in the rating team through discussion. (ii) A first-round Delphi survey was conducted in which the participants were asked to score the candidate professionals using a 5-point Likert Scale in terms of their suitability for evaluating particular ICF-RS categories (20, 21). Table I presents the list of categories. A first draft of the team roster was then prepared according to the results combined with discussion among the experts. (iii) A secondround survey was then conducted in which the participants were asked to rate their agreement with the category assignments. Ethics approval for this procedure was obtained from the ethics committee of Sun Yat-sen Memorial Hospital (Guangzhou, China). Written informed consent was obtained from all participants.

Table I

International Classification of Functioning, Disability and Health Rehabilitation Set (ICF-RS) categories

Body functions	Activities	Participation
b130 Energy and drive functions	d240 Handling stress and other psychological demands	d230 Carrying out daily routine
b134 Sleep functions	d410 Changing basic body position	d470 Using transportation
b152 Emotional functions	d415 Maintaining a body position	d660 Assisting others
b280 Sensation of pain	d420 Transferring oneself	d710 Basic interpersonal interactions
b455 Exercise tolerance functions	d455 Moving around	d770 Intimate relationships
b620 Urination functions	d450 Walking	d850 Remunerative employment
b640 Sexual functions	d465 Moving around using equipment	d920 Recreation and leisure
b710 Mobility of joint functions	d510 Washing oneself
b730 Muscle power functions	d520 Caring for body parts
	d530 Toileting
	d540 Dressing
	d550 Eating
	d570 Looking after one’s health
	d640 Doing housework

Screening basis

An expert panel was set up to screen professionals for the rating team. The experts were all from the Rehabilitation Department of Sun Yat-sen Memorial Hospital, Sun Yat-sen University (Guangzhou, China). They were a senior physician, a senior nurse, a senior physiotherapist, and a senior occupational therapist. All of the experts had at least 5 years of experience working in a rehabilitation department and had worked with the ICF for more than 3 years.

The candidate professionals were evaluated through item-by-item discussion of each category. Two main questions were proposed to help the screening: (i) What kinds of professionals are in contact with or pay attention to this category’s content in their daily work? (ii) What kinds of professionals are involved with this category using other scales in a rehabilitation clinic? The discussion was based on the definition and evaluation content of each category, but constrained, to some extent, by the availability of professionals in rehabilitation departments in China.

First-round Delphi survey

Participants. Purposive snowball sampling was used to enrol participants (22). The sample size of participants was not clearly defined for the Delphi survey, and many studies reported the participant numbers ranged from 10 to 50 (23–25). This study aimed to enrol 25–30 experts for the Delphi survey. The initial recruiting exploited an online expert group specialized in ICF research in China. Group members were asked to volunteer. Those who volunteered were then asked to recommend other relevant ICF experts. Some experts who had participated in a previous Delphi survey about the psychometric properties of Chinese ICF-RS (13) were also selected. The members of the expert panel did not participate in the 2 rounds of Delphi surveys.

Geographical representation was taken into account in the recruitment process. Those included were all registered senior physicians, senior nurses, attending or senior therapists. All had at least 10 years of clinical experience in hospitals, including at least 5 years of clinical experience in rehabilitation and also 2 years of experience related to the ICF. One nurse who was not at the senior level was included because the ICF-RS was the topic of her doctoral research.

Design

An invitation letter, describing the survey’s purpose, method, content and matters needing attention, was sent to the participants by e-mail or WeChat (Weixin, Beijing, Tencent Technology Co., LTD), inviting them to participate. A questionnaire was then sent to those who agreed to participate. The questionnaire had 2 parts: expert consultation for team evaluation; and personal information about the respondent.

In the expert consultation part, the respondents were asked to evaluate if a physician, a nurse, or a therapist was suitable for assessing each ICF-RS category. The sub-questions for each category were designed based on the types of candidate professionals. The respondents used a 5-point response format from “strongly unsuitable” (quantified as 1) to “strongly suitable” (quantified as 5) for each question. Only one option could be selected in response to each question. The evaluation content and evaluation options for each category of the ICF-RS were provided for the respondents’ reference (Fig. 1).

Fig. 1

An example of the questionnare in the first-round survey. Four sub-questions for the category “b280 Sensation of pain”.

The personal information collected about the participants included their name, age, gender, education, profession, title, years using the ICF and (self-assessed) degree of familiarity with the ICF.

It took approximately 10–15 min to complete the questionnaire. The respondents were asked to return it within 2 weeks. If no reply had been received by 3 days before the deadline, the respondent was reminded by the coordinator. All of the responses collected were checked, and if there were any omissions the respondent was contacted and asked to complete the responses. Eventually, a questionnaire with missing responses was deemed invalid.

If only one type of professional was rated as “basically suitable”, “suitable” or “strongly suitable” (a score of 3, 4 or 5) for assessing a particular category by more than 75% of the respondents, and if the mean suitability score was ≥ 3.6, that type of professional was considered appropriate for that category without any further discussion (13, 26, 27). In most cases, however, more than one type of professional met that standard. In those cases, the category assignment was considered through discussion among the expert panel. The mean suitability scores, the balance of workload and the relationship with a profession’s routine work were major factors to be considered in assignment. If no profession met that standard, then the profession rated as “basically suitable”, “suitable” or “strongly suitable” (a score of 3, 4 or 5) was chosen by the majority of the respondents.

Second-round Delphi survey

The respondents who participated in the first-round survey received a second questionnaire by email or WeChat. That questionnaire reported the results of the first-round survey and the initial category assignments. The respondents were asked to consider to what extent they agreed with the assignments. They replied using a 5-point Likert scale: 1, “strongly disagree”; 2, “disagree”; 3, “doesn’t matter”; 4, “agree”; and 5, “strongly agree”. Two weeks were allowed to return their responses, and a reminder was sent 3 days before the deadline by a coordinator. According the standard for consensus generally used in such research (13, 26, 27), an assignment was accepted if it had a mean agreement score ≥ 3.6 and more than 75% of the respondents chose “agree” or “strongly agree” (Likert scores of 4 and 5). Any category with a mean agreement score below 3.6 or total agreement among the respondents of less than 75% was re-assigned in discussion among the expert panel considering the results from the first Delphi round and based on their clinical experience.

Statistical analysis

Version 20.0 of the SPSS software suite (IBM, Armonk, NY, USA) was used for statistical analysis. The suitability and expert agreement scores for each category were calculated and expressed as mean (standard deviation (SD)). The higher the mean, the better the agreement about a category assignment.

RESULTS

Screening of professionals

The Chinese version of the ICF-RS was generated to be used in rehabilitation institutions in China, but many of these institutions do not employ personnel across the full set of professional categories. As a result, some of the work of speech therapists, psychotherapists and social workers is, in fact, done by other types of professionals. Through discussion, it was decided that a speech therapist was only related to category “d550 Eating”. A psychotherapist was related to the category “b152 Emotional functions”, and a social worker was related to category “d850 Remunerative employment”. All of those categories could, it was decided, also be evaluated by other sorts of professionals instead. Thus, speech therapists, psychotherapists and social workers were excluded from the evaluations. Physicians, nurses, physiotherapists and occupation therapists were eventually selected to participate in the rating team.

Results of the first-round survey

Of the 31 experienced rehabilitation professionals initially recruited for the Delphi survey, 30 (96.8%) completed the questionnaire adequately (11 physicians, 9 nurses and 10 therapists). The demographics and qualifications of these professionals are shown in Table II.

Table II

Demographic characteristics and experience of the professionals

Items	Frequency	Percentage, %	Mean (SD)
Gender
Male	14	44
Female	16	56
Age, years			40.5 (6.94)
≤ 39 years	15	50
40–60 years	15	50
Profession
Physician	11	36.67
Nurse	9	30
Physiotherapists	6	20
Occupational therapists	4	13.33
Professional title
Attending	8	26.67
Vicesenior	16	53.33
Senior	6	20
Working years			17.8 (8.10)
10–15 years	13	43.33
15–45 years	17	56.67
Working with ICF, years			7.07 (3.63)
3–5 years	11	36.67
6–15 years	19	63.33
Familiarity with ICF Rehabilitation Set
Mildly familiar	9	30
Familiar	15	50
Very familiar	6	20

ICF: International Classification of Functioning, Disability and Health; SD: standard deviation.

Among the 30 categories, there were 22 in which only one type of professional got a suitability percentage ≥ 75% and a mean suitability score ≥ 3.6. Those selections were accepted without further discussion. Of the 22, there were 4 categories for which a physician was clearly considered the most suitable. 2 was assigned to a nurse, 9 were assigned to a physiotherapist, and 7 to an occupational therapist. There were 8 categories in which at least 2 types of professionals got a suitability percentage ≥ 75% and a suitability score ≥ 3.6. Through discussion, the expert panel assigned them to be evaluated by a physician or a nurse in an attempt to share the workload. “d240 Handling stress and other psychological demands” was assigned to the physicians. “d510 Washing oneself”, “d520 Caring for body parts”, “d530 Toileting” and “d550 Eating” were assigned to nurses. The nurses also received “d230 Carrying out daily work” and “d570 Looking after one’s health”, although 3 types of professionals were considered suitable. The category “b130 Energy and drive functions” met that standard with 4 types of professionals, but it was assigned to the physicians (Fig. 2). The details of the initial category assignments, the suitability rate and the mean suitability scores are listed in Table III.

Fig. 2

Results of the first round of the Delphi survey. The 8 categories in which 2 or more professions met the standard were assigned through expert panel discussion

Table III

First round suitability scores using a 5-point response scale

Profession	Categories	Suitability		Scores
		Frequency	Percentage, %	Mean	(SD)
Physician	b134 Sleep functions	29	96.7	4.30	(0.84)
	b152 Emotional functions	27	90	4.07	(0.98)
	b280 Sensation of pain	28	93.3	4.20	(0.89)
	b640 Sexual functions	28	93.3	4.20	(0.96)
	d240 Handling stress and other psychological demands	28	93.3	3.78	(0.86)
	b130 Energy and drive functions	28	93.3	3.87	(0.97)
Nurse	d770 Intimate relationships	26	86.7	3.83	(1.09)
	b620 Urination functions	29	96.7	4.43	(0.77)
	d230 Carrying out daily routine	29	86.7	3.87	(1.18)
	d570 Looking after one’s health	28	90.0	3.87	(1.04)
	d510 Washing oneself	25	83.3	3.97	(1.13)
	d520 Caring for body parts	26	86.7	4.20	(1.03)
	d530 Toileting	28	93.3	4.17	(0.91)
	d550 Eating	28	93.3	4.07	(0.94)
Physiotherapist	b455 Exercise tolerance functions	28	93.3	4.57	(0.86)
	b710 Mobility of joint functions	30	100	4.63	(0.49)
	b730 Muscle power functions	30	100	4.70	(0.47)
	d410 Changing basic body positon	29	96.7	4.60	(0.77)
	d415 Maintaining a body position	29	96.7	4.53	(0.78)
	d420 Transferring oneself	29	96.7	4.47	(0.82)
	d450 Walking	28	93.3	4.60	(0.81)
	d465 Moving around using equipment	29	96.7	4.53	(0.73)
	d455 Moving around	29	96.7	4.57	(0.73)
Occupational therapist	d640 Doing housework	30	100	4.73	(0.52)
	d660 Assisting others	29	96.7	4.50	(0.78)
	d470 Using transportation	28	93.3	4.27	(0.91)
	d710 Basic interpersonal interactions	29	96.7	4.44	(0.96)
	d920 Recreation and leisure	28	93.3	4.43	(0.90)
	d540 Dressing	30	100	4.70	(0.60)
	d850 Remunerative employment	27	90	4.17	(1.12)

SD: standard deviation.

Results of the second-round survey

The 30 respondents who participated in the first expert survey also received the second-round questionnaire. Of these, 29 (96.7%) responded adequately, including 10 physicians, 9 nurses and 10 therapists. In the second round, 29 categories received more than 75% agreement, with the mean agreement scores ranging from 3.79 to 4.72 (Table IV). Category “d230 Carrying out daily routine” received less than 70% agreement, so it was assigned to the occupational therapists in view of the fact that it was the assignment that received the highest mean score in the first-round survey.

Table IV

Final assignments after the second round of Delphi survey

Profession	Category assignment	Agreement		Scores
		Frequency	Percentage, %	Mean	(SD)
Physician	b134 Sleep functions	28	96.6	4.14	(0.86)
	b152 Emotional functions	27	93.1	4.07	(0.80)
	b280 Sensation of pain	28	96.6	4.38	(0.68)
	b640 Sexual functions	27	93.1	4.38	(0.62)
	b130 Energy and drive functions	26	89.7	4.14	(0.69)
	d240 Handling stress and other psychological demands	26	89.7	4.07	(0.65)
Nurse	d770 Intimate relationships	23	79.3	3.79	(1.11)
	b620 Urination functions	26	89.7	4.28	(0.92)
	d570 Looking after one’s health	25	86.2	4.10	(0.98)
	d510 Washing oneself	24	82.8	3.93	(1.00)
	d520 Caring for body parts	24	82.8	4.00	(1.04)
	d530 Toileting	23	79.3	3.93	(1.10)
	d550 Eating	23	79.3	3.90	(1.08)
Physiotherapists	b455 Exercise tolerance functions	28	96.6	4.62	(0.68)
	b710 Mobility of joint functions	28	96.6	4.69	(0.66)
	b730 Muscle power functions	28	96.6	4.69	(0.66)
	d410 Changing basic body positon	27	93.1	4.48	(0.95)
	d415 Maintaining a body position	26	89.7	4.41	(1.05)
	d420 Transferring oneself	25	86.2	4.31	(0.97)
	d450 Walking	27	93.1	4.41	(0.73)
	d465 Moving around using equipment	26	89.7	4.35	(0.86)
	d455 Moving around	27	93.1	4.41	(0.73)
Occupational therapists	d230 Carrying out daily routine	20	69	3.62	(1.21)
	d640 Doing housework	30	100	4.72	(0.46)
	d660 Assisting others	27	93.1	4.45	(0.74)
	d470 Using transportation	28	89.7	4.31	(0.97)
	d710 Basic interpersonal interactions	26	93.1	4.31	(0.85)
	d920 Recreation and leisure	27	93.1	4.59	(0.68)
	d540 Dressing	28	96.6	4.59	(0.63)
	d850 Remunerative employment	27	93.1	4.45	(0.91)

SD: standard deviation.

DISCUSSION

This study attempted to develop a team approach to facilitate use of the Chinese version of the ICF-RS in evaluations. Team evaluation provides an alternative approach for busy professionals, especially when used to evaluate patients with complex problems or poor communication ability. The professional analysis assigned 6 categories to physicians, 7 to nurses, 9 to physiotherapists and 8 to occupational therapists.

The Delphi technique used in this study successfully developed a consensus among the experts contributing through repeated information exchange and feedback (28). Unlike with the classic Delphi technique, the modified Delphi method used here did not apply the conventional 4 rounds (29–31). The consultation ended after 2 rounds, as the experts had reached a consensus. A review of 30 studies using the Delphi technique reveals they reached consensus after 1–5 rounds. Among them are 14 studies which reported requiring only 2 Delphi rounds (32). They were mostly generating standards, guidelines and scales (33–35). Some reports described using intermediate face-to-face meetings between Delphi survey rounds (36) or the involvement of different expert panels in the consensus process (37). As there were no reports about how to divide the same scale into sections to generate another rating version, in this study expert panel discussion was combined with 2 rounds of Delphi survey to successfully develop the team approach.

Finally, relatively few categories were assigned to the physicians and nurses. However, as the goal of the team approach was to share out the evaluation tasks to facilitate the process while ensuring the relevance and the accuracy of the results, which was considered satisfactory. All of the assignments met the consensus standard and seemed relevant to actual job responsibilities in a clinic.

The category “d240 Handling stress and other psychological demands” refers to regulating and controlling one’s mental state in order to accomplish tasks that may be complicated by stress and/or distraction. It emphasizes completing tasks under stress rather than simple physical movement and coordination. Physicians pay more attention to patients’ psychological adjustment than therapists normally would, so assigning this category to physicians received good support in the second round of the survey. The category “b130 Energy and the drive function” refers to physical fitness and the executive initiative required to achieve general goals and meet special needs. It too emphasizes the patient’s mental state, but also physical strength and energy. Because low energy and fatigue in the rehabilitation process are often related to inconsistent training intensity, or perhaps to heart failure or other factors, physicians can better grasp the patient’s basic difficulty and its causes and carry out more comprehensive and accurate assessments. The categories “d510 Washing oneself”, “d520 Caring for body parts”, “d530 Toileting” and “d550 Eating” were developed based on the content of the Modified Barthel Index. That index is a rating scale for ability in the activities of daily living, which is commonly used in rehabilitation in China. Research has shown that Barthel Index scoring by nurses agrees well with that by physicians (38).

In the second round of the survey the assignment of category “d230 Carrying out daily routine” generated less than 70% agreement. It was eventually assigned to the occupational therapists. The category refers to planning, scheduling and completing the daily routine, including controlling one’s activity level. The goal of occupational therapy is to create meaningful and satisfying lives through targeted and well-designed activities. Much of occupational therapy relates to patients’ daily lives and work, improving their quality of life and returning them to their families and society (39). Hence, d230 was considered consistent with the scope of practice and professional responsibilities of occupational therapists.

While this study related entirely to China’s rehabilitation context, medical professionals and their skills and patients and their problems have many similarities worldwide. This study’s assignments may therefore be helpful for rehabilitation personnel attempting to apply the ICF-RS in many different settings. The professionals who completed the questionnaire all had rich experience and deep understanding of rehabilitation practice. It might, however, have been better to have provided a job responsibility guide for their reference. That might have helped them to make easier and more consistent choices. In addition, although consensus was reached in 2 rounds, a third round of Delphi surveying would perhaps have been useful for research purposes. It is possible that some deviation from the consensus might have emerged in a third round. More research is needed to demonstrate that the team evaluation approach developed here has validity consistent with that of the conventional single-rater procedure. This will be the next stage of our research.

CONCLUSION

A new team approach to applying the ICF-RS in evaluation was generated in this study. The whole set of 30 evaluation categories was divided among a hypothetical team consisting of a physician, a nurse, a physiotherapist and an occupational therapist, with 6 categories assigned to the physician, 7 to the nurse, 9 to the physiotherapist and 8 to the occupational therapist. A limited number of categories can be finished easily during a professional’s work day without undue pressure. Improving the evaluation efficiency of the individual team members inevitably facilitates the implementation of the ICF-RS in the clinic. The team evaluation approach promises to better share the evaluation workload, perhaps improving the accuracy of evaluations and strengthening interdisciplinary collaboration in the clinic.