Usability and consistency in findings of the work support needs assessment tool.

BACKGROUND: Structured work support needs assessment could facilitate professionals and increase assessment consistency.
OBJECTIVES: Evaluating usability of the Work Support Needs Assessment Tool and test if professionals' (labour experts, coaches) findings become more consistent after a tool training. The tool includes a 21 item checklist for assessing work support needs of people with disabilities.
METHODS: Usability was explored through 28 interviews with professionals. Consistency was evaluated in an experimental pre-post study design, in which thirty-nine other professionals assessed work support needs of standardized clients before and after a protocolized training. Quantitative content analysis was conducted. Consistency of findings between professionals covered three categories: type (client-focused coaching), focus (topics to be addressed) and duration of support. An increase in consistency was defined as a decrease in the total number of different sub-categories of findings in each category.
RESULTS: Nineteen professionals indicated that the tool was useful, as they gained relevant information and insights. Regarding consistency, the number of findings differed pre- and post-training for type of support (8 vs 9) and focus of support (18 vs 15 and 18 vs 17).
CONCLUSIONS: Participants had positive experiences with the tool. Increased consistency in findings of professionals after the training was not demonstrated with the current study design.

Introduction

Disability affects many people, with the global average prevalence rate being estimated at 16% of the adult population [1, 2]. These prevalence rates are combined with high levels of unemployment among people with disabilities leading to high social security costs for society [3]. Employment can have beneficial effects on health, including feeling more autonomous and having an increased sense of wellbeing [4]. Moreover, especially for people with a disability, having a job is associated with a higher level of functioning and a better quality of life [5, 6]. Therefore, it is important that people with disabilities are given the opportunity to enter and remain in the labour market.

People with disabilities encounter problems in finding and holding onto a job [7]. However, when they receive support, these problem can be alleviated [8, 9]. Frequently, vocational rehabilitation professionals are required to assess the type and intensity of support that a person with a disability will need in order to function at work. The type of support can vary from job search assistance (support in finding suitable jobs, writing resumes or job interviews) to job coaching (support when people start work), job training (teaching job-related tasks) or workplace adaptation [2, 3, 7, 10, 11] In order for a professional to make a decision about the necessary support during work, referred to below as ‘work support needs’, the professional must assess what kind of support will be needed in which situations.

Work support needs are often assessed before the person with a disability has a job, because the required amount and type of support plays an important role in the process of matching the right person with a suitable job and workplace. Therefore it is often not feasible to search for a job when work support needs are not known. This makes the assessment of work support needs complex, as the level of support needed is partly dependent upon the actual workplace and the specific job tasks. Additionally, the assessments are often made for young people with little or no work experience. In such cases the professional has to look for practical examples from a person’s daily life to determine the areas in which support is needed, translating these into work situations. A tool that facilitates these work support needs assessments by providing the professional with more structure and guidance could therefore be useful. Furthermore, an universal structured approach in assessing work support needs might increase transparency and consistency in assessments between professionals. To the best of our knowledge, currently no universal tool exists for assessing work support needs of people with disabilities when the future workplace is unknown.

Therefore, we developed the Work Support Needs Assessment Tool. This tool facilitates vocational rehabilitation professionals (labour experts, job coaches) in the process of assessing work support needs of people with disabilities when the future workplace is still unknown. The 21 items included in the tool cover four categories: pre-conditions for work; skills; personal characteristics; and cognitive and executive functions. The tool guides the professional in deciding which items are deemed relevant for the work support needs assessment and in making an overall decision on the work support needs. It is important to evaluate whether professionals feel the tool can be used to assess work support needs in practice (i.e. does it provide the professional with relevant information) and whether the tool can be incorporated in the work process of a professional. These views of professionals concerning the usability of the Work Support Needs Assessment Tool are still unclear. Furthermore, it is not known if professionals’ assessment of work support needs becomes more consistent after using the tool.

The first phase of the study explored the usability of the Work Support Needs Assessment Tool, while the second phase evaluated the consistency of work support needs assessments. Training programmes have been found to be effective in enhancing knowledge and skills as well as guideline adherence of professionals [12, 13]. It is likely that training in the use of our tool will lead to better knowledge and skills in using the tool. Therefore, participants in our study completed a training programme on how to use the Work Support Needs Assessment Tool. Subsequently, we evaluated the influence of this training programme on the consistency of the professional findings generated by using the work support needs tool. The following research questions were addressed: What are the views of professionals regarding usability of the Work Support Needs Assessment Tool?; Do the findings of professionals become more consistent after completing a training programme on the Work Support Needs Assessment Tool?

Methods

Design and procedures

The first phase of the study on the usability of the tool had a qualitative design, with Dutch vocational rehabilitation professionals participating in semi-structured interviews. Prior to the interviews, participants were asked to use the tool to assess work support needs with at least two clients from their caseload.

The consistency of findings on work support needs was evaluated in an experimental pre-post study design using two client vignettes played by an actor. Participants assessed the work support needs of one client before the start of the training programme and without using the tool. The participants then completed the training programme before assessing the work support needs of a second client with use of the tool. The order of the client vignettes alternated between the four training groups.

Ethics

The research was conducted in accordance with the Helsinki declaration. The research proposals were submitted to and approved by the Medical Ethical Committee of the Academic Medical Center, which judged that a comprehensive evaluation was not required since these studies were not subject to the Medical Research Involving Human Subjects Act (reference numbers W17_235#17.273 and W17_ 298 # 17.351).

Participants

Various types of vocational rehabilitation professionals (labour experts, consultants, job coaches) participated in the study. In the first phase, professionals were eligible to participate if the assessment of future work support needs of clients with disabilities was part of their job, and if clients in their caseload did not yet have a job. For the second phase of the study, professionals were eligible to participate if they had not been involved in prior studies in which the tool was developed and if they were experienced in assessing future work support needs of clients with disabilities.

Participants were recruited through flyers that were distributed on professional networks, websites, through a newsletter and by approaching professionals and organizations from the researchers’ network. The recruitment procedure was aimed at various types of vocational rehabilitation professionals from different organizations. All participants gave their written informed consent prior to study participation.

The work support needs assessment tool

The tool development started in 2015, as part of another study (study not published). The aim of the study was to develop a checklist with items that were deemed relevant for assessing work support needs of people with disabilities, when the future workplace is still unknown. A concept mapping approach was used to collect, integrate and conceptualize views and appraisals of the consulted experts (i.e. vocational rehabilitation professionals) [14]. The checklist development followed three stages: (1) literature search (conducted by researchers), (2) a focus group and (3) prioritizing and organizing of items. The literature search resulted in a list of 66 items relevant for assessing work support needs. In the second stage 15 vocational rehabilitation professionals participated in a focus group. Prior to the focus group, participants could online submit relevant items, this resulted in a list of 55 items. All items from the literature search and the online submittal were discussed in the focus group and consensus was reached on a list with 50 items. These items were organized in clusters and prioritized by relevancy online, on a 0–5 scale (0 = not very important; 5 = very important). Items with a mean score of 3.5 or higher were included in the final list. This resulted in a checklist of 21 items. In a second focus group with the same professionals, consensus was reached about the definitions of these 21 items (study not published).

The checklist was further developed into the Work Support Needs Assessment Tool in 2017. The core of the tool is the 21-item checklist (see Table 1). The first step of the tool is to systematically assess every item by answering several questions. These recurring questions guide the professional in deciding whether the item is relevant to the assessment. The second step of the tool facilitates the professional in the process of making an overall work support needs judgement by obtaining answers to several concluding questions (see Table 1). Additionally, professionals can consult an appendix, which provides an overview of the items, corresponding definitions and suggestions on how to question clients on a certain item. See supplement material 1 and 2 for the paper-based (translated into English) and print screens of the digital version of the tool (in Dutch).

Table 1

21 items and the questions included in work support needs assessment tool

Cluster		Item	Definition
Pre-conditions for work	1	A place to live	A place to sleep, access to sanitary facilities and sufficient food
	2	Ability to cope independently	Physical, intellectual and mental ability to manage situations independently in personal life
	3	Social network	The opinions, help and expectations of parents, brothers, sisters and friends that positively or negatively influence the choices that a person makes
Skills	4	Perseverance	Having the physically and mentally capacity to persevere in job related tasks
	5	Working independently	The ability to start, execute and finish job related activities without the help and support from others
	6	Work agreements	Being on time and adhering to work agreements
	7	Dealing with time limits	Being capable of achieving specific work tasks in a certain amount of time and dealing with time limits
Personal characteristics	8	Dealing with other people	Being capable of showing appropriate behavior and attitude towards colleagues and costumers (i.e. implicit rules of conduct and behavior)
	9	Dealing with authority	Being able to work for a boss and accept instructions or assignments from people with authority
	10	Conflict management	Dealing with conflicts with or between others
	11	Dealing with resistance	Being able to deal with feedback, criticism or resistance
	12	Motives and motivation for work	Motives in life and the intrinsic motivation to work
	13	Sense of reality	Being able to separate fantasy (including delusions and hallucinations) from reality
	14	Self-knowledge	Knowing your own strengths and weaknesses and boundaries
	15	Determination	The ability to carry on and not to give up, even when something is very difficult or does not work out as intended
Cognitive and executive functions	16	Ability to concentrate	The ability to completely focus on processing of information and to block off distraction
	17	Effectiveness	Being able to perform activities in a coordinated manner in order to reach a certain goal, by structuring and prioritizing tasks and by keeping oversight
	18	Information processing	Being able to filter an process what we perceive through our senses
	19	Apprehension	Being able to understand job tasks and job instructions
	20	Dealing with unexpected or new situations	Dealing with unexpected or new situations in such a manner that it does not affect the quality of work
	21	Memory	The ability to store, remember and reproduce information
			Answer options
Questions for assessing item	Does the person have capacities concerning this item that will help with functioning in work?		Yes, sufficient
			No, insufficient
			Not yet clear at this moment
	Is support and/or adjusting workplace/job tasks regarding this item necessary?		Yes support is necessary
			Yes, adjustment workplace/job tasks is necessary
			Yes support and adjustment to workplace/job tasks is necessary
			No, cannot be solved by support or adjustment workplace/job tasks
	What did you observe (or read in client’s file) that made you think support is necessary?		Open answer
	What kind of adjustment in workplace/job tasks would be necessary?		Open answer
Concluding questions	What are the best chances for participation in work and what kind of support will enhance these chances?		1-7 Open answers
	On which items is coaching/training/supervision necessary, by whom and with what aim?
	Which areas where support is needed should be addressed first and why?
	Duration of support: How many hours a week or month, during which period and why?
	What is the view of the client on support?
	When is re-evaluation of work support needed? Other remarks?

Phase one: Usability of the work support needs assessment tool

The interviews were audio-recorded and conducted by the first author (MS) and another member of the research team (SV). Interviews were conducted with the use of a topic list developed by the research team. The topic list included open-ended questions regarding demographic information, how the tool was used, usability of the tool (i.e. whether the tool assisted in obtaining the right information and generated sufficient insight regarding the support needs of the client) and whether the tool could be implemented in the assessment routines of the participant.

Phase two: Consistency of the findings regarding work support

Training programme

The consistency of the findings of professionals on work support needs were evaluated after completing a protocoled training programme. The learning objectives of the training programme were: 1) the participant understands the 21 items and related definitions 2) the participant is able to formulate suitable questions to assess relevant items, 3) the participant can use the tool during work support needs assessment. The three-hour training consisted of the following components: 1) general introduction (stating aims and learning objectives), 2) short specific introduction on the tool development process, 3) interactive exercise in pairs (aimed at interiorizing several items and definitions), 4) general discussion about interactive exercise, 5) demonstration of how to use the tool. The training was supervised by the second author (KN), a professionally qualified and experienced teacher.

Client vignettes

The two client vignettes were played by an actor (see Box 1 for short vignettes description). During the session, participants were given the opportunity, as a group, to ask the actor questions, so that they would be able to assess the work support needs. In order to minimize the possibility of differences in the roles played, the actor used a fixed script. After questioning the actor, the participants individually recorded their findings regarding the work support needs of the client.

Outcome measure

The outcome measure was operationalized as the findings of the work support needs assessment and was measured with the following question: ‘Describe the necessary support for the client: include in your assessment the type of support, areas on which the support should focus and the level of support needed’.

Analysis

Phase one: Usability of the work support needs assessment tool

The interviews were transcribed by MS and SV and were randomly checked for inconsistencies by SA. Transcriptions were categorized according to views about usability and implementation in the work processes and summarized for each category by SV. The categorizing and summarizing of the data were checked and discussed by MS, KN and SA until inconsistencies were resolved and adjustments made accordingly. The final analysis was then discussed by the whole research team.

Phase two: Consistency of the findings regarding work support needs

Assessors involved in analysing the work support needs assessments were blinded for group allocation and whether the work support finding was pre- or post-training. To analyse the differences in the consistency of the findings, a quantitative content analysis was conducted in three steps [15]. In Step 1, one assessor (MS) coded all of the findings independently, according to the predetermined codes (type, focus and duration of the support). If necessary, new codes were added. A second assessor (KN) independently coded 10% of the findings. Differences in coding and newly emerging codes were compared and discussed by both assessors until consensus was reached. This resulted in a final coding scheme that was discussed with the other authors (SA, HW, MD). Step 2 involved organizing the codes according to the main and sub-categories by MS and KN (for instance, coaching client or coaching colleagues/manager as sub-categories of the main category ‘type of support’). Lastly, the codes from the anonymous findings on work support needs were connected to the timing of measurement and to client A or B.

Step 3 comprised counting of the different type of findings within the three main categories: type of support (e.g. coaching client or coaching colleagues/manager), the focus of the support (e.g. support aimed at teaching certain skills or adjustments in task instruction) and the duration of the support (e.g. certain number of hours of coaching during a specific period). Within these three categories, the total number of different sub-categories of findings were counted. The total number of different sub-categories of findings in each of the three main categories were then compared across pre- and post-training measurement.

Greater consistency of findings on work support needs was defined as a decrease in the total number of different sub-categories of findings in the post-training measurement, as this would indicate fewer differences in findings with less variation among the participants. An increase in the total number of different findings would indicate the findings were becoming less consistent and more incongruous among the participants.

Results

A total of 28 participants were interviewed between July-October 2017. Of these 28 participants, 13 were male and 14 were working as labour experts. Six participants were job coaches and eight worked as vocational rehabilitation consultants or coaches. The mean interview duration was 45 minutes.

Phase 2: Consistency of the findings regarding work support needs

A total of 41 participants completed the training programme in September 2017. Two participants did not provide a pre- or post-training measurement, therefore only 39 participants were included in the analysis. Of these 39 participants, 17 were male and 19 were labour experts, 15 were vocational rehabilitation consultants and five were job coaches.

Phase one: Usability of the Work Support Needs Assessment Tool

A small majority of the participants (n = 19) stated that the tool was useful for obtaining relevant information and gaining insights regarding the work support needs of their clients. It is necessary to gather in-depth information on a variety of subjects and settings (e.g. a client’s everyday life, previous work and education experiences) for a professional to make a judgement about work support needs. The tool was found to assist the professionals in collecting this information by addressing different subjects across the 21 items and by the optional in-depth questions for each item. Participants stated that by following the tool format, client conversations became more structured.

You are forced to touch on everything. That helps a lot and gives structure. You reach a deeper level and get a better insight into the person and where the guidance is needed. (Participant 22).

I find it helpful that the instrument provides insight and structure. That is an improvement on my current way of working. If someone is new to the job I can imagine that an instrument like this is very useful. It gives you something to hold on to. (Participant 9)

Participants mentioned that certain items from the tool were very important to address, something they were made aware of or reminded of by using the tool.

The tool addresses aspects that I wouldn’t normally ask about in an intake interview, but which are certainly important in the guidance. Concerning cognitive and executive functions, for example ability to concentrate and comprehension, those are certainly important things to take into account, or in employment reintegration mediation, to see what positions are suitable. (Participant 14)

Two participants did not consider the tool to be useful because it did not provide them with information they needed. They considered it overly focused on work support, or found that certain items which they normally would consider were lacking in the tool. One participant stated that building trust and developing a bond between the professional and the client was also an important element in their work and that the tool did not facilitate this. Another participant stated that the tool did not sufficiently focus on the living conditions or the nature of the disability of the client.

It’s very standardized. That means there’s no room for grey areas. (Participant 2)

It’s the sum of things, the total picture that there may be a need for guidance. Only the 21 aspects makes for a very analytical approach. (Participant 15)

The majority (n = 26) of the participants stated that the tool could be used to assess the work support needs. The participants also appreciated the optional item-related example questions that were included, as they gave guidance on how the tool items can be converted into questions during clients’ interviews. These example questions also led to more in-depth information about the client.

Normally intake interviews are more general (what kind of work would you like, why are you motivated to find work) and now the questions go much deeper, so you get far more information about the client. For example, dealing with criticism, that’s something that most clients say they have no problem with, but now you really get to grips with things through the questions (example questions, appendix) so you get a far more honest answer. (Participant 14)

A small majority of the participants (n = 20) believed that it was possible to incorporate the tool into their own work process. However, a few preconditions were mentioned; for example, some suggested that the use of the tool should not take too much time, while others said they would only use the tool if colleagues were also using it.

Phase two: Consistency of the findings on work support needs

The results on consistency in the findings on work support needs are presented below according to three main categories: 1) type of support, 2) focus of support and 3) the duration of the support. The results regarding the second category are presented separately for Client A and Client B due to differences in the content of the vignettes.

Work support findings regarding type of support

Before the training programme, the participants produced eight different sub-categories of findings regarding the type of work support, while after the programme, the participants provided findings that covered nine different sub-categories. This indicates that consistency in findings about the type of support did not improve after completing the programme. See Table 2 for an overview of the findings on the type of support.

Table 2

Different sub-categories of findings on the type of support

	Type of support	Pre-training N (%)*	Post-training N (%)*
1	Solely coaching/support of client	19 (27%)	19 (26%)
2	Job coaching (not further specified)	16 (23%)	13 (18%)
3	Solely coaching of colleagues/employer	10 (14%)	17 (23%)
4	Sheltered work environment	7 (10%)	11 (15%)
5	Support with finding a job	8 (11%)	5 (7%)
6	Support with living/everyday life	5 (7%)	1 (1%)
7	Supervise client	4 (6%)	3 (4%)
8	Specified regulation 1	0 (0%)	3 (4%)
9	Specified regulation 2	0 (0%)	1 (1%)
10	Preparing client for work	1 (1%)	0 (0%)

*Percentage of the total number of findings on the type of support (pre-training: N = 70; post-training: N = 73). The categories are not mutually exclusive.

Findings regarding the focus of work support for Client A: Autism

In the pre-training measurement for Client A, there were a total of eighteen different sub-categories of findings regarding the focus of the support. In the post-training measurement, this number decreased to fifteen. This indicates a small increase in the consistency of the findings regarding the focus of support for Client A after the programme (see Table 3 for an overview).

Table 3

Client A: Different sub-categories of findings on the focus of support

	Focus of support	Pre-training N (%)*	Post-training N (%)*
1	Adjusted task instruction	11 (11%)	16 (21%)
2	Defined/singular/repetitive tasks	18 (18%)	15 (20%)
3	Supervision at the workplace	14 (14%)	11 (14%)
4	Coaching/teaching skills	11 (11%)	3 (4%)
5	Teaching/increasing interpersonal competences	7 (7%)	0 (0%)
6	Teaching/increasing intrapersonal competences	4 (4%)	6 (8%)
7	Individual tasks	3 (3%)	0 (0%)
8	Facilitating client	7 (7%)	3 (4%)
9	Other advice about workplace	0 (0%)	2 (3%)
10	Fixed point of contact in the workplace	6 (6%)	3 (4%)
11	Further examination of client necessary	3 (3%)	1 (1%)
12	Informing and supporting colleagues and employers	2 (2%)	3 (4%)
13	Structured/quiet workplace	3 (3%)	4 (5%)
14	Accessibility between home and workplace	3 (3%)	3 (4%)
15	Support client in area of personal life/living/network	4 (4%)	2 (3%)
16	Characteristics of supervisor	3 (3%)	3 (4%)
17	Not having deadlines in work tasks	1 (1%)	0 (0%)
18	Own workplace	1 (1%)	0 (0%)
19	Other advice	1 (1%)	1 (1%)

*Percentage of the total amount of findings on the focus of support (pre-training: N = 102; post-training: N = 76). The categories are not mutually exclusive.

Findings regarding the focus of work support for Client B: Intellectual disability

Before the training programme, eighteen different sub-categories of findings concerning the focus of support were offered for Client B. This number decreased to seventeen after the programme. Since the difference is very small, an improvement in consistency in findings regarding focus of support for Client B cannot be assumed. See Table 4 for an overview of the findings for the focus of support for Client B.

Table 4

Client B: Different sub-categories of findings on the focus of support

	Focus of support	Pre-training N (%)*	Post-training N (%)*
1	Teaching/increasing interpersonal competences	12 (12%)	17 (17%)
2	Informing/supporting colleagues and employer	15 (15%)	12 (13%)
3	Defined/singular/repetitive tasks	12 (12%)	11 (11%)
4	Teaching/increasing intrapersonal competences	11 (11%)	11 (11%)
5	Support with finding a job	10 (10%)	4 (4%)
6	Coaching/teaching skills	1 (1%)	6 (6%)
7	Fixed point of contact in the workplace	7 (7%)	3 (3%)
8	Individual tasks	1 (1%)	4 (4%)
9	Structured/quiet workplace	8 (8%)	6 (6%)
10	Support client in area of personal life/living/network	0 (0%)	2 (2%)
11	Own workplace	4 (4%)	5 (5%)
12	Facilitating client	4 (4%)	3 (3%)
13	Supervision at the workplace	1 (1%)	0 (0%)
14	Other advice about workplace	1 (1%)	0 (0%)
15	Adjusted task instruction	5 (5%)	5 (5%)
16	Further examination of client necessary	2 (2%)	2 (2%)
17	Other advice	2 (2%)	2 (2%)
18	Characteristics of supervisor	2 (2%)	2 (2%)
19	Not having deadlines in work tasks	1 (1%)	1 (1%)

*Percentage of the total number of findings on the focus of support (pre-training: N = 99; post-training: N = 96). The categories are not mutually exclusive.

Findings regarding duration of support

The participants described the duration of support in various ways. Due to the many different kinds of responses given on the duration of support, it was not possible to analyse and codify the majority of the findings and thus it was not possible to draw conclusions regarding the consistency of these findings.

Discussion

This study aimed to explore professionals’ views about the usability of the Work Support Needs Assessment Tool. Most of the participants considered the tool to be useful for assessing the work support needs of clients with disabilities. By using the tool, they obtained useful insights and in-depth information to assist in their assessment of work support needs. The tool also gave them structure and guidance during consultations with clients. A few participants did not find the tool useful, due to its standardized and analytical format or because of missing items. The majority of the participants believed that the tool could be incorporated into their assessment routines.

An overall significant increase in consistency in the professionals’ work support needs judgements could not be demonstrated in this study. The results indicate that findings about the type of support did not become more consistent after the training programme, as the total number of sub-categories of findings on types of support increased. In relation to the findings on the focus of support, a very small increase in consistency was found.

Another similar work-related support needs assessment instrument is the Work-ability Support Scale. This tool has demonstrated acceptable levels of accuracy and reliability and is considered adequate for use in clinical practice [16]. The Work-ability Support Scale can be used in the assessment of vocational rehabilitation needs when a client already has a job or is expected to start in a job. This tool addresses specific items that are also included in our tool, for instance dealing with managers, clients and colleagues, motivation to work, personal support, planning and organizing work tasks and having realistic expectations regarding work or abilities. Differences between both tools relate to the timing of using the tool in the vocational rehabilitation process. Our Work Support Needs Assessment Tool is developed to be used at the start of the guiding process, when the future workplace of the client is still unknown. It therefore produces an initial and generic work support needs judgement that can be used to guide the job searching and matching process. When there is a potential workplace in sight, the initial support needs judgment should be specified and adjusted to the actual workplace. This revised support needs judgement can be conducted by using a more specific tool, such as the Work-ability Support Scale, in which the job tasks and workplace are taken into consideration [16]. Although the majority of consulted professionals in our study had positive views about the Work Support Needs Assessment Tool, caution should be exercised when using the tool as more research is required to increase the psychometric properties of the tool.

Limitations

The Work Support Needs Assessment Tool seemed acceptable for assessing work support needs according to the majority of the professionals in our study. However, our study did not demonstrate an improvement in the consistency of findings regarding work support needs after a training programme. Explanations for this finding might be found in the study design and the way data were analysed. First, the training programme was focused on participants internalizing the items and related definitions included in the tool, along with a method of assessing these items during a client interview. For practical reasons, it was not feasible for participants to individually interview the two clients. This meant it was not possible to allow every participant to complete an individual work support needs assessment overview, as for which the tool was designed. A training programme that focused on judging the work support needs through such an individually generated overview of work support needs assessment might have had other effects on the consistency of the findings regarding work support needs. Second, consistency in work support findings was assessed through quantitative content analysis [15]. Although this type of analysis has a quantitative element (i.e. counting the times a certain word or phrase is used), in our study the sub-categories for type of support and focus of support were based on a qualitative analysis preceding the quantitative analysis. Another method of measuring differences in assessments might produce different results, for instance letting the participants tick fixed multiple choice answer categories and assessing inter-rater reliability through statistical analysis. Another limitation is that due to the small amount of findings in each sub-category of the assessments, it was not possible to conduct stratified analyses based on the type of professional. It is possible that a job coach could have assessed work support needs different from a labour expert.

One strength of this study is the large number of participants included in the entire study, and the variation in type of professions such as labour experts, job coaches or consultants. The participants appeared to be a representative sample of the different type of professionals who are involved in assessing work support needs. Furthermore, using an actor to play the role of the client in two vignettes was also a strength, as this strategy has been associated with the initiation of active learning processes, allowing trainees to experience and reflect through assignments with the actor. Active learning has been found to be an effective method for obtaining new skills or changing behaviour [17]. In addition, vocational rehabilitation professionals use all kinds of sources when collecting information about the functioning of a client. An interview with the client is the most important source of information, as it gives the professional the opportunity to ask questions and to observe, and sometimes test the clients’ functioning, responses and behaviour. A written case description of a client or a video of them would not make it possible for participants to ask questions and would therefore not be a realistic simulation of the use of the tool in practice.

Future research should focus on exploring other suitable and quantitative methods of assessing consistency of findings that follow from using the Work Support Needs Assessment Tool. An increase in consistency of work support needs findings would contribute to the reliability of work support needs assessments. Moreover, further development of the tool should also be considered; for example, the tool might be embedded in a mobile app. An app could possibly offer users more flexibility in relation to their needs and preferences and might also reduce the time it takes to fill out the questions included in the tool. This could further increase the usability of the tool.

Conclusion

The Work Support Needs Assessment Tool offers valuable assistance in the structural assessment of work support needs of clients with disabilities at the start of the guiding process, when the future workplace is still unknown. The tool provides professionals with structure and guidance during client consultations. However, in the current study design we were not able to demonstrate an increase in the consistency of professionals’ findings regarding work support needs after a training programme in the use of the tool.

Conflict of interest

The authors declare that they have no conflict of interest.

Data availability

The datasets generated during and/or analysed during the current study are not publicly available as these include Dutch text data, but are available from the corresponding author on reasonable request.

Funding

This study has been funded by a SUWI grant from ‘The Dutch Social Security Institute: the Institute for Employee Benefits Schemes (UWV)’ Grant number: K1617a.

Box 1. Description of client vignettes

Vignette 1. Client with autism	Vignette 2. Client with an intellectual disability
Gender: Male	Gender: Male
Age: 26 years old	Age: 21 years old
Education: diploma in intermediate vocational education (ICT specialization)	Education: special needs secondary education
	Previous work experience: internship in catering (canteen)
Previous work experience: various short ICT-related employment contracts	Living situation: assisted living
	Main concerns: Low IQ and limited self-reliance in areas of living, school and work. Capable of carrying out simple and defined tasks; however, needs someone who will explain and show him how to execute the task. Difficulties with prioritizing and multi-tasking and working under time pressure.
Living situation: lives on his own
Main concerns: Encounters problems with maintaining a job due to limitations in social interaction, conflict management and taking orders from colleagues with authority. Cannot deal with unexpected changes in work tasks or in the workplace.