The suitability and usability of a tool to assess contact status from COVID-19 exposures in the workplace.

OBJECTIVE: To assess the usability of a self-assessment COVID-19 exposure tool for workplaces.
METHODS: A COVID-19 exposure tool for workplaces was developed using five risk criteria. Public Health Unit (PHU) assessors who administered the tool documented when they administered the tool, the time taken for finalisation of the assessment and ease of administration. The System Usability Scale was used for workplace managers' perceptions on tool use. Data were assessed using both quantitative and qualitative analysis.
RESULTS: Eighty-four workplaces used the tool to assess COVID-19 exposure risk. Of those, the outcome provided by the tool did not require modification by the PHU assessor in 70% of workplaces. Eighty per cent of the assessments were completed by the next day. PHU assessors rated the overall ease of administration of the tool as 'easy' or 'very easy' for 85% of workplaces and indicated they would employ the tool across a number of settings including complex workplaces. The mean System Usability Scale was 82. Workplace managers were predominately positive regarding its suitability.
CONCLUSION: The tool provides an easy-to-use assessment of SARS-CoV-2 exposure in the workplace. IMPLICATIONS FOR PUBLIC HEALTH: The tool's adoption will empower workplace managers and improve the capacity of public health units to prevent further transmission of SARS-CoV-2 in workplaces.

In mid‐2021, New South Wales (NSW), Australia experienced a fourth wave of severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) infections. The three previous waves began in January, July and December 2020, and were all successfully controlled through public health intervention. The fourth wave proved more challenging, driven by the Delta variant of SARS‐CoV‐2 being more transmissible than previous variants of SARS‐CoV‐2.

The fourth wave began in Sydney on 16 June 2021. It was seeded from a single case, a worker whose job involved transporting pilots arriving from international flights to their accommodation. As of 18 September 2021, 46,250 cases had been associated with this single event. In an attempt to control the fourth wave, NSW Health invoked public health measures including shutting down non‐essential work and most retail workplaces on 28 June 2021 in greater metropolitan Sydney. Despite lockdown measures, numerous workplaces were still legally able to operate throughout the fourth wave.

Between 16 June and 28 August 2021, the South Eastern Sydney Public Health Unit (PHU), a public health unit that services the population and workplaces in eastern Sydney, had assessed more than 2,000 SARS‐CoV‐2 workplace exposures. Each of these assessments was site‐specific and involved a detailed analysis of the workplace by a PHU assessor. To undertake these assessments, the PHU assessor engaged with the workplace manager on numerous occasions over potentially a number of days. With a growing number of COVID‐19 workplace exposures generated from the Delta variant, the PHU's ability to deliver a timely assessment of these workplaces was challenged. Therefore, the need for an easily administered workplace SARS‐CoV‐2 exposure assessment tool that could quickly determine the initial exposure status of employees was required. A search of peer‐reviewed literature and Australian government agencies found a number of risk tools for COVID response, but none specifically for workplace assessment. In response, the PHU developed such a tool based on five risk factors in acquiring SARS‐CoV‐2 infection: vaccination status, indoor or outdoor exposure, mask‐wearing, contact distance and exposure time (Figure 1). , , , ,

Figure 1

The combination of risk factors that inform the decision making of the assessment tool.

PHU assessors provided the tool to workplace managers so they could undertake an initial assessment by entering the five factors in an excel spreadsheet for each employee. The spreadsheet was programmed to provide contact status (close, casual or monitor for symptoms) in line with Figure 1, based on data entered by the manager (Figure 2). In this way, the workplace manager had an instantaneous contact status which was used to inform their employees of the immediate actions each employee must take prior to connecting back with the PHU for review and finalisation of the assessment.

Figure 2

Example of the contact status output from the assessment tool.

Essential to the successful implementation of the tool was its usability by workplace managers. The System Usability Scale is a questionnaire developed to provide a measure of the usability of a given product or service. It has been administered extensively in usability studies and has several advantages including its ability to assess a wide range of interface technologies, its ease of administration, and its provision of a single score on a scale easily understood by a wide range of people.

The aim of this study was to evaluate the utility of the tool for both the workplace manager and PHU assessor.

Methods

A workplace was eligible for study inclusion if the workplace was notified to the PHU for assessment from 29 August to 19 September 2021 and had only one case linked to it at the time of notification. PHU assessors had complete discretion to decide whether or not to use the tool in eligible workplaces they were assigned to. PHU assessors were provided with a package that included a basic script, email template, user guide and the tool (Supplementary Files) – no additional training or instruction was provided. PHU assessors were asked to record the following factors when undertaking all workplace assessments during the study period: use of tool (Yes/No); and industry (construction, courier/delivery, manufacturing, small retail, large retail, office building, transport/airport or other). If the tool was used, the following information was also recorded: time taken on initial call from PHU assessor to workplace manager; whether the assessment provided by the tool required clarification or modification by the PHU assessor prior to finalisation; the date and time the assessment was sent, returned and completed; and overall ease of administration. At the end of the study period, all PHU assessors who used the tool were invited to undertake a survey regarding their experience with the tool. Questions included the features of the venue that led to the decision to use the tool and if they believed using the tool required more mental effort than not using the tool when undertaking an assessment.

All workplace managers who were given the opportunity to use the tool were invited to participate in a survey regarding the usability of the tool. This survey included the Systems Usability Scale, a ten‐statement, five‐point Likert scale survey where higher scores reflect greater usability, and an open‐ended question asking for ‘any comments you would like to make’. The survey invitation was provided to the workplace managers within a week following their workplace assessment finalisation with a follow‐up invitation provided for those who did not respond to the first invitation. The surveys were administered online through Redcap version 11.3.0.

Quantitative and qualitative analysis was undertaken including descriptive statistics, inferential statistics (chi‐square analysis) and thematic analysis. Quantitative analysis was undertaken in SAS Enterprise Guide version 8.3. Qualitative analysis was undertaken using inductive constant comparison techniques developed by Corbin and Strauss.

Results

PHU assessor

From 29 August to 19 September 2021 the PHU assessed 205 workplaces that met the eligibility criteria. PHU assessors employed the tool for 84 workplaces (41%). Analysis of tool use versus industry showed that the PHU assessors were significantly less likely to use the tool for large retail and significantly more likely to use the tool for transport/airport venues when compared to all other venues combined (Table 1). Fifty‐nine (70%) venue assessments did not require any further clarification or modification of employee exposure status prior to finalisation; 23 required some modification (28%), and the workplace managers at two venues (2%) did not return their assessments so were unable to finalise their assessment. Of those assessed, the mean initial time on the phone to the workplace manager was 12 minutes, with 80% of assessments completed on either the same or the next day (Table 2). The PHU assessors rated the overall ease of using the tool as ‘very easy’ or ‘easy’ for 85% of all venues (Table 3). The PHU assessor's choice to use the tool was based on two broad categories: workplace parameters and workplace manager capacity. PHU assessors reported they would use the tool when a workplace was large (including the number of employees) and complex. The PHU assessors would also use the tool if it appeared a workplace manager could fill out a spreadsheet, was requesting transparency in the decision making process, didn't have their own processes for identifying exposed employees, or used other systems that could integrate with the assessment tool. All but one PHU assessor (94%) found using the tool required less mental effort when undertaking the assessment, with no PHU assessor believing the tool required more mental effort.

Table 1

Assessor's tool use, and Workplace manager's response rate and System Usability Scale (SUS) mean scores per industry, with SUS interpretive scales.

Industry	Assessors tool use			Workplace manager's Response rate		Workplace manager's System Usability Scale rating
	Used (n)	Not used (n)	P value a	Total surveyed (n) b	Response rate	Mean score	Percentile rank c	Grade c	Adjective rating scale c
Construction	21	24	0.4	20	75%	86.0	97	A+	Excellent
Courier / Delivery	1	1	0.8	1	0%	N/A	N/A	N/A	N/A
Large Retail (e.g. Coles, Woolworths)	5	23	0.007	4	50%	78.8	83	B+	Good
Manufacturing	1	4	0.3	1	100%	67.5	50	C	OK
Office Building / Environment	17	19	0.4	14	64%	89.2	99.8	A+	Excellent
Small Retail	19	32	0.5	19	26%	78.0	83	B+	Good
Transport / Airport	14	0	>0.01	10	80%	72.8	67	B‐	Good
Other (e.g. Veterinary, Interior Decorating)	6	18	0.09	6	67%	80.0	88	A‐	Good
Total	84	121		75	59%	82.0	90	A	Good

Notes:

a: compared to all other industries combined

b: Some workplace managers were responsible for multiple workplaces – only the first workplace was surveyed

c: Further explanation of these scales can be found in Sauro et al. 2016 & Bangor et al. 2009

Table 2

Time from initial contact to completion.

Time	N	%	Cumulative %
Same day	28	34	34
Next day	38	46	80
Two days	10	13	93
Three days	4	5	98
Four days	2	2	100

Note:

a: Two assessments were not returned

Table 3

Assessors perceived overall ease of administration per venue.

Ease	N	%	Cumulative %
Very easy	49	60	60
Easy	21	25	85
Neither easy or hard	9	11	96
Hard	3	4	100
Very hard	0	0	100

Note:

a: A response was not provided for two venues

P value

Total surveyed (n)

Percentile rank

Grade

Adjective rating scale

%

%

Workplace manager

The 84 workplaces where the tool was used had 75 unique workplace managers. Forty‐four (59%) of these managers responded to the follow‐up survey with a range of 0–100% depending on industry (Table 1). The mean System Useability Scale for all workplaces was 82.0 with a range of means of 67.5–89.2 (Table 1).

Qualitative analysis

Qualitative analysis of the workplace manager's comments revealed the core category of suitability. All workplace managers provided feedback on the suitability of the tool through one of the discovered categories. Suitability of the tool emerged as the core category because it was sufficiently abstract so it could be used as the overarching explanatory concept tying all the other categories together. Figure 3 provides a conceptualisation of the categories and concepts and how they impact the decision of suitability, with Table 4 providing further comments which informed these categories.

Figure 3

Suitability of the tool – categories leading to its suitability.

Table 4

Further comments informing the formation of the categories and concepts.

Category / Concept		Informing comments
Praise / Support		Great tool in order for the business to identify close/casual contacts very quickly [Construction 3] Great initiative from PHU [Construction 5] A great tool for future use [Small retail 4] It's a great tool and would love to be able to use this across districts [Large retail 1]
Functionality	Individual circumstance	I found the spreadsheet really simple to use [Office building / environment 5] I also found the tool to be very user friendly [Construction 3] You should consider adding a column for physical barriers … as we have had COVID Marshalls behind screens [Construction 1] … it has unfortunately created industrial mayhem through it's use [Transport / Airport 6]
	Issues / Improvements	The coding of this could be simplified to allow bulk data dumps [Large retail 1] Whowever is developing this spreadsheet should make contact with ‘Contact Harold’ (the company that is providing the contact tracing to majority of the major projects around the city) [Construction 8] Would make it a lot easier if you could access the spread sheet using an iPad [Small retail 2]
	Operationalisation	… it's calculations might be too focused on ‘direct contact/meetings’ with a positive case [Office building / environment 5] The most significant benefit is that (while being fairly conservative) removes the subjectivity of assessing a persons contact status particularly by those not of the medical / health profession (or by those who are focused on operational deliverables) [Construction 10]
Need	Timeliness	… this spreadsheet has been requested to be completed too late [Construction 2] Please send this spreadsheet earlier! [Office building / environment 1] This spreadsheet would of been more useful and effective if provided immediately after calling NSW Health [Other 1]
	Access	… have been nagging the SESLHDPHU team for permission to use it for other exposures [Office building / environment 4] … would be good to distribute to industry so that we can conduct this tracing as soon as we are aware of a positive case to help expedite the process [Office building / environment 5] I think it should be on NSW Health website to help people when there is a case at their work [Small retail 1] This should be a mandatory document across all of NSW if the onus is on the employer to notify staff if they are casual/close [Small retail 3]
	Absence	I think it is a waste of time for me as we follow guidelines to isolate any contacts whether close or casual to positive cases [Construction 2]
Vaccination		Also, NSW Health as a whole should allow employers to use this for determining who initial close contacts are … especially in relation to vaccinated workers [Construction 1]
Adaptability	Risk tolerance
	Practice behaviours	Master Builders Association would be very interested in seeing the tool [Construction 7]
Use		You should use it for all Enquiries [Construction 7] I don't see the point in this for our business [Construction 2]

Praise or support

This category was a positive impact on suitability and was the most mentioned category:

This tool is an absolute game‐changer in terms of non‐healthcare incident managers being able to accurately classify staff following a COVID exposure…I would be happy to be contacted for further information and to provide any support I can to get this rolled out more broadly. [Office building/environment 4]

Functionality

Functionality had positive, negative/constructive and neutral impacts on suitability. It consisted of three concepts: how the functionality of the tool impacted the workplace manager's individual circumstances; comments on issues or improvements of a general nature; and the operationalisation of the tool. In considering the workplace manager's individual circumstances, the feedback was positive as well as negative/constructive.

The spreadsheet was very easy to use. [Small retail 4]

Allowing companies to put in employee numbers, and workplace address where positive individual has worked on more than one site would be helpful. [Transport/Airport 3]

There were some neutral and constructive comments on issues or improvements of a general nature.

Would have been nice to be able to sort even if it was on a separate tab. [Construction 11]

The operationalisation concept relates to a workplace manager's cognitive understanding of how the tool functions.

It ensures a consistent approach across sites and reduces subjectivity and individual bias in the assessment. It also eliminates concern WRT ensuring that we are using the latest information. The process was very efficient, extremely logical, and well‐designed to cover a range of variables (vax status, location, exposure time/distance) without creating complexity or confusion. [Office building/environment 4]

Need

Category of need relates to the workplace manager's perceived necessity for the tool. The concepts related to this category included timeliness, access and absence. Timeliness was predominately negative and involved the need for the tool sooner.

Wish I had spreadsheet on the day I needed to make decision on who needed to isolate. [Other 2]

Access was closely related to timeliness but wasn't specific to the workplace manager's circumstance, rather on a general level about the need for the community to access the tool.

NSW Health as a whole should allow employers to use this for determining who initial close contacts are. [Construction 1]

Absence involved the workplace manager's perceived lack of need for their situation.

The provided spreadsheet is just a double up as we have our own spreadsheet. [Construction 8]

Vaccination

At the time of the study, there was no clear guidance from the New South Wales Government regarding how the reduced risk of infection due to vaccination should be considered when classifying exposure status. Thus some workplace managers raised the suitability of the tool in light of this fact.

Given that it considers vaccination status as a factor, this is not supported in any official info of which we're aware…In turn, employees and their union have assumed that it is the company making an arbitrary decision to use vaccination status, which in turn has led to confusion, anger and industrial backlash. [Transport/Airport 6]

Adaptability

The category of adaptability consisted of two concepts, the adaptability of the tool as risk tolerance changes and adaptability of the practice behaviours because of the tool. It has been noticed that a change in risk tolerance, such as a decrease in separation distance, could be accommodated into the tool.

A very good tool that can be easily updated as the health orders or information in regards to transmissibility, etc. [Transport/Airport 1]

Workplace managers identified changes in workplace practices following the advent of the tool, but also the need to access the tool for this to happen.

Very useful and incorporated into our company BCP and COVID response. [Transport/Airport 1]

Use

The category of use represented comments regarding future use of the tool.

Would be comfortable to continue using the spreadsheet in the future. [Construction 4]

Discussion

The suitability and useability of the tool have been considered from the perspective of those who administered it and those who used it. We aimed to understand how the tool would be used by PHU assessors with little background briefing other than a script, template email, user guide and the tool. Overall, the PHU assessors used the tool for 40% of the workplaces they assessed during the trial period, which we consider acceptable, given it was a new process with no direction on when the tool should or should not be used. Curiously, the PHU assessors reported a complex workplace would be a criterion for using the tool. This is contrary to what the authors hypothesised the tool would be used for, believing a complex workplace would trigger a site‐specific assessment. The fact that the tool was used in that way and the predominately positive feedback on the tool bodes well for the use of the tool for complex workplaces. Another criterion for use nominated by PHU assessors was the transparency of process, suggesting that – at least in the PHU assessor's eye – the tool provided transparency of their decisions to the workplace manager. This transparency aspect was also raised as a positive point for the tool by some workplace managers.

The tool worked well, rapidly delivering an acceptable contact risk assessment in 70% of venues assessed, with 80% of workplaces assessments finalised within 48 hours. It was easy to implement, with PHU assessors reporting it as easy or very easy to implement for 85% of venues assessed. It was less commonly applied to large retail sites such as supermarket chain store assessments. However, these chains had established contact tracing processes in place that had been agreed to in part by the NSW Ministry of Health, making our tool redundant. A more common application of the tool was for the transport/airport industry, which is surprising. This industry includes those associated with the importation of goods and the workforce is highly unionised. It is possible that the tool was used to provide transparency to the workforce, which is not always provided with site‐specific risk assessments.

The PHU assessors predominately perceived less analytic input was required when using the spreadsheet to undertake an assessment. It is possible that this perception could be biased if the PHU assessors used the tool in less‐complex workplaces where less analytic input was required. However, the fact that the tool was commonly deployed to complex workplaces means this bias is unlikely.

The tool was assessed using the System Usability Scale with a mean score of 82, placing it in the top 10% of 446 studies that have used the System Usability Scale. In other words, it is perceived as more useable than 90% of the products tested in those studies, and on a grading scale would be assigned an ‘A’. An adjective rating scale has also been developed for the mean scores of the System Usability Scale, with our tool fitting the description of ‘Good’ (Table 1). The rating did vary between ‘Excellent’ for office building/environment and construction industries to ‘OK’ for Manufacturing industries, and it is possible that those in office environments and construction have a greater familiarity with spreadsheet applications, therefore finding it more intuitive.

When commenting on the tool, the workplace managers were ultimately concerned about the suitability of the tool to assess workplaces. Their comments endorsed the overall suitability of the tool, with the exception of one category regarding the absence of need. Many of the categories that emerged from the analysis were positive including praise and support for the tool, positive need for the tool, endorsing future use of the tool, operationalisation of the tool and willingness to adapt practices to encompass the tool. Other categories provided endorsement through constructive feedback to improve the tool's implementation through improved functionality and endorsement of taking into account the protective effect of vaccination.

Negative comments mainly related to the explicit inclusion of vaccination status as a mitigating risk factor. As this was the first time fully vaccinated workers were formally considered at lower risk than their similarly exposed colleagues, there was apparently concern in some workplaces, whereas others welcomed the initiative.

This study has limitations. The response rate of the workplace managers’ survey was 59% overall but only 26% for small retail. It is possible that the lower response rate in small retail could reflect their ability to use computer‐based applications and therefore potential difficulty with using the spreadsheet, which has not been captured by the survey. On a broader scale, it is possible that those who did not respond may represent those who have lower English proficiency or limited computer skills or computer access. This survey was not anonymous, as it was important to understand issues across the different industries. This may have led to those who identified negative issues being less likely to respond.

[Addendum – The tool has proven to be adaptable and flexible to changing definitions and risk criteria over time. For example, NSW Health no longer considers contacts as close or casual, rather as high, medium and low risk. Since the completion of this study, this tool has been successfully adapted to changed risk settings and adopted by the New South Wales Government as its key tool for the assessment of COVID‐19 exposure in workplaces. ]

Conclusions and implications for public health

The tool provides an easy to use public health assessment of SARS‐CoV‐2 exposure in the workplace. The adoption of this tool will facilitate a rapid assessment of workplaces and earlier intervention to stop the transmission of SARS‐CoV‐2. Further, the tool could extend beyond SARS‐CoV‐2 exposure. With modifications, the tool could be applied to other infectious diseases in the workplace such as hepatitis A.

Acknowledgements

The authors thank Lara Cooke, Ally Stagni, Wade Dillon, Michelle Carvalho‐Mora, Kate Sewell, Natalie Tuffin, Stefanie Dietsch, Irene Sangadi, Kristie‐Lee Smith, Angelique Butler, Samara Lewis, Tom Munro, Freya Grove, Nafisa Hai, Lisa Maude of the South Eastern Sydney Public Health Unit ‐ Venue Assessment Team and all the workplace managers who participated in this investigation.

Additional supporting information may be found in the online version of this article:

Supplementary File 1: Email template.

Supplementary File 2: Initial workplace assessment use guide

Supplementary File 3: Initial workplace assessment.

Supplementary File 4: Script.

Supporting Information

Click here for additional data file.

Supporting Information

Click here for additional data file.

Supporting Information

Click here for additional data file.

Supporting Information

Click here for additional data file.