Solutions for Shared Resource Lab Remote Quality Control and Instrument Troubleshooting during a Pandemic.

The COVID-19 pandemic has dramatically affected shared resource lab (SRL) staff in-person availability at institutions globally. This article discusses the challenges of ensuring reliable instrument performance and quality data output while facility staff and external service provider on-site presence is severely limited. Solutions revolve around the adoption of remote monitoring and troubleshooting platforms, provision of self-service troubleshooting resources specific to facility instruments and workflows, development of an assistance contact policy, and ensuring efficiency of limited in-person staff time. These solutions employ software and hardware tools that are already in use or readily available in the SRL community, such as remote instrument access tools, video hosting and conferencing platforms, and ISAC shared resources.

Health Insurance Portability and Accountability Act

quality control

standard operating procedure

shared resource laboratory

The COVID‐19 pandemic has put cytometry shared resource lab (SRL) managers in the difficult situation of needing to ensure reliable performance of their instruments, while simultaneously mitigating the safety risk to staff and users (1). Many facilities around the globe have responded to this situation by continuing to provide access to instrumentation and ensuring quality data output, but often with strict limits on the number of lab occupants and the hours that staff members are able to physically be present in the lab (2). With these limitations on in‐person staff time, SRLs have had to rely more heavily on the end‐user to assist in the quality control (QC) process and on remote support staff to troubleshoot issues. Another challenge has been in the procurement and installation of new instrumentation. As the pandemic has advanced, strategies beyond simply delaying facility improvement has become increasingly necessary.

This communication discusses resources for self‐service instrument troubleshooting and QC, modes of remote support between SRL users and staff, and remote support between staff and external service providers. This includes an evaluation of available communication tools, a workflow for determination of successful resolution of issues and QC, and the presentation of an effective example of vendor‐provided remote assistance during instrument installation. We propose that with a collaborative approach relying on synergistic contributions from users, staff, and external service providers, SRLs can maintain a high level of instrument uptime, performance, and can expand their instrumentation and services during times of pandemic.

Resources for End‐User Self‐Service and Troubleshooting

The ability of end‐users to perform instrument QC and identify, diagnose, and correct issues is of critical importance during a pandemic. Self‐service troubleshooting can be considered the first line of defense when concerns arise. Instrument vendors have also made great advances in user‐friendliness of instrument operation and QC, facilitating end‐user independence (3). Establishing a system within SRLs to empower end‐users to assist in these roles will be advantageous as more SRLs strive to provide expanded services and throughput without a commensurate expansion of staffing (4, 5, 6). However, SRL managers must also set clear limits on which procedures should be attempted by users and which should require staff consultation.

The foundation of effective collaboration between SRL stakeholders is a solid communication strategy (7). While face‐to‐face communication and demonstrations may have comprised a large portion of SRL user and staff interactions, alternative means of communication have increased in importance. SRLs should not rely on one single communication platform, but rather spread their messaging across multiple modalities including email listservs, SRL websites, physical signage, lab management software platforms, presentations, social media, and personal communication. Despite reliance on multiple means of communication, it is paramount that SRL managers ensure the information presented is current and consistent.

An essential component of user independence and self‐reliance is a robust training program and accessible standard operating procedures (SOPs). Considerations for user and staff training during COVID‐19 is covered in a concurrent article in this issue (8). Two important resources SRLs should develop and distribute are (1) written instrument SOPs that include QC and troubleshooting considerations and (2) video SOPs.

Written SOPs

The format and content of SOPs have been covered extensively in the ISAC SRL Best Practices publication (9). A new resource developed by the ISAC SRL Services Committee is the SRL SOP Repository (https://archive.org/details/@isac_srl). This repository contains SOPs on instrument operation, QC, and troubleshooting for a variety of cytometers across multiple institutions. The materials uploaded to the repository are reviewed for appropriateness by the SRL Services Committee, ensuring this resource serves as a powerful, contemporary, and high‐quality template for SOP development. In addition to SOPs followed under normal operational conditions, SOPs generated during the pandemic to address issues such as reduced in‐person staffing, workspace decontamination, and access and spacing restrictions must be considered and are covered in another article in this issue (10).

Video SOPs

As a supplement to written SOPs, video SOPs of basic instrument setup, QC, and troubleshooting have proven to be extremely useful for visually oriented SRL users. Video SOPs functionally distinguish themselves from the many theoretical training videos that discuss the techniques of cytometry in abstract terms. Video SOPs should present a brief step‐by‐step process for instrument startup, QC, shutdown, and/or troubleshooting of commonly encountered issues. Care must be taken to ensure the video SOPs do not contradict written SOPs. Video SOPs should have similar information to written SOPs including:

Title slide or accompanied written description

Purpose

Date of establishment

Revision number and date of revision

Table of contents—included as a timestamp of content transitions

Link or reference to a written SOP

While some video SOPs may be helpful to users across institutions, it is advisable for each facility to generate videos that accurately reflect each facility's instruments and SOPs. Videos should be housed on reliable and accessible web video hosting platforms or a secure local source if public viewability is not desired. YouTube serves as a good hosting platform due to its ubiquitous nature and free channel hosting ability. For instruments that are not networked, storage of video files in a readily accessible location on the workstation is also an effective option.

Workflow for Remote QC Performance

Developing detailed SOPs is critical for self‐service operation for users. However, equally important is making SOPs accessible and known to users. A major component of this will come in the training stages of instrument use where trainees should be instructed on how to locate and utilize instrument SOPs (8). Printed, laminated SOPs may be affixed on or near each instrument to ensure ease of access when troubleshooting needs arise. Additionally, if using video SOPs, placing an easy‐access link to them on the workstation computer as well as on the SRL website is recommended.

SRL users should perform QC prior to use of SRL instrumentation. Each SRL must develop QC procedures for each of their instrument types guided by accepted best practices (11, 12, 13, 14). For each instrument, clear criteria for determining if the instrument has passed QC must be communicated to users. This will entail reference to a baseline performance report often generated using fluorescent bead standards and comparison of measures such as coefficient of variation values for each channel. It is important that the result of this metric is relayed back to SRL staff in order to confirm instrument performance specifications are passing standards set by both the SRL and instrument manufacturer. User QC measurement can be accomplished by:

Ensuring the SOP and reagents for QC are available for ease of use

Providing users with metrics for determining QC pass/fail

Facilitating user submission of the QC report to the SRL staff for review and record keeping

Following a workflow that is provided by SRL Staff (Fig. 1)

Fig 1

Workflow of user‐performed instrument quality control. When users are entrusted to perform instrument QC, the workflow is laid out in the SOP with a decision tree to guide the user through the QC process. In case of a QC fail, the user shall contact the SRL staff who can perform remote troubleshooting to achieve a QC pass or decide what further action to take if QC continues to fail. [Color figure can be viewed at wileyonlinelibrary.com]

Staff Support

Each SRL must clarify which procedures and troubleshooting measures are allowable to be performed by their users, and this must be effectively communicated to them. Care must be taken when balancing the need to facilitate self‐service with the risk that users may worsen issues, jeopardize the integrity of their data, or damage instrumentation. It is advisable to develop a decision tree for troubleshooting that guides users through the proper sequence of interventions and allowed self‐service measures. When self‐service is not appropriate, SRL staff must be able to support user needs through a combination of in‐person and remote support.

Staff In‐Person Support

As both SRL staff and end users may be subjected to local travel restrictions, room occupancy limits, and curfews, it is imperative that clear scheduling of times when SRL staff will be on site is communicated (15). This will assist in both scheduling of experiments for which SRL staff will be required, when instruments may need to be taken offline for hands‐on troubleshooting or maintenance, and when QC can be verified in person. Required information includes:

When staff will be physically present on site

Which staff operated instruments will be available when staff are on site

Which instruments will be unavailable when staff are on site

Which instruments cannot be simultaneously utilized due to proximity

The purpose of this is to allow the SRL staff to perform robust QC, maintenance as required and provide in‐person assistance that has been requested by end‐users.

Remote Support

While remote support by facility staff may have been commonplace for many SRLs prior to COVID‐19, the pandemic has made it an increasingly essential service. A successful SRL must be extremely responsive and be able to utilize technological aids to best serve users remotely.

Staff contact policy

It is imperative that SRLs clarify to users how their remote support functions and make that information widely accessible. This information should be available to all SRL users on the SRL website and include the following information:

The hours of remote support availability and by whom

Contact information for remote support

Estimated response times

What technological aids may be required for remote support

Remote support aids—Hardware

While some facilities may find that telephone, text, or email are sufficient to handle most issues that arise for users when staff are remote, other problems can be more effectively solved with the aid of advanced technological support.

Webcams

Generally, workstations connected to instruments do not have integrated webcams. Therefore, to provide remote troubleshooting solutions, SRLs can invest in webcams to promote efficient visualization of system components between users and off‐site SRL staff. The desirable key features of webcams for remote troubleshooting are:

High definition camera—HD or Full HD

Robust autofocus

Built‐in Microphone for two‐way communication

At least 2 m of cable to allow the on‐site user to provide a real time visualization of the entirety of the instrument

Webcams with the above‐described features are usually available at low to moderate cost, but in the COVID‐19 environment may be difficult to source due to increased demand and supply chain issues. This lack of availability is a potential limiting factor for SRLs that need to provide remote support. Figure 2A illustrates the use of a webcam providing a constant video feed, enabling the remote SRL staff to observe and instruct on‐site personnel how to perform basic optical adjustments to a flow cytometer.

Fig 2

Remote support tools for instrument troubleshooting. (A) A webcam connected to the instrument workstation allows users and SRL staff to troubleshoot instrument hardware issues without the need for additional mobile devices. (B) Applications, such as TeamViewer Pilot, on a dedicated troubleshooting device enable SRL staff to remotely diagnose and troubleshoot issues using tools such as augmented reality. This is especially relevant when instrument workstations operate on outdated operating systems or are not networked. [Color figure can be viewed at wileyonlinelibrary.com]

Dedicated troubleshooting smart device

Another solution to visualizing cytometry hardware issues is through the use of tablets or phones. While nearly every SRL user will have a personal smartphone, there are challenges to relying on their device to connect. These challenges include:

Incompatible operating systems or installed apps

Unreliability of mobile data networks or Wi‐Fi to user devices

Unfamiliarity of staff or users with each other's technology

Institutional regulations applicable to personal mobile devices

When instrument troubleshooting is needed, it can already be a highly stressful and frustrating situation that should not be compounded by technological issues with the remote support tools. For this reason, each SRL is advised to develop a SOP for remote troubleshooting that utilizes workstation‐connected hardware/software or a SRL‐provided troubleshooting device. This will be highly dependent on institutional preferences and support options of various remote meeting platforms, as discussed below. The remote troubleshooting SOP should be extensively tested between facility staff under as diverse circumstances as possible prior to adoption with users. Below is an example of a solution to minimize software and hardware compatibility issues:

SRL provides a single dedicated mobile troubleshooting device, for example, a tablet or smartphone that is connected to your institutional Wi‐Fi network

Ensure that the troubleshooting device is stored in an assigned area, always charged, and the login credentials are easily available to all users

Provide a sturdy protective case for the device to avoid accidental damage by users and that can be decontaminated per laboratory protocol

Install facility‐preferred troubleshooting platform and make sure it is easy to access

Develop a remote device troubleshooting SOP and make it readily available to users in proximity to the device, on the SRL website, and/or via other communication methods

Remote support aids—Software

Web‐based meeting and sharing tools are now a ubiquitous part of life: they are used daily for personal and/or business use. Considering institutional preferences and software licenses, personal preferences, and end‐user platform familiarity, there are many meeting and sharing platforms that can be chosen by SRLs. Another aspect to consider is whether the platforms work on local network or must communicate with external servers. Here we briefly discuss some of the platforms regularly utilized in SRLs with specific attention to their features in the context of remote troubleshooting and QC (Table 1). This is not an exhaustive list, but rather a reflection of the predominant SRL‐utilized platforms during the COVID‐19 pandemic.

Table 1

Software platforms utilized in SRLs for effective remote troubleshooting and QC

Software platform	Key features	Screen and file sharing	Remote control	Multiple mobile OS access	Security and privacy measures	Weblink
AnyDesk	Online meetings, presentations and sharing; Enterprise version can operate on local network without internet	Yes	Yes	Yes	Remote connection; secured end to end; verified connections	https://anydesk.com/en/enterprise
Google Meet	Online meetings without additional software to install	Yes	Yes *need to install Google Meet Chrome extension	Yes	Data encryption; anti‐hijacking controls; meeting codes; anonymous users not allowed	https://support.google.com/meet
Microsoft Teams	Online meetings full integration with Office 365	Yes	Yes	Yes	Data encryption; multifactor authentication	https://support.microsoft.com/teams
TeamViewer	Remote access and support; Video and audio communications	Yes	Yes	Yes	Remote connection; fully encrypted; password encryption	https://www.teamviewer.com
TeamViewer Pilot	Video and audio communications with augmented reality	Yes	Yes	Yes	Remote connection; secured end to end	https://www.teamviewer.com/solutions/augmented‐reality‐remote‐support
Webex	Video conferencing and online meetings with HD video and audio	Yes	Yes	Yes	Data encryption; multilayer security	https://help.webex.com
Zoom	Online meetings with HD video and audio	Yes	Yes	Yes	Data encryption; meeting password; meeting access control	https://support.zoom.us

Generally, these platforms offer online meetings that can facilitate audiovisual communication between SRL users and staff and enable sharing of the entire desktop, certain applications, and files. These platforms also allow remote access and control of instrument workstations, which are useful features to effectively troubleshoot instrument issues. In addition, when operated through a workstation, these platforms can permit a HD camera to be hand operated by the user, enabling SRL staff to assist in troubleshooting instrument hardware issues.

Most platforms work across numerous operating systems and devices including Windows, macOS, Linux, iOS, iPadOS, and Android. However, it is important to verify the compatibility of these software platforms with versions of operating systems running in instrument computers. Legacy operating systems such as Windows 7 and Windows XP may not support these platforms. Regarding security and privacy, these platforms allow secure meetings with data encryption, password, and access controls.

AnyDesk

AnyDesk is a remote desktop software and the Enterprise version has the key advantage of working solely on a local network and not having to communicate with external servers. This configuration may be advantageous to SRLs that handle sensitive data.

Google Meet

As a response to the COVID‐19 pandemic, Google Meet advanced features such as screen sharing and remote control, that previously were accessible only to enterprise and education customers via G Suite, became available to the general public via installation of the Google Meet Chrome extension.

Microsoft Teams

Microsoft Teams is integrated with Microsoft's cloud‐based identity and access management service, Azure Active Directory. For remote desktop control in Microsoft Teams, as in Zoom, SRL users can give remote control to staff only following desktop sharing.

TeamViewer

TeamViewer is a free platform (for noncommercial use) that allows full remote control of instrument workstations. TeamViewer connections are fully encrypted and this platform implements password encryption with the secure remote password (SRP) protocol. A new random password is generated for each session. However, with an account, SRL staff can consistently connect to saved partner devices without the need to re‐enter session passwords.

TeamViewer Pilot

TeamViewer has recently released an augmented reality application called TeamViewer Pilot. The user is able to share a video feed from their mobile device that can be annotated by both the SRL staff member and user in real time with augmented reality overlay tools, such as drawings, arrows, and text (Fig. 2B). This can be effectively used to visualize and correct issues with instrument hardware.

Webex

Webex is a platform that allows real‐time HD video sharing across workstations and smart devices. It has rigorous data security and enables tools such as screen and file sharing, an interactive whiteboard which can be stored on the cloud. SRLs may find it especially useful if their institution already relies on Cisco services for their IT infrastructure.

Zoom

Zoom has the benefit of being a broadly familiar platform that is user intuitive and has readily available online resources such as a help center, live and recorded trainings, and a blog. The SRL user can hand over control of a workstation to staff; however, remote control is not allowed when using an iPad or Android device.

Remote support caveats and security

For effective troubleshooting, a meeting and sharing platform should offer not only efficient and secure video communication but also secure remote access and control. A key limitation to utilization of web‐based support is the prevalence of antiquated operating systems or lack of connectivity of many cytometer workstations. Institutional IT departments should always be consulted prior to networking workstations to ensure they comply with their security policies. All remote connectivity tools used should also comply with local IT department guidelines. Due to the plethora of meeting and sharing platforms available, each SRL should choose the most appropriate one based on features, institutional preferences, and compatibility with end‐users. It is important to consider that the one‐on‐one privacy expectation during video sessions between users and off‐site staff may be difficult to enforce. Remote access and control of cytometer workstations is extremely useful. However, it must be used cautiously due to the sensitive nature of the resident data or other proprietary information. For this reason, remote access to workstations may be limited to facility staff or vendor support.

External Service Providers

SRLs recognize the importance of skilled external service providers to ensure dependable operation of their cytometry instrumentation. Whether the SRL utilizes the instrument manufacturer, third party service providers, or a combination of the two, this pandemic has necessitated the development of procedures for facilitating safe in‐person service visits as well as an increased reliance on remote vendor support.

Options for External Support

While many SRLs have delayed nonessential services such as preventative maintenance visits, the need for emergency service has continued as SRLs continue to serve their clients. Emergency in‐person service visits are continuing in most regions of the world. When interfacing with service technicians in‐person, SRLs must:

Ensure external service providers are allowable on‐site personnel at their institution

Share any pertinent operational information for the facility, such as required health screening, personal protective equipment, or distancing measures

Accommodate any vendor requirements, such as safety documentation

Remote service has emerged as an important tool for SRLs and service providers. In some regions, it is the only available option based on travel restrictions. The following are important considerations about vendors' remote service platforms:

Security and patient privacy:

Is the communication session encrypted and with what encryption standard?

Is it compliant with laws protecting sensitive patient information such as the US Health Insurance Portability and Accountability Act (HIPAA), the European Parliament Directive 95/46/EC, or other local patient privacy requirements?

Diagnostics and remote control:

What instrument diagnostics can be collected?

To what extent can the service provider remotely control the system?

Can software be remotely managed and updated?

Cost and Availability:

Is the remote service platform availability geographically restricted?

What are vendor technical support hours and how do they align with various time zones?

Is remote service only available to customers with active service contracts?

Is there a fee for remote service?

Table 2 highlights these key considerations regarding remote service platforms from several major cytometer manufacturers. All vendors included provide secure platforms that are internationally available with the purchase of an instrument service contract.

Table 2

Vendor platforms for remote instrument assistance

Cytometer vendor	Platform name	Passive instrument diagnostics collected	Remote control access	HIPAA compliant
BD Biosciences	Assurity Linc	Yes	Yes	Yes
Beckman Coulter Life Sciences	Beckman Connect	No	Yes	Yes
Bio‐Rad Laboratories	BRiCare	Yes	Yes	Yes
Cytek Biosciences	Third party remote connection tools	No	Yes	Possible
Luminex Corporation	Third party remote connection tools	No	Yes	Possible
Sony Biotechnology	Third party remote connection tools	No	Yes	Possible
Thermo Fisher Scientific	Digital Service Innovations	Yes	Yes	Yes

A Successful Example of Remote Service

Instances of successful utilization of vendor remote support platforms have exemplified how instrument manufacturers have supported SRLs during the COVID‐19 pandemic. Where allowed on‐site, service engineers have successfully followed institutional requirements such as appropriate use of Personal Protective Equipment. However, travel restrictions have required some services to occur fully remotely. Highlighting the importance of support via remote, web‐based troubleshooting to assist SRL operations, many instrument manufacturers have used virtual platforms such as BD Assurity Linc (BD Biosciences), BeckmanConnect (Beckman Coulter), and BriCare (BioRad). Others such as Luminex Corporation, Thermo Fisher Scientific, and Cytek Biosciences have used third party communication web tools for secure audio and video interactions during instrument troubleshooting (Table 2).

In a specific example, full remote support was required during the installation of a BD FACSAria III cell sorter at the National Center for Primary Care and Allergy Research at the University of Sri Jayewardenepura, Sri Lanka. This was accomplished by BD Biosciences, along with their local channel partners, using a combination of tools to facilitate the entire installation process. These tools included remote desktop control for continuous instrument monitoring and the use of a third‐party platform, Help Lightning, to create a virtual interactive presence utilizing on‐site mobile devices (Fig. 3A). The two software platforms allowed a remote engineer to collaborate with on‐site technicians to effectively facilitate remote instrument and workstation management, in addition to the live video feed to successfully install the instrument. A combination of different web‐based tools (Microsoft Teams and BeckmanConnect) have been utilized by Beckman Coulter for instrument troubleshooting their customers' SRLs (Fig. 3B). Remote vendor service is important beyond a restricted travel environment, including situations where in‐person service engineer presence is not feasible due to laboratory accessibility issues.

Fig 3

On‐site and remote service provided by BD Biosciences and Beckman Coulter. (A) BD Service using online video calling support to instruct local engineers on BD FACSAria III installation through the Help Lightning remote access tool. (B) Beckman Coulter providing support by taking remote control access of the institution's cytometer through BeckmanConnect and communicating with the SRL staff through Microsoft Teams. [Color figure can be viewed at wileyonlinelibrary.com]

Conclusion

QC and instrument troubleshooting are fundamental responsibilities of the SRL. Here, we propose a three‐tiered system toward providing continuity of instrument performance during a pandemic. First, SRLs must empower users to know which tasks are allowed to be performed in the absence of on‐site staff and provide reliable SOPs to accomplish them. Second, SRLs must provide staff support, both in‐person and remotely, when self‐service is not appropriate. And third, SRLs must work with external providers to ensure safe and reliable service options that leverage vendor expertise and remote service tools.

During a fluid situation, SRLs must continuously evolve their strategies to address QC and troubleshooting based on their institutional guidance and conditions on the ground as the pandemic progresses. Feedback from users, staff, and other stakeholders is an important component of this process. Communication and solicitation of feedback from these groups in the form of email, surveys, and advisory committee recommendations should guide the decision making of SRL managers.

The COVID‐19 pandemic has forced SRLs to confront operational challenges extremely rapidly. It has made obvious the need for remote support options and has galvanized SRLs to broadly adopt new SOPs and communication technologies. While these needs existed before COVID‐19, this pandemic has provided the spark for many SRLs to strengthen their operations and establish policies and procedures that will serve them well into the future.

Author Contributions

David Gravano: Project administration; writing‐original draft; writing‐review and editing. Uttara Chakraborty: Writing‐original draft; writing‐review and editing. Isabella Pesce: Writing‐original draft; writing‐review and editing. Michael Thomson: Supervision; writing‐original draft; writing‐review and editing.