Effect Analysis of "Four-Step" Training and Assessment Tool in the Prevention and Control of COVID-19.

Purpose: To improve the ability of infection prevention and control (IPC) of medical staff during the COVID-19 epidemic period, the "four-step" mode of whole staff training and assessment was used.
Methods: During the period from March 9 to March 18, 2020, 5425 medical staff from The First Affiliated Hospital of Nanjing Medical University were selected as the objects of this study. There are four stages in the training assessment mode. The first stage is the basic assessment stage; the second stage releases the electronic version of the knowledge point manual; the third stage conducts online exercises; the fourth stage conducts the final assessment.
Results: In the first stage, the participation rate of medical staff was 95.04%. In the fourth stage, the participation rate of medical staff was 98.01%. The average score of female medical staff in the first stage and the fourth stage was higher than that of males (P< 0.001). The average score of medical staff under 30 years old in the first stage and the fourth stage was higher than that of other age groups (P< 0.05). In the fourth stage, the correct rate of each part of exercises in the knowledge points of IPC was higher than that in the first stage (P< 0.001). In the two stages, the two parts of "COVID-19 prevention and control" and "multi-drug resistant bacteria prevention and control" had the highest accuracy, while the "disinfection and sterilization" and "infectious disease management" had the lowest accuracy (P< 0.001).
Conclusion: The "four-step" infection control training assessment mode has realized "full participation" and "effective training", and the level of medical staff's IPC has been significantly improved.

Introduction

The Corona Virus Disease 2019 (COVID-19) has spread rapidly around the world, World Health Organization (WHO) considers COVID-19 to be a pandemic and the spread of the COVID-19 pandemic is accelerating.1 In this severe struggle without smoke and mirrors, medical staff is the backbone of the fight against the epidemic, Infection Prevention and Control (IPC) work has been further given high priority.2 In the Notice on Further Strengthening Infection Prevention and Control in Medical Institutions during the Epidemic (National Health Office Medical Letter [2020] No. 226) issued by the National Health Committee, it is stated that IPC in medical institutions should be further strengthened, cross-infection should be minimized. Besides, the Basic System of Infection Prevention and Control in Medical Institutions (Trial Implementation) (National Health Office Medical Letter [2019] No. 480) should be seriously implemented, and strictly carry out pre-post IPC training for all staff, involve the standard prevention concept into all daily work, do a good job of disinfection, isolation, prevention of occupational exposure, medical waste management and other work, and reduce the risk of nosocomial infections.

Healthcare-associated infections (HAIS) patients have long hospital stays, high morbidity, and mortality rates, which provide an increased economic burden on health care systems.3 With the improvement of IPC standards in healthcare facilities, the proportion of preventable HAIS may decrease with the strengthening of IPC interventions, and the incidence of HAIS associated with multilevel IPC interventions has the potential to be significantly reduced by 35% to 55%.4 The IPC knowledge of the medical staff is the basis for compliance with all standard practices and reduction of HAIS, so training of medical staff on IPC knowledge is crucial.5

The current IPC training methods in medical institutions are usually theoretical combined with practical teaching methods, and research has shown that the rate of active learning and interest of medical staff in this type of training is not high, and the lack of supervision mechanism through the online training makes the training effect poor. To explore and carry out a more effective method of IPC training in medical institutions and to improve the effectiveness of all medical staff training on hospital-acquired IPC, the First Affiliated Hospital of Nanjing Medical University has gradually formed a “four-step” training assessment mode for all staff training on infection control, which has greatly improved the IPC awareness rate of all medical staff in the hospital, reduced the incidence of HAIS in the hospital, and had achieved good results in the hospital. The content of the “four-step” training assessment mode includes 1) mapping assessment of the medical staff’s mastery of IPC knowledge; 2) electronic version of knowledge points of IPC for medical staff; 3) online practice of the knowledge points of IPC; 4) final assessment for all medical staff. The first stage, the third stage, and the fourth stage are the same question bank (details shown in Method). However, the effectiveness and application frequency have not been fully confirmed.

In this study, we aim to apply this “four-step” training and assessment tool to the IPC training of all medical staff during COVID-19 and provided the reference for IPC training in other medical institutions, providing an original approach for medical staff training to prevent and control COVID-19.

Research Object and Method

Research Object

From March 9 to 18, 2020, a total of 5425 medical staff (1669 doctors, 3129 nurses, and 627 medical technicians) from the First Affiliated Hospital of Nanjing Medical University were included as the research object for the training of knowledge related to IPC. The training included all clinical and medical technical departments (Department of Radiology, Department of Pharmacy, Department of Medical Laboratory Science, Department of Pathology, Department of Ultrasound Diagnosis, Department of Blood Transfusion, Department of Nuclear Medicine, Department of Nutrition, Forensic Science Center, etc.), while other staff (administrative staff, property staff, logistical support staff, etc.) were not mandatory. The basic information such as gender, department, work number, age, and title of all research objects was collected, but medical staff with incomplete or incorrect basic information were excluded.

Research Method

According to the General Office of the National Health Commission “on the Issuance of the COVID-19 Prevention and Control (Sixth Edition)” “the COVID-19 Diagnosis and Treatment (Trial Seventh Edition)” and the relevant norms, standards and guidelines released in recent years at home and abroad requirements, 12 infection management professionals established IPC-related knowledge exercises and “Should Be Known and Done” knowledge points. The department (Infection Management Division) discussed collectively and finally verified that there were no errors in formulating the basic knowledge of IPC (100 questions), the knowledge of COVID-19 prevention and control (66 questions) and the electronic version of the manual of IPC “Should Be Known and Done” knowledge points. Powered by , the question bank was designed and produced with QR code, and the notification (QR code and “Should Be Known and Done” electronic manual) was sent through the WeChat group of doctors/nurses and the hospital intranet OA system, requiring all clinical and medical-technical departments to organize medical staff to use cell phones to scan the QR code and participate in the training and assessment. The test data were uploaded to the online examination system of “”, the relevant information such as gender, age, title, and correct rate of IPC knowledge points, quantitative data were collected and analyzed.

Training Assessment Content

The training question bank has 166 questions, including a total of 9 parts: 1) COVID-19 prevention and control (knowledge about epidemiology, personal protection, disinfection, and isolation, etc.), 2) basic knowledge of hospital infection (hospital infection judgment, standard precautions, outbreak management, three management prevention and control (grid-based management services, refined management services, and humane management services), aseptic operation, surgical prophylaxis, ICU management, etc.), 3) hand hygiene, 4) occupational exposure prevention and management, 5) disinfection and sterilization, 6) medical waste management, 7) standardization of sample testing, 8) prevention, and control of multi-drug resistant bacteria, 9) infectious disease management, etc. The electronic version of the manual “Should Be Known and Done” includes knowledge of prevention and control of COVID-19, basic knowledge of hospital infection (hospital infection and its related regulation, three management prevention, and control, surgical site infection, infection outbreak, etc.), hand hygiene, sterilization, isolation, occupational exposure and protection, preventive and control measures of key sites, basic knowledge of multi-drug resistant bacteria, medical waste, prophylactic application of antimicrobial drugs in the perioperative period, infectious disease management, etc.

Training and Assessment Tool

The “four-step” mode of whole staff training and assessment (the “four-step mode”) is divided into four stages: the first stage (March 9–11) is to conduct a mapping assessment of the medical staff’s mastery of IPC knowledge; the second stage (March 12–14) is to release the electronic version of knowledge points of IPC for medical staff to study; in the third stage (March 15–16), the online practice of the knowledge points of IPC was conducted, and medical staff could practice as many times as they wanted; in the fourth stage (March 17–18), the final assessment was conducted for all medical staff. The first stage, the third stage, and the fourth stage are the same question bank. The participation of the fourth stage assessment and the assessment results will be included in the year-end assessment of the department and the department director (Figure 1).

Figure 1

“Four step” training and assessment mode flow of all staff.

Training Assessment Rules

Through the “” online examination system setting assessment rules: 1) each assessment of 20 questions randomly issued in the question bank, each question 5 points, out of 100 points; 2) each assessment time limit of 20 minutes; 3) the network will be blocked during the assessment process. The first stage and the fourth stage (assessment stage) can only be assessed once per person, and the third stage (question bank practice stage) has an unlimited number of assessment exercises.

Statistical Analysis

The data of medical staff assessment and practice were exported from the online examination system of “”, statistical analysis was performed using stata 15.0 software. Comparative analysis was performed to assess the indicators of medical staff’s performance in the two phrases before and after the training, including gender, age, title, and correct rate of IPC knowledge points. Quantitative data were expressed as mean ± standard deviation, and t-test or ANOVA was used for comparison between groups; qualitative data were expressed as a percentage, and χ2 test was used for comparison between groups. All statistical tests were two-sided, and P < 0.05 was considered a statistically significant difference.

Results

(i) The participation of all staff training and assessment of IPC

The total number of questionnaires with complete basic information in the first stage was 5325, in which 5156 medical staff participated, and the participation rate of medical staff was 95.04% (5156/5425). A total of 16,261 questionnaires were completed in the third stage of the questionnaire investigation. The total number of questionnaires with complete basic information in the fourth stage was 5577, in which 5317 medical staff participated, and the participation rate of medical staff was 98.01% (5317/5425). The composition ratio of the physician between the fourth and the first stage was statistically significant, the proportion of physicians with higher professional titles was larger in the fourth stage than in the first stage (P < 0.05, Table 1).

Table 1

Infection Prevention and Control Training Assessment Participants

Classification	Stage 1		Stage 4		χ2	P-value
	Quantity	Proportion (%)	Quantity	Proportion (%)
Gender					0.655	0.418
Male	1028	19.31	1111	20.86
Female	4297	80.69	4466	83.87
Age (years)					7.034	0.071
<30	2373	44.56	2366	44.43
30~	1959	36.79	2078	39.02
40~	598	11.23	674	12.66
50~	395	7.42	459	8.62
Physicians	1388	26.07	1351	25.37	2.723	0.099
Residents	484	34.87	334	24.72	43.693	<0.001
Attending Physician	368	26.51	408	30.2
Associate Chief Physician	285	20.53	321	23.76
Chief Physician	211	15.2	267	19.76
Other	40	2.88	21	1.55
Nursing staff	3232	60.69	3369	63.27	0.093	0.76
Nurse	2075	64.2	2187	64.92	1.637	0.802
Supervising Nurse	852	26.36	867	25.73
Associate Chief Nurse	182	5.63	189	5.61
Chief Nurse	49	1.52	59	1.75
Other	74	2.29	67	1.99
Medical Technicians	536	10.07	597	11.21	1.194	0.274
Technician	257	47.95	277	46.4	0.608	0.962
Supervising Technician	157	29.29	173	28.98
Associate Chief Technician	68	12.69	83	13.9
Chief Technician	27	5.04	31	5.19
Other	27	5.04	33	6.16
Other personnel	169	3.17	260	4.88	15.962	<0.001
Junior	50	29.59	70	26.92	3.779	0.437
Intermediate	36	21.3	77	29.62
Associate High	31	18.34	40	15.38
Senior	7	4.14	9	3.46
Other	45	26.63	64	24.62

(ii) Comparison of pre-and post-assessment scores of the medical staff of different genders and ages

The average score of all medical staff in the fourth stage (90.34±10.28) was significantly higher than that of the first stage (81.35±11.75), with an increase of 8.99 points (P<0.001). The mean scores of the medical staff who participated in the assessment of the fourth stage were improved in all the subgroups regardless of gender or age, compared with the first stage (P<0.001). Moreover, the further analysis showed that the mean scores of female medical staff were significantly higher than those of male medical staff in the first and fourth stages (P<0.001), and the mean scores of medical staff aged less than 30 years old in the first and fourth stages were higher than those of other age groups (P<0.05) (Table 2).

Table 2

Comparison of Pre- and Post-Assessment Scores of Medical Staff of Different Genders and Ages

Classification		Stage 1 (n=5325)	Stage 4 (n=5577)	Improved Score	t-value	P-value
Average medical staff score		81.35±11.75	90.34±10.28	8.99	−42.582	<0.001
Gender	Male	78.59±12.81	87.84±12.25	9.24	−17.051	<0.001
	Female	82.01±11.38	90.97±9.62	8.96	−39.823	<0.001
	t-value	−8.433	−9.158
	P-value	<0.001	<0.001
Age	<30	81.73±111.59	90.55±8.82	8.82	−27.965	<0.001
	30~	81.45±11.60	90.48±9.93	9.03	−26.620	<0.001
	40~	80.73±11.74	90.09±10.52	9.36	−14.999	<0.001
	50~	79.56±13.24	89.05±12.27	9.50	−10.873	<0.001
	F-value	4.51	2.99
	P-value	0.004	0.030

(iii) Comparison of pre-and post-assessment scores of medical staff with different titles

The mean scores of physicians, nursing, medical technology, and other staff in the fourth stage of assessment were higher than those in the first stage, and the improvement scores showed that other staff > medical technology staff > nursing staff > physicians (P<0.001). The mean scores of physicians with junior titles were higher than those of physicians with other titles, and the mean scores of physicians with different titles in the fourth stage were higher than those in the first stage (P<0.001). The difference among the mean scores of nursing staff with different titles in the fourth stage was statistically significant (P<0.05), and the mean scores of nursing staff with different titles in the fourth stage were higher than those in the first stage (P<0.001). Besides, the mean scores of medical technologists with different titles in the fourth stage were higher than those in the first stage (P<0.001), and the mean scores of medical technicians with different titles (except intermediate) in the fourth stage were higher than those in the first stage (P<0.001) (Table 3).

Table 3

Comparison of Pre- and Post-Assessment Scores of Medical Staff with Different Titles

	Stage 1 (n=5325)		Stage 4 (n=5577)		Improved Scores	t-value	P-value
Classification	Average Score	Standard Deviation	Average Score	Standard Deviation
Physician Staff	80.13±12.12		88.89±11.03		8.76	−19.768	<0.001
Residents	79.27±12.45		88.99±10.48		9.73	−11.71	<0.001
Attending Physician	91.17±11.83		89.34±11.41		8.17	−9.784	<0.001
Associate Chief Physician	81.19±11.34		89.19±10.58		8	−8.976	<0.001
Chief Physician	79.45±12.64		88.56±11.28		9.1	−8.303	<0.001
F value	2.63		5.16
P-value	0.033		<0.001
Nursing staff	83.08±10.98		92.26±8.69		9.18	−37.766	<0.001
Nurse	82.92±11.00		92.07±8.90		9.15	−29.915	<0.001
Supervising Nurse	83.29±10.80		92.95±7.99		9.66	−21.102	<0.001
Associate Chief Nurse	83.9±11.56		92.49±8.48		8.59	−8.178	<0.001
Chief Nurse	83.67±11.89		92.03±9.06		8.36	−4.145	<0.001
Other	82.57±10.34		89.10±9.77		6.54	−3.849	<0.001
F-value	0.51		3.87
P-value	0.727		0.004
Medical Technicians	76.32±11.34		85.57±12.11		9.25	−13.224	<0.001
Technician	76.63±11.14		85.11±12.07		8.47	−8.413	<0.001
Supervising Technician	75.76±12.09		86.16±11.47		10.39	−8.011	<0.001
Associate Chief Technician	74.85±11.13		85.36±13.01		10.51	−5.265	<0.001
Chief Technician	77.41±9.84		87.41±13.72		10.01	−3.15	0.003
Other	79.26±10.53		85.15±12.21		5.89	−1.976	0.052
F-value	0.94		0.4
P-value	0.438		0.811
Other personnel	74.32±15.34		83.98±12.95		9.66	−7.015	<0.001
Junior	74.1±15.93		85.5±11.43		11.4	−4.567	<0.001
Intermediate	77.64±14.51		82.14±13.82		4.5	−1.588	0.115
Associate High	71.77±13.76		83.50±11.94		11.73	−3.84	<0.001
Senior	73.57±14.35		82.22±18.93		8.65	−1.01	<0.001
Other	73.78±16.66		85.08±13.20		11.3	−3.946	<0.001
F-value	0.65		0.8
P-value	0.628		0.528
Fa value	84.93		133.1
Pa value	<0.001		<0.001

Notes: The Pa value represents the comparison of scores between the two stages for different occupations.

(iv) Comparison of correct rates of different IPC knowledge points

The correct rates of IPC knowledge points in the fourth stage were significantly higher than those of the first stage (P<0.001). The correct rates of “prevention and control of COVID-19” and “prevention and control of multi-drug resistant bacteria” in the IPC knowledge points were the highest, while “disinfection and sterilization” and “management of infectious diseases” had the lowest correct rates (P < 0.001) (Table 4).

Table 4

Comparison of Correct Rates for Different Knowledge Points

Knowledge Points (Number of Questions)	Stage 1 (n=5325)			Stage 4 (n=5577)			χ2	P-value
	Number of Times Correct	Number of Assessment	Correct Rate (%)	Number of Times Correct	Number of Assessment	Correct Rate (%)
COVID-19 prevention and control (66)	36,888	42,605	86.58	42,404	46,932	90.35	313.339	<0.001
Basic knowledge (18)	8890	11,525	77.14	10,645	11,981	88.85	573.968	<0.001
Occupational exposure prevention and management (24)	11,903	15,454	77.02	14,388	16,239	88.60	750.88	<0.001
Hand hygiene (20)	10,392	12,930	80.37	12,019	13,346	90.06	491.091	<0.001
Sterilization (10)	4622	6450	71.66	5860	6774	86.51	443.211	<0.001
Medical waste management (10)	5148	6400	80.44	6003	6755	88.87	180.859	<0.001
Microbiological sample testing (7)	3496	4464	78.32	4233	4745	89.21	202.388	<0.001
Multi-drug resistant bacteria prevention and control (6)	3173	3850	82.42	3734	4087	91.36	140.532	<0.001
Infectious disease management (5)	2341	3194	73.29	2875	3319	86.62	181.313	<0.001
χ2	1.7*103			171.777
P	<0.001			<0.001

Discussion

The prevention and control of COVID-19 in China have obtained satisfactory results to some extent, however, the COVID-19 plague still causes severe consequences worldwide, maintaining a colossal menace on international public health.6 The complications such as lung injuries, venous/arterial thrombosis, heart injuries, cardiac/brain stroke and neurological injuries are the most frequent late complications of COVID-19.7 WHO issued an interim guidance document, namely “Infection Prevention and Control in Healthcare Facilities in the Event of Suspected COVID-19 Infection”, stating that national healthcare facilities should establish IPC planning, have well-trained full-time teams, provide adequate training for health care workers, monitor health care workers’ compliance with standard precautions and establish improvement mechanisms as appropriate, etc.6 Till now, researchers have identified several SARS-CoV-2 variants changing clinical manifestations and increasing the transmissibility, morbidity, and mortality of COVID-19,8 and several genes related to the immune system’s response were related to the severity and susceptibility to the COVID-19 combing through the genome.9 However, the effectiveness and safety of vaccination are still debatable.10–12 Therefore, there is an urgent need for healthcare facilities to train all medical staff in IPC during the COVID-19 outbreak.

The application of health protocols to prevent COVID-19 could also help to reduce the incidence of other infectious diseases such as influenza, pneumonia, and Mycobacterium tuberculosis.13 The traditional lecture method fails to mobilize the initiative and motivation of medical staff, which is relatively dogmatic and boring, and lacks intuitiveness and vividness.14 Recently, the case teaching method combining text and video is emerging, which can illustrate and explain HAIS typical cases with the features of teaching vivid and interesting.15 Case study method, problem-oriented teaching method, and e-learning method are widely used in foreign countries, however, they lack effective training effect assessment mechanism and are limited by training venues, staff, and time, which cannot achieve full participation and most importantly, or quantitatively assess the level of HAIS knowledge of medical staff.16 Hence, how to use more effective training methods during the COVID-19 epidemic to improve healthcare workers’ awareness, concepts, and practice of hospital-acquired IPC has drawn the attention of IPC managers.

In this study, the “four-step” training assessment mode adopts the form of “mapping assessment - online training - question bank practice - effectiveness evaluation”, with 2~3 days per stage, which is of short training and assessment period and enables medical staff to quickly acquire knowledge of hospital-acquired IPC during the epidemic. The “four-step” training assessment mode can be finished on the cell phone by scanning the QR code from “” to acquire the questionnaire. It is user-friendly, and medical staff is not limited by time and venue, reducing the gathering of people and is in line with the control measures in public places during COVID-19. The assessment process is set up with a time limit and blocking network, which greatly improves the compliance of medical staff and ensures the acquisition of IPC knowledge by medical staff during COVID-19. Our analysis results showed that the participation rate can reach 95.04% in the first stage of the training and assessment, which was not included in the year-end assessment of the department head and the department, reflecting both the high awareness of the medical staff to the COVID-19 epidemic as well as the nosocomial infection and control. In the fourth stage of effect evaluation, other staff such as administrative staff, property staff, and logistics support staff also actively participated in this assessment, and the assessment coverage rate increased to 102.45%, ensuring that full training was achieved. We further investigated the examination scores in the first, third and fourth stage, the average score of medical staff was (81.35±11.75), which was higher than the predicted level, we speculated that the COVID-19 epidemic has greatly increased the medical staff’s attention to IPC, the average score was higher than we expected, which has laid a solid foundation for overcoming COVID-19 epidemic. After the second and third stages of study, the scores of medical staff in the fourth stage increased to (90.34±10.28), which was slightly lower than the results of a study on the meta-analysis of IPC training methods and effects17 due to the high basic level of medical staff’s knowledge of hospital-acquired infections.

After this short-term training, the mean scores of medical staff in all categories of stratification (gender, age group, and title) were significantly higher than in the first stage, and all medical staff had significantly improved their knowledge of IPC, truly achieving effective training for all staff.18 In addition, it was found that the mean scores of female medical staff were higher than those of male medical staff in both stage one and stage four, and the mean scores of nursing staff were also higher than those of other medical staff, which was analyzed because the nursing staff were mostly female and had the higher initiative and compliance.19 Interestingly, the mean scores of the first and fourth stages were relatively higher in younger medical staff with the age less than 30 years old, and the junior title of physician staff improved their scores more than others. Although previous studies showed that the foundation of IPC knowledge of physician staff of younger age and lower titles were relatively weak due to the lesser number of hospital infection courses offered in medical schools, which may be related to the fact that this training is a short-term training, the younger physicians have a stronger ability to learn in the short term and accept new things such as the COVID-19 epidemic more quickly.20 These results suggested that future training should focus on men, physicians, and medical technicians, and the long-term effects of IPC training could be further investigated.21,22

In the present study, the correct rate of IPC knowledge points in all 9 parts of the fourth stage was higher than that in the first stage, and the correct rate of IPC knowledge points in both stages was highest in the parts of “prevention and control of COVID-19” and “prevention and control of multi-drug resistant bacteria”, indicating that all medical staff paid more attention to the prevention and control of COVID-19. After training, the correct rate of “prevention and control of multi-drug resistant bacteria” increased significantly, suggesting that the basic level of multi-drug resistant bacteria prevention and control was relatively weak among medical staff, but it was significantly improved after short-term training, which has certain reference value for the prevention and control management of multi-drug resistant bacteria in hospitals. Besides, the lowest correct rates were found in the two stages of “disinfection and sterilization” and “infectious disease management”, the content of these parts mainly focus on COVID-19 survival condition. Correspondingly, medical preprint website med Rxiv reported that COVID-19 was detected on the surface of the nurses’ station in the isolation area of the suspected patient, as well as in the air of the ICU, all of which indicate that COVID-19 is present both on the surface of objects and in the air and that there is an urgent need to strengthen the implementation of sterilization and isolation measures and training on sterilization and isolation and prevention and control in healthcare institutions. Droplet and contact with contaminated surfaces are the most frequent transmission modes of COVID-19, however, fecal excretion, environmental contamination, and fluid pollution may also contribute to viral transmission.23 Disinfection and sterilization are important cornerstones of IPC, and infectious disease management is a key area of concern for medical staff, and it is even more important for healthcare facilities at all levels to strengthen targeted training in these two areas during COVID-19.

There are some limitations here, such as the small number of single assessment questions, the large difference in the number of questions of each part of IPC knowledge, and the lack of evaluation mechanism for the long-term effect of training, etc.

Conclusion

Altogether, the “four-step” assessment mode can be applied to train medical-relevant staff to acquire important information about IPC during the COVID-19 plague. Through the short-term training and assessment of “four-step”, medical staff has quickly learned the basic level of their IPC knowledge, thus using this information to effectively improve IPC capabilities and actively implement various IPC measures. Due to there are still some limitations here, it is necessary to further design a more meticulous training mode to explore the long-term effect of hospital IPC knowledge training, thus reducing the hospital infection rate effectively.