Is knowledge retained by healthcare providers after training? A pragmatic evaluation of drug-resistant tuberculosis management in China.

OBJECTIVES: Considering the urgent need of training to improve standardised management of drug-resistant infectious disease and the lack of evidence on the impact of training, this study evaluates whether training participants' knowledge on multidrug-resistant tuberculosis (MDR-TB) is improved immediately and a year after training. SETTING AND PARTICIPANTS: The study involved 91 MDR-TB healthcare providers (HCPs), including clinical doctors, nurses and CDC staff, who attended a new MDR-TB HCP training programme in Liaoning and Jiangxi provinces, China. MAIN OUTCOME MEASURES: A phone-based assessment of participants' long-term retention of knowledge about MDR-TB management was conducted in July 2017, approximately 1 year after training. The proportion of correct responses in the long-term knowledge assessment was compared with a pretraining test and an immediate post-training test using a χ2 test. Factors influencing participants' performance in the long-term knowledge assessment were analysed using linear regression.
RESULTS: Across both provinces, knowledge of definitions of drug-resistant TB, standardised MDR-TB case detection protocols and laboratory diagnosis was improved 1 year after the training by 14.5% (p=0.037), 32.4% (p<0.001) and 31% (p<0.001) relative to pretraining. However, compared with immediately after training, the knowledge of the three topics declined by 26.5% (p=0.003), 19.8% (p=0.018) and 52.7% (p<0.001) respectively in Jiangxi, while no significant decline was observed in Liaoning. Additionally, we found that obtaining a higher score in the long-term knowledge assessment was associated with longer years of clinical experience (coefficient=0.51; 95 CI% 0.02 to 0.99; p=0.041) and attending training in Liaoning (coefficient=0.50; 95% CI 0.14 to 0.85; p=0.007).
CONCLUSION: Our study, the first to assess knowledge retention of MDR-TB HCPs 1 year after training, showed an overall positive long-term impact of lecture-style group training on participants' knowledge. Knowledge decline 1 year after training was observed in one province, Jiangxi, and this may be partly addressed by targeted support to HCPs with fewer years of clinical experience. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The study addressed the information needs of the Chinese policymakers in assessing the long-term impact of a pilot healthcare worker training and provided evidence for them to make decisions on future investments.

A phone-based survey approach was used to avoid participants using external sources to complete the long-term knowledge assessment.

A high response rate was achieved in both immediate post-training and long-term knowledge assessments.

Because pretraining and immediate post-training tests were completed anonymously, the results of the three assessments were unable to be linked at individual level, and a group comparison between the knowledge assessments was conducted.

Introduction

Providing good quality health services for the diagnosis and management of the growing burden of drug-resistant infections is particularly challenging as healthcare providers (HCPs) need to be trained on management of clinical complexities associated with drug resistance; this has often not been covered in detail during initial clinical training as attention to drug resistance has grown in recent years.1 2

Globally, it is estimated that 30% of all deaths related to antimicrobial resistance (AMR) are caused by drug-resistant tuberculosis (TB).3 Multidrug-resistant (MDR)-TB control in China is central in the global response to AMR since China has the second highest MDR-TB case load in the world, with an incidence of 73 000 in 2016.4 Even though TB and MDR-TB services are provided free of cost at the point of care at selected public sector health facilities in China, because MDR-TB services are concentrated in centralised, urban health facilities, there are substantial financial barriers for patients in rural areas to access MDR-TB care.5 6 Improved access to HCPs that can provide high quality MDR-TB care across all parts of China is essential, as approximately 80% of patients with MDR-TB live in rural areas that are far from centralised hospitals that are currently set-up to provide MDR-TB services.7–9 To improve physical access to health services for new patients with MDR-TB, and to prevent the emergence of bacterial strains that are resistant to even more anti-TB drugs (which studies indicate already occurring10) improving and standardising management of MDR-TB in health facilities across the country are key priorities for China.11

In response to the WHO’s call for UHC and the Chinese government’s determination to tackle rising MDR-TB, the Chinese Center for Disease Control and Prevention (China CDC) and Chinese Medical Association (CMA) implemented a comprehensive pilot programme to provide universal care for patients with MDR-TB, including the use of rapid diagnostic tests, standardised treatment and financial protection against catastrophic health costs.12 Within the four pilot cities, there were reductions in delay to diagnosis and treatment initiation, improved case detection and treatment outcome rates and reduced patient expenses for diagnosis and treatment.13 14

A major barrier to scaling up the programme nationwide is ensuring that clinical staff at peripheral health centres are appropriately trained in providing quality MDR-TB services—including implementing new diagnostic protocols—according to the national guidelines. The China CDC is particularly focused on improving early detection on MDR-TB using the new diagnostic protocols as the MDR-TB case detection rate in China was only 7.7% in 2014,15 and one of the major factors for the low detection rate is lack of trained HCPs.16–18 Health policymakers in the China CDC and CMA are therefore considering investments in countrywide training programmes to ensure standardised programmatic management of MDR-TB. However, a recent systematic review showed that even though the number of TB HCP training programmes has surged since 2000, there is only limited evidence of impact from training evaluation studies; in particular, there was a dearth of assessments on changes in on-the-job practices among training participants and long-term knowledge retention of trainees.19 In line with the literature review, our qualitative investigation of information that Chinese policymakers would require to make decisions on future investments in training programmes indicated that there are concerns that knowledge about MDR-TB management gained during training would quickly be forgotten; objective data on long-term retention of knowledge imparted during training is therefore needed to make decisions about investments into HCP training in China and beyond.20

This study therefore aims to provide evidence on long-term knowledge retention of HCPs trained during the pilot MDR-TB training programme implemented by China CDC. Our two study objectives are to: (i) assess whether there are differences in HCP knowledge about clinical and laboratory diagnosis of MDR-TB cases before, immediately after and 13–14 months after the training and (ii) investigate factors that influence long-term knowledge retention after training.

Methods

Description of the training programme

The training programme that we assessed was designed and managed collaboratively by the China CDC and Lilly MDR-TB Partnership, which is a public–private initiative initiated in 2003.21 It aimed to help standardise MDR-TB diagnosis, treatment and laboratory testing of MDR-TB by TB HCPs in China. In order to inform decisions on countrywide implementation of the training programme, pilot sites were established by the CDC in six provinces across China: Fujian (Southern China), Jiangxi (Central China), Liaoning (Northeast China), Sichuan (Southwest China), Xinjiang (Western China) and Hebei (Northern China). In each province, HCPs (clinical doctors, nurses and local CDC staff) involved in delivering care to patients with TB in the following organisations were included in the training programme: provincial CDC, TB specialist hospitals and general hospitals that contained TB divisions.

A total of seven training sessions were conducted between July 2015 and November 2016 in the six pilot provinces (table 1). Eligible HCPs treating patients with TB from each involved hospital were selected by the managers/supervisors at the hospital to participate in a one-time training session organised in their local province. Each training session had the same content and format, and no formal refresher training was provided. Training was delivered solely in the form of oral lectures with the aid of PowerPoint to highlight key points by experienced TB clinical experts over 4–5 days in a conference room, and identical training material was used across the training sessions. The training material was adapted from the ‘Guidelines for the Implementation of National TB prevention and Control’ and ‘Companion handbook to the WHO guidelines for the programmatic management of drug-resistant TB’ and provided to the participants at the start of the training session.22 23 Training covered diagnosis, treatment and management of MDR-TB cases, which was directly relevant to the clinical practice of training participants, and additionally covered broader topics such as epidemiology, strategies for disease control, mechanisms of developing resistance to anti-TB drugs, responsibilities of disease control departments and surveillance and information management. The number of participants attending training sessions ranged between 38 and 54.

Table 1

List of training sessions conducted between 2015 and 2016

Date of training session	Training sites (city, province)	Number of participants
27–31 July 2015	Fuzhou, Fujian	38
30 May–3 June 2016	Ganzhou, Jiangxi	52
20–24June 2016	Dalian, Liaoning	54
15–19 August 2016	Chengdu, Sichuan	46
22–26 August 2016	Yili, Xinjiang	50
30 October–3 November 2016	Zhangzhou, Fujian	39
14 November–18 2016	Tangshan, Hebei	45

Pretraining and immediate post-training assessments on the day of training

MDR-TB knowledge among participants was assessed three times: pretraining, immediately post-training and 1 year after training. However, to comply with permissions granted by the study participants, no identifying information was collected during testing and therefore individual test papers could not be linked longitudinally. A baseline pretraining assessment was conducted at the beginning of each training session through an anonymous test lasting 1 hour. The test, developed by the China CDC training programme management team, contained 15 multiple choice questions and covered key knowledge areas on diagnosis, treatment and management of MDR-TB cases in line with national and international guidelines (online supplementary file 1). The short-term effect of training on participants’ MDR-TB knowledge was assessed by an immediate post-training test conducted on the last day of the training session. The immediate post-training test contained the same set of test questions as the pretraining test and it was answered anonymously within 1 hour. Tests were marked by the China CDC training programme management team in Beijing, and raw data on test scores were recorded in an excel sheet. In line with the agreement between training participants and the CDC, scores were not linked at the individual level.

Long-term knowledge survey

To assess long-term knowledge retention of participants after the completion of training, we conducted a phone-based survey on MDR-TB knowledge in July 2017 to assess knowledge retention of the training participants approximately 1 year after the training. This 1-year time frame was chosen for two reasons. First, in a systematic review of training evaluation studies, we found most of the training evaluation used pretraining and post-training tests to assess short-term effectiveness of training programmes, but there is very limited evidence on the long-term impact (more than 6 months) of training participants.24 Therefore, 1 year seemed to be a suitable time period for evaluating the long-term effectiveness of a training programme. Furthermore, TB doctors and nurses in China are required to attend at least one training every year. An annual training programme could fit in with their busy schedule. Thus, the stakeholders needed evidence to make decisions on whether the training programme can be implemented on a yearly basis. In line with the study objectives, priorities of the China CDC and constraints on time that could be requested from trainees for additional knowledge testing, we focused on assessing knowledge related to clinical and laboratory diagnosis of MDR-TB. We identified two provinces where the training sessions were conducted in May and June 2016: the Liaoning and Jiangxi province. Liaoning provinces is located in the northeast China with about 40 million residents. Jiangxi province is located in the central China. Similar to the size of Liaoning province, Jiangxi province has about 40 million residents. All participants attending the training sessions were eligible and included in the study.

The survey was predesigned by the research team (SW, MSK) to be consistent with the pretraining and post-training test, with input from an international TB clinical expert about the technical accuracy of the questions and response choices. Prior to implementation, it was translated into Chinese by a native Chinese researcher (SW) and piloted in Chinese. The survey was divided into two parts and was designed to be completed within 20 min. The first part of the survey included questions on participants’ demographic information, such as age, gender, highest education and years of TB clinical experience. The second part contained three questions on MDR-TB knowledge selected by the China CDC training programme management team, the designer and organiser of the training programme, from the pretraining test as they believed that the evaluation of these three topics was key to determine whether the intended objective of the training programme was achieved. The specific question topics were: standard case definitions of drug-resistant TB, clinical case detection and laboratory diagnostic protocols. Listed in box 1, the three questions in the phone-based survey questionnaire were exactly the same as questions in the pretraining test in both content and format.

Which of the following is the definition of extensively drug-resistant TB (XDR-TB)?

Multidrug-resistant tuberculosis (MDR-TB) plus resistance to at least three drugs out of six injectable second line drugs.

Resistance to all first line drugs.

MDR-TB plus resistance to quinolone drugs and all injectable drugs.

Resistance to all first line drugs and quinolone drugs and at least one injectable second line drug.

MDR-TB plus resistance to one of the quinolone drugs and one injectable second line drug.

A new patient with TB who received 5 months of standardised type I TB treatment gets smear-positive results, which of the following explanation is possibly true?

Non-tuberculosis mycobacteria.

MDR-TB.

Susceptible to all drugs.

Dead bacteria, patient will cure eventually.

All of the above explanations are possible.

Which of the following statement is correct?

Bacterial strain identification is not suitable for MDR-TB diagnosis.

All laboratory MDR-TB diagnosis should include culture.

The molecular diagnostic test for isoniazid resistance has very high sensitivity.

The molecular diagnostic test for rifampin resistance has very high specificity.

Data were collected by trained Chinese data collectors under the supervision of the first author. To ensure familiarity with the survey and standardisation in survey implementation, data collectors were provided a 3-hour training session detailing how to conduct the phone-based survey and record responses of participants. At the beginning of the long-term knowledge survey, participants were informed about the objectives of the study. Responses by participants to each question were recorded by the data collectors in a predesigned electronic survey form with built-in data validation checks to minimise data entry errors. Data in Chinese were translated into English, and responses were marked by the researcher (SW) and verified by the China CDC managing team. We counted the total correct answers out of the three knowledge questions for each participant. Age and years of clinical experiences were categorised such that an approximately balanced number of participants were distributed across the categories.

Statistical analysis

We obtained raw data collected during pretraining and immediate post-training tests from the China CDC. These raw data, as well as data from the long-term knowledge survey, were analysed using STATA V.14.1.

To address the first study objective, we compared the proportion of participants correctly answering each of the three questions in the pretraining test, immediate post-training test and long-term knowledge survey using a χ2 test. We had to compare proportions because the pretraining and immediate post-training assessments were conducted anonymously without unique identifiers, and we were therefore unable to link the test scores at the individual level. The correlation between correct responses across the three questions was assessed using the Spearman’s correlation coefficient before comparing results of the pretraining, immediate post-training and long-term knowledge tests. A coefficient greater than 0.7 suggested a strong correlation between the variables.25 26 To further investigate whether there were differences in the training outcomes between provinces, a comparison of the proportion of correct responses for each test question was performed separately for Liaoning and Jiangxi provinces.

To identify which group of participants had lower scores for knowledge retention on MDR-TB diagnosis —to address the second study objective—a regression analysis was conducted. The test score for each participant was the sum of total correct answers out of the three questions; this was used as the outcome variable in all regression models. The distribution of the test score was assessed using the Shapiro-Wilk test. We first performed a univariable analysis to estimate the association between the test score and each explanatory variable: age, gender, occupation, years of clinical experience, training province, highest education obtained and whether they attended any other training programmes in the past 12 months. Correlations between explanatory variables were examined using the Spearman’s correlation coefficient. Variables with a correlation coefficient greater than 0.7 were considered to meet the assumption of multicollinearity.25 26 Multiple linear regression modelling was then used to study the relationship between explanatory and outcome variables while adjusting for other covariates. Forward stepwise selection was performed to identify explanatory variables to be included in the regression model.27 Selection was based on the Akaike Information Criterion (AIC): the added variable would be included if the model gave a smaller AIC value compared with the one without the added variable, suggesting that the model has better fitness with less complexity.28 The regression coefficient of each included variable, the associated 95% CI and the two-sided p value were reported.

Patient and public involvement statement

Patients were not involved in this study.

Results

Demographic characteristics of participants

Of the 102 HCPs completing the training and pretraining test in Liaoning or Jiangxi provinces, 91 (89%) participated in the long-term knowledge survey. Eleven participants could not be included in the long-term knowledge survey, of which five refused to participate when contacted and six could not be contacted on the phone despite at least three attempts.

Data on demographic characteristics of our study participants are presented in table 2. There was almost equal representation from both provinces and of male and female HCPs. The majority of participants were clinical doctors (n=66, 72.5%) with a smaller number of CDC staff involved in laboratory testing and data management (n=16, 17.6%) and even fewer nurses (n=9, 9.9%). Most of the training participants were aged between 40 and 49 (n=40, 44.0%), with less participants who were less than 39 years old (n=29, 31.9%) or greater than 50 years old (n=22, 24.2%). The majority (n=65, 71.4%) had not attended any other training on MDR-TB apart from the CDC-Lilly training described in the study Setting section.

Table 2

Characteristics of training participants and association with long-term knowledge test score.

	Participants' characteristics, number (%) or mean (SD)	Long-term knowledge survey results	Univariable analysis		Adjusted for training province		Multiple linear regression
		Meanscore (SD)	Coefficient (95% CI), %	P value	Coefficient (95% CI), %	P value	Coefficient (95% CI), %	P value
Total participants	91 (100)	1.84 (0.89)
Gender
Male	49 (53.8)	1.76 (0.92)	Reference	–	Reference	–
Female	42 (46.2)	1.93 (0.84)	0.17 (−0.20 to 0.54)	0.354	0.06 (−0.30 to 0.42)	0.746
Age
≤39	29 (31.9)	1.79 (0.81)	Reference	–	Reference	–
40–49	40 (44.0)	1.83 (0.96)	0.03 (−0.40 to 0.47)	0.884	−0.15 (−0.58 to 0.28)	0.489
≥50	22 (24.2)	1.91 (0.81)	0.12 (−0.39 to 0.62)	0.647	0.05 (−0.43 to 0.53)	0.843
Occupation
Clinical doctor	66 (72.5)	1.80 (0.88)	Reference	–	Reference	–
Nurse	9 (9.9)	1.67 (0.87)	−0.14 (−0.76 to 0.49)	0.667	−0.07 (−0.63 to 0.58)	0.814
CDC/hospital staff (non-clinical)	16 (17.6)	2.06 (0.93)	0.26 (−0.23 to 0.75)	0.297	0.07 (−0.42 to 0.56)	0.782
Years of clinical experience
≤7	23 (25.3)	1.65 (0.88)	Reference	–	Reference	–	Reference	–
8–15	29 (31.9)	1.69 (1.00)	0.04 (−0.44 to 0.52)	0.877	0.04 (−0.42 to 0.51)	0.853	0.07 (−0.40 to 0.53)	0.775
16–23	11 (12.1)	1.64 (0.50)	−0.01 (−0.64 to 0.61)	0.96	−0.13 (−0.75 to 0.48)	0.668	−0.17 (−0.77 to 0.44)	0.587
≥24	18 (30.8)	2.21 (0.78)	0.56 (0.08 to 1.04)	0.023	0.42 (−0.05 to 0.90)	0.081	0.51 (0.02 to 0.99)	0.041
Training province
Liaoning	46 (50.5)	2.11 (0.95)	Reference	–			Reference	–
Jiangxi	45 (49.5)	1.56 (0.72)	−0.55 (−0.91 to –0.20)	0.002			−0.50 (−0.85 to –0.14)	0.007
Highest education
Junior college or secondary vocational degree	16 (17.6)	1.63 (0.96)	Reference	–	Reference	–	Reference	–
Master or bachelor’s degree	75 (82.4)	1.88 (0.87)	0.26 (−0.23 to 0.74)	0.298	0.25 (−0.21 to 0.71)	0.283	0.40 (−0.07 to 0.86)	0.092
Attended other training
No	65 (71.4)	1.77 (0.91)	Reference	–	Reference	–	Reference	–
Yes	26 (28.6)	2.00 (0.80)	0.23 (−0.18 to 0.64)	0.26	0.30 (−0.09 to 0.69)	0.132	0.33 (−0.06 to 0.71)	0.094

Comparison of proportions of correct responses between three knowledge assessments

Our first analysis compared the proportion of participants who answered each of the three questions correctly in the pretraining, immediate post-training and long-term knowledge assessments across both provinces (figure 1A). We observed no strong correlation between the three test questions. In the pretraining test, 57.4%, 43.4% and 7.5% of the training participants correctly answered the definitions of drug-resistant TB (question 1), MDR-TB clinical case detection protocols (question 2) and laboratory diagnosis of MDR-TB (question 3) respectively. Compared with the pretraining test, the proportion of correct responses in the immediate post-training test increased by 27.7% (p<0.001), 47.2% (p<0.001) and 46.6% (p<0.001) for question 1 to 3 respectively. Similarly, in the long-term knowledge survey, the percentage of participants who answered question 1, 2 and 3 correctly was 14.5% (p=0.037), 32.4% (p<0.001) and 31% (p<0.001) higher than in the pretraining test. For question 3 (about laboratory diagnosis of MDR-TB), even though there was a large increase in participants answering correctly compared with the pretraining test, less than 40% of participants answered the question correctly in the long-term knowledge survey, compared with 69.2% and 75.8% of participants correctly answering questions 1 and 2. When comparing to the immediate post-training test, we found a decline in the proportion of correct responses 1 year after the training for all three questions by 13.2% (p=0.043), 14.8% (p=0.009) and 15.6% (p=0.037).

Figure 1

Comparison of proportions of correct responses to each test question between the three knowledge assessments: (A) overall training participants; (B) Liaoning province and (C) Jiangxi province.

Conducting the same analysis separately for Liaoning and Jiangxi provinces revealed differences between the provinces. In Liaoning province (figure 1B), the proportion of correct responses for questions 1 to 3 was 51.9%, 46.3% and 9.3% respectively in the pretraining test. The proportion of correct responses increased significantly to 73.2% (p=0.035), 90.2% (p<0.001) and 34.2% (p=0.003) in the post-training test. Comparing the long-term knowledge survey with the immediate post-training test, the proportion of correct responses was similar for the questions about MDR-TB case definition and clinical MDR-TB diagnosis (questions 1 and 2). For question 3 on laboratory diagnosis, the proportion of correct responses increased by 22.3% (p=0.037) in the long-term knowledge survey compared with the immediate post-training test.

In Jiangxi province (figure 1C), the correct response proportions for questions 1 to 3 in the pretraining assessment were similar to the pretraining test results in Liaoning province. When comparing to the pretraining test, the percentage of correct responses increased significantly to 90.9% (p<0.001), 90.9% (p<0.001) and 72.7% (p<0.001) in the immediate post-training test. However, the percentage of correct responses for questions 1 to 3 decreased in the long-term knowledge survey compared with the immediate post-training test by 26.5% (p=0.003), 19.8% (p=0.018) and 52.7% (p<0.001) respectively. Overall, 64.4%, 71.1% and 20% of participants in Jiangxi had correct knowledge about MDR-TB case definition, clinical diagnosis and laboratory diagnosis 1 year after training.

Factors for participants’ performance in the long-term knowledge survey

Results from the linear regression analysis in which we investigated factors that influence the performance of participants in the long-term knowledge survey are shown in table 2. In the univariable analysis, we found that the long-term knowledge test results were strongly associated with training province and participants’ years of clinical experience. Participants who attended training in Jiangxi province scored 0.55 points (95% CI −0.91 to −0.20, p=0.002) lower than those who attended training in Liaoning province. Participants who had clinical experience more than 24 years scored 0.56 points (95% CI 0.08 to 1.04, p=0.023) higher than those who had less than 7 years of clinical experience. After adjusting for training province, we could still observe a weak association between long-term knowledge test score and participants’ years of clinical experience.

The multiple linear regression analysis model included the following variables: years of clinical experience, the province where the participant attended training, highest education level and whether an additional training programme was attended in the past 12 months. Multivariable analysis results indicated that provincial differences were substantial even after adjusting for other variables; participants’ test scores in Jiangxi province were significantly lower than Liaoning province by 0.50 (95%CI −0.85 to −0.14, p=0.007). We also found strong evidence to indicate that participants with more than 24 years of clinical experience scored higher in the long-term knowledge survey than those who had less than 7 years of clinical experience. There was weak evidence of an association between long-term knowledge test score and participants’ education level after adjusting for other explanatory variables; participants who obtained a bachelor or master’s degree scored 0.40 (95CI% −0.07 to 0.86, p=0.092) points higher than those who had lower education level. Similarly, there was a weak association between long-term knowledge test score and participants’ attendance at additional training programmes.

Discussion

Our study, the first to assess the long-term knowledge retention of HCPs after training to improve management of drug-resistant TB, showed clear findings with important policy implications. Overall, participants’ knowledge of MDR-TB was improved immediately after the training and 1 year after training compared with before training. In Jiangxi province, there was a decline in knowledge from immediately after training to 1 year after training, even though participants’ knowledge was still substantially higher in the long-term knowledge survey than before training. Contrary to the findings in Jiangxi province, no significant decline in participants’ knowledge was observed 1 year after the training in Liaoning province compared with the immediate post-training assessment. Considering that training participants in both provinces had similar MDR-TB knowledge test scores in the pretraining and immediate post-training tests, the differences in knowledge decline during the year following training between the provinces suggests that participants’ knowledge retention may be influenced by their working environment.

In terms of specific training topics, participants’ knowledge of MDR-TB laboratory diagnosis was very low before training and significantly improved immediately after training in both provinces. However, we observed a substantial decrease in correct responses among participants in Jiangxi province a year after training, while the correct responses improved among participants in Liaoning province. The decline observed in Jiangxi province 1 year after the training may be because HCPs in Jiangxi province had fewer opportunities to perform MDR-TB laboratory diagnostic tests in their daily jobs than those in Liaoning province; a study has found that the availability of diagnostic tests for drug-resistant TB varies among different geographical regions in China.18

Our analysis of characteristics associated with a lower score in the long-term knowledge assessment indicated that participants who had fewer years of TB clinical experience could be supported with follow-up training, on-the-job mentorship or tools to assist with adherence to guidelines. Follow-up training could be integrated into the nationwide internet-based training network which has been established as part of capacity building initiatives in China.29 Furthermore, since the use of multiple training methods can facilitate the learning process of training participants,30–32 other training methods such as case-based group discussion or hands-on practice can be used. Follow-up training would also be useful because of rapid developments on international guidelines on the management of drug-resistant infections, which means that knowledge of HCPs needs to be updated regularly.

Consistent with other TB HCP training evaluations that assessed knowledge improvement immediately after training, our study showed an increase in the proportion of participants with higher test scores in the immediate post-training test, which adds to the evidence that training programmes are effective in improving factual TB/MDR-TB knowledge in short term.33–35 Because no previous evaluation has assessed long-term knowledge retention of TB HCP training programmes, we were unable to compare the knowledge changes observed a year after training in our study with other similar assessments. Considering evidence from studies that evaluated the long-term knowledge retention from training programmes on health areas other than TB, we found that it was common for participants’ knowledge to improve immediately after the training programmes and gradually declined over time.36–40 The degree of knowledge decline, as summarised in a systematic review, was found to be associated with the time interval of retention, whether the knowledge or skill has been repeated or practised beyond the initial proficiency, the training methods, and individual differences in learning ability.31

An important new finding from our study was that knowledge can decline or increase a year after training, as observed in Jiangxi and Liaoning province respectively. This difference in knowledge uptake could be explained by factors from three aspects: the training programme delivery, the working environment and the local context. Different trainers and facilitators conducted training sessions in different provinces considering the language and cultural differences across the provinces, and their quality of teaching may have differed. However, to ensure the quality of the training across the pilot provinces, the China CDC had a managing team to design and monitor the training programme, and training of trainers and facilitators was provided before the commencement of the training programme. Differences in the working environment following training also have a substantial influence on knowledge retention. Others have found a positive impact on knowledge retention if training participants are able to practise what they learnt regularly.31 However, because we did not have data on the number of MDR-TB cases managed by the participants in the past year or other differences in the working environment in these two provinces, we are unable to verify whether this influenced differences in knowledge retention between Jiangxi and Liaoning. Furthermore, contextual factors, such as the MDR-TB epidemics in the province, the healthcare infrastructure and the number of healthcare professionals available to provide TB and MDR-TB services, may also contribute to the variation we observed in the two provinces. We were unable to find published information about contextual differences across the provinces to explore how these might have played to explain our findings about provincial differences.

An important strength of our study is the high response rate achieved in both immediate post-training and long-term knowledge assessments. When assessing the long-term knowledge retention, we used a phone-based survey approach to avoid participants using external sources when completing the survey. To ensure consistency in the conduct of the survey by the data collectors, training was provided at the beginning of data collection, and the correct answers to the knowledge survey were not provided to the data collectors to minimise interviewer bias. Finally, we involved the China CDC managing team in the process of designing and implementing the evaluation study, which may facilitate the uptake of our study results in the policy-making process.41

Despite generating important new evidence on knowledge retention following training on MDR-TB, a limitation of our study is that we were unable to link the results of the three assessments on an individual basis as assessments were completed anonymously. Furthermore, because the long-term knowledge assessment was conducted via phone while the pretraining and immediate post-training tests were paper based, we acknowledge that how the assessment was administered might influence participants’ performance. Another limitation is that the study was only conducted in two provinces, in which the patterns of knowledge retention varied. It is possible that the results on knowledge retention could be different from the results in the two provinces if participants from all pilot provinces were included in the long-term knowledge assessment. Our study should therefore be replicated in other settings.

We acknowledge that our results on knowledge improvement do not guarantee changes in clinical practice of HCPs, and that substantial knowledge deficits remained even after training. As summarised in a systematic review, the determinants of HCPs’ behaviour change are multifactorial, such as their confidence in practising new skills, barriers in the working environment and habits from previous practices.42 One way to address the gap between knowledge gain and behaviour change is incorporating teaching styles other than oral lecture to emphasise the application of knowledge and skills on the job. Additionally, enhancing the working environment, through updating of medical equipment and providing desk guides or visual aids to act as on-the-job reminders, may also be useful.43

We acknowledge that exploring the causes for the variations of knowledge retention among training participants in different provinces is key to designing effective MDR-TB education programmes to be implemented nationwide. Therefore, future studies that examine the differences on healthcare infrastructure, MDR-TB epidemics and services could be conducted to understand contextual factors that influence the observed difference in knowledge uptake in our study. Qualitative analysis of the participants’ experiences of the training may also help to differentiate barriers in the training conducted in different contexts. Additionally, studies that investigate the correlation of change in clinical practice and knowledge gain are also needed.

Conclusions

In summary, our study provides important and timely information for planning future training TB programmes, addressing a gap in evidence about long-term knowledge retention following training. Our finding that participants’ knowledge of MDR-TB was substantially higher 1 year after training than it was before training suggests an overall positive long-term impact of the training programme on participants’ knowledge; however, knowledge decline from immediately after training to 1 year after training is a justified concern which may be partly addressed by targeted support to HCPs with fewer years of clinical experience. Further evaluations are needed on the factors that facilitate knowledge retention after training and whether improved knowledge is translated into daily practice.