Three methods of delivering clinic-based training on syndromic management of sexually transmitted diseases in South Africa: a pilot study.

INTRODUCTION: The South African National Department of Health sought to improve syndromic management of sexually transmitted infections (STIs). Continuing medical education on STIs was delivered at primary healthcare (PHC) clinics using one of three training methods: (1) lecture, (2) computer and (3) paper-based. Clinics with training were compared with control clinics.
METHODS: Ten PHC clinics were randomly assigned to control and 10 to each training method arm. Clinicians participated in on-site training on six modules; two per week for three weeks. Each clinic was visited by three or four unannounced standardised patient (SP) actors pre-training and post-training. Male SPs reported symptoms of male urethritis syndrome and female SPs reported symptoms of vaginal discharge syndrome. Quality of healthcare was measured by whether or not clinicians completed five tasks: HIV test, genital exam, correct medications, condoms and partner notification.
RESULTS: An average of 31% of clinicians from each PHC attended each module. Quality of STI care was low. Pre-training (n=128) clinicians completed an average of 1.63 tasks. Post-training (n=114) they completed 1.73. There was no change in the number of STI tasks completed in the control arm and an 11% increase overall in the training arms relative to the control (ratio of relative risk (RRR)=1.11, 95% CI 0.67 to 1.84). Across training arms, there was a 26% increase (RRR=1.26, 95% CI 0.77 to 2.06) associated with lecture, 17% increase (RRR=1.17, 95% CI 0.59 to 2.28) with paper-based and 13% decrease (RRR=0.87, 95% CI 0.40 to 1.90) with computer arm relative to the control.
CONCLUSIONS: Future interventions should address increasing training attendance and computer-based training effectiveness. TRIAL REGISTRATION NUMBER: AEARCTR-0000668. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

RCT Entities:   Population Interventions Outcomes

Introduction

Syndromic management for sexually transmitted infections (STIs) diagnosis and treatment in primary care settings is recommended and widely used in many countries of Africa.1–9 Syndromic management is a presumptive diagnosis of STIs based on symptoms and easily recognised signs of infection and is typically directed by national treatment protocols.10 Protocols are regularly updated based on microbiological studies, as the prevalence of aetiological agents and treatment resistance profiles for STIs change. The South African National Department of Health (NDOH) updated the national STI guidelines in 2008 to reflect changing gonorrhoea resistance profiles by removing ciprofloxacin for gonorrhoea treatment and to add acyclovir as part of first-line therapy for genital ulcer syndrome.11

The NDOH sought to train health professionals on the revised STI national guidelines. The NDOH supported a decentralised approach to continuing medical education, which is implemented at the clinic level, such as educational outreach.12 13 The approach builds capacity and minimises absenteeism due to training. An STI training programme with three clinic-based training methods was developed: lecture, computer and paper-based.

Several articles have reported on the effectiveness of classroom-based STI syndromic management training,9 14 15 but less is known about the effectiveness of clinic-based programmes. In general, evidence shows that online learning is as effective as classroom methods.16 A pilot study of the STI training programme was conducted prior to national implementation with two objectives: (1) to test the feasibility of using each of the training methods for clinic-based teaching and (2) to conduct preliminary tests of their effectiveness. The pilot study was conducted at 40 primary healthcare (PHCs) clinics in North West Province that were randomly assigned to one of four arms; one for each training method and a delayed implementation arm that served as a control during the pilot.

Feasibility was assessed by the percentage of clinicians who attended training sessions. Effectiveness was tested with two measures pre-training and post-training: (1) STI knowledge test and (2) patient simulation by unannounced standardised patient (SP) actors. The STI knowledge test was completed by participants, but not by clinicians in the control arm. We tested the hypothesis that STI knowledge would increase post-training. SPs visits occurred at intervention and control clinics and measured quality of care. We tested the hypothesis that the pre-training to post training change in STI tasks performed by clinicians during SP encounters would be larger at clinics where STI training occurred relative to control clinics.

Methods

Sites and participants

In total, 40 stationary PHC clinics out of 74 in three subdistricts were purposefully selected for the pilot study and randomly assigned to four arms as shown in figure 1. PHC clinic size varied from 2 to 33 assigned clinicians and 40–230 patients per day. Two of the selected PHCs in one subdistrict provide 24 h services, three in the second and four in the third. Most other PHCs operate from 07:30 to 16:00, and a few operate 12 h per day, on weekdays. The district management team selected clinics with wider perceived gaps in STI management. Three exclusion criteria were: national STI clinical surveillance sites, clinics at correctional facilities, and non-governmental organisation clinics. Training programme participants were doctors and nurses assigned to the PHC clinics.

Figure 1

Participant flow diagram. Flow of clinics and training participants. HWC, health care worker; PHC, primary healthcare; SP, standardised patient.

Intervention

The six modules of the STI training programme and their learning objectives are listed in table 1. The modules were organised around clinical cases based on several recent reviews that concluded that interactive training was more effective than didactic.17 18 Each module was designed to be completed in 1 h. Computer-based modules are available at http://edgh.uw.edu/series/sexually-transmitted-infections and on compact disc. All training materials are in English, which is 1 of 11 official languages of South Africa.

Table 1

Overview of sexually transmitted infection (STI) curriculum and learning objectives for each module

Topics	Modules
	1—STI introduction and patient history	2—STI physical exam and counselling	3—Male urethritis syndrome	4—Vaginal discharge syndrome	5—Genital ulcer syndrome	6—Lower abdominal pain
STI introduction and patient history
Review the epidemiology of STIs	x
Discuss characteristics of a sensitive patient encounter for STIs	x
Describe the importance of patient history to making an accurate STI diagnosis	x
List the most important patient history questions for an STI screen or complaint	x
STI physical exam and counselling
Review importance of a physical exam to making the appropriate STI diagnosis		x
Discuss elements of both a male and female physical exam		x
Describe counselling topics related to STIs and how to discuss them sensitively		x
Objectives for specific syndromes or diagnosis
Identify the primary symptoms related to syndrome or diagnosis			x	x	x	x
List the most important patient history questions for an STI visit			x	x	x	x
Ask questions in a manner that will elicit the responses needed			x	x	x	x
Accurately use the NDOH syndromic algorithm for the syndrome or diagnosis			x	x	x	x
Recognise signs and symptoms that require urgent referral			x		x	x
Describe a targeted physical exam for syndrome or diagnosis			x	x	x	x
Discuss appropriate approach to caring for children with suspected child abuse				x
Discuss appropriate approach to caring for the elderly						x

Training frequency and time allocated was the same for all methods: two modules per week for 3 weeks for a total of six. Tests were given before and after the allocated time. The 2 h sessions were offered twice each week with the intention of training at least 90% of the clinicians at each facility.

All participants received a printed version of the national STI guidelines. For the lecture arm, a trainer taught two modules and gave participants handouts for those modules. For the computer arm, computers and assistance with navigation were provided. For the paper-based arm, participants received printed modules, which they could keep. No training or materials were provided to staff in the control clinics.

Outcomes

Feasibility was assessed by the number of clinicians who completed the pre-test for each module as a percentage of the number of clinicians at the clinic who consented to participate in the study. STI knowledge was measured with five, case-based, multiple choice questions per module. Module 4 was the exception because one question was withdrawn. Case-based questions began with a description of a case that the participants might encounter, and then posed questions about managing that case.19

We measured whether or not clinicians completed five STI management tasks during an unannounced SP encounter, and the primary outcome was total number completed. SPs are widely used in medical and nursing education in the USA,20 and unannounced SPs are considered the gold standard for measuring quality of care.21 Unannounced SPs have been used to measure outcomes of interventions for syndromic management of STIs in pharmacies22 23 and clinics.9 22 24–26

Professional actors who were fluent in Setswana attended a 3-day SP training and 1-day pilot in September 2013. Actors were trained on a standard script for vaginal discharge syndrome for women and male urethritis syndrome for men. All scripts were adapted to the actor's age, and other personal characteristics. To prevent potential bias due to acting quality, the SPs encounters were balanced, meaning that each SP visited a similar number of clinics in each arm during each time period.

After the encounters, SPs provided information about five STI tasks: correct medication, HIV test, condoms, partner notification slip and genital exam (see table 3 for exact wording). To avoid ingesting or removing drugs from the clinic, SPs disclosed at the end of the encounter. The clinicians recorded the drug name, dose, mode of administration, frequency and duration on a Medication Slip, and all five elements had to be correct.

Table 3

Sexually transmitted infection (STI) tasks completed by time and arm (percentage)—completed visits only

			Training method
	Arm 1 Control	All training methods	Arm 2 Lecture	Arm 3 Computer-based	Arm 4 Paper-based
Complete cases
Pre-training	29	99	33	34	32
Post-training	28	86	30	29	27
Were you offered correct medications?
Pre-training	10 (34.5)	30 (30.3)	11 (33.3)	11 (32.4)	8 (25.0)
Post-training	9 (32.1)	27 (31.4)	10 (33.3)	9 (31.0)	8 (29.6)
Were you offered an HIV test?
Pre-training	18 (62.1)	51 (51.2)	18 (54.6)	17 (50.0)	16 (50.0)
Post-training	17 (60.7)	51 (59.3)	24 (80.0)	10 (34.5)	17 (63.0)
Were you provided with condoms?
Pre-training	13 (44.8)	22 (22.2)	9 (27.3)	6 (17.7)	7 (21.9)
Post-training	10 (35.7)	26 (30.2)	12 (40.0)	5 (17.2)	9 (33.3)
Were you given partner notification slips?
Pre-training	10 (34.5)	29 (29.3)	16 (48.5)	7 (20.6)	6 (18.8)
Post-training	11 (39.3)	24 (27.9)	12 (40.0)	5 (17.2)	7 (25.9)
Were you offered a genital exam?
Pre-training	11 (37.9)	39 (39.4)	10 (30.3)	14 (41.2)	15 (46.9)
Post-training	13 (46.4)	37 (43.0)	15 (50.0)	12 (41.4)	10 (37.0)

Randomisation

Forty PHC clinics were randomised to four parallel arms (1:1:1:1 balance): arm 1 was control, arm 2 was lecture, arm 3 was computer and arm 4 was paper-based. Sites were randomised in strata to control for two characteristics of PHC clinics: subdistrict and operating hours, meaning 24 h services versus fewer hours. The randomisation was conducted on 30 September 2013 before the pre-training SP visits and knowledge tests.

Data management

A research coordinator met each SP after each encounter to record information about the five STI tasks on the SP Encounter Form and conduct a debriefing. Data from the SP Encounter Form, Medication Slip and SP Debriefing Questionnaire were entered using REDCap Software, V.5.8.2 (Vanderbilt University, Nashville, Tennessee, USA) and verified by comparing entered data against the paper forms. All medication data were checked by a clinician for accuracy. Total STI knowledge test scores for each clinic, module and time period were entered on a Microsoft Excel spreadsheet (Microsoft, Redmond, Washington, USA).

Statistical analysis

Descriptive statistics were calculated on training attendance and STI tasks performed in each time period and arm. STI knowledge test scores were compared pre/post with a two-sample t test. The hypothesis about changes in STI tasks completed was tested with the post training/pre-training relative risk (RR) of the number of tasks completed during SP encounters in the training arms compared to the RR of the control arm (see below). Analyses were performed with Stata V.11 (Statacorp, 2009 College Station, Texas, USA).

The comparison of STI knowledge test scores was conducted with complete cases, where the unit of analysis was the clinic module. A complete case meant that at least one person at the clinic completed the pre-test for a module and one person completed the post-test. A sensitivity analysis was conducted in which an average score of zero was assigned to clinic modules when no one at a clinic completed a test for the module and all clinic modules were included in the analysis.

Complete cases for SP encounters were analysed as binomial counts of the number of STI tasks performed correctly. We used a generalised linear model with a Poisson family and log link and main effects for arms, time period and their interaction, controlling for gender of the SP. The coefficients for arm and time were RR and the coefficient for Post*Arm was the ratio of RR (RRR) of the post training/pre-training RR in the training arm to the post training/pre-training RR in the control arm. All regression analyses were clustered on the clinic with robust SEs to adjust for using the Poisson instead of the binomial family. Results are presented with 95% CIs.

Two sensitivity analyses were conducted: controlling for clinic operating hours, and incomplete visits were coded as tasks not completed and all SP encounters were included in the analysis.

Results

Participant flow

Figure 1 reports the flow for PHC clinics and training participants. Clinic managers and clinicians were recruited and enrolled from August to October 2013. Total number of SP encounters and number of visits by individual SPs were similar across arms and time periods because of their balanced distribution. There were two exceptions: (1) 32 incomplete visits were not included in the main analysis, and (2) two actors who visited PHC clinics before training were not available after training and were replaced by two actors who used their scripts. Reasons for incomplete visits by arm are reported in figure 1.

Feasibility

As shown in figure 1, an average of 22 of 71 (31%) of clinicians who consented to participate completed the pre-tests in the lecture arm, 15 of 59 (25%) in the computer arm and 25 of 65 (38%) in the paper-based arm. Participation on the pre-test was 17% at clinics with 24 h services and 40% at other clinics.

STI knowledge

When all training methods were combined, average test scores increased from 52.5% pre-test to 65.7% post-test, an absolute increase of 13.2% (p<0.001), as shown in table 2. The increases were statistically significant for all participants for each module and for total score for each arm. The increase in total score was largest for the lecture arm. In sensitivity analyses, results were substantially the same as the complete cases analysis, but average test scores were lower especially for the post-test. Scores were missing for 37 of 240 clinic modules for the post-test compared with 10 for the pre-test.

Table 2

Effect of sexually transmitted infection (STI) training on case-based test results

Module		Training (all methods combined)					Lecture
	N	Pre (%)	Post (%)	Change	p Value	n	Pre (%)	Post (%)	Change	p Value
1 STI introduction and patient history	22	57.2	71.1	13.9% (7.1% to 20.7%)	<0.001	9	51.3	78.7	27.4% (20.5% to 34.2%)	<0.001
2 STI physical exam and counselling	21	53.7	65.5	11.8% (3.9% to 19.8%)	0.006	9	52.1	73.9	21.7% (11.6% to 31.9%)	0.001
3 Male urethritis syndrome	22	57.9	75.7	17.8% (9.6% to 26.0%)	<0.001	10	63.5	77.3	13.8% (1.3% to 26.2%)	0.034
4 Vaginal discharge syndrome	20	56.8	61.7	4.8% (−2.8% to 12.5%)	0.200	10	49.7	62.5	12.8% (−0.1% to 25.7%)	0.051
5 Genital ulcer syndrome	19	46.0	62.5	16.6% (6.0% to 27.1%)	0.004	7	43.3	61.9	18.6% (−12.7% to 49.8%)	0.196
6 Lower abdominal pain	18	40.9	54.7	13.8% (2.5% to 25.1%)	0.019	7	27.6	45.2	17.6% (−4.1% to 39.3%)	0.094
Total	122	52.5	65.7	13.2% (9.8% to 16.5%)	<0.001	52	49.2	67.7	18.5% (13.1% to 23.9%)	<0.001
Module		Computer-based training					Paper-based training
1 STI introduction and patient history	3	74.0	63.3	10.7% (−6.1% to 2.7%)	0.958	10	60.6	63.3	2.8% (−5.7% to 11.2%)	0.480
2 STI physical exam and counselling	2	52.0	52.0	0.0% (−50.8% to 50.8%)	1.000	10	55.5	60.8	5.3% (−7.9% to 18.5%)	0.387
3 Male urethritis syndrome	5	53.2	74.7	21.4% (6.3% to 36.4%)	0.017	7	53.1	74.0	21.0% (−1.8% to 43.7%)	0.065
4 Vaginal discharge syndrome	3	63.2	54.2	−9.0% (−32.4% to 14.3%)	0.238	7	64.3	63.7	−0.6% (−9.3% to 8.1%)	0.873
5 Genital ulcer syndrome	5	38.0	58.0	20.0% (2.4% to 37.6%)	0.034	7	54.3	66.3	12.1% (1.5% to 22.8%)	0.031
6 Lower abdominal pain	4	31.7	53.8	22.1% (−32.8% to 76.8%)	0.290	7	59.5	64.8	5.2% (−6.6% to 17.1%)	0.320
Total	22	48.5	62.1	13.6% (5.0% to 22.3%)	0.004	48	57.9	65.1	7.2% (2.5% to 11.9%)	0.004

STI tasks completed

The majority of SP encounters were with female clinicians (93%) and all but two clinicians were nurses (99.2%). Before training, the average number of STI tasks completed was 1.63 (median=1, IQR 0–3). Clinicians gave correct medications to 30% of SPs, offered an HIV test to 51%, provided male and/or female condoms to 22%, gave partner notification slips to 29% and offered genital exams to 39%. Despite the random assignment of PHC clinics to arms, there were differences in percentages of tasks completed across arms before training. Table 3 reports results by arm. Consequently, the ratio of RR or difference-in-difference analysis was necessary.

After training, the average number of STI tasks completed was 1.73 (median=2, IQR 0–3). There was no change in the number of STI tasks completed in the control arm. When all training arms were combined, there was an 11% increase in the number of STI tasks completed relative to the control arm (RRR=1.11, 95% CI 0.67 to 1.84). Looking at effects across training arms, there was a 26% increase (RRR=1.26, 95% CI 0.77 to 2.06) in STI tasks completed in the lecture arm relative to the control arm, a 17% increase (RRR=1.17, 95% CI 0.59 to 2.28) in the paper-based arm and a 13% decrease (RRR=0.87, 95% CI 0.40 to 1.90) in the computer arm.

Incorrect medications were prescribed for 88 of 121 (73%) women and 78 of 121 (64%) men. Among incorrect medications, 42 (48%) women and 25 (32%) men did not receive any drugs to treat an STI. The clinician prescribed ciprofloxacin instead of cefixime to 4 (5%) women and 35 (45%) men. Duration of doxycycline prescription was incorrect for 11 (13%) women and 3 (4%) men. Dose and frequency of metronidazole was incorrect for 21 (24%) and 13 (15%) women, respectively, and it was incorrectly prescribed to 10 (13%) men.

In sensitivity analyses, results were the same when controlling for clinic operating hours. Clinics with 24 h services were associated with a 16% decrease in the number of STI tasks completed (RR=0.84, 95% CI 0.58 to 1.21). Results were substantially the same in analysis of all SP encounters in which incomplete visits were coded as tasks that were not completed.

Discussion

The pilot study identified challenges with attendance at clinic-based sessions across all training methods. Despite working closely with clinic managers to schedule training at convenient times, participation in the training programme overall was only 31%, and in the computer-based method was only 25%. Some clinicians may have been off duty during the trainings, as suggested by the relatively low participation at PHC clinics with 24 h services. Some clinicians were unfamiliar with computers and may have found this method challenging. STI knowledge test scores increased, however, suggesting that the training methods may have been effective among participants.

Lecture and paper-based training methods were associated with improvements in syndromic management of STIs. Computer-based training was associated with a decrease in the number of STI tasks completed. None of the effects were statistically significant. Some lectures were delivered in Setswana, whereas paper-based and computer-based trainings were in English.

The clinic-level design was based on 90% participation in training, with the expectation that most clinicians who treated a SP post-training would reflect the effects of the training programme. In practice, SP encounters were not necessarily with clinicians who participated in training. Although we did not identify the clinicians and link them to training records, future SPs could simply ask the clinicians if they participated in the training when they disclose at the end of the encounter.

Limitations

The preliminary test of the effect of the training on STI tasks completed could be characterised as “absence of evidence, rather than evidence of absence” 27 because the sample size was too small. The pre-training mean number of STI tasks completed was 1.63 out of 5, with an SD of 1.48. In future tests, at least 251 SP visits would be needed per arm to detect an absolute increase of 0.5 STI tasks performed to 2.13, and 84 visits to detect an absolute increase of 1.0 STI tasks performed. Accounting for clustering of clinicians at facilities would increase the sample size. A difference-in-difference analysis as opposed to a comparison of means would also increase the sample size.

Similarity of the scripts across SPs of the same gender may have contributed to some clinicians suspecting the SPs' identity before she/he disclosed. During debriefings, 11 SPs (4.5%) reported that the clinician suspected his/her identity. In these encounters, clinicians may have performed better than normal, but even their best practice left room for improvement. Also, the clinics in the sample had wider perceived gaps in STI management than other clinics. Effects could be larger at other clinics that more readily adopt new skills or smaller at clinics with less room for improvement.

Comparison with previous studies

The quality of STI care was low. In previous research, a national PHC study reported that STI care was a well-established component of PHC services and available in >90% of PHC clinics in 1998, 2000 and 2003.28 A national survey of the quality of care reported that ciprofloxacin was available in 86% of PHC clinics and condoms were available in 92% and 79% of urban and rural PHC clinics, respectively.29 During SP visits in 2002, the percentage of STI tasks performed correctly was lower than in table 3 for three tasks: (1) physical exam was offered to 10% of women and 21% of men, (2) HIV test was offered to 8% of SPs and (3) partner notification slips were given to 18%.30 Condoms were provided without asking to 69% of SPs in 2002 compared with 32% among all SP post-training encounters.

Implications for future STI training

Clearly additional effort to improve the quality of STI care is warranted. Both paper-based and computer-based training methods could be used to train new staff and clinicians who work on shifts outside normal training hours. During the pilot study, a participant in the paper-based arm could access the content whenever and wherever she/he wanted, even though the pre-tests and post-tests were limited to the 2 h sessions at clinics. If the PHC clinic had routine access to a computer, participants could complete basic computer tutorials to improve their computer skills and then complete computer-based training. The voiceover for the computer-based content could be available in multiple languages.

Although the SP encounter was a data collection method for this pilot study, SPs are primarily a training method20 that supports participants to apply new clinical skills. In the future, when SPs disclose at the end of the encounter, they could give clinicians a handout about STI tasks. Or a training coordinator could provide constructive feedback to clinicians after the SP encounter.

Finally, the national STI training programme could be expanded to address health system challenges, such as stock-outs of cefixime, condoms and partner notification slips. A subsequent, national evaluation of STI service delivery found that oral cefixime was only available in 45.2% of health facilities (Pamela Kohler, personal communication). One solution tested by Colvin et al25 would be to provide ‘syndrome packets’, in addition to training, with correct antibiotics, 10 condoms, a partner notification card, and an information leaflet.

The South African National Department of Health supported clinic-based training for health professionals on revised sexually transmitted infection (STI) national guidelines. Three training methods were compared: lecture, paper and computer-based.

Attendance at clinic-based training was lower than expected, averaging 31% of clinicians.

STI knowledge scores increased on case-based tests across all three training methods, suggesting that the trainings were effective among participants.

The average number of STI tasks completed correctly was 1.63 out of 5 pre-training and increased by 11% post-training.