Republished: unlocking the potential: longitudinal audit finds multifaceted education for general practice increases HIV testing and diagnosis.

BACKGROUND: This longitudinal study aimed to evaluate the impact of a multifaceted educational intervention (Sexual Health in Practice, SHIP) on general practice HIV testing rates in a high prevalence London area. INTERVENTION: SHIP offered training in sexual health clinical skills to general practitioners (GPs) and practice nurses (PNs) in Haringey. SHIP training aims to break down stigma in sexual health and provide sexual history and communication tools (e.g. differential diagnosis), and provides resources to practices (including condoms).
DESIGN: Numbers of GP HIV tests were collected from laboratories for 24 months prior, 19 months during and 5 months after training. Attendance data and practice list sizes were obtained.
RESULTS: 39 of 51 practices had at least one trained individual. These `trained' practices conducted an average 526 HIV tests p.a. before training began which rose to a projected 1556 p.a. (on the basis of the last 6 months of data). Testing rates of trained and untrained practices increased from 2.29 to 6.66 and 1.54 to 1.90 tests/1000 registered patients/year (p=0.0016 and p=0.5195) respectively. The rate of positive diagnosis was high in the trained group (18.0 and 16.7 positives/1000 tests before and after training began; p=0.7908). This equates to a rise from 9.5 to 22 new diagnoses p.a.
CONCLUSIONS: The training intervention has been found to significantly increase general practice HIV testing rates in the absence of financial incentives. Positivity rates are substantially higher than that found in pilots of screening in London, suggesting that the training nurtured and supplemented complex clinical skills.

Introduction

A quarter of those with HIV in the UK are undiagnosed.1 Those diagnosed late are more likely to die and to develop serious illness.2 There is also a higher risk of onward transmission of the virus, which can be prevented with early treatment.3 HIV testing in the UK must, therefore, increase substantially so as to lead to improvement.

It is recognised that general practice is highly accessible and well used by people, including those in key groups at higher risk of HIV,2 4–6 as well as by people with relevant symptoms.5 7 In response to UK national HIV testing guidelines,8 screening of newly registering patients in general practices in high prevalence areas has been piloted. However, a Department of Health review of the findings9 concluded that there should be further investigation of models for testing in general practice ‘since not all primary care facilities conduct new patient health checks’. In a recent House of Lords report on HIV,10 11 it was noted that ‘professionals, most notably general practitioners (GP), must become more confident and competent in [HIV testing]. Training and education are important tools to use to achieve this … [and] must incorporate efforts to address HIV-related stigma’. Similar problems are identified, and solutions proposed, at a Europe-wide level.12

Guidelines and educational approaches8 13–15 have been used to try to increase GP HIV testing rates. Written material is highly valuable, as it brings consensus and sets standards. However, Cochrane evidence reviews conclude that there is no evidence that ‘printed educational materials’ had an impact on patient outcomes,16 and the effects of educational meetings on complex clinical behaviours is small.17 By contrast, multifaceted interventions combining, for instance, didactic and interactive education with other approaches (eg, resource provision, feedback to clinicians, audit and use of local clinical leads) may effect change,18–21 and a close link with the clinical context may help.22 One UK study of a Sexually Transmitted Infection Foundation course23—a mixed interactive and didactic course for those with a clinical interest—found no increase in HIV testing post-training. There is currently no basis for assuming that any intervention will change a complex clinical behaviour, such as GP HIV testing.

Intervention

In this paper, we report on the effect of a multifaceted educational intervention, SHIP (Sexual Health in Practice, Heart of England NHS Trust) on GP HIV testing rates. SHIP addresses all aspects of sexual health including sexually transmissible infections, health promotion and contraception. With respect to HIV, it aims to remove the barriers to HIV testing and teaches a systematic clinical approach to testing (see online supplementary document).

This paper evaluates the effect of the SHIP intervention on general practice HIV testing rates. For this audit, attendance at the sessions relevant to HIV testing only were considered (see online supplementary document).

SHIP teaches a cohesive clinical model for sexual health care in general practice. This includes approaches to the four situations in which HIV tests may be offered or carried out:

Patient request

Opportunistic testing for those found to be at risk through history taking

Diagnostic testing for selected patients with relevant symptoms

Screening (eg, those referred for abortion or, in high prevalence areas, seeking contraception).

To provide HIV tests, GPs and practice nurses (PNs) need good clinical knowledge, effective communication skills and also risk assessment and sexual history-taking skills. To apply these, confidence and motivation is needed. There are multiple perceived and actual barriers to all forms of HIV testing in general practice.12 15 Overcoming these is vital to improving testing rates.15 24

Kirkpatrick outlined a hierarchy for the evaluation of clinical learning in response to educational interventions,25–28 number 1 being the least valuable:

Participation: numbers and types of participants

Reaction of participants

Learning: knowledge, skills, attitudes, confidence, commitment

Changes in practice and behaviour

Outcomes for patients

Return on expectations—what degree of change would indicate success? What is the cost-benefit analysis?

This hierarchy offers a stark reminder that the responses of participants, while they may help improve and develop training, are in fact a weak test of effectiveness.

In an earlier publication,29 SHIP demonstrated the value of its training by the use of Kirkpatrick's outcomes 1 through 3. In Haringey, general practice laboratory HIV testing rates were selected as an outcome measure because they are:

clinically significant, with potential to reduce undiagnosed HIV (if positivity rates are high)

not ‘self-reported’ by practices

easily measured

an indication of reduction in stigma associated with HIV testing and improved communication in consultations.

An increase in GP HIV testing rates might, therefore, fulfil Kirkpatrick's Levels 4 and 5.

Birmingham SHIP was commissioned by Haringey Primary Care Trust (PCT), North London, to support their Sexual Health Strategy (see online supplementary document). The 51 medical practices within Haringey are in four collaboratives (approximately geographical groups). Haringey PCT has a population of around 270 000 registered for healthcare, and has the 12th highest prevalence of HIV nationally (6.8/1000 adults aged 15–59 in 2009).30 Incentives to attend training included access to free condoms and pregnancy testing kits and locum costs. No incentives were offered for HIV testing.

SHIP training in Haringey included five rounds of four afternoon sessions. These were run in March and June of 2010 and January, June, September and October 2011 (see online supplementary document).

Methodology

The total registered population in Haringey was 271 697 in April 2011. A PCT list of Haringey practices and clinical staff and their numbers of registered patients (list size) was obtained.

Haringey GP HIV tests are sent to one of three laboratories: North Middlesex, Whittington or Homerton hospitals. Monthly, numbers of tests by practice were requested from laboratory managers for 24 months retrospectively, then prospectively through 19 months of training, and 5 months post-training, totalling 48 months of data. Equivocal results were deleted on the assumption they were repeated. Laboratories were asked to identify repeat positives (ie, with identical patient ID) which were removed.

Relevant training for GPs consisted of two afternoon sessions, the first addressing clinical and communication skills relevant to sexual health, and the second focusing on HIV. PNs attended three afternoon sessions so that sexual health promotion skills could also be taught (see online supplementary document). Training attendance data was collected from sign-in sheets at the training sessions, and time of departure of early leavers noted. For the sake of comparison, practices were defined as either ‘untrained’ practices (those practices with no clinical staff to complete all relevant sessions, even if some had attended one session) or ‘trained’ (any practice with at least one health professional who had attended all relevant sessions). Three practices merged during the follow-up period, and their data was merged and treated as one practice throughout. One practice of <1000 patients closed during the study. It was excluded from analysis.

A linear regression analysis was performed to investigate whether the monthly number of Haringey HIV tests increased over time. An interaction term was fitted to investigate whether the change in the number of tests performed was significantly different once SHIP training was introduced. Next, the rate of HIV testing was calculated by dividing the total number of tests performed in a month by the list size of the practice. These rates were then rescaled to give the rate of testing per 1000 patients per month. The rate of testing at each practice in the period before SHIP was stratified by whether practices went on to become ‘trained’ practices or not. The rates of HIV testing for the latest 6 months of data were then calculated, and compared with the preintervention rates using Wilcoxon and Mann–Whitney tests, as the rates were not normally distributed. The proportion of all HIV tests that were positive before and after the intervention was calculated and compared using a χ2 test. Finally, Poisson regression was used to investigate the impact of SHIP training on the rate of testing taking into account the month that the practice trained. A term was fitted to investigate the average change in the practice rate of testing once they had attended SHIP. Generalised Estimating Equations were included to account for the fact that each practice contributed data in more than 1 month. March 2010 data was a transitional month because relevant skills for HIV testing were taught on the 4th but no one had completed training until the 25th. This month was removed from all statistical data analysis except from the graph of monthly tests, the formal test for interaction, and the Poisson distribution analysis. The statistical package used was SAS V.9.3 (SAS Institute Inc, Cary, North Carolina, USA).

At no point was patient-identifiable information collected, and there was no randomisation.

Results

Table 1 summarises attendance by staff over five rounds of training. Of a total of 150 participants, three participants left a training session early, but were classed as full attendance. By the end of five rounds of training, 19 of the 39 trained practices had multiple attenders. Table 2 compares trained with untrained practices by population coverage.

Table 1

Number and percentage of doctors trained, nurses trained and practices ‘trained’*

Training round date	March 2010	June 2010	January 2011	June 2011	Sept–Oct 2011
Doctors trained of 190 (Cumulative %)	14 (7.4)	11 (13)	10 (18)	9 (23)	8 (27)
Nurses trained of 126 (Cumulative %)	6 (4.8)	6 (9.5)	11 (18)	4 (21)	1 (22)
Cumulative number of practices ‘trained’ of 51 (Cumulative %)	15 (29)	21 (41)	30 (59)	37 (73)	39 (76)

*At least one member of staff having completed training.

Table 2

Comparing trained practices and untrained practices by population coverage—using April 2011 list size data

	Trained practices	Untrained practices
Number	39	12
Total population covered	195761	37968
Mean practice list size	5020	3164

Results of the linear regression model suggest that, during the 24-month pretraining period, the number of HIV tests performed in Haringey was slowly increasing at a non-significant rate, with an extra 0.1 tests performed per month (95% CI −0.3, +0.6; p=0.59). When considering the time period after the introduction of SHIP, the number of tests performed increased at an estimated extra 3.5 tests per month (95% CI +2.7, +4.4; p<0.0001). A formal test for interaction considering the entire follow-up period demonstrated that introduction of SHIP was associated with a significant increase in the number of HIV tests performed in Haringey PCT (p=0.0004). It can also be observed from figure 1 that testing appears to increase after each training round, apart from round 1. The number of tests conducted dipped in December 2010 and December 2011 (figure 1), probably due to a reduction in non-urgent clinical activity around Christmas.

Figure 1

Training rounds (vertical bars aligned with the start of each training round) and monthly number of HIV tests conducted by Haringey practices (March 2008 to February 2012 plotted at the end of their corresponding month). Data for graph is available in online document ‘figure 1 data’.

Table 3 shows that the rate of testing among trained practices increased between the 24 months prior to SHIP and the last 6 months, a statistically significant change (Wilcoxon test p=0.0016). By contrast, there was no evidence of an increase in HIV testing rates in the untrained group (p=0.5195). The total testing rate for Haringey PCT also significantly increased (p=0.0014). Prior to SHIP, there was a significant difference in testing rates between the untrained and trained practices (Mann–Whitney U test p=0.0137).

Table 3

Testing rate of trained and untrained practices

	Trained (39 practices)	Untrained (12 practices)	Total (51 practices)
	Tests/1000 pts/year	Tests/1000 pts/year	Tests/1000 pts/year
March 2008–February 2010 (24 months)	2.29	1.54	2.18
September 2011–February 2012 (6 months)	6.66	1.90	5.99
Increase (%)	+191	+23.4	+175

Based on first four rounds of training before training and in the 6-month period after the first four rounds of training were complete.

A Poisson distribution analysis, taking into account the staggered nature of the intervention and using all 48 months of data, showed that there was a significant effect of completion of training on the rate of tests performed. A 14% increase in the number of tests performed is observed every 6 months after training started (rate ratio=1.14; 95% CI 1.07 to 1.22; p=0.0002).

Table 4 illustrates the wide range in diagnosed HIV prevalence within Haringey, with northeast and southeast collaboratives serving extremely high prevalence areas. Estimates of undiagnosed HIV in London were thought to add another 26% to these figures in 2009.30

Table 4

Participation in and response to Sexual Health in Practice training by Haringey collaborative (geographical area) and approximate local diagnosed HIV prevalence

	West	Central	Northeast	Southeast	Haringey Total
Total number of practices	14	12	15	10	51
Approximate diagnosed HIV prevalence (per 1000 adults aged 15–59)*	0–4	4–10	6–20	4–20	6.8 (Actual, not approximate (30))
Total population served (to nearest 100)	86600	60500	78800	45900	271700
Number of trained practices (minimum one trained nurse or doctor)	11	10	11	7	39
Percentage population served by trained practices	86	92	86	78	86
Number of high performing trained practices†	5	5	7	3	20
Percentage population served by high performing trained practices	63	55	66	46	59
Number of high performing practices with additional trained staff	4	2	6	1	13
Number of trained but lower performing practices (of which non-responders‡)	6 (4)	5 (4)	4 (2)	4 (2)	19 (12)
Percentage affected population (served by non-responders) (%)	23 (14)	37 (16)	19 (5)	32 (12)	27 (11)
Number of low performing (of which non-responding) practices with additional trained staff	2 (1)	1 (1)	3 (1)	0 (0)	6 (3)
Total collaborative testing rate (/1000 registered patients/p.a. before training)	2.44	1.27	2.31	2.68	2.18
Total collaborative testing rate (/1000 registered patients/p.a. in last 6 months)	5.08	2.51	10.5	4.49	5.99
% increase	108	98	355	68	175

Testing rates were calculated by taking mean numbers of registered patients in the month of April in each relevant year.

*Collaborative diagnosed HIV prevalence data was estimated from Middle Superior Output Area HIV prevalence map.30

†High performers defined as practices that have increased their testing by over 50%.

‡Non-responding practices are those with no change, or a decrease, in their testing.

The number of positive HIV tests in trained practices increased from 19 over the 2 years before training to 39 in the 2 years since training began. There is no evidence of a difference between the rate of HIV positive tests over time (χ2 test statistic 0.07, p=0.7908), though numbers are small (table 5).

Table 5

Total number of tests and total number of positives in Haringey PCT over the period of study

	Trained (39 practices)
	Number of tests done	Number of positives	Positives/1000 tests
March 2008–February 2010 24 months pre-training (annual average number)	1056 (528 p.a.)	19 (9.5 p.a.)	18.0
March 2010–February 2012 24 months after training began (annual average number)	2333 (1167 p.a.)	39 (19.5 p.a.)	16.7
September 2011–February 2012 Latest 6 months of data (projected annual average number)	778 (1556 p.a.)	11 (22 p.a.)	14.1

Discussion

The SHIP intervention produced a substantial effect. With the training of 27% of doctors and 22% of nurses in Haringey, SHIP was associated with an increase in HIV testing rates (p=0.0004) and a high rate of positives (16.7/1000). Furthermore, a large proportion of all Haringey practices (20/51) serving 59% of Haringey's registered population, increased their testing rates by more than 50%. The biggest increases in testing rates (350%) were seen in the northeast collaborative, the larger of the two most deprived areas of Haringey, and with the highest prevalence of HIV with only 11 of 15 having had a member of staff trained.

These findings support the conclusion that SHIP, as an educational intervention, led to changes in clinical practice and behaviour (Level 4 Kirkpatrick's Hierarchy). Furthermore, because additional HIV-infected patients appear to have been diagnosed as a result of the SHIP intervention, improved outcomes for patients have been demonstrated (Level 5). It is unusual for clinical educational interventions to achieve this level. It is not possible, however, to state whether any of these HIV diagnoses were achieved early (which would be of greatest benefit to the individual and the population). In addition, only a controlled study, including economic evaluation, could fulfil Kirkpatrick's Level 6—a cost-benefit analysis. The prevalence of HIV in the area would affect any such analysis, although SHIP training may be of value in lower prevalence areas because late diagnosis of HIV is more common,31 and training may benefit other aspects of sexual health.

This longitudinal clinical audit has a number of strengths. The study involved the collation of data over an extended period of time and was able to reveal trends for 2 years before and after training. The selected outcome measure (numbers of general practice HIV tests) is meaningful to patient outcomes. The intervention increased general clinical HIV testing in general practice without financial incentive to test (only to train). Indeed, it is possible that the provision of payment per test, outside a screening programme, does not actually help practitioners overcome some of the substantial barriers to testing.

There are several limitations to this study. The effect of the training intervention was assessed by longitudinal clinical audit rather than a randomised control trial (RCT). However, this was the best method available within the timescales and funding of this commissioned intervention. Furthermore, there are advantages to measuring the impact of an educational intervention in the real-life primary care environment, potentially not measurable with a tightly controlled RCT. The ‘first-come-first-served’ system of enrolment for SHIP training attracted staff from practices with a slightly higher baseline testing rate, meaning untrained practices were not necessarily comparable and could not, therefore, act as a control group. The number of tests per month conducted in Haringey demonstrated a small increase just before training began. This might be explained by SHIP advertising and correspondence from December 2009, as has been shown in the past with educational material sent by post.32 The broader trend does not appear to be explained by other interventions. While, for example, UK National Guidelines for HIV Testing8 have been introduced, neither this nor any other local or national intervention we could find coincided with the introduction of SHIP in Haringey. The intervention was spread over a 19-month period, making ‘before-after’ comparison difficult.

Recruitment to SHIP training varied, and the last course in the period of study showed a dip in attendance, although the most recent training (run after the end of the study period of this paper) showed a return to the usual numbers of attendees.

The 6 months’ data used in some subanalyses as the ‘after’ period may not be representative of a year's testing data, and is likely to underestimate yearly testing rates due to the observed annual fall in HIV testing in December. In addition, while individual GP and PN attendance data was collected, only whole-practice HIV testing data was available. This means that the effect of training may have been diluted in the practice setting, particularly in the larger practices if only one person had completed training. Finally, some practices had not yet sent a clinical member of staff to be trained, and some did, but showed no response to training (some of these performing no HIV tests). This study is not able to elucidate why this was the case.

Unlike other interventions,23 SHIP aims to attract GPs and PNs whether or not they have an interest in sexual health. The evidence of benefit presented here echoes the conclusions of Cochrane reviews on educational interventions, and suggests that a systematic tackling of health professionals’ perceived barriers to testing with the support of peer-educators, are important factors.

It is likely that further gains will be made in HIV testing, if SHIP training continues, although the rate of increase may slow if the most motivated staff have trained first. Thus, 65% of the 20 higher-performing trained practices and 32% of 19 lower-performing trained practices had multiple attenders at training, suggesting that having more than one trained individual in a practice reinforces change.

The cumulative increases in testing over time suggest that the impact of each round of training is sustained. Individual practice HIV testing rates may also be responsive when a new diagnosis of HIV is made in the practice, reinforcing the value of changed clinical practice. Several new positives were found in the northeast collaborative, and this may be one factor in the dramatic increases in testing rates seen in this very high prevalence area. Numbers of new HIV positives are not high enough to enable closer examination of this issue. Qualitative work could give insights into this.

Rates of positives were substantially above levels achieved with screening trials.9 However, while duplicate positives were removed from our dataset by the laboratories, it is possible that some ‘new’ diagnoses of HIV counted here were in fact in patients who already knew they were HIV positive. This cannot be explored here due to the simple study design, but we think it unlikely to account for many—or indeed any—of the cases. However, if some patients used new opportunities for HIV testing to help them ‘disclose’ their HIV status to their GP, this is likely to be a good thing for their future clinical care. The high rates of positivity, and high increases in testing rates in the highest prevalence area, may reflect the nature of the intervention, which aims to teach a range of clinical skills to help clinicians select patients who will be offered a HIV test. These skills include employing strategies for the clinical management of those with relevant symptoms and the use of risk assessment for opportunistic offers of testing.

The findings from this study have implications for policy makers. Commissioning of SHIP training for local practices seems likely to support the implementation of increased HIV testing and diagnosis in the primary care setting. Evidence given here suggests that SHIP training also changes clinical practice most in the areas with the highest need. Considering a recent statement from the House of Lords that the removal of stigma around HIV will be a slow process over a number of years,10 our findings suggest a faster route, at least towards its reduction.

The SHIP measure of HIV testing rates (the number of tests per annum per 1000 registered patients) provides a simple shorthand rate that practice staff can calculate for themselves, and forms a useful basis for discussion and education. Such rates are outside the remit of existing antenatal, new patient or other screening programmes. On the basis of the individual Haringey practices testing at the highest rate, a guide figure for high prevalence areas might be over 30 tests/1000 patients per annum. This proposed benchmark takes no account of population age distribution, or local HIV prevalence. It is simply a guide figure to stimulate reflection for practitioners considering what might be appropriate for their own practice, given its own individual circumstances. It is not intended as a measure of quality of care.

Further investigation into this educational intervention is indicated, perhaps using a step-wedged randomised—or randomised controlled—trial. Areas for further study include a comprehensive cost-effectiveness analysis of SHIP training (encompassing other sexual health outcomes of SHIP); qualitative research to better understand clinicians’ response to training (including non-response); and whether the training can have an impact on early diagnosis of HIV infection.

Sexual Health in Practice is a multifaceted training programme for general practices implemented in a high prevalence area of North London

It was associated with a substantial increase in the number of HIV tests done over a 19-month period of five training rounds (formal test for interaction: p=0.0004).

Numbers of positives identified by general practice rose from an average 9.5 per annum before training to a projected 22 per annum (on the basis of the last 6 months’ data).

The highest increases in HIV testing were seen in the locality with the highest prevalence of HIV.