Preventive medical care in remote Aboriginal communities in the Northern Territory: a follow-up study of the impact of clinical guidelines, computerised recall and reminder systems, and audit and feedback.

BACKGROUND: Interventions to improve delivery of preventive medical services have been shown to be effective in North America and the UK. However, there are few studies of the extent to which the impact of such interventions has been sustained, or of the impact of such interventions in disadvantaged populations or remote settings. This paper describes the trends in delivery of preventive medical services following a multifaceted intervention in remote community health centres in the Northern Territory of Australia.
METHODS: The intervention comprised the development and dissemination of best practice guidelines supported by an electronic client register, recall and reminder systems and associated staff training, and audit and feedback. Clinical records in seven community health centres were audited at regular intervals against best practice guidelines over a period of three years, with feedback of audit findings to health centre staff and management.
RESULTS: Levels of service delivery varied between services and between communities. There was an initial improvement in service levels for most services following the intervention, but improvements were in general not fully sustained over the three year period.
CONCLUSIONS: Improvements in service delivery are consistent with the international experience, although baseline and follow-up levels are in many cases higher than reported for comparable studies in North America and the UK. Sustainability of improvements may be achieved by institutionalisation of relevant work practices and enhanced health centre capacity.

Background

A variety of interventions to improve the delivery of preventive services in primary care have been shown to improve coverage and quality of care in North America and the UK. [1] However, there is little information on the sustainability of the impacts of such systems, or of the impact of such systems in remote settings where poor socio-economic and environmental conditions and relative lack of infrastructure impose particular challenges. Previous relevant reports that address sustainability have been limited by relatively short follow-up periods [2,3] or do not provide empirical data. [4] Recent health reform initiatives in two remote regions of the Northern Territory of Australia focussed in part on improving delivery of preventive medical services through the implementation of locally developed best practice guidelines supported by an electronic care planning, recall and reminder system. [5]

The guidelines were developed by an experienced multidisciplinary team of health professionals. [5] The incorporation of the guidelines into a computerised clinical information system enabled the generation of individualised care plans with a schedule of services specified according to one of four client age groups. The care plans are required to be updated through the entry of new clinical information at the time of each visit.

The implementation of the guidelines and information system took place in the context of a broader health reform initiative in these two regions. Other aspects of the so-called "coordinated care trials" included the establishment of a health board in each region with the responsibility for planning and purchasing a wide range of health services for the population in each region, and the associated transfer of government health department funds to these health boards. [6,7] The establishment of the two health boards had to draw on different representative and administrative structures in each region. However, the boards have followed broadly similar trajectories: rather than remaining content to purchase services from the government and other providers, they have each moved to become providers in their own right, moving to take over the community health centres: three on the Tiwi Islands and eight in the Katherine West region.[6,7] Each has developed community based programs in public health and other areas of prevention and health promotion.

The aim of this paper is to examine the levels of, and trends in, delivery of preventive medical services prior to and following implementation of the guidelines.

Method

Study setting and population

The two trial settings differed in significant respects. The Tiwi Islands cover 7,900 sq km and are located approximately 60 km north of Darwin. The population in 1996 was estimated at 2,032 persons, 90% of whom were Indigenous.[6] The two islands contain three main communities and a number of smaller settlements. The Katherine West region covers 162,000 sq km and includes several large Aboriginal communities and smaller outstations, a number of cattle stations, and a small township. The southernmost community, Lajamanu, is approximately 890 km from Darwin. The population in 1996 was estimated by the Australian Bureau of Statistics to be 2,800 persons although local government estimates are in excess of 3,000. Some 85% of the regional population are Aboriginal, drawn from ten language groups. By Australian standards the health and socio-economic profiles of these regions are very poor, and can be typified by the life expectancy of Indigenous Australians of about 20 years less than the rest of the Australian population.[8,9]

Study design and sampling

Follow up with repeated measures of a panel sample over a three-year-period was conducted to assess changes in clinical practice following the health reforms described above. Panel samples for the audits were drawn from all consenting trial participants at the end of the first intensive recruitment phase of the trial (1205 consenters from a population of approximately 2000 for Tiwi, and 1340 consenters from a population of approximately 3000 for Katherine West). The target samples of 90 in the population care plan group (all age groups together) for each trial were designed to provide an estimate within 10% of the true population rate at a 95% confidence level for each trial site. The panel samples were followed across all five audit periods reported. A stratified random sample design was used to ensure the sample included participants from each community and care plan age group. At the time of the 3 year audit the samples were supplemented by further random sampling of consenting participants to make up for movement between age groups and loss-to-follow-up. The number of participants at each audit is detailed in Table 1.

Table 1

Sample by age category and audit period

Age Category	Baseline	6 months	1 year	2 years	3 years a	3 years b
0–3 years	51	51	47	33	54	0
4–15 years	80	80	79	81	53	13
16–49 years	88	87	85	91	109	62
50+ years	46	46	46	44	55	31
Total	265	264	257	249	271	106

a Supplemented random sample (includes original panel); b remaining clients of original panel sample; Note: some of the panel sample moved into older age categories during the course of the study period.

Intervention

The clinical guidelines, electronic systems to support these guidelines, and associated staff training were introduced progressively starting in late 1998. There were three aspects of the intervention that were expected to possibly impact directly on clinical practice. First, each health centre received evidence based clinical guidelines targeted at four categories of population (0–3, 4–15, 16–49, 50+ age groups). [5] Details of guidelines are in Figure 1 and Figure 2. These standard guidelines (care plans) were developed by a multidisciplinary team of health professionals using National Health and Medical Research Council "Guidelines for the Development and Implementation of Clinical Practice Guidelines" as well as adopting local prevalence data and relevant social and cultural information. Guidelines were incorporated into a computerised information system to assign preventive services to individuals within population groups with respect to basic measurements, clinical examination and screening, immunisations / deworming, and counselling.

Figure 1

Summary of clinical guidelines for 0–3 years and 4–15 years population

Figure 2

Summary of clinical guidelines for 16–49 years and 50+ years population

Second, a computerised Coordinated Care Trial Information System (CCTIS) was installed in community health centres. This computer system maintained a database of all consented trail participants and their preventive services. CCTIS was designed to be used in 'real time' at client consultation, in order to serve as an effective reminder system in the delivery of health services according to the embedded age-group-specified guidelines. Two education officers were employed to provide on-site training to health centre staff in application of the software packages.

Third, findings from the audits were fed back to, and their interpretation discussed with, health centre clinicians and managers. This usually occurred within a few months of the conduct of each audit, and within a timeframe which allowed a response to feedback to influence findings of the next audit.

Audit method

The audits were conducted on the records of a sample of the clients who had consented to make their health records available for the trial evaluation. The audits followed the detailed guidelines (in Figure 1 and 2) and were conducted on five occasions over a three year period: baseline; and at approximately 6, 12, 24 and 36 months after baseline. Baseline audits were conducted around the same time as the introduction of the guidelines, and recorded services delivered over the preceding 12 months – i.e. before the introduction of the guidelines could have had effect.

Both the paper record and the electronic record for each client were checked for the most recent record of delivery of each service according to a schedule specified in the best practice guidelines. A service was recorded as delivered if there was a record of delivery within one month of the due date for services scheduled before 12 months of age, within 2 months for services scheduled at 12 months and up to 4 years of age, within 6 months for services scheduled at 4 years and up to 16 years, and within the past year for services scheduled for 16 years and older, except where recommended service intervals were more than one year (pap smear, mammography, pneumococcal and tetanus/diphtheria vaccine).

Information on attendance at health centres within the last 6, 12, or 24 months for each client, depending on age group, was also collected at year 1, year 2 and year 3 audits in order to determine the proportion of the population attending the health centre within specified times, and the extent of possible missed opportunities for delivery of preventive services at times when clients were attending for other reasons.

Outcome measures

The outcome of interest was adherence to guidelines in terms of delivery of scheduled services. The assessment of delivery of each specific service at each audit period was defined as a dichotomous variable (delivered/not delivered). The assessment of delivery of services for each client at each audit period was defined as a continuous variable (delivered services as a percentage of all scheduled services for that client). An example is given to demonstrate the calculation as follows: the scenario is an 11-month-old boy undergoing a baseline audit. According to the clinical guidelines, the most recently scheduled services for this infant are weight measurement, haemoglobin test, hearing examination, ear disease check, and deworming at the age of 9 months. If any one of above services was done within one month of due date (i.e. when he was 9 months old), the service was labelled as delivered, otherwise as undelivered (service based dichotomous variable). If 3 of these 5 scheduled services were carried out, the percentage of delivered services for the boy was 60% (3/5) (client based continuous variable).

Statistical methods

To determine the effectiveness of interventions over the total follow up period, multilevel analysis was conducted. [10] In view of our panel data with 5 rounds of audits, inherent dependency structure existed in repeated measures from the same individual, and clients were nested within community health centres. Therefore, the client and the health centre were defined respectively as level one and level two units.

The parameter of interest was the regression coefficient of the time variable (value of 0, 0.5, 1, 2, 3 for each audit respectively), indicating the average magnitude of improvement in delivery of scheduled services per year from baseline. The p value of the regression coefficient was used to assess whether there was a significant increasing or decreasing trend in delivery of services from year to year across the full period of follow-up.

A logistic (random effects) model was used for the dichotomous outcome for each service in each care plan group and regression coefficients were transformed to odds ratios (OR). A linear random effects model was employed for client-based continuous outcome (client's percentage of delivered services). Separate analyses were conducted for each care plan group and each community. The means of the continuous outcome measures (and associated confidence intervals) were also calculated for each audit (see data tables for each figure).

All multilevel analyses were performed using Stata version 7.0 (commands xtlogit and xtreg with defined iis for level two and tis for level one identifiers) and were adjusted for recent attendance at the health centre.

Results

Sample by age category and audit interval

The total sample at baseline included 265 clients (Table 1).

Loss to follow-up

Loss to follow-up was most marked between the year 2 and year 3 audits and in the 0–3 and 4–15 age groups (Table 1). In the 0–3 age group, and to a lesser extent in the 4–15 group, the main reason for loss to follow up was through aging of the cohort. In the older age groups the main reasons were movement out of the region, death, and lost records.

Attendance at health centres

Of clients in the 0–3 year sample, between 89 and 97% had a record of attendance at the health centre within the 6 months preceding each of the year 1, year 2 and year 3 audits. For the 4–15 year sample the corresponding figures for the 12 months preceding each audit were between 87 and 94%. The corresponding figures for the 16–49 year sample were consistently 96%, and for the 50+ sample between 96 and 98% for the 24 months preceding each audit. Data on attendance were not collected at the first two audits.

Trends in service delivery over study period

For each care plan age group there was a significant improvement in percentages of delivered services delivered between baseline and the audits at 1 year and/or 2 years (Figure 3 and Table 6). For the 0–3 age group the improvement in service delivery was sustained to the 3 year audit. However, for the other three age groups there was a decline in service delivery by year 3. In the case of the 16–49 year age group this decline had begun at the time of the 2 year audit, and by year 3 had dropped below baseline.

Figure 3

Means of the percentages of services delivered to clients by care plan group and audit period

Table 6

Means of the percentages of services delivered to clients by care plan group and audit period

Care plan group	Baseline	6 months	1 year	2 years	3 years a	3 years b	β* (95% CI)	P for trend
	Means % (95%CI)
0–3 years	44 (37–51)	53 (45–61)	65 (56–74)	70 (58–82)	69 (61–78)	n/a	9.07 (5.64, 12.49)	0.001
4–15 years	49 (39–59)	33 (24–42)	56 (46–66)	62 (52–72)	53 (45–61)	63 (47–78)	5.21 (0.92, 9.50)	0.017
16–49 years	40 (34–46)	43 (37–49)	53 (47–60)	49 (43–56)	39 (33–44)	40 (32–47)	0.70 (-1.37,2.77)	0.508
50+ years	40 (34–47)	43 (36–50)	54 (46–61)	56 (48–63)	47 (40–54)	44 (34–54)	2.44 (0.11, 4.77)	0.040

a Based on supplemented random sample (includes original panel); b Based on remaining clients of original panel sample; * β (coefficient) represents the average change in percentage of delivered services to each client with the increase of one unit of time (1 year), adjusted for the recent attendance at health centres using two-level linear random-effects model.

Proportion of scheduled services delivered to 0–3 age group, and trends

The increase in the proportion of services delivered to the 0–3 age group showed a highly significant trend for all services together across the study period (Figure 3 and Table 6). The most marked improvement occurred between the 6 month and year 1 audits, and improvement was sustained to year 3. Improving trends occurred between baseline and year 1 for almost all services (Table 2). There was wide variation in the level of delivery of different services at baseline and across the study period (Table 2). Those services with high baseline levels of coverage had less room for improvement and as expected were less likely to show improvement. Services starting from a lower baseline were more likely to show significant improvement. In general services requiring basic measurements or procedures were delivered at higher levels than those requiring more advanced clinical skills.

Table 2

Percentage of scheduled services delivered to clients in 0–3 age care plan group

Services*	Baseline	6 months	1 year	2 years	3 years	OR§ (95%CI)	P value for trend
Basic measures
Weight	92% (51)	92% (51)	94% (47)	91% (32)	87% (54)	0.82 (0.55–1.24)	0.348
Height/Length	39% (49)	51% (45)	65% (40)	61% (23)	70% (54)	1.47 (1.15–1.89)	0.002
Head circumference	41% (17)	38% (13)	50% (4)	- (0)	68% (19)	1.51 (0.95–2.40)	0.078
Haemoglobin	53% (49)	53% (51)	77% (47)	66% (32)	80% (50)	1.48 (1.12–1.96)	0.006
Hearing test	17% (46)	15% (39)	44% (32)	36% (11)	51% (49)	1.77 (1.31–2.39)	0.001
Clinical examination
Vision test	15% (46)	9% (34)	28% (32)	36% (11)	53% (49)	1.99 (1.47–2.68)	0.001
Hips/gait examination	11% (45)	15% (34)	35% (31)	30% (10)	52% (48)	2.02 (1.48–2.77)	0.001
Heart auscultation	17% (24)	10% (21)	29% (14)	14% (7)	24% (21)	1.35 (0.81–2.26)	0.246
Testes examination	14% (7)	0% (5)	0% (2)	0% (1)	60% (5)	2.71 (0.98–7.49)	0.054
Ear examination	54% (46)	61% (38)	76% (34)	58% (12)	76% (49)	1.32 (0.99–1.76)	0.062
Doctor examination	42% (33)	36% (28)	59% (17)	n/a	41% (27)	1.10 (0.76–1.58)	0.619
Vaccination
Immunisation up-to-date	88% (32)	79% (24)	85% (20)	100% (5)	80% (54)	0.87 (0.59–1.28)	0.474
Deworming	57% (49)	65% (51)	72% (47)	73% (33)	80% (50)	1.40 (1.05–1.86)	0.023

* Denominators in brackets are the number of most recent scheduled services (within the preceding 12 months of each audit) according to guidelines, which varied at each audit depending on the age of the child and the scheduled frequency of the service. §Odds ratio (OR) represents the average changing of probability for the delivery of each service with the increase of one unit of time (1 year), adjusted for the recent attendance at health centres using two-level logistic random-effects model.

Proportion of scheduled services delivered to 4–15 year age group, and trends

The same general pattern of higher service delivery levels at years 1 and 2, with levels at 6 months well below baseline and at year 3 similar to baseline, was seen for almost all individual services (Table 3). No individual services showed a significant improving trend across the study period, except for deworming. There was wide variation in the level of delivery of different services at baseline and across the study period. Overall, there was a significant increasing trend for delivery of services across the study period (Figure 3 and Table 6).

Table 3

Percentage of scheduled services delivered to clients in 4–15 age care plan group

Services*	Baseline	6 months	1 year	2 years	3 years	OR§ (95%CI)	P value for trend
Basic measures
Weight	87% (30)	61% (31)	92% (26)	90% (29)	75% (51)	0.96 (0.69–1.32)	0.793
Height	53% (30)	23% (30)	78% (23)	83% (29)	45% (53)	1.07 (0.83–1.39)	0.601
Hearing	25% (12)	20% (10)	100% (9)	64% (11)	23% (22)	0.89 (0.59–1.36)	0.603
Urinalysis	33% (6)	13% (8)	67% (9)	67% (9)	25% (12)	1.01 (0.58–1.74)	0.980
Clinical examination
Vision test	30% (23)	25% (20)	71 (14)	60% (15)	24% (42)	0.90 (0.66–1.23)	0.502
Ear examination	73% (11)	63% (8)	88% (8)	100% (14)	91% (22)	1.97 (0.99–3.90)	0.053
Vaccination
Immunisation	83% (30)	72% (29)	83% (24)	86% (29)	66% (53)	0.76 (0.54–1.06)	0.104
Deworming	49% (80)	35% (80)	52% (79)	56% (81)	62% (53)	1.31 (1.06–1.61)	0.013

* Denominators in brackets are the number of most recent scheduled services (within the preceding 12 months of each audit) according to guidelines, which varied at each audit depending on the age of the child and the scheduled frequency of the service. §Odds ratio (OR) represents the average changing of probability for the delivery of each service with the increase of one unit of time (1 year), adjusted for the recent attendance at health centres using two-level logistic random-effects model.

Proportion of scheduled services delivered to 16–49 year age group, and trends

Almost all individual services followed the same general trend with higher levels of service delivery recorded at the year 1 and year 2 audits, with a decline at year 3 to levels close to or below baseline (Table 4). Notable exceptions are pap smears where the highest levels of delivery were recorded at the year 3 audit. As for other age groups, the levels of service delivery varied widely between different services, with counselling services being recorded at lower levels than basic measures. Overall, there was no significant trend for delivery of services across the study period (Figure 3 and Table 6).

Table 4

Percentage of scheduled services delivered to clients in 16–49 age care plan group

Services*	Baseline	6 months	1 year	2 years	3 years	OR§ (95%CI)	P value for trend
Basic measures
Weight	68% (88)	75% (87)	80% (85)	70% (91)	56% (109)	0.80 (0.66–0.97)	0.020
Blood pressure	80% (88)	83% (87)	84% (85)	81% (91)	74% (109)	0.87 (0.70–1.08)	0.216
Waist circumference	38% (88)	32% (87)	45% (85)	50% (90)	38% (109)	1.12 (0.93–1.34)	0.225
BMI	33% (88)	34% (87)	41% (85)	47% (90)	31% (109)	1.05 (0.87–1.26)	0.612
Blood sugar (random)	56% (88)	53% (87)	64% (85)	60% (91)	43% (109)	0.89 (0.75–1.06)	0.188
Urinalysis	59% (88)	57% (87)	64% (85)	59% (91)	47% (109)	0.84 (0.71–1.01)	0.058
Clinical examination
Pap smear	66% (47)	52% (46)	60% (45)	68% (50)	75% (57)	1.29 (0.99–1.67)	0.055
Clinical breast examination	21% (48)	34% (47)	23% (47)	37% (51)	25% (57)	1.05 (0.81–1.35)	0.722
Vaccination
Pneumococcal vaccine	n/a	n/a	n/a	48% (89)	65% (107)	n/a	n/a
Counselling
Family history	19% (88)	17% (87)	33% (85)	15% (91)	4% (108)	0.64 (0.49–0.83)	0.001
Alcohol	23% (88)	31% (87)	53% (85)	38% (91)	21% (109)	0.94 (0.78–1.14)	0.543
Smoking	22% (88)	36% (87)	53% (85)	42% (91)	23% (109)	0.98 (0.82–1.18)	0.823
Exercise	13% (88)	21% (87)	36% (85)	31% (91)	18% (109)	1.05 (0.79–1.39)	0.748

* Denominators in brackets are the number of most recent scheduled services (within the preceding 12 months of each audit) according to guidelines, which varied at each audit depending on the age of the client and the scheduled frequency of the service. §Odds ratio (OR) represents the average changing of probability for the delivery of each service with the increase of one unit of time (1 year), adjusted for the recent attendance at health centres using two-level logistic random-effects model.

Proportion of scheduled services delivered to 50+ year age group, and trends

Most individual services followed the same general trend with higher levels of service delivery recorded at the year 1 and year 2 audits, with a decline at year 3 (Table 5). Notable exceptions to this pattern included the adult immunisations where levels were highest at year 3. Counselling services tended to be reported at lower levels than basic measures, and immunisations at higher levels. No individual services showed a significantly improving trend across the study period other than BMI and pneumococcal vaccination. Overall, there was a significant increasing trend for delivery of services across the study period (Figure 3 and Table 6).

Table 5

Percentage of scheduled services delivered to clients in 50+ age care plan group

Services*	Baseline	6 months	1 year	2 years	3 years	OR§ (95%CI)	P value for trend
Basic measures
Weight	70% (46)	70% (46)	78% (46)	77% (44)	69% (55)	1.01 (0.78–1.32)	0.934
Blood pressure	83% (46)	83% (46)	91% (46)	82% (44)	82% (55)	0.94 (0.69–1.29)	0.705
Waist circumference	33% (46)	37% (46)	41% (46)	45% (44)	42% (55)	1.13 (0.88–1.45)	0.336
BMI	22% (46)	26% (46)	33% (46)	43% (44)	40% (55)	1.39 (1.05–1.83)	0.020
Blood sugar (random)	57% (46)	54% (46)	67% (46)	64% (44)	56% (55)	1.02 (0.80–1.29)	0.903
Urine	52% (46)	48% (46)	57% (46)	64% (44)	44% (55)	0.95 (0.75–1.20)	0.663
Clinical examination
Vision	28% (46)	35% (46)	45% (44)	44% (32)	35% (55)	1.06 (0.84–1.35)	0.610
Clinical breast examination	5% (19)	11% (19)	11% (19)	13% (15)	5% (19)	0.97 (0.48–1.95)	0.925
Pap smear	25% (16)	25% (16)	38% (16)	47% (15)	42% (24)	1.32 (0.87–1.99)	0.191
Mammogram	13% (16)	7% (15)	19% (16)	15% (13)	18% (22)	1.46 (0.70–3.03)	0.309
Vaccination
Pneumococcal vaccine	65% (46)	70% (46)	72% (46)	80% (44)	82% (55)	1.39 (1.03–1.86)	0.030
Tetanus/diphtheria	83% (46)	78% (46)	74% (46)	84% (43)	88% (52)	1.23 (0.90–1.69)	0.192
Influenza vaccine	77% (22)	73% (26)	71% (35)	80% (25)	84% (32)	1.41 (0.94–2.13)	0.098
Counselling
Physical activity	28% (46)	26% (46)	56% (41)	39% (33)	27% (55)	0.97 (0.76–1.25)	0.832
Home accidents	20% (46)	13% (46)	46% (41)	30% (33)	27% (55)	1.15 (0.89–1.50)	0.292
Nutrition	28% (46)	35% (46)	63% (41)	42% (33)	44% (55)	1.15 (0.91–1.46)	0.252
Deafness	15% (46)	22% (46)	38% (42)	36% (33)	27% (55)	1.19 (0.92–1.54)	0.194
Exercise	26% (46)	28% (46)	41% (46)	41% (44)	33% (55)	1.11 (0.86–1.43)	0.435
Smoking	24% (46)	41% (46)	41% (46)	52% (44)	35% (55)	1.10 (0.85–1.43)	0.471
Alcohol	26% (46)	35% (46)	33% (46)	48% (44)	24% (55)	0.96 (0.74–1.25)	0.751

* Denominators in brackets are the number of most recent scheduled services (within the preceding 12 months of each audit) according to guidelines, which varied at each audit depending on the age of the client and the scheduled frequency of the service. §Odds ratio (OR) represents the average changing of probability for the delivery of each service with the increase of one unit of time (1 year), adjusted for the recent attendance at health centres using two-level logistic random-effects model.

Service delivery and trends by trial region and community

Tiwi communities tended to have higher levels of service delivery at baseline and for the duration of follow-up (Figures 4 and 5, Table 7 and 8). The general pattern of improvement in levels of service delivery from baseline to year 1 and/or year 2, with a decline at year 3, is seen for most communities. However, two communities in Katherine West buck this trend (Figure 4). In both there is a decline in service delivery from year 1 to year 2, and a subsequent increase to year 3. Overall, 2 of the 3 Tiwi communities, and one of the Katherine West communities, showed a significantly increasing trend in service delivery across the study period.

Figure 4

Means of the percentages of services delivered to clients by health center in Katherine West

Figure 5

Means of the percentages of services delivered to clients by health center in TIWI communities

Table 7

Means of the percentages of services delivered to clients by health center in Katherine West

Katherine West community health centres	Baseline	6 months	1 year	2 years	3 years	β* (95% CI)	P for trend
	Means % (95%CI)
A	41(32–50)	42 (33–51)	42 (31–53)	45 (32–58)	31 (21–42)	-1.92 (-5.64, 1.80)	0.312
B	44 (30–58)	38 (23–52)	49 (32–65)	53 (37–69)	36 (25–47)	0.02 (-4.88, 4.93)	0.992
C	40 (23–56)	37 (20–54)	38 (19–58)	31 (13–49)	49 (33–66)	2.01 (-3.89, 7.91)	0.504
D	30 (20–40)	34 (24–45)	47 (33–60)	46 (31–61)	56 (38–73)	7.17 (2.47, 11.86)	0.003

* β (coefficient) represents the average change in percentage of delivered services to each client with the increase of one unit of time (1 year), adjusted for the recent attendance to health centres using linear random-effects model.

Table 8

Means of the percentages of services delivered to clients by health center in TIWI communities

Tiwi community health centres	Baseline	6 months	1 year	2 years	3 years	β* (95% CI)	P for trend
	Means % (95%CI)
E	52 (44–60)	46 (38–54)	74 (67–82)	67 (59–75)	54 (49–60)	1.61 (-1.37, 4.59)	0.289
F	47 (37–57)	44 (35–54)	63 (54–71)	62 (52–72)	59 (48–70)	5.70 (1.82, 9.57)	0.004
G	35 (25–45)	39 (29–50)	47 (34–60)	68 (55–80)	61 (53–68)	9.77 (5.80, 13.74)	0.001

* β (coefficient) represents the average change in percentage of delivered services to each client with the increase of one unit of time (1 year), adjusted for the recent attendance at health centres using linear random-effects model.

Discussion

The implementation of care coordination through the care planning process, best practice guidelines and computerised recall and reminder systems was associated with a general improvement in delivery of specified preventive services in the study population. However, the sustainability of the improvements was variable. While the overall improvement in service delivery at year 3 had dropped below levels achieved at years 1 and 2, service delivery was still significantly higher than at baseline for 3 out of the 4 population age groups.

In general the services for which the highest levels of service delivery were achieved were those supported by the best quality evidence and the strongest recommendations [11] (eg immunisation in children and older adults). The clearest exceptions to this trend were mammography in the 50+ women and counselling of adults on alcohol use, smoking, exercise, and home accidents, where there were low levels of service delivery in the face of particularly strong recommendations. Counselling services may be more likely to not be recorded when they are delivered, and this may partly explain low recorded levels of service. However, a low level of recording of services is a problem in itself, in that it impedes effective, efficient and coordinated service delivery.

It is remarkable that the delivery of preventive services in these remote health centres is in most cases as good as or better than reported by North American and UK studies with comparable data. [12-22] Possible explanations may be a recent wider acceptance of the value of evidence-based preventive care, and the strong public health orientation of health workers in the environment in which these trials were conducted.

The patterns of service delivery over time raise questions about the sustainability and nature of systems required to support a strategy that aims to enhance delivery of key prevention and health promotion services across all age groups. It is apparent from the data that these systems had some impact on improving delivery of care in the first one to two years following the introduction of the care coordination model, and that subsequently the systems to maintain the improvement in practice failed, or that the practitioners responded less effectively to these systems. Interviews with health centre staff and health service management and community specific patterns of service delivery suggest that the reasons include:

1) discontinuities in staffing

2) lack of ongoing training and support for the care coordination process

3) lack of development of relevant work practices

4) lack of clear priorities to guide delivery of care plan services

5) lack of capacity to monitor service activity through timely analysis of service data

6) reluctance on the part of patients to accept offered services when presenting for acute, non-scheduled services.

The implementation of guidelines and computerised recall and reminder systems will not of itself lead to sustained improvement in delivery of preventive services. Their impact clearly depends on effective use by health centre staff. The introduction of recall for preventative services resulted in many items listed as overdue and consequently required development of a strategy to integrate active and opportunistic recall for these services with acute care and other activities. It demanded attention to the recording of greater numbers of services in the systems of electronic and paper records. Health centres appear to have not been able to systematically develop work practices in such a way as to efficiently cope with the additional demands. Inadequate training and work practice support for the use of care plans beyond initial induction in their basic use appears therefore to have limited their impact.

The high levels of attendance at health centres for all age groups indicates that low service levels were not the result of poor access, but rather of clients not being offered, or not taking up preventive services when they attended the centres. Client behaviour, including an unwillingness to accept care plan services offered opportunistically, is almost certainly a factor in sustaining delivery of preventive services to adults. For those with health problems, there may be a great number of services due on any given occasion. Clients were reportedly often unhappy to wait for a second consultation, particularly for preventive activity unrelated to the initial reason for presentation. However, these difficulties may not be insurmountable, as the trends to improvement noted in the first and second years of the trials have indicated.

The low levels of services seen at six months for the 4–15 year age group was a result of school screening programs being conducted later than usual at some sites, and after the audit. Most services to this age group are provided as part of a school screening program in these communities. Most services scheduled on the care plans for this age group are also specific to children age 5, 10 or 15 years, and were therefore only relevant to a proportion of the sample. The numbers on which most estimates are based for this sample are therefore relatively small, and the estimates unstable.

Systematic differences between the original sample and the sample of additional participants recruited for the year 3 audit is unlikely to be an explanation for the declining trend between year 2 and year 3. The mean level of service delivery for those in the original sample was very close to that of the overall sample for both adult samples at the year 3 audit (see Table 6). While the corresponding difference for the 4–16 age group was larger, as stated above, the estimates for this group are relatively unstable.

Finally, sustained improvement in the delivery of preventive care services will depend on the institutionalisation of the processes of care at health centre level, and the development of capacity to support these processes at the level of management, policy and training supports.[23,24] Institutionalisation could be supported by linking care plan protocols more firmly to work practices and work flow, and specifically supporting opportunistic delivery of preventive services. Increased involvement of clients in reviewing preventive care plans, and thereby enhancing their understanding of the value of preventive services, may also increase service uptake. However, sustainable improvement is unlikely to follow simply from practitioner and health centre response to a set of protocols. Management needs to actively support strategies for sustained delivery of preventive services.

Competing interests

The authors were contracted as independent evaluators of the two trials that are the subject of this report.

Authors' contributions

RB designed the clinical audit method, oversaw the audit data collection and analysis, and prepared the manuscript for publication.

ST assisted with audit design, took primary responsibility for data collection in one study region, and took a lead role in data analysis.

DS made a major contribution to the data analysis.

GR and Pd'A played a major role in project leadership and design, in data collection and interpretation of findings.

All authors contributed to, have read and approved the final manuscript.

Pre-publication history

The pre-publication history for this paper can be accessed here: