Development and Evaluation of GIS-Based Chlamydia Trachomatis Intervention Policy in Illinois.

Chlamydia trachomatis is the most prevalent infectious disease in the United States. Complications include pelvic inflammatory disease (PID), ectopic pregnancy, and infertility. The cost of PID in 1998 was estimated at greater than $1.9 billion. Screening intervention strategies are often consumed by those at low risk. The objective of this study is the development of a more cost-effective intervention strategy by employing Geographic Information Systems and Census Bureau demographic data in selected Local Health Departments in Illinois. Case studies of intervention activities at the state and local level were performed. An evaluability assessment model of current inputs, processes and outcomes was created. A proposed model utilizing additional state inputs was developed and tested. Interventions were evaluated for effectiveness in reducing the incidence of Chlamydia trachomatis. Societal cost effectiveness analysis was also performed. The proposed model was tested in 2006. Results indicate that only minimal changes in annual incidence are required for GIS-augmented interventions to be cost-effective.

Introduction

Chlamydia trachomatis (CT) is the most prevalent infectious disease in the United States, and the most prevalent sexually transmitted bacterial infection worldwide with an estimated 50 million cases in 1999.( There were 877,478 reported cases in the United States in 2003.( State case rates range from 78.7 to 462.3 per 100,000. There were 48,294 reported cases in Illinois in 2003.( While greater than 70% of CT cases are asymptomatic,( long-term morbidity is observed in increased pelvic inflammatory disease (PID) which can lead to ectopic pregnancies, infertility and other morbidity. The national financial cost of PID in 1998 was estimated at greater than $1.9 billion.( Public sources paid 30% of these direct costs while managed care groups, private insurance, and individuals paid the remainder.( While interventions at both the state and federal levels include screening tests, these are often consumed by those at low risk (approximately 30% of all tests in Illinois). There is a need to define a process and policy by which resources may be more directly applied to the population at greatest need.

The utility of geographic information systems (GIS) to identify sexually transmitted disease (STD) concentrations and assist intervention and control has been shown in multiple studies.( This is consistent with the core theory of STD transmission and corroborates recent work in the field of core group and bridge population descriptions.( In the study of sexual behavior patterns, researchers have seen that STD endemic levels depend upon a “core” group of people who are frequently infected. This core is usually a small proportion of the population and their level of mixing with “bridge populations” influences the spread of an STD in the general population (. While the impact of bridge populations on overall morbidity varies according to local population and disease characteristics, identifying the high-prevalence disease clusters may be beneficial in designing an effective intervention. It has also been show that directing resources to a geographic area can be an effective complement to traditional intervention methods (.

The objective of this study is the development of a more cost-effective CT intervention strategy. We describe the existing policy and its limitations and compare it to a new policy and its potential benefits. An additional state input, consisting of GIS-based information and Census Bureau demographic data (GIS/CB), is utilized as a tool for targeting resources to populations at increased risk of becoming infected with CT. We also construct an evaluability assessment (Logic) model of the current and existing policies, test the proposed model, and perform a cost-effectiveness analysis (CEA). A well-described Logic model will help us identify program resources, associated activities and outcomes, and the relationships involved.( The study seeks to show that the introduction of GIS technology and Census Bureau demographic data ( GIS/CB) into CT intervention policy is effective in reducing the incidence of CT and also cost- effective (CE) from society’s perspective.

Materials and Methods

The current policy concerning CT interventions entails the local health department (LHD) devising and implementing their own, often unique, intervention. This is in addition to the standard screening and testing performed by each LHD. The Illinois Department of Public Health’s Sexually Transmitted Diseases Section (ISTD) staff provided testing resources, medications, materials and literature, and advice to support the local efforts. The extent and effectiveness of these interventions are largely dependent upon the locally available staff and resources, and vary widely by department. The majority of LHD STD sections are managed by nurses, many of whom do not have any formal training in public health. In addition, an Association of State and Territorial Health Officials (ASTHO) survey reported that the nursing occupational class is the area in public health with the greatest shortage of workers.( In many instances, the result is an understaffed STD section attempting activities for which they have had no formal training.

A case study approach was used to describe current intervention policy elements in Illinois. The use of other qualitative and quantitative data sources added insight to structure and gave more depth to program descriptions.( For a state-level description of policy, the ISTD Section Chief and the Surveillance and Evaluation Coordinator were interviewed on multiple occasions between June 2005 and January 2006. They were able to describe state policy and ISTD activities, provide copies of the ISTD budget, organization charts and Section Outbreak Response Plan. They were also able to speak at length concerning their interactions with LHDs, and the relative strengths and weaknesses of the current system.

The ISTD section is involved in a variety of activities ranging from data collection to active participation in local intervention efforts. Such activities can be grouped into three broad categories:

Local, state and national data collection, monitoring, reporting and evaluation.

Indirect services such as technical assistance to health care providers; policy development; legislative and administrative rule work; training.

Direct services such as data reporting and interpretation, the provision of STD testing and screening, the supply of medication, provision of materials and literature, intervention design and implementation advice, and direct disease intervention.

For a local-level description of policy, STD staffs in thirteen LHDs were interviewed in November and December 2005. There are 95 certified health departments in Illinois serving 99 counties. Three counties are serviced by the ISTD directly. LHDs are funded through a Local Health Protection grant, local taxes, and additional specific grants. The departments selected for study were chosen to be representative of the range of circumstances and resources encountered throughout Illinois.

STD staff in each LHD were asked to discuss their structure and activities in six areas: STD Staffing (those involved with CT); Intervention Activities; Collaborations (with community organizations); Educational and Promotional Materials; Interactions (with the ISTD); Actions Taken (in response to a STD outbreak). One reported area, Interactions, produced uniform responses from all LHDs. All reported that such interactions involved the provision of case data to and from the ISTD, advice, training and materials by the ISTD, and a high degree of assistance and customer service from the ISTD to any kind of requested help. Table 1 compares and contrasts LHD responses to the remaining five areas. The close intertwining of HIV and other STD work, even though they are often funded separately, makes it difficult to clearly separate many duties and staff. Thus, the numbers of staff assigned to non-HIV STD work are only close approximations.

Table 1

Summary of Case Study Responses (Data as of 12/05)

Area	Criteria	Measure	High Value/Yes	Low value/No
STD Staffing	FTEs	number	153	0
	PTEs	number	14	0
	MD present	yes/no	8	5
Intervention Activities	On-site clinic	number	12	1
	Off-site clinic	number	3	10
	Internet outreach	yes/no	3	10
	Paid advertising	yes/no	3	10
	Health fairs	yes/no	8	5
	Unique**	yes/no	8	5
Collaborations	Family Planning	yes/no	7	6
	Local clinics, hospitals	yes/no	12	1
	Drug/alcohol rehab	yes/no	7	6
	Comm. service agencies	yes/no	7	6
	Faith based	yes/no	5	8
	Detention centers	yes/no	9	4
	School/University	yes/no	10	3
	Unique	yes/no	7	6
Educational and Promotional Materials	Free from ISTD/CDC	yes/no	13	0
	Purchased	yes/no	7	6
	Created in-house	yes/no	9	4
Actions Taken	Increased clinic/lab	yes/no	9	4
	Local provider notification	yes/no	12	1
	Developing additional education/outreach	yes/no	11	2
	More aggressive partner notification	yes/no	12	1
	Unique	yes/no	4	9

The following county health departments were surveyed for this study: Adams, Alexander, Champaign, Jackson, Kankakee, McLean, Macon, Peoria, Vermilion, Winnebago. Also included were the cities of Chicago and Springfield (now incorporated into Sangamon County), and the East Side Health District.

Unique criteria represents an activity in an area that only one health department performs.

Logic (Evaluability Assessment) Model of the CT Intervention Strategy

A key component to the successful integration of additional state inputs into intervention development is the determination of a program theory and creation of an evaluability assessment model. The program theory represents actual activities and was obtained from case studies, site visits and various documents. It identifies program resources and activities and the causal relationships between them and outcomes. To assess the effectiveness of the current theory, a model is created with input from identified stakeholders, managers and data users. Figure 1 presents the Logic Model for the current and proposed policies.

Figure 1

Evaluability assessment models of the current and proposed policies

The proposed policy indicates where GIS/CB might be effectively added, where processes may be evaluated, and where outcomes are monitored to influence future activities. These models are intended to clarify the assumed relationships among the resources, activities and outcomes. The models can be used to identify opportunities to change program components or utilize information to improve performance and outcomes.( The evaluability assessment models of the current and proposed CT screening policies.

Program Input

State Inputs (current policy)

The ISTD section has several inputs including CT case data summaries and other reports, advice, condoms and treatment medications and educational materials. These inputs are directed to the LHD, which then utilizes them in a locally-derived intervention.

State Inputs (proposed policy)

The proposed additional state input consists of the provision of GIS-based targeting information and Census Bureau demographic data to the LHDs. This information is derived from county-level case and demographic data and is used to target local intervention resources to a specific geographic area, aid intervention development and enlist partners. Identified areas may harbor core transmitters, and their identification and treatment may significantly decrease local incidence and prevalence. The proposed policy enables ISTD to have directly contributed to the intervention development policy.

The proposed additional state input consists of the provision of GIS-based targeting information and Census Bureau demographic data to the LHDs. This information is derived from county-level case and demographic data and is used to target local intervention resources to a specific geographic area, aid intervention development and enlist partners. Identified areas may harbor core transmitters, and their identification and treatment may significantly decrease local incidence and prevalence. The proposed policy enables ISTD to directly contribute to the intervention development policy.

LHD Inputs (current policy)

Each LHD implements intervention strategies in its jurisdiction. The types and degree of interventions depend greatly upon LHD inputs that include the following: CT case data; staffing levels; budget; equipment; and the knowledge, skills and abilities of individual staff members. LHD inputs and the local incidence directly determine the nature and level of local intervention development. Interventions are developed and implemented based upon the population demographics, the level of current local capabilities and the available budget.

Program Process

The process in this model consists of new and continuous intervention activities. Associated with this process are intervention and evaluation activities. Of the thirteen LHDs involved in case studies, twelve of them indicated intervention the desire for state intervention activities in addition to the standard screening.

Intervention activities (current):

These were unique to each LHD studied. Strategies for additional outreach and community engagement included: increased screening beyond the recommended guidelines; increased clinic hours; additional methods of literature distribution; on-site outreach through health fairs or mobile clinics; public service announcements and other regular interactions with print and video media; community outreach through collaborations with local groups. It was found that no two LHDs had the same strategy, just as no two LHDs had the same skills and resources.

Intervention activities (proposed):

The proposed intervention activity would consist of the use of additional state inputs to devise and implement an intervention targeted towards a geographic area.

Evaluation activities (current and proposed policies):

Most LHDs infrequently perform only informal evaluations, if any. No LHD interviewed indicated that they had any formal evaluation process in place. The proposed policy utilizes CEA as an integral portion of the intervention. Once an intervention based upon the GIS/CB has been implemented, an evaluation will be conducted.

There have been many studies of the cost-effectiveness of CT interventions, and most of them have concerned the utility of various screening strategies.( Work has been done to show which type of modeling (stochastic versus deterministic) is most accurate for this situation.( For our purposes, the simpler deterministic model is more useful.( The CEA utilizes the following three types of data:

Direct costs – Includes: pro-rated time and salary of involved personnel, costs of printing and advertising, automobile mileage;

Indirect costs – Includes: pro-rated personnel retirement and administrative costs, time denied other projects;

Intervention results – Resultant incidence rates post-intervention.

Program Outcomes

The primary outcome is a decreased level of disease incidence in the LHD’s jurisdiction. This will be measured directly by case reporting to the ISTD. The goal is to determine effective interventions utilizing GIS/CB technology. The outcome is measured in decreased local incidence. The cost-effectiveness of these interventions will also be done to determine if the net societal benefit in averted costs is greater than the intervention costs.

Methodology

Five Illinois counties agreed to test this model. To illustrate how GIS and maps may be used, Figure 2 shows example maps of Peoria County. In each figure, Chlamydia cases for all of 2005 have been geocoded and mapped (due to confidentiality rules, maps of actual geocoded cases are not shown). The census block groups (CBG) for each county were then stratified by total case count and case concentration to locate the areas of highest case concentrations. The ISTD provided the LHD GIS maps accompanied by Census Bureau demographic data specific to the identified CBGs. This data was then be used to devise a CBG-specific intervention that was implemented in one or more of the identified CBGs. Five control counties were chosen and assigned to cases based on annual incidence rate similarity.

Figure 2

Example maps of Peoria County

Once the intervention was completed, the incidence rate for that county, and its assigned control, would be observed. If the model is effective, the incident rate for the targeted county should have a resultant incidence lower than that of the control. The model would then be evaluated by CEA. The ISTD would assist the LHD in collecting information concerning all costs incurred during the intervention and measure them against the expected savings due to averted diseases.

Success of the model would be its iterative use to reduce the incidence of the entire county to an acceptable level. Since Chlamydia is unlikely to be eradicated, it is proposed that lowering the county incidence to the level of the state average be considered acceptable. In this approach, the LHD and ISTD perform iterative models, mapping and interventions targeting CBGs with decreasing case counts and concentrations until the desired incidence is reached.

Results

Participating LHDs received their maps and data in September 2006. They devised and implemented their interventions by November 14, 2006. The first evaluation of the process is qualitative as each LHD performed a limited, and unique, intervention. Intervention consumption was directly observed through personal interactions, receipt of educational materials, and increased clinic visits and partner referrals.

Statistical analysis of available data (through February, 2007; estimated at >95% CI) indicates that Vermilion county experienced a significant decrease in incidence not matched by its control (see Table 2). Two other control counties also experienced incidence decreases. The decline in Adams County may be explained by the excessive time observed for reporting results. Of the 14 cases reported for January, six of them required more than 42 days to be reported to the ISTD and another 4 required more than 74 days. The decline in Sangamon County may be related to the consolidation of the city and county health departments during 2006.

Table 2

Comparison of post-intervention and pre-intervention data

County/Status	Data	Cases		t-test for Equality of Means Independent Samples
		mean	range	t	df	sig.
Champaign/CaseMacon/Control	post-intervention	88.3	79–100	–0.233	6	0.823
	pre-intervention	90.3	72–105
	post-intervention	53.0	51–55	–1.021	6	0.347
	pre-intervention	58.0	45–68
McLean/CaseAdams/Control	post-intervention	40.3	36–46	–1.820	6	0.119
	pre-intervention	50.8	3 6–64
	post-intervention	9.5	4–14	–3.130	6	0.020
	pre-intervention	18.3	15–22
Peoria/CaseJackson/Control	post-intervention	124.8	115–138	–1.602	6	0.160
	pre-intervention	138.5	119–152
	post-intervention	35.5	30–39	–0.253	6	0.809
	pre-intervention	36.5	32–46
Vermilion/CaseKankakee/Control	post-intervention	31.5	26–3 9	–2.63 5	6	0.039
	pre-intervention	40.5	39–45
	post-intervention	43.0	38–45	0.23 1	6	0.825
	pre-intervention	42.3	34–46
Winnebago/CaseSangamon/Control	post-intervention	119.5	92–155	–0.147	6	0.888
	pre-intervention	121.5	115–130
	post-intervention	67.3	61–82	–5.468	6	0.002
	pre-intervention	94.8	92–96

Costs involved with the project involving both the author and LHDs staff, have been collected (data not shown). Chlamydia complication rate estimations and their associated costs have been collected from the literature.( From these data we estimate the number of averted cases needed for a county’s intervention to be cost-effective. Estimations are made utilizing a range of progression rates (untreated chlamydia to PID) and lifetime costs of PID taken from the literature (Table 3). CE for most counties, and most progression rates and costs, may be attained with an incidence decrease of <2%. The specific analysis for Vermilion County (Table 4) indicates that the intervention produced a net societal benefit in the range of $2,002–$56,061.

Table 3

Minimum change required for cost-effectiveness utilizing low-to-high values of disease progression and lifetime cost

County		Champaign	McLean	Peoria	Vermilion	Winnebago
Intervention costs		$1,680	$1,892	$10,564	$1,179	$7,448
2005 total cases		1,133	493	1,292	368	1,522
Number of averted cases required for CE	10%; $1,060	20	20	100	20	80
	25%; $2,150	4	4	20	4	16
	50%; $3,180	2	2	8	2	6
Percent decrease from 2005 for CE	10%; $1,060	1.8%	4.1%	7.7%	5.4%	5.3%
	25%; $2,150	<1.0%	<1.0%	1.5%	1.1%	1.1%
	50%; $3,180	<1.0%	<1.0%	<1.0%	<1.0%	<1.0%

Minimum averted cases required for CE was determined using the range of values found in the literature for both disease progression in untreated chlamydia infection (10–50%; 25% median) and lifetime cost ($1,060–$3,180; $2,150 median).

Table 4

Cost-effectiveness of Vermilion County Intervention

Cases CT averted	Progresion to PID	Cases PID averted	Average cost of PID	Savings	Intervention cost	Net Societal benefit/loss
36	10%	3	$1,060	$3,180	$1,179	+$2,001
	50%	18	$3,180	$57,240	$1,179	+$56,061

Table 3 shows the minimum change required for cost-effectiveness utilizing low-to-high values of disease progression and lifetime cost, while Table 4 presents the cost-effectiveness of Vermilion county interventions

Discussion

The study has found that systematic modeling and evaluation can assist in the development of a more cost-effective chlamydia intervention strategy. As seen in the logic model of the current policy, intervention development is largely dependent on the local STD staff who often lack training and resources to develop interventions and evaluate them for effectiveness. The proposed policy showed where additional state inputs (GIS/CB) could assist in intervention development. Process evaluation is done to assess intervention implementation. Finally, the entire intervention is evaluated for effectiveness and cost-effectiveness. The process of developing the model, testing it in five counties, and performing the evaluation shows that there was a significant effect in at least one county, and that it was cost-effective. Most interventions and estimations show CE being achieved with a reduction <2%. Utilizing the lowest estimation for progression to PID and highest average lifetime cost has all counties achieving CE at a 7.7% reduction.

Evaluability models have been in use for more than twenty years. They have been used for such purposes as evaluating work flow in large organizations and describing outpatient care alternatives. The utility of GIS to assist in STD intervention development has been documented for both syphilis and gonorrhea. GIS has also been used to describe the extent of chlamydia in areas ranging in size from military bases to a Canadian province. Cost-effectiveness analysis is quite well documented in a wide variety of fields. Most of the CEAs done for chlamydia focus on alternative screening strategies and subsequent costs for different treatments and sequalae.

Literature searches returned no studies applying program theory development and evaluability assessment to STD intervention policies. This study describes the partners, roles, inputs, processes and outcomes for a STD policy at the state and local level. We have been able to determine an area where an additional input may be incorporated into a process which results in a more favorable outcome. This systematic approach allows for evaluation at different stages, engages partners who have the authority to act of results and outcomes, and has identified a new policy (state input) which may be both more effective (at reducing local chlamydia incidence) and cost-effective (societal benefit exceeds costs).

There are three main limitations of this study. The first was the lack of supplemental funding available to the participants to more aggressively act on the additional state input. All agreed to participate in this project while utilizing existing budgets and resources. As a result, the interventions were generally small in scale, utilized only existing staff with other duties, and lacked community partners and wide- scale activities. In spite of these limitations, one county had a significant incidence decrease. It is unknown if other counties would have had a decrease with additional resources.

A second limitation is its scale. It was tested in only five counties during one short portion of the year. The five test counties were chosen based upon a single large city within their jurisdiction and a high chlamydia rate. It is unknown if performing the study during a different time (e.g. summer), or if measuring incidence for a longer time period post-intervention, would have had different results. It is also not known if providing the same state inputs into more, and more diverse counties (in terms of size, population and location) would have returned similar results.

Finally, the assignment of control counties to participants (cases) is imperfect at best. While Vermilion’s decrease was not matched by its assigned control, two other control counties did experience significant decreases. More work will have to be done in examining Vermilion County’s intervention, and its subsequent incidence, before a better determination can be made as to its effectiveness.