The Cost-effectiveness of a Point-of-Care Paper Transaminase Test for Monitoring Treatment of HIV/TB Co-Infected Persons.

BACKGROUND: Persons with HIV and tuberculosis (TB) co-infection require transaminase monitoring while on hepatotoxic medications. A novel paper-based, point-of-care transaminase test is in development at an anticipated cost of $1 per test.
METHODS: To project long-term clinical outcomes and estimate the cost-effectiveness of using a paper-based fingerstick test to monitor for drug-induced liver injury (DILI), as compared with automated testing and with no laboratory monitoring. The design was a decision analytic model, including deterministic and probabilistic sensitivity analyses. Data sources were observational cohorts and a validation study of the paper-based test. The target population was HIV/TB co-infected persons in South Africa on antiretroviral therapy who were initiating TB therapy. Interventions: (1) clinical (no laboratory) monitoring; (2) monitoring using the paper-based test with a ≥120 IU/mL threshold for positivity; (3) monitoring using the paper-based test with a ≥200 IU/mL threshold for positivity; (4) monitoring using the paper-based test using 1 of 3 categories: <120 IU/mL, 120 to 200 IU/mL, and >200 IU/mL ("bin placement"); (5) monitoring using automated ALT testing using the same 3 categories ("automated testing"). The outcome measures were discounted quality-adjusted life-years (QALYs), costs, and incremental cost-effectiveness ratios (ICERs).
RESULTS: The ICER of automated testing was $5180/QALY. Use of the paper-based test with the bin placement strategy was cost-effective compared with clinical monitoring alone.
CONCLUSION: At its current performance, monthly DILI monitoring by bin placement using the paper-based test was cost-effective, compared with clinical monitoring, in HIV/TB co-infected persons in South Africa.

Monitoring liver transaminases for detection of drug-induced liver injury (DILI) is recommended for persons co-infected with HIV and tuberculosis (TB), as many of the medications are hepatotoxic [1]. Automated testing requires equipment for sterile phlebotomy, a phlebotomist, centrifugation, and testing on an automated platform. Transaminase monitoring while on hepatotoxic medications is routine in resource-rich settings. In resource-limited settings without necessary laboratory infrastructure and reliable power supply, the World Health Organization recommends only clinical monitoring for persons on hepatotoxic TB medications [2].

A paper-based transaminase test has been developed to yield a semiquantitative ALT measurement from a fingerstick sample [3]. This device, developed by Diagnostics For All (Cambridge, Massachusetts) is based on “patterned-paper” technology, in which a wax-based printer is used to create a series of hydrophobic barriers and hydrophilic channels that guide fluid wicking through the paper. As fluid flows through the layers of patterned paper, it contacts zone-specific assay reagents, allowing multiple reactions to be performed in parallel on a single sample. This test is approximately the size of a postage stamp and uses capillary whole blood to yield an ALT value in 30 minutes at an anticipated cost of $1 per test [3]. This device has the capacity to bring point-of-care (POC) DILI monitoring to settings where lab infrastructure is limited. While potentially more feasible than automated testing, questions remain regarding the best means of employing the paper-based ALT test. First, what threshold values should one use for clinical decision-making? Given imperfect accuracy, should this test be “binary,” providing either a positive or negative result, or would it be more valuable as a “multiple-level” test utilizing more than 1 cutoff? Second: how accurate does the test need to be to improve outcomes compared with clinical monitoring, and is the paper-based test cost-effective?

We use decision analytic modeling to evaluate optimal positivity thresholds for the paper-based test, as well as the comparative- and cost-effectiveness of DILI monitoring using the paper-based test, as compared with clinical monitoring alone and with automated ALT measurement in HIV/TB co-infected persons in South Africa. South Africa was chosen as it is a setting with limited resources and a high burden of HIV-TB co-infection.

METHODS

Analytic Overview

We created a Markov model to evaluate 5 strategies for monitoring DILI among a hypothetical cohort of HIV/TB co-infected persons in South Africa on antiretroviral therapy (ART) who were initiating TB treatment:

Clinical monitoring alone (current standard of care);

Monitoring ALT monthly with the paper-based test using any value ≥120 IU/mL as the threshold for positivity (“≥120 threshold”);

Monitoring ALT monthly with the paper-based test using any value ≥200 IU/mL as the threshold for positivity (“≥200 threshold”);

Monitoring ALT monthly with the paper-based test and using that test to assign patients to 1 of 3 categories: <120 IU/mL, 120 to 200 IU/mL, and >200 IU/mL (“bin placement”);

Monitoring ALT monthly using automated ALT testing and assigning patients to 1 of 3 categories: <120 IU/mL, 120–200 IU/mL, >200 IU/mL (“automated testing”).

The model assumed a health sector perspective on cost and employed a 1-month Markov cycle. Outcomes included quality-adjusted life-years (QALYs) and lifetime medical costs, both discounted at 3% annually [4]. We calculated incremental cost-effectiveness ratios (ICERs) of each strategy by dividing the additional cost by the additional QALYs gained compared with the next less expensive strategy [4, 5]. We interpreted ICERs using a willingness-to-pay threshold [4] based on the 2013 South African per capita GDP [6]. Strategies with an ICER < per capita GDP ($6617/QALY) were “highly cost-effective,” while those with an ICER < 3× per capita GDP ($19851/QALY) were “cost-effective” [6]. Testing strategies that provided fewer QALYs at higher total cost than an alternative, as well as strategies that provided fewer QALYs at a higher cost per QALY gained, were inefficient (dominated) and excluded from the final comparisons [7, 8]. The decision analytic model was programmed in TreeAge Pro version 2013 (Williamstown, Massachusetts).

We performed extensive deterministic sensitivity analyses, in which we varied the value of each input parameter through its plausible range to determine the impact on cost-effectiveness conclusions. A priori–defined sensitivity analyses of particular interest included investigation of paper-based test accuracy, rate of transaminitis, and cost of the paper-based test. We used a second-order Monte Carlo simulation to perform probabilistic sensitivity analysis around uncertainty related to the test characteristics of the paper-based test. We used data from a recent validation study of the paper-based test [3] and the mathematical properties of the beta distribution to develop probability density functions around the probability of being categorized in a given ALT “bin,” conditional upon the true ALT level. We then performed 10 000 iterations of the simulation, each time drawing parameter values for accuracy of the paper-based test from defined density functions. We display the results of probabilistic sensitivity analyses using cost-effectiveness acceptability curves [9].

Model Structure

TB Therapy and Monitoring

Patients in the hypothetical cohort modeled were HIV and TB co-infected in South Africa, on suppressive ART, and initiating standard 6-month TB therapy.

The model distinguishes between a patient’s true ALT value and clinical awareness of the ALT level. Clinical decision-making in the model is based solely on clinically available data, such that unrecognized elevations of ALT do not alter clinical management. With every month on treatment, patients have a probability of developing an elevated ALT (transaminitis). Those with transaminitis also face a probability of associated symptoms of hepatotoxicity and DILI-related death. Having transaminitis does not itself impact quality-of-life. Only when symptoms of hepatotoxicity occur is there a quality of life decrement.

Transaminitis cannot be detected unless 1 of the monitoring strategies is used. Test characteristics for the paper-based test are imperfect, such that false-positive and false-negative results are possible. We assume that automated, lab-based ALT measurement is the gold standard, with 100% accuracy. We assume that health care providers act correctly on test results.

The algorithms for each of the 5 monitoring strategies are as follows:

All 5 Monitoring Strategies Include Clinical Monitoring for Symptoms of Hepatotoxicity

If one develops symptoms of hepatotoxicity while on TB medications, medications are stopped and one undergoes confirmatory automated liver testing, with ALT and total bilirubin. In strategies using routine monthly monitoring by the paper-based test, elevated ALT leads to confirmatory automated testing. For all confirmatory automated testing, an ALT <120 IU/mL is considered a normal value, and ≥120 IU/mL is considered elevated. If confirmatory ALT is <120 IU/mL despite symptoms of hepatotoxicity or a positive initial paper-based test (false positive), all patients return to firstline therapy immediately. If confirmatory ALT is ≥120 IU/mL (true ALT elevation), medications are stopped and ALT is monitored 3 times weekly by automated testing until <120 IU/mL [10]. While firstline medications are stopped, the following (less hepatotoxic) TB regimen is started: ethambutol, moxifloxacin, and streptomycin. Once ALT normalizes, 63% of patients restart firstline TB medications, and 37% continue the regimen of ethambutol, moxifloxacin, and streptomycin [11]. For those on this modified regimen, the continuation phase consists of moxifloxacin, ethambutol, and rifampin or isoniazid for 12 months [10]. The modified regimen is slightly less effective and more expensive than firstline therapy [12].

Clinical Monitoring Alone

Clinical moitoring alone utilizes clinical monitoring for symptoms and automated confirmatory testing as above.

≥120 Threshold and ≥200 Threshold Strategies

Patients are monitored monthly with the paper-based test, regardless of symptoms. Any result exceeding the threshold value is a “positive” test, and medications are stopped (regardless of symptoms) given concerns for DILI; results are confirmed by automated testing as described above.

Bin Placement and Automated Testing Strategies

Patients are tested monthly with either the paper-based test (bin placement) or automated testing, regardless of symptoms. Per ATS TB treatment guidelines [1], any ALT value <120 IU/mL is a “negative” test, and patients continue with TB therapy. Values between 120 IU/mL and 200 IU/mL without symptoms of hepatotoxicity are also “negative” tests. Values between 120 and 200 IU/mL with symptoms of hepatotoxicity are “positive” tests and prompt stopping of TB medications and rechecking ALT by automated testing 3 times weekly. Any ALT value >200 IU/mL, regardless of symptoms, is considered “positive” and results in stopping initial TB meds, initiation of a modified less hepatotoxic regimen, thrice-weekly automated ALT testing as described above, and re-initiation of TB medications once ALT normalizes.

In all strategies, individuals with a false-positive paper test result experience a short-term negative quality-of-life impact (as additional testing is pursued) and increased cost (for confirmatory automated ALT testing). Those with true DILI experience an associated short-term negative quality-of-life impact, elevated cost related to management, and increased probability of death secondary to DILI.

See the Supplemental Methods for further details regarding TB disease progression and HIV therapy and monitoring.

Model Data

Demographic and clinical characteristics were derived from a South African cohort of 969 HIV-infected patients who initiated ART between 2007 and 2010 [13]. The mean age for this cohort was 34 years; 68% were female (Table 1). The median baseline CD4 was 128 cells/µL (interquartile range, 61–186 cells/µL), and all were on ART for at least 12 months. The prevalence of asymptomatic transaminitis, symptomatic hepatotoxicity, and DILI-related mortality was taken from a South African prospective cohort study of HIV-infected persons initiating ART between 2002 and 2005 [14]. The paper-based ALT test characteristics were derived from the published validation study of the paper-based test in persons with liver disease (Table 2) [3].

Table 1.

Input Parameters for an Analysis of the Cost-effectiveness of Transaminase Monitoring for HIV/TB Therapy

Variable	Base Case	Range Evaluated in Sensitivity Analyses	Reference
Baseline cohort characteristics
Mean age, y	34	18–54	[13]
Sex, female, %	68	50–80	[13]
Rate of transaminitis with ALT of 120 IU/mL to 200 IU/mL, rate/100 PYs	19.7	0–24.0	[14]
Rate of transaminitis where ALT > 200 IU/mL, rate/100 PYs	7.7	0–10.5	[14]
Proportion with ALT 120 IU/mL to 200 IU/mL who have symptoms of hepatotoxicity, rate/100 PYs	0.16	0–0.32	[14]
Proportion with ALT > 200 IU/mL who have symptoms of hepatotoxicity, rate/100 PYs	0.35	0–0.70	[14]
Rate of DILI-related mortality, deaths/100 PYs	0.025	0.01–0.04	[14]
Rate of TB-related mortality, deaths/100 PYs	9.3	5.0–19.0	[15]
Rate of mortality from partially treated TB	9.8	5.0–25.0	[16]
Paper-based test characteristics	See Table 2		[3]
Costs, 2015 USD
Background medical costs, US$/mo
HIV infected, TB-uninfected	207.35	120–270	[17]
HIV-infected, TB-infected	222.63	180–614	[12]
TB treatment costs, $/mo
Firstline medications	7.72	3–25	[12]
Firstline retreatment medications	38.53	12–92	[12]
Cost of liver monitoring
Paper-based ALT test	1.00	0.10–6	[3, 18]
Automated ALT testing	3.33	3–7	[19]
Total bilirubin testing	2.42	2–4	[19]
1-wk hospitalization for DILI	481.86	210–860	[12]
Background, non-TB mortality
Standardized mortality risk for all HIV-infected persons (excluding risk of TB)			[20]
1-y survival for HIV infected (no TB)	92.5%
2-y survival for HIV infected (no TB)	85%
50% of HIV-infected died by, y	8.17	6–10
HIV and TB related mortality			[15]
1-y survival for HIV/TB co-infected	89%
2-y survival for HIV/TB co-infected	79%
50% of HIV/TB co-infected die by	5.5 years
Quality-of-life weights (0 = death, 1.0 = best possible health)
HIV on ART with TB	0.819	0.64–0.93	[11, 14, 21]
TB on firstline retreatment medications	0.85	0.70–0.93	Assumption
Alive with partially treated TB	0.70	0.65–0.80	Assumption
False-positive test, holding medications	0.75	0.65–0.85	Assumption
Hospitalization	0.5	0.4–0.6	[11]
Asymptomatic rise in ALT	1	1	Assumption
ALT 120 IU/mL to 200 IU/mL with symptoms of hepatotoxicity	0.75	0.65–0.85	[22, 11, 23]
ALT > 200 IU/mL with symptoms of hepatotoxicity	0.65	0.40–0.70	[22, 11, 23]

Abbreviations: ART, antiretroviral therapy; DILI, drug-induced liver injury; PY, person-years; TB, tuberculosis.

Table 2.

Paper-Based Test Characteristics [3]

	Piccolo (automated testing)
	<120 IU/mL	120 to 200 IU/mL	>200 IU/mL	Total
Paper-based text
<120 IU/mL	58	7	0	65
120 to 200 IU/mL	6	9	2	17
>200 IU/mL	0	2	4	6
Total	64	18	6	88

See the Supplementary Methods regarding rates of TB, HIV/TB mortality, monthly background costs, quality-adjusted life weights, and component costs of the paper-based test. All costs were converted to US dollars (conversion $1 US dollar = 13.47 Rand) and adjusted to 2015 currency year. The total cost of the paper-based ALT test was $1.00.

RESULTS

Base Case

With clinical monitoring alone, discounted QALYs lived was 8.9 years, and discounted lifetime medical costs were $79 152 (Table 3). Using the paper-based test with bin placement strategy resulted in $0.36 increased cost, 0.18 increased quality-adjusted life-months (QALMs), and an ICER of $20/QALY compared with no monitoring. Using the paper-based test with a 200 IU/mL positivity threshold decreased QALYs lived while increasing lifetime medical costs compared with the bin placement strategy, meaning it was never an appropriate use of resources and should not be considered in cost-effectiveness conclusions. At the 200 IU/mL positivity threshold, the sensitivity of the testing strategy was poor, resulting in missed cases of DILI, poor outcomes, and higher costs. The automated testing strategy extended quality-adjusted life expectancy by 0.03 QALMs compared with the paper-based bin placement strategy, at an incremental cost of $13, resulting in an ICER for automated ALT testing compared with the paper-based test of $5200/QALY. Finally, using the paper-based test with a 120 IU/mL positivity threshold was the most costly strategy and had the worst outcomes. Using the test with a 120 IU/mL positivity threshold was poorly specific, such that too many patients advanced to second-line therapy and outcomes suffered.

Table 3.

Cost-effectiveness Analysis of Monitoring for DILI

Monitoring Strategy	Cost, $	Incremental Cost, $	Efficacy, QALM	Incremental Efficacy, QALM	ICER, $/QALY
Clinical monitoring	79 151.65	—	106.84	—	—
Paper test, bin placement	79 152.01	0.36	107.02	0.18	20
Paper test, 200 cutoff	79 154.36	2.35	107.00	–0.02	Dominated
Automated testing	79 164.95	12.94	107.05	0.03	5180
Paper test, 120 cutoff	79 222.74	68.8	104.44	–2.61	Dominated

ICERs were rounded to the nearest $10.

Abbreviations: ICER, incremental cost-effectiveness ratio; DILI, drug-induced liver injury; QALM, quality-adjusted life-month; QALY: quality-adjusted life-year.

Sensitivity Analyses

Test Performance

Paper-based test accuracy was important to cost-effectiveness conclusions, but not all types of error had equivalent impact. Avoiding systematic testing error that overcalls a positive test result (type I error) was an important driver of clinical outcomes and cost. When we doubled the probability that the paper-based test would incorrectly categorize a normal ALT as above the positivity threshold, QALYs lived was 0–3% lower for strategies employing the paper-based test, and costs were higher. Overcalls of a positive test result (type 1 error) using the paper-based test resulted in inappropriately stopping medications and additional costly lab testing. In comparison, when we doubled the probability that a patient with a true ALT elevation was miscategorized as having a negative test (“undercall”), QALYs lived was only 0–1% lower for strategies that used the paper-based test.

Drug-Induced Liver Injury

Incidence of transaminitis, associated symptoms of hepatotoxicity, and associated mortality all influenced cost-effectiveness conclusions (Figure 1). As the probability of symptomatic DILI increased, automated testing was no longer cost-effective and using the paper test with bin placement was the most efficient strategy. As mortality from DILI increased, the importance of identifying early ALT elevations increased, such that the most efficient method for ALT monitoring was automated testing (Supplemental Table 12). In all scenarios, using the paper-based test with the bin placement strategy resulted in better outcomes than clinical monitoring alone.

Figure 1.

This tornado diagram depicts the results of multiple 1-way sensitivity analyses. The horizontal axis is denominated in terms of the incremental cost-effectiveness ratio ($/quality-adjusted life-year) of using the paper-based ALT test with “bin placement” interpretation compared with the next best alternative. Each bar represents the range of ICERs observed when varying a single model parameter through its plausible values. Long bars demonstrate parameters that have a large impact on ICERs. *Parameters for which the range of values tested included feasible values that resulted in automated serum testing becoming a dominating strategy. Abbreviation: QALY, quality-adjusted life-year.

Cost of the Paper-Based Test

The cost-effectiveness of paper-based testing was robust to a wide range of test costs. As long as the paper-based test cost was ≤$3.10—that is, approximately the cost of automated testing—it was preferred from a cost-effectiveness perspective (Figure 2).

Figure 2.

Results of sensitivity analysis of the cost-effectiveness of the paper-based transaminase test on the incremental cost-effectiveness ratio of serum ALT monitoring by automated testing. Below a cost of $3.10, monthly monitoring using the paper test with bin placement interpretation is cost-effective compared with monthly automated testing. Beyond $3.10, the paper-based test is no longer cost-effective compared with automated testing. Abbreviation: QALY, quality-adjusted life-year.

Additional Deterministic Sensitivity Analyses

No other parameters impacted the qualitative conclusions about cost-effectiveness when varied through their feasible range, including age, gender of the cohort, mortality from HIV, mortality from TB, and cost of care other than TB care (see the supplementary tables).

Probabilistic Sensitivity Analysis

In probabilistic sensitivity analysis, the paper-based test using bin placement always resulted in better life expectancy and more QALYs lived than clinical monitoring alone, with an ICER that was less than the South African willingness-to-pay threshold in >99% of simulations. Assuming a willingness to pay of 3× the South African per capita GDP ($20 000/QALY), the bin placement strategy was the cost-effective option in 13% of simulations and it was never a dominated option, while automated testing resulted in higher life expectancy and was cost-effective in 87% of simulations (Figure 3).

Figure 3.

Cost-effectiveness acceptability curve for paper-based testing vs automated testing. Abbreviation: QALY, quality-adjusted life-year.

DISCUSSION

We used Markov modeling to investigate the clinical outcomes and cost-effectiveness of a novel paper-based POC ALT test. Compared with the current standard of clinical monitoring alone, the paper-based test identifies cases of DILI earlier and prevents morbidity. In our base-case scenario, monthly automated testing provided very little clinical benefit over monthly testing with the paper-based test. If resources that could be invested in expanding the availability of automated ALT measurement for routine monitoring were instead used to broadly implement paper-based monitoring with automated confirmatory testing only as needed, the reach of monitoring ALT would likely increase and population-level outcomes could improve.

Our sensitivity analyses demonstrate 2 important points: First, when automated testing is not broadly feasible for routine monitoring of HIV/TB co-infection treatment, the paper-based test using the bin placement strategy is clearly effective and cost-effective. In >99% of our simulations, using the paper-based test with bin placement strategy resulted in better life expectancy and more QALYs lived than clinical monitoring alone. Using the bin placement strategy for the paper-based test is important as employing the ≥120 threshold interpretation of the paper-based test resulted in poor outcomes due to the low threshold for this “positive” result inappropriately classifying asymptomatic patients with ALT >120 as having DILI and inappropriately advancing patients to higher-cost, lower-efficacy second-line TB medications. Thus, the specificity of the monitoring strategy is at least as important as its sensitivity.

Second, probabilistic sensitivity analyses demonstrate that when automated testing is widely available, it is likely the optimal choice. In 86% of simulations where automated ALT monitoring was most efficient, the marginal effectiveness of automated ALT monitoring was small. Policy-makers can feel confident that with limitations regarding feasibility and sustainability of routine automated ALT monitoring, the use of the paper-based test with bin placement strategy and automated confirmatory testing provides benefit. Finally, our results also address the sustainability of the paper-based platform; only when the test costs over $3.00 (3 times the anticipated cost) did cost-effectiveness conclusions change.

Limitations of our analysis include that data to inform rates of DILI, associated symptoms of hepatotoxicity, and probability of death from DILI were taken from a single South African cohort study [14]. Probability of DILI may vary by region and by prevalence of other co-infections such as viral hepatitis. We conducted a priori–defined sensitivity analyses around toxicity parameters, however, and found that across a broad range of assumptions about rates of DILI, monitoring with the paper-based test was cost-effective. Furthermore, we used mathematical assumptions to estimate the benefit from transaminase monitoring. Though routine monitoring for DILI is the standard of TB care, there is no conclusive evidence that such monitoring actually reduces mortality. In the pessimistic scenario in which DILI monitoring continues to be recommended as the standard of care but provides no real benefit, using the paper-based test despite being standard of care routinely would at least be cost-saving, as the paper-based test is less expensive than automated testing. Additionally, South Africa is unique as it has the highest rate of HIV/TB co-infection; it also has a higher GDP per capita compared with other sub-Saharan African countries. Our findings may not be generalizable to the continent. However, we expect that in countries with a lower GDP per capita and fewer laboratory resources, the paper-based test would be an even more valuable investment. Paper-based test performance data [3] included a relatively small number of patients with elevated ALT, potentially limiting the assessment of device accuracy in that ALT range. We used automated testing in each strategy to confirm paper-based test results. Thus, our findings are not generalizable to settings where automated testing is entirely unavailable. Similar to other cost-effectiveness analyses, this study uses estimates of health state utilities from the medical literature, which may not accurately reflect the experience of HIV/TB co-infected patients in South Africa. Furthermore, this model did not account for those with a cholestatic pattern of DILI without transaminase elevations. Finally, this model was specific to monitoring DILI in HIV/TB co-infected patients. The paper-based test could potentially be used in other contexts, such as in newly diagnosed viral hepatitis, but the conclusions from our study would not be generalizable to such a scenario.

This analysis demonstrates the utility of modeling to query optimal performance characteristics and test usage strategies for a novel POC test in development—analysis that can optimize the use of test development resources and strategic planning for early test deployment. In our analysis, we used Markov modeling to define the best methods of test use. At an anticipated cost of $1 per test, and at its current performance, monthly DILI monitoring using the paper-based test with a bin placement strategy is an effective and cost-effective approach that could potentially replace routine automated ALT monitoring. This paper-based ALT test has the potential to bring routine monitoring for DILI to remote sites that have not previously had sufficient laboratory infrastructure, thus reducing morbidity and mortality associated with DILI among HIV/TB co-infected persons.

Supplementary Data

Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Click here for additional data file.