Early Economic Evaluation to Identify the Necessary Test Characteristics of a New Typhoid Test to be Cost Effective in Ghana.

BACKGROUND: In Ghana, there are issues with the diagnosis of typhoid fever; these include delays in diagnosis, concerns about the accuracy of current tests, and lack of availability. These issues highlight the need for the development of a rapid, accurate, and easily accessible diagnostic test. The aim of this study was to conduct an early economic analysis of a hypothetical rapid test for typhoid fever diagnosis in Ghana and identify the necessary characteristics of the test for it to be cost effective in Ghana.
METHODS: An early cost-utility analysis was conducted using a decision tree parameterized with secondary data sources, with reasonable assumptions made for unknown parameters. The patient population considered is individuals presenting with symptoms suggestive of typhoid fever at a healthcare facility in Ghana; a time horizon of 180 days and the Ghanaian national health service perspective were adopted for the analysis. Extensive sensitivity analysis was undertaken, including headroom analysis.
RESULTS: The results here show that for a hypothetical test to perform better than the existing test (Widal) in terms of QALYs gained and cost effectiveness, it is necessary for it to have a high specificity (at least 70%) and should not be priced more than US$4. The overall value of conducting research to reduce uncertainty (over 5 years) is US$3287.
CONCLUSION: The analysis shows the potential for the hypothetical test to replace the Widal test and the market potential of developing a new test in the Ghanaian setting.

Key Points for Decision Makers

Introduction

Typhoid fever is an acute infection caused by the bacteria Salmonella typhi and paratyphi [1]. Transmission occurs via the consumption of food or water contaminated with the pathogens rather than person-to-person [2], and humans serve as the only natural host and reservoir for the causative organisms [3]. Typhoid fever remains an important cause of disease worldwide [4], but particularly, in resource-limited countries [5]. This is as a result of poor hygiene practices [6] and an inadequate diagnostic laboratory capacity to differentiate typhoid fever from other febrile conditions [7].

In Ghana (a resource-limited country), reported annual incidence of typhoid fever in 2017 was 60,892 cases [8]. Typhoid fever ranks among the 20 leading causes of outpatient illness in the country, accounting for 0.92% of all hospital admissions [9]. The two tests for typhoid fever diagnosis in Ghana are the Widal test and blood culture [10]. Compared with blood culture, the Widal test is easier and requires fewer resources to perform; however, the Widal test is an inaccurate indicator of typhoid fever [11, 12], resulting in inappropriate treatment frequently being administered [10]. The treatment of individuals falsely testing positive for typhoid has led to the development of resistance of Salmonella typhi and paratyphi to treatments such as chloramphenicol, and this has resulted in an increase in the use of the more expensive ciprofloxacin in Ghana [10]. The alternative to the Widal test, blood culture, lacks sensitivity [7], meaning that the disease will be missed in many patients, resulting in lost opportunities for treatment, which might lead to increased morbidity and mortality. Furthermore, culture requires equipment, supplies, trained laboratory personnel, and electricity, which may not be available in primary healthcare facilities in Ghana, and when performed, can take 2–3 days before test results are obtained. Thus, diagnosis may be delayed and patients may end up with complications such as intestinal perforation resulting in death if treatment is postponed or patients are lost to follow up [13]. Furthermore, those without typhoid fever may receive unnecessary and inappropriate antimicrobial treatment if treated presumptively whilst awaiting results [5]. Another issue is that typhoid fever most closely clinically resembles other febrile conditions such as malaria and is easily misdiagnosed without laboratory confirmation [14], which could lead to an overprescribing of antimalarial therapies associated with a huge economic impact on the Ghanaian economy [15]. Therefore, the development of a test for typhoid fever in Ghana that is accurate, simple, and affordable with a quick turnaround time for typhoid fever has an obvious attraction.

The aim of this study was to conduct an early economic evaluation to examine the potential cost effectiveness of a hypothetical test (hereafter referred to as the HT-test) in Ghana, with specific focus on the estimation of the maximum price and the minimum test performance required for the HT-test to be cost effective in comparison with the Widal test. In a context of ever-increasing demand to demonstrate value for money, technology developers can no longer assume that products developed will be adopted and funded. Thus, research and development of new technologies should be driven by considerations of value to payers of healthcare, and early economic evaluation has much conceptual attractiveness in this regard. Early economic evaluation is a tool that is useful for technology developers to help them understand the value of a new technology to payers, the expected commercial viability, the risks, and the potential return on investment in the technology. The value of such analyses is that they can be cost saving by directing future research development into areas of greatest need, and avoid wasting money on unnecessary research if the early economic evaluation shows that the price of the new technology will need to be too low to make a profit. Headroom analysis is one of the key components of early economic evaluation. It considers whether there is sufficient unmet need for a new technology with regards to a specific condition to support a price consistent with an acceptable return on investment. Clearly, early economic evaluation has potentially profound advantages for technology developers and encouraging companies to use this approach has the potential to guide the development of new technologies on the basis of need rather than trying to find the best fit for them. This analysis considers only the Widal test (but not blood culture) because it is the better of the two tests currently available in Ghana (in terms of accuracy) [16]; therefore, by adopting the Widal test as the comparator, the analysis provides a more robust examination of the HT-test.

Methods

We conducted an early cost-utility analysis to examine the potential economic value of the HT-test in the Ghanaian setting. In this analysis, the HT-test is the test to be evaluated, and the comparator intervention against which the performance of the HT-test is compared is the Widal test. The patient population considered is individuals presenting with symptoms suggestive of typhoid fever (diagnosis is unknown at presentation) at a healthcare facility in Ghana. A time horizon of 180 days was chosen because all pathways eventually lead to successful treatment and the time horizon needs to be at least as long as the longest test–treat pathway. In this instance, the longest test–treat pathway was 34 days. Thus, 180 days was chosen to be an appropriate time over which patients will benefit from the effects of testing and treatment for typhoid fever [17]. Also, by taking the time horizon to 180 days, it allows other feasible options to be added to the model that could have a longer treatment time. The Ghanaian national health service perspective was adopted for the analysis because the health service is the direct payer for healthcare services and the aim of this analysis is to support decision making for the Ghanaian health service [18]. No discounting of costs and benefits was undertaken because the time horizon did not exceed 12 months [19]. The findings are expressed in terms of cost, quality-adjusted life-years (QALYs), and net-monetary benefit (NMB), where NMB is defined for each intervention as

In Ghana, a cost-effectiveness threshold of < US$104–US$951/QALY is generally considered to be cost effective by policy makers [20]. For this analysis, the upper limit of the cost-effectiveness threshold was chosen because the lower limit is practically too low and adopting the lower threshold value means most interventions could never be cost effective.

Model Choice

A static model rather than a dynamic model was adopted, although it is acknowledged that typhoid fever is an infectious disease. Underpinning this decision is the fact that, unlike vaccination where the role of the intervention is to reduce the incidence of typhoid transmission (which is best captured by a dynamic model) [21], there is no evidence in support of early diagnosis and treatment affecting onward transmission [22]. In the case of this analysis, the emphasis was on evaluation of the direct benefits of testing and treatment to an individual (i.e., survival and quality of life) rather than benefits to the population via prevention of onward transmission. Furthermore, all the patients in this model are assumed to be treated successfully, and because the means of typhoid transmission is not direct person-to-person [2], even a delay of 3 days until cure is unlikely to contribute substantially to onward transmission. In conclusion, under these circumstances, a static model was considered appropriate. Decision trees were employed because they are particularly suited to acute clinical conditions and typhoid fever is an acute condition.

A key determination in test evaluation is the best placement and role (replacement, triage or add-on) of a new test on existing pathways. Therefore, as a comparator to the Widal test, the HT-test was evaluated in three possible roles, resulting in the construction of three decision trees (Fig. 1a–c). Each decision tree has two arms (current practice and the hypothetical testing strategy). The test–treat pathway for a suspected typhoid case, for each of the different roles, is as shown in the different decision trees. In the replacement role, the HT-test is evaluated to determine whether it should be used as the main investigative test without any further testing with the Widal test. In the triage role, the HT-test is evaluated to determine whether it should be used to test patients first to decide those requiring further testing with the Widal test. In the add-on role, the HT-test is evaluated to determine whether it should be used to confirm diagnosis after testing patients first with the Widal test (Fig. 2).

Fig. 1

a Model structure for evaluating hypothetical test in the replacement role. b Model structure for evaluating hypothetical test in the triage role. c Model structure for evaluating hypothetical test in the add-on role

Fig. 2

The role and placement of tests in existing care pathways: rapid test for typhoid

Test Accuracy

In this analysis, each test is considered to provide either a positive or negative test result. This allows the application of sensitivity and specificity values to be considered in the analysis. The resulting possible test statuses for patients in each pathway are as follows: true positive (TP) (has typhoid and tests positive for it), true negative (TN) (does not have typhoid and tests negative for it), false positive (FP) (does not have typhoid but tests positive for it) and false negative (FN) (has typhoid but tests negative for it). Given that the HT-test is a hypothetical test, there was no information on the sensitivity and specificity of the test. Test accuracy values used for the Widal test are shown in Table 1.

Table 1

Model parameters used in the economic analysis along with the data source

Parameter	Estimate	Data source
Sensitivity and specificity values for the Widal test
Sensitivity of Widal test (CI)	0.69 (0.61–0.75)	[16]
Specificity of Widal test (CI)	0.83 (0.77–0.88)	[16]
Probability estimates utilized in the economic analysis
Probability of successful first typhoid treatment	0.95	[28]
Prevalence of typhoid in the patient population presenting	0.48	[29]
Parameter estimates used to inform QALY values
Utility when experiencing typhoid symptoms (CI)	0.867 (0.81–0.912)	[17]
Perfect health utility	1	By definition: [19]
Time horizon of model (days)	180	[17]
Number of days per typhoid treatment	14	[10]
Mean recovery time for successful typhoid treatment (SD)	3.68 (0.92)	[28]
Number of days per malaria treatment	3	[10]
Mean recovery time for successful malaria treatment (SD)	1.83 (0.95)	[30]

CI confidence interval, OPD outpatient department, QALY quality-adjusted life-year, SD standard deviation, VOI value of information

Model Assumptions

In order to implement a working model, the following assumptions were made:

For a patient presenting with suspected typhoid, if the patient does not have typhoid, then it is assumed they have malaria because malaria is the most probable differential diagnosis [7].

The results obtained from retesting a patient following a treatment failure with the same test are independent of those from the first test.

If the first round of typhoid treatment fails in individuals who actually have typhoid, then the second round will always be successful. This is because ciprofloxacin has been found to be highly active against Salmonellae in vivo and repeated treatment is usually associated with a 100% cure rate [23].

For any patient that has malaria, every first treatment is successful. This is justified because artemisinin-based combination treatments (ACTs) (the first-line drug for uncomplicated malaria in Ghana) are rapid and effective (effectiveness usually exceeds 95%) [24].

There is no uncertainty about the length of treatment for typhoid and malaria. These are informed by the standard treatment guidelines in Ghana.

All patients experiencing treatment failure return for retesting immediately after completion of a treatment course.

Delays on the test–treat pathway (i.e., delay receiving test results, delay receiving treatment) are negligible.

Patients experience the same utility at the different phases of treatment as they do when experiencing untreated typhoid symptoms but return to perfect health once treatment has been completed. This assumption was made because of the paucity of evidence on the impact of treatment on patient quality of life.

Utility when experiencing typhoid symptoms is the same as when experiencing malaria symptoms as both diseases have similar symptoms during the acute phase of illness [25].

Model Description

Any action taken is dependent on the information available to the clinician at that point in time (i.e., test results and response to treatment). For example, as shown in the current practice arm (Fig. 1), individuals presenting with symptoms suggestive of typhoid who test positive and actually have typhoid (TP) are given a first round of typhoid treatment, some of whom will be successfully treated and others not. TPs experiencing treatment failure are retested and re-test TPs are given a second round of typhoid treatment, which in the model is assumed to be successful. Retested negatives (re-test FN) will be given malaria treatment on the basis that typhoid treatment has been unsuccessful. However, because re-test FN patients actually have typhoid, the malaria treatment will be ineffective. They are then given a second round of typhoid treatment following the malaria treatment and this second round of treatment is assumed to be successful.

Parameterization

A range of secondary sources was utilized to parameterize the model. These secondary sources were identified through extensive searching of published literature, websites of government departments in Ghana (Bank of Ghana), and the National Health Insurance Scheme tariff of Ghana (2016). Table 1 shows the parameters used in the economic analysis along with the data source. Given that this is an early economic evaluation, the parameters describing the characteristics of the HT-test were unknown, and thus these parameters were subject to extensive sensitivity analysis to draw conclusions about the characteristics of this test necessary for it to be cost effective.

Measurement of Effects

The primary outcome measure was the QALY. Total QALYs were estimated for each branch based on the parameters described in Table 1.

For every round of unsuccessful treatment, the total number of days per treatment is included in the estimation, and for every successful treatment, the mean recovery time for successful treatment is also included. For example, total QALYs for a branch where a patient receives two rounds of typhoid treatment (i.e., one unsuccessful and one successful) was calculated as [(U1T1) + (U1T2) + (U2T3)]/365 days.

Where U1 is the utility when experiencing typhoid symptoms, T1 is the number of days per typhoid treatment, T2 is the mean recovery time for successful typhoid treatment, U2 is the perfect health utility, and T3 is the number of days in perfect health after two typhoid treatments.

The time horizon (TH) has been incorporated into the above calculation as follows:

Costs

Total cost for each branch was estimated based on the perspective of the Ghanaian national health service for the 2018 cost year. All costs were converted to US dollars (US$) by using the exchange rate (Gh₵1 = US$0.225) adopted from the website of the Bank of Ghana at the time of analysis (29 March 2018, Bank of Ghana, 2018). Table 1 shows the costs used in the economic analysis. Ciprofloxacin and artemether/lumefantrine are the first-line drugs of choice for typhoid and malaria treatment, respectively, in Ghana.

The total cost for a branch where a patient has visited a health facility once, received the Widal test, and then typhoid treatment is estimated as follows:

The OPD (outpatient department) tariff is an amount reimbursable to the service provider per visit to cover other costs like staff time and consultation. On any pathway where initial treatment is unsuccessful, there will be more than one visit and, in such instances, the OPD tariff is multiplied by the number of visits and added to the other costs incurred to estimate the total cost for that branch.

Dealing with Uncertainty

There was no information on test accuracy or cost of the new test. Thus, each of these parameters was a separate focus of sensitivity analysis. Headroom analysis (i.e., maximum cost at which the new technology is still cost effective at a given WTP threshold) using NMB was conducted. The price at which the NMB of using the HT-test equals the NMB of using the Widal test gives information on the headroom price of the HT-test compared with the Widal test. Furthermore, sensitivity and specificity pairs for the HT-test were varied and the maximum price for each pair determined. This analysis gave information on how good the test needs to be (in terms of accuracy), and at what price, for it to be cost effective. By varying accuracy pairs for the HT-test, the incremental effectiveness for each accuracy pair could be determined and information on which accuracy pairs provide the most benefit to patients within cost-effectiveness constraints could be identified.

Probabilistic sensitivity analysis (PSA) was also conducted. A probability distribution was defined for each uncertain parameter (Table 1). A beta distribution was assigned to all uncertain parameters except the test accuracy parameters of the Widal test. Table 1 shows the beta distributions assigned to each of the uncertain model parameters. Sensitivity and specificity are correlated and therefore required a different method that accounts for the correlation between these parameters [26]. Data from a meta-analysis of typhoid diagnostic accuracy studies [16] were utilized to obtain the hierarchical summary receiver operating characteristic (HSROC) curves for the Widal test. Using the Stata function, metandi, to obtain the HSROC curve and the following equation that links sensitivity to any given specificity, it was possible to take a value of specificity and calculate a corresponding sensitivity along the HSROC curve.

where lambda is the mean of the accuracy parameter and beta is the shape parameter.

These distributions were then sampled for the PSA. The model was run 1000 times, each time randomly selecting a value for each parameter from their respective distributions. Mean costs and mean QALYs were calculated by averaging across all 1000 simulations. Cost-effectiveness acceptability curves (CEACs) were drawn to gain a greater understanding of realistic price and test accuracy parameters.

Value of Information (VOI) Analysis

Value of information (VOI) analysis was conducted to provide insights into the value of conducting further research from the perspective of the healthcare provider in Ghana. In the absence of information on test accuracy parameters of the HT-test and specific guidance on VOI analysis at this early stage of analysis, the assumption was made that the same distributions for accuracy characteristics could be used for the HT-test as the Widal test, sampling independently for the two sets. The VOI analysis was repeated for different prices in each role as informed by the results of the headroom analysis. Table 1 shows the parameters used in the VOI analysis.

In VOI analysis, insight into the parameters that cause the most decision uncertainty and the potential value of reducing the uncertainty by collecting more data were examined using expected value of partial perfect information (EVPPI). In this study, EVPPI was estimated using SAVI (Sheffield Accelerated Value of Information) [27].

Results

Headroom Analysis Using Net-Monetary Benefit (NMB)

As shown in Fig. 3, taking all parameters at baseline, at a WTP of US$951 per QALY, the HT-test can be priced up to US$3.85 and still be cost effective against the Widal test in the replacement role. The maximum prices at which the HT-test can be priced and still be cost effective against the Widal test in the triage and add-on roles are US$1.94 and US$1.47, respectively (see Appendix in the electronic supplementary material [ESM]).

Fig. 3

Headroom of the HT-test vs. Widal test at a WTP of $951/QALY (replacement role)

Incremental Effectiveness (QALYs) Varying Sensitivity and Specificity Pairs

Tables 2 and 3 show the incremental effectiveness and the maximum price at which the HT-test is still cost effective at each sensitivity and specificity pair for the HT-test vs. Widal test in the replacement role. The tables for the triage and add-on roles are presented in the Appendix (see ESM).

Table 2

Incremental effectiveness (QALYs) at each sensitivity and specificity pair for the HT-test vs Widal test (replacement role)

Table 3

Maximum price (US$) at which the HT-test is still cost effective at each sensitivity and specificity pair for the HT-test vs Widal test (replacement role)

NV represents negative values for price and implies that the HT-test cannot be cost effective at these sensitivity and specificity pairs and the end user of the test would have to be paid for using the test. Shaded black cells represent cases where either the test is completely useless, or it would be better to take the reverse of the test results

Values in bold format represent price at sensitivity and specificity pairs at which the HT-test is still cost effective and has a positive incremental effect, and would normally be of interest to a decision maker

Values in italic format represent price at sensitivity and specificity pairs at which the HT-test is still cost effective but is slightly less effective than the Widal test

As shown in the set of tables (Tables 2, 3), the HT-test can be priced up to US$3.85 and still be cost effective against the Widal test in the replacement role when the HT-test has both sensitivity and specificity of 100%. In this case, the incremental effectiveness is 0.0010. When the HT-test has both sensitivity and specificity of 70%, it can be priced up to US$0.95 and still be cost effective; however, here the incremental effectiveness is − 0.0002. This means that as the accuracy of the HT-test decreases, the maximum price of the HT-test for it to be cost effective against the Widal test in the replacement role reduces with a decrease in incremental effectiveness. The same observation is made for the triage and add-on roles.

Cost-Effectiveness Acceptability Curves (Replacement Role)

Using a reasonable price range for the HT-test of US$1.00–US$4.00, which was informed by findings shown in Tables 2 and 3, the CEACs for the PSA of the HT-test vs. the Widal test in the replacement role are presented below. The set of figures for the triage and add-on roles are presented in the Appendix (see ESM). In the CEACs, the price is fixed whereas sensitivity and specificity are varied using the following pairs of values (1:1; 0.6:1; 1:0.6; 0.9:0.9; 0.7:0.9; 0.9:0.7; 0.8:0.8; 0.7:0.7).

The results show that specificity is a major driver of the probability of the HT-test being cost effective at the threshold value adopted. For any price above the estimated headroom, the HT-test was always < 50% likely to be cost effective at the WTP threshold, even for the most optimistic scenario for the HT-test (i.e., both sensitivity and specificity of 100%). For example, the headroom price for the HT-test in the replacement role as shown in Table 3 is US$3.85. From Fig. 4a, at a price of US$4 (which is above the headroom), the HT-test is < 50% likely to be cost effective, even when both sensitivity and specificity are 100%. The results further show that when the price is low (below the price of the Widal test) (in this case US$1), then all the pairs with positive expected incremental effectiveness have a 100% probability of being cost effective (Fig. 4b).

Fig. 4

a CEAC (with variations in the sensitivity and specificity of the HT-test at a fixed price of $4.00) at different WTP values (replacement role: the HT-test vs. Widal test). b CEAC (with variations in the sensitivity and specificity of the HT-test at a fixed price of $1.00) at different WTP values (replacement role: the HT-test vs. Widal test)

VOI Results

Using sensitivity and specificity for the HT-test sampled from the same distribution for the Widal tests and a price of US$1 informed by the CEACs, the results of the VOI analysis for the replacement role are presented below. The results for the triage and add-on roles are presented in the Appendix (see ESM). As shown in Fig. 5, at the WTP threshold adopted, the overall expected value of perfect information (EVPI) for the specified decision relevance time horizon of 5 years is US$3287.

Fig. 5

VOI analysis of the HT-test vs. Widal test in the replacement role at a price of $1.00

Table 4 shows the EVPPI for each of the uncertain model parameters at the adopted WTP threshold. As shown in Table 4, specificity of the HT-test and Widal test are the two parameters causing most of the decision uncertainty. The standard error values also indicate that sufficient replications were undertaken.

Table 4

Single-parameter EVPPI

Model parameters with uncertainty	Per person EVPPI (US$)	Standard error	EVPPI for Ghana over 5 years (US$)
Mean recovery time for successful typhoid treatment	0	0	0.00
Mean recovery time for successful malaria treatment	0	0	0.00
Prevalence of typhoid in the patient population presenting	0	0	0.00
Probability of first successful typhoid treatment	0	0	0.00
Utility when experiencing typhoid symptoms	0	0	0.00
Specificity of Widal test	0.000222	0	62
Sensitivity of Widal test	0	0	0.00
Specificity of the HT-test	0.001010	0	283
Sensitivity of the HT-test	0	0	0.00

The standard error shows whether there have been enough replications in the model

EVPPI expected value of partial perfect information

Table 5 shows the percentage likelihood of the HT-test being cost effective compared with the Widal test at a given price and accuracy pair for the HT-test at the adopted WTP threshold. For example, at a price of US$2 and test accuracy parameters of sensitivity 60% and specificity 100%, the HT-test is 63% likely to be cost effective in a replacement role compared with the Widal test. The results also show that, as the price of the HT-test drops, more and more pairs of sensitivity and specificity values become cost effective.

Table 5

Summary of headroom and percentage likelihood of cost-effectiveness results for the HT-test vs Widal test

Role	Headroom price (US$)	Incremental effectiveness (QALYs) (positive range)	Price range (US$) (across range of positive incremental effectiveness)
Replacement	3.85	0–0.0010 (specificity at least 70%)	1.64–3.85 (specificity at least 70%)
Triage	1.94	0–0.0003 (specificity at least 90%)	0.34–1.94 (specificity at least 90%)
Add-on	1.47	0–0.0002 (specificity at least 90%)	0.76–1.47 (specificity at least 90%)

Table 6 shows that for the assumptions made about the HT-test, the EVPI per person is US$0.00 when the HT-test is US$2.00; thus, the overall EVPI for 5 years is also US$0.00. This indicates that research into other parameters to resolve uncertainty is not worth doing based on the assumptions made about the HT-test. The EVPPI results also show that at this price (US$2.00) there is no uncertainty associated with the individual model parameters as an inevitable consequence of the overall EVPI being zero.

Table 6

Summary of VOI analysis for the HT-test vs. the Widal test

Role	Price (US$)	EVPI per person	Overall EVPI for the decision-relevant time horizon (5 years)
Replacement	2.00	0.00	0.00
	1.00	0.011723	3287
Triage	1.00	0.001029	289
	0.50	0.066283	18585
Add-on	1.00	0.000082	23
	0.50	0.001449	406

EVPI expected value of perfect information, EVPPI expected value of partial perfect information, VOI value of information

The results further show that, regardless of the scenario, mean recovery time for successful typhoid treatment, mean recovery time for successful malaria treatment, probability of successful first typhoid treatment and the sensitivity of the HT-test do not contribute to the decision uncertainty. Overall, the prevalence of typhoid in the patient population, the specificity of the Widal test, and the specificity of the HT-test can be seen to be the parameters contributing to most of the decision uncertainty in the analysis.

Discussion

Using a model-based economic evaluation with the outcome measure of the QALY, this study sought to identify the key characteristics of a hypothetical test for typhoid diagnosis for it to be cost effective in the Ghanaian context compared with the Widal test. A particular focus of this evaluation was the position of the HT-test on the testing pathway.

The headroom analysis (where the assumption is made that the HT-test is 100% accurate) shows that the HT-test can be priced up to US$3.85 and still be cost effective against the Widal test in the replacement role. The maximum prices at which the HT-test can be priced and still be cost effective against the Widal test in the triage and add-on roles are US$1.94 and US$1.47, respectively. Thus, if it is not possible to develop the HT-test so that it can be sold below these prices and make a profit, then resources should not be committed to its further development. Furthermore, it can be concluded that a test developer will gain most from the new test replacing current practice. It is worth stating that although a higher price allows for a greater chance of a return on investment, it also increases the possibility of rejection on budget impact grounds.

Acknowledging that the headroom price (when the HT-test is 100% accurate) is probably going to be unrealistic, the focus turns to how accurate the HT-test needs to be to be cost effective. Additional information on the value of the HT-test compared with the Widal test in each of the roles was considered based on effectiveness (QALYs) and headroom prices. For example, it is shown that the HT-test can be priced up to US$3.85 and still be cost effective against the Widal test in the replacement role when the HT-test has sensitivity and specificity of 100%. In this case, the incremental effectiveness is 0.0010. When the HT-test has a sensitivity and specificity of 70%, the maximum price at which it can still be cost effective is US$0.95 but with an incremental effectiveness of − 0.0002. This indicates that as the accuracy of the HT-test decreases, the maximum price at which the HT-test can be priced and still be cost effective against the Widal test in the replacement role must also decrease. This implies that greater potential returns on investment are possible with a better test. Although care must be taken when recommending a test with characteristics that lead to a negative effectiveness, as further factors beyond cost effectiveness may need to be considered for the test to be acceptable to decision makers (e.g., small decrease in effectiveness coupled with large cost savings). The maximum price at which the test can be sold after development when both sensitivity and specificity are 100% is far greater and more attractive to investors compared with when both parameters are 70% (US$3.85 compared with US$0.95) and the incremental effectiveness is also more attractive from the patient perspective (i.e., 0.0010 compared with − 0.0002). Similar conclusions can be drawn for the other roles of the HT-test vs. Widal test.

The PSA results indicate that as the price of the HT-test drops, more and more pairs of sensitivity and specificity become cost effective (i.e., the PSA supports the main findings). The PSA results help to provide a greater understanding of a realistic price and test accuracy parameters for the HT-test to be cost effective. For example, in the replacement role the headroom price is US$3.85; however, at this price the PSA reveals that the HT-test is likely to be cost effective only when both sensitivity and specificity are 100%. Acknowledging the fact that no test is likely to be 100% accurate, it is not realistic for investors to invest in the development of a new test at this price and with these accuracy pairs.

The PSA results also show that specificity (at least 70%) is a major driver of the probability of the HT-test being cost effective at the adopted WTP threshold. This observation is explained by the prevalence of typhoid in the patient population and what happens to FPs and FNs following the test–treat pathway for a suspected typhoid case as shown in the different decision trees. From the models, FP is worse for patients than FN because of the extra testing and treatments patients undergo before they are eventually cured. Thus, a high specificity is required in order to avoid the unnecessary testing and treatment associated with these pathways.

It is shown in the VOI analysis (in the replacement role) that for the specified decision relevance time horizon of 5 years, and for an annual incidence of 60,000, the overall expected value of removing decision uncertainty when the HT-test is US$1.00 would be US$3287. This implies that research or data collection exercises costing more than this amount would not be a cost-effective use of resources to resolve this decision uncertainty. This is because the cost of making a wrong decision by the decision maker, as measured by the cost savings of enabling the decision maker’s ability to switch and select other strategies when evidence obtained reduces decision uncertainty, is expected to be no higher than US$3287. From this value, it can be concluded that there might be no value in resolving this uncertainty as any research studies on further data collection are likely to cost more than this amount. The VOI analysis further shows that different parameters (prevalence of typhoid in the patient population, utility when experiencing typhoid symptoms, sensitivity and specificity of the Widal test, specificity of the HT-test) contribute to the decision uncertainty. These results will be of interest to a technology developer as it shows where future research should be focussed. The calculations presented here assume immediate and 100% uptake. This is because a recent qualitative study by this study team shows that, in the Ghanaian setting, patients are most likely to take a test requested by their clinician [32] and, because the new technology would be administered in the same way as the current technology, uptake is likely to be the same for both tests, and therefore its exclusion should not impact on conclusions drawn from the model.

This work demonstrates the importance of early economic evaluation as a tool in priority setting by providing insight into whether to fund the further development of a new product, thereby helping to allocate limited budget more efficiently. Moreover, this study has demonstrated how modelling can be used to consider different test–treat pathways and draw conclusions about the optimum placement of a new technology on a care pathway. Specific to the context of this study, it has shown that in the developing world context, the cost-effectiveness threshold is very low, and thus, does not provide much headroom in which to develop new technologies. The findings of this study have wider implications for the early economic evaluation of diagnostic tests by contributing to a limited evidence base and identifying areas requiring further research.

Early economic evaluations are typically characterized by a lack of data to populate models. In the absence of such evidence, expert judgement may be relied upon for model parameterization and assisting in model structure conceptualization. However, unless the early economic model is for a specific technology in development, where experts can provide insights into the characteristics of the technology (e.g., cost and effectiveness), then expert elicitation to inform parameters related to the new technology may not be necessary.

Strengths and Limitations of the Study

A strength of this analysis is that extensive sensitivity analysis has been conducted to identify the impact of key parameters on the conclusions drawn from the models. It also highlights the usefulness of early cost-effectiveness modelling embedded within the test development pathway.

A limitation of this analysis is that there was no specific guidance on the best approach to VOI analysis at such an early stage of development to be adopted and how to manage parameters when they are completely unknown and hypothetical. Thus, the approach to VOI analysis adopted in this study may not be the most appropriate, highlighting the need for further work in this field. Another limitation of this study is the completely hypothetical nature of the test. Having some form of data on the new test would have been far better than not having any data.

Conclusion

This evaluation has presented a complete analysis of the potential benefits of the HT-test compared with the Widal testing pathway in each of three different roles. The analysis shows that at certain prices and accuracy pairs for the HT-test, the HT-test has potential to be more effective compared with the Widal test in terms of QALYs gained and cost effectiveness. A high specificity (at least 70%) is a key test characteristic requirement for the HT-test to be able to improve current practice, and the HT-test is likely to improve current practice when used to replace the currently used test. The overall value of conducting research to reduce uncertainty (over 5 years) is US$18,585.

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 115 kb)

This study has wider implications for the early economic evaluation of diagnostic tests by contributing to a limited evidence base and identifying areas requiring further research.

A high specificity (at least 70%) is a key test characteristic requirement for the hypothetical test (HT-test) to be able to improve current practice, and the HT-test is likely to improve current practice when used to replace the current test rather than as triage or add-on.