Have Reforms Reconciled Health Rights Litigation and Priority Setting in Costa Rica?

The experience of Costa Rica highlights the potential for conflicts between the right to health and fair priority setting. For example, one study found that most favorable rulings by the Costa Rican constitutional court concerning claims for medications under the right to health were either for experimental treatments or for medicines that should have low priority based on health gain per unit of expenditure and severity of disease.32 In order to better align rulings with priority setting criteria, in 2014, the court initiated a reform in its assessment of claims for medicine. This paper assesses this reform's impact on the fairness of resource allocation. It finds three apparent effects: (1) a reduction in successful claims for experimental medication, which is beneficial; (2) an increase in the success rate of medication lawsuits, which is detrimental because most claims are for extremely cost-ineffective medications; and (3) a decline in the number of claims for medicine, which is beneficial because it forestalls such low-priority spending. This paper estimates that, taking all three effects into account, the reform has had a modest net positive impact on overall resource allocation. However, it also argues that there is a need for further reforms to lower the number of claims to low-priority medicines that are granted.

Introduction

The right to health is enshrined in supranational covenants and in more than 100 national constitutions. To satisfy its demands, governments must allocate substantial resources toward meeting people’s health needs. However, the right to health does not require that each person receive all necessary medical treatment, no matter what the cost. Instead, when, due to resource constraints, it is not possible to meet everyone’s needs, it requires that decisions about whose health needs should be met are made fairly and accountably. While there is a range of reasonable opinion on how to specify criteria for fair priority setting in detail, there is broad agreement that these should track the proper aims of the health sector, which are to increase population health and reduce inequalities in access to health services and in health-related quality of life.

In many Latin American countries, it has proven challenging to establish a process for judicial decision making on claims under the right to health that is fully consistent with this imperative to set priorities fairly and accountably. Costa Rica provides a case in point. While Costa Rica’s Constitution does not contain an explicit right to health, the Constitutional Chamber of the Supreme Court (known as Sala IV) has derived a right to health from the right to human life (art. 21), the right to social security protection (art. 73), and international human rights treaties. This right entitles citizens to public and preventative health services as well as medical care.

In a landmark case in 1997, the court ruled in favor of the provision of antiretroviral therapy for people living with HIV/AIDS. This ruling led to an explosion of health rights litigation. The decision regarding antiretroviral therapy has been cited as an example of how litigation can redress discriminatory decisions. Furthermore, later evaluation of the intervention has suggested it was consistent with fair priority setting. However, subsequent cases have highlighted the potential for conflict between court decisions and reasonable priority setting. For example, in 2001, Vera Salazar Navarro challenged the Costa Rican social security institution (Caja Costarricense de Seguro Social, referred to as CCSS) for refusing to cover the branded medication she was prescribed for her multiple sclerosis. (The CCSS had offered to reimburse a less expensive generic alternative instead.) The court ruled in favor of Navarro, arguing that the CCSS had breached the patient’s right to health by refusing to pay for the exact drug prescribed. In this case, the court’s ruling undermined effective and fair priority setting, since its decision allocated substantial resources to a use with virtually no incremental benefit. In another controversial case in 2003, the court obliged the CCSS to pay for the treatment of Gaucher disease for a young girl at an annual cost of US$160,000—equivalent at the time to 38 times the GDP per capita—for the remainder of the patient’s life, overruling the CCSS’s medical experts, who had judged that the cost was far out of proportion to the benefits. In 2007, the court even appeared to declare explicit priority setting illegal by arguing that the CCSS could not decline treatment for “eminently economic reasons.”

Cases such as these are not incidental. In an important study in this journal, Ole Frithjof Norheim and Bruce Wilson classified a random sample of medication-related lawsuits filed in Costa Rica in 2008; the authors used common priority setting criteria, including size of the predicted health gain, cost-effectiveness, size of the disease burden, and the quality of evidence available for how a medication performed on these criteria. They found that 73% of successful claims were for experimental or otherwise low-priority medications. This result is concerning, since it is estimated that 9% of all CCSS spending on medicines is driven by court orders.

These findings lent support to complaints by the CCSS that the court’s health rights decisions were harming its capacity to manage scarce resources fairly because magistrates lacked the knowledge necessary to evaluate medications. In response to this criticism, in 2014, with support from the World Bank, the court initiated a joint program with the Cochrane Collaboration (a UK-based nonprofit organization of medical experts who cooperate to produce credible and accessible health information). Before the reform, magistrates would typically accept the evidence of the treating physician. After the reform, the court can (and in nearly three-quarters of cases does) refer the case to one of ten forensic clinics around the country, where a doctor investigates the patient’s medical records, performs a physical examination, and evaluates the appropriateness of the prescribed medication using the Cochrane medical databases. (These investigations are done free of charge to the claimant. This process has made the court more reliant on evidence-based medicine.) But it is important to note that the Cochrane reviews only summarize the quality of evidence for the effectiveness of a medication; they do not include evidence on cost-effectiveness or severity of disease.

Another recent study in this journal—published by Olman Rodríguez Loaiza, Sigrid Morales Carrasco, Norheim, and Wilson—evaluates the impact of this reform. They compare claims for medication post-reform (from 2016) with Norheim and Wilson’s pre-reform sample (from 2008). They find that this reform has had two effects: (1) among successful claims, a shift from experimental to higher-priority medications and (2) an increase in the success rate of medication lawsuits. They tentatively conclude that the new process has led to modest gains in fairness.

The data these authors have generated is of great value, especially given the paucity of information on the impacts of ways of institutionalizing the right to health in Latin America. However, their analysis has three shortcomings. First, they do not establish the statistical significance of the differences they observe. Second, they miss an important potential effect of the reform, which is that it may have changed the number of medication cases brought to the court. Third, they do not provide a detailed analysis of the joint impact of the possible effects of the reform. Here, we address these shortcomings. We first analyze Rodríguez Loaiza et al.’s data and show that the two effects they identify are indeed statistically significant. We also demonstrate that the number of medication cases brought to court declined significantly post-reform. We then evaluate the joint impact of these potential effects on the assumption (which we argue is plausible) that the CCSS focuses its non-court-mandated spending on high-priority interventions. We show that if one considers only the change in the success rate and the reduction in approved claims to experimental (unproven) medications, then, contrary to Rodríguez Loaiza et al.’s judgment, post-reform health resource allocation is likely to be less fair. (This is primarily because, even post-reform, low-priority medications make up the majority of successful claims. Consequently, the increase in the success rate of claims generates a net increase in expenditure on low-priority drugs.) Furthermore, we show that if one also considers the reduction in the number of medication claims after the reform, then it is plausible that overall resource allocation is fairer post-reform. We end by suggesting that to reduce the tension between judicial decision making on individual claims and fair priority setting, the process of evaluating claims should take account of cost-effectiveness and individual disease burden.

Data analysis

In 2008, pre-reform, the Sala IV dealt with 192 claims for medications. Norheim and Wilson randomly selected 37 of the winning cases and classified them according to the effectiveness of the medicine, the severity of the condition it addresses, its cost-effectiveness, and the quality of the evidence available. In 2016, post-reform, the Sala IV dealt with 128 claims for medications. Rodríguez Loaiza et al. analyzed the entire population of successful cases using the same criteria, which were operationalized in the manner explained in Table 1.

Table 1.

Rodríguez Loaiza et al.’s criteria for priority classification

Criterion	Measure	Grading
Effectiveness	QALY gain#	I > 1 QALYII < 1 & > 0.5 QALYIII < 0.5 QALY
Severity of disease	QALY loss##	I > 5 QALY lossII > 1 QALY loss < 5 QALY lossIII < 0.5 QALY loss
Cost-effectiveness	Cost per QALY gained	I < 1 GDP per capitaII > 1 GDP per capita < 3 GDP per capitaIII > 3 GDP per capita
Quality of evidence	Types of published evidence	I Meta-analysis or randomized trialII Observational, non-comparative studiesIII Single case reports

Note: QALYs are quality-adjusted life years, a measure of health-related quality of life, in which one year in perfect health (or its equivalent) is 1, death is 0, and a year in a condition that impairs quality of life without rendering it not worth living is rated between 0 and 1, depending on the severity of impairment.

# Compared to standard intervention

## Compared to normal healthy life expectancy

Rodríguez Loaiza et al. draw on these criteria to create an overall priority ranking into four classes: I (high priority); II (medium priority); III (low priority); and IV (experimental). They do so as follows. A medicine qualifies as high priority if and only if it scores in category I on all four criteria. It qualifies as medium priority if and only if it scores in category II on at least one criterion and in category I or II on all other criteria. It qualifies as low priority if and only if it scores in category III on at least one criterion and in category I, II, or III on all other criteria. Drugs are classified as experimental if there is insufficient evidence of effectiveness as judged by a trustworthy health technology agency.

Table 2 reports Rodríguez Loaiza et al.’s findings regarding the distribution of successful cases across priority classes. (We remove from their post-reform data the five cases they could not classify, leaving 93 observations.) This explains why the percentages in Table 2 differ slightly from the ones they report. We tested whether the distribution of cases across priority classes is different before and after the reform, using a chi-square test of independence, with the null hypothesis being that the two distributions are the same and the alternative hypothesis that they are not. The p-value of this test is 0.046. Accordingly, we can reject the null hypothesis with a reasonable degree of confidence. (Although we use a chi-square statistic, a Fischer’s exact test might be appropriate instead, since if the pre- and post-reform samples are drawn from the same population, the expected frequencies in two cells are marginally smaller than 5. Fortunately, the choice of test does not affect our conclusion, as the exact test gives a p-value of <0.01.) The final row reports the contribution to this finding made by the shift in the share within each priority class. It reveals that the overwhelming majority of the shift is driven by the change in the share of successful cases in priority classes I (high priority) and IV (experimental). These findings support the claims made by Rodríguez Loaiza et al. about the distributional shift that has coincided with the reform.

Table 2.

Distribution of successful cases across priority classes

	Share of cases in priority class (%)				p-value
	I	II	III	IV
Pre-reform (2008, N = 37)	2.7	27.0	48.6	21.6	0.046
Post-reform (2016, N = 93)	16.1	18.3	55.9	9.7
Contribution to chi-square statistic (%)	48.5	12.2	3.2	36.1

Table 3 reports their findings regarding the success rate of all claims for medicines, supplemented by the results of our analysis. We tested Rodríguez Loaiza et al.’s claim that there was an increase in the success rate of medication lawsuits using a two-proportions Z-test, with the null hypothesis that the pre-reform litigation success rate is greater than or equal to the post-reform success rate. The p-value of this test is 0.0004. (Some might be concerned about the sample sizes not being very large, and argue instead for the use of a Fisher exact test. Reassuringly, the Fisher exact test gives almost identical p-values.) Accordingly, we reject the null hypothesis and conclude that the post-reform success rate is indeed higher.

Table 3.

Success rate of litigation for the provision of medicine

Period	Number of cases	Success rate	p-value
Pre-reform (2008)	192	57.9%	0.0004
Post-reform (2016)	128	76.6%

We now consider a further difference between pre- and post-reform data that is not discussed by Rodríguez Loaiza et al.: that the reform, by typically adding a stage to evidence gathering—a forensic doctor’s report—might have affected the number (and share) of medication claims. Even though such a report comes at no direct financial cost to claimants, it involves further time and effort, including being examined by a new doctor. These represent burdens and therefore possible deterrents. Moreover, the existence of this new process may have made citizens less likely to advance claims for medications of unproven effectiveness. These factors would lead one to expect a decline in claims for medication. As Table 4 reveals, this is indeed what happened. It reports the results of a two-proportions Z-test, with the null hypothesis being that the pre-reform share of medication claims is smaller than or equal to the post-reform share. The probability (p) of finding the observed proportions under this hypothesis is 0.0005. We conclude that the post-reform claims to medication are a significantly lower proportion of all cases.

Table 4.

Litigation for the provision of medicine

Period	Amparo cases (for protection of constitutional rights)	Claims for medication	Share of medication cases	p-value
Pre-reform (2008)	16,345	192	1.2%	0.0005
Post-reform (2016)	15,782	128	0.8%

Source: Sala Constitutional de Costa Rica, Consolidado de datos generales: Estadísticas de asuntos votados por la Sala Constitucional, desglosada por año, tipo de asunto, término y tema. The 2008 data was found at https://www.poder-judicial.go.cr/salaconstitucional/index.php/2016-06-27-17-08-39/item/64-4-cantidad-de-asuntos-entrados-por-tipo-de-asuntos-ampa [accessed July 22, 2019, but the linked data was later removed; the 2008 number corresponds to 2008 data mentioned in Norheim and Wilson (see reference 1)]. The 2016 data is from https://salaconstitucional.poder-judicial.go.cr/index.php/estadisticasv1.

Effects of the reform on resource allocation

We have found three significant differences between the pre- and post-reform situation. Of course, these differences may have causes other than the reform. But if we assume that they were due to the reform, how should we judge its impact? In this section, we answer this question. We proceed step by step, starting with the impact of the change in the distribution of accepted claims in isolation and then adding the increase in the success rate and the decrease in the number of cases brought.

If we focus on accepted claims alone, it is straightforward to see that the shift away from purely experimental drugs (class IV) documented in Table 2 leads to fairer resource allocation. One way of establishing this is to consider the cumulative distribution of accepted claims across priority classes, which is given in the “accepted claims” section of Table 5. The post-reform situation constitutes an improvement over the pre-reform situation if the post-reform cumulative distribution over priority classes ordered from I through IV is, at every point, at least as great as the pre-reform cumulative distribution, and strictly greater at some point. Table 5 reveals that the post-reform distribution is indeed an improvement in this sense.

Table 5.

Proportion of claims leading to spending in each priority class (%)

	Deterred	Rejected	Accepted
Priority class	I#	I#	I	II	III	IV
Accepted
Pre-reform			2.7	27.0	48.6	21.6
Post-reform			16.1	18.3	55.9	9.7
Cumulative pre-reform			2.7	29.7	78.4	100
Cumulative post-reform			16.1	34.4	90.3	100
Rejected and accepted
Pre-reform		42.1	1.6	15.6	28.1	12.5
Post-reform		23.4	12.4	14.0	42.8	7.4
Cumulative pre-reform		43.7		59.3	87.4	100
Cumulative post-reform		35.8		49.8	92.6	100
Deterred, rejected, and accepted
Pre-reform	0.0	42.1	1.6	15.6	28.1	12.5
Post-reform	31.0	16.2	8.5	9.7	29.6	5.1
Cumulative pre-reform	43.7			59.3	87.4	100
Cumulative post-reform	55.6			65.3	94.9	100

# By assumption

Because fairness depends on multiple criteria (including health gain and reduction of inequality), it is more difficult to establish just how much of an improvement the post-reform distribution of accepted claims is. For simplicity, we focus on generating a rough estimate of the reform’s impact on the health gain criterion.

The idea is to estimate how much total health gain is purchased for a given quantity of resources that are spent in line with the pre-reform and post-reform distributions of accepted claims. To this end, we require an estimate of the gain in QALYs for each successful claim in a priority class. We used the average individual QALY gain from the data provided in Norheim and Wilson and in Rodríguez Loaiza et al., filling in some lacunae in their data. Furthermore, we required an estimate of cost per QALY gained in each priority class. We again took the averages in the data provided by Norheim and Wilson and Rodríguez Loaiza et al. (All cost-effectiveness measures were expressed in GDP per capita per QALY for the relevant base year.) Next, we required an estimate of the share of expenditure devoted to each priority class. We assumed that for each successful claim in a class, the expenditure is the estimated individual QALY gain in that class (the “quantity of health purchased”) multiplied by the indicative cost per QALY in that class (the “purchase price”). Finally, we assumed that expenditure in each class is transformed into QALYs by dividing the volume of expenditure by the indicative cost per QALY for that class.

The upshot is reported in the top two rows and the “accepted claims” section of Table 6. (The calculations underlying it are in our online appendix.) It is noteworthy that the indicative cost per QALY for class III (low priority) is very high, in excess of 8 times GDP per capita per QALY, and that for class IV, it is extremely high, in excess of 32 times GDP per capita per QALY. This implies that expenditure on these medications is many times less cost-effective than expenditure on high-priority interventions. It is also noteworthy that, post-reform, the estimated expenditure shares shift principally from class IV (experimental) toward class III (low priority). To estimate the effect of this shift, we calculate the total number of QALYs generated for a given amount of expenditure on accepted claims pre- and post-reform, taking 100 times GDP per capita as an illustrative amount. The blue highlighted cells in the right-hand side of the “QALY gain” rows give the upshot: there is a marked, roughly 50% improvement in the number of QALYs gained per unit of expenditure on accepted claims. Equivalently, one can say that the estimated cost-effectiveness of expenditure on accepted claims falls from close to 9 times GDP per capita per QALY to close to 6 times GDP per capita per QALY. In this respect, the reform is a success.

Table 6.

Indicators of cost-effectiveness, individual gain, expenditure, and overall QALY gain for claim-related expenditure

	Deterred	Rejected	Accepted
Priority class	I	I	I	II	III	IV	Total
GDP per capita per QALY	0.5#	0.5#	0.3##	2.3##	8.2##	32.6##
QALY gain per individual			2.4##	1.6##	1.1##	0.7##
Accepted
Expenditure share pre-reform (%)			0.2	9.1	41.8	49.0	100
Expenditure share post-reform (%)			1.5	7.9	62.2	28.4	100
QALY gain per 100 GDP per capita, pre-reform			0.6	4.0	5.1	1.5	11.2
QALY gain per 100 GDP per capita, post-reform			4.8	3.5	7.6	0.9	16.8
Rejected and accepted
Expenditure share pre-reform (%)		42.1	0.1	5.3	24.2	28.4	100
Expenditure share post-reform (%)		23.4	1.1	6.0	48.1	21.4	100
QALY gain per 100 GDP per capita, pre-reform		84.2	0.4	2.3	2.9	0.9	90.7
QALY gain per 100 GDP per capita, post-reform		46.8	3.7	2.7	5.8	0.7	59.6
Deterred, rejected and accepted
Expenditure share pre-reform (%)	0.0	42.1	0.1	5.2	24.2	28.4	100
Expenditure share post-reform (%)	31.0	16.2	0.8	4.2	32.9	15.0	100
QALY gain per 100 GDP per capita, pre-reform	0.0	84.2	0.4	2.3	2.9	0.9	90.7
QALY gain per 100 GDP per capita, post-reform	61.9	32.3	2.5	1.9	4.0	0.5	103.1

# Mid-point of range for priority class specified by Norheim and Wilson

## Averages drawn from data supplied by Norheim and Wilson and Rodríguez Loaiza et al. and further research

However, this result of course fails to consider the impact of the increase in the acceptance rate of claims. Even post-reform, low-priority and experimental medications together make up nearly two-thirds of successful claims and, by our estimate, more than nine-tenths of the expenditure on approved claims. Consequently, the increase in the success rate of claims will lead to a corresponding increase in expenditure on low-priority and experimental medications, which may counteract the salutary effects of the reduction in the share of experimental treatments among approved claims. Further analysis is required to establish the net effect of these opposing forces.

Such analysis must rely on an assumption about the use to which the CCSS puts those resources that it is not compelled to spend on meeting successful claims. While, as in every health system, there is in Costa Rica scope for improvements in efficiency, there is also reason to assume that the CCSS generally and effectively directs resources toward improving population health and reducing health-related inequalities. Expert reviews of the health system, which is to a large extent administered by the CCSS, have noted the attainment of near-universal health coverage for an extensive package of services. They also emphasize Costa Rica’s excellent population health indicators (such as life expectancy and maternal mortality) in comparison with countries with a similar GDP per capita. Moreover, these reviews document the health system’s focus on primary care, which typically provides high-priority interventions that reach all parts of the population. Public sector health resource allocation appears to be substantially pro-poor. For example, the poorest make the greatest use of public health services, and 30% of government health spending goes to the poorest 20% of the population. This likely contributes to Costa Rica’s comparatively low inequality in lifetime health. More specifically in relation to medicines, the CCSS’s Official Medicines List is drawn up by expert doctors and pharmacists in line with reasonable criteria, including efficacy, safety, the ratio of costs to benefits, impact on the financial sustainability of the system, and the ability to ensure that all segments of the population have access to these medicines and can be expected to use them as prescribed. This list has been held up by the World Health Organization as an example of the optimal use of scarce resources.

A recent study on the cost-effectiveness of marginal health expenditure by governments around the world supports the idea that it qualifies as high priority on at least one central criterion. It estimates that the cost-effectiveness of marginal health spending in middle- and high-income countries (of which Costa Rica is a member) lies between 0.18 and 0.71 of per capita GDP per QALY. These assessments fall well within Rodríguez Loaiza et al.’s <1 of per capita GDP per QALY requirement for high-priority interventions.

These findings are consistent with an assumption that the CCSS focuses on interventions that fall within the high-priority category. On this assumption, and supposing that court orders do not lead to new resources being added to the CCSS budget, resources that the CCSS remains at liberty to allocate because a person’s claim is rejected flow to high-priority interventions, while court-mandated spending on anything other than high-priority medications displaces high-priority spending. Some evidence of such displacement is provided by interviews with senior health officials in Costa Rica, who mention that health providers need to reallocate resources to accommodate the court’s rulings. Further evidence for the likelihood of such displacement comes from the CCSS’s stated aim to keep spending on medicines to 8–10% of the health budget. This suggests that court-mandated spending may well impair the provision of drugs on the Official Medicines List, or the expansion of this list to include new high-priority medications.

We shall therefore proceed under the assumption that ordinary CCSS spending is on high-priority interventions. For concreteness, we shall assume that expenditure on such interventions, on average, comes in at 0.5 GDP per capita per QALY (this is the mid-point of priority class I, and a reasonable estimate given the aforementioned study on the cost per QALY of marginal government expenditure on health in middle- and high-income countries; our qualitative results are, fortunately, robust to changing this assumption to a cost-effectiveness of general spending of slightly less than 3 GDP per capita). We can then, again, use our two tests to compare the fairness of pre- and post-reform claim-related expenditure. Let us start with the claim-related distributions by priority class, which are reported in the “rejected and accepted claims” section of Table 5. Clearly, the post-reform distribution does not constitute an unambiguous improvement. Indeed, the distribution associated with the pre-reform policy is superior for priority classes I and II, and only somewhat worse after that. Our first test is therefore inconclusive.

Our second test is a comparison of the benefits generated per unit of claim-related expenditure, which is reported in the “rejected and accepted claims” section of Table 6. The way to interpret it is as follows. Suppose that one sets aside 100 GDP per capita to deal with claims. If a claim is rejected, money flows toward unconstrained high-priority expenditure. If a claim is accepted, money is spent in line with the formula outlined above for accepted claims in each priority class (in proportion to both the individual QALY gain per claim and the claim’s indicative cost per QALY). Because unconstrained spending is, according to our indicators, far more cost-effective than expenditure on accepted claims, the detrimental effect of the increase in the acceptance rate swamps the beneficial effect of the reduction in expenditure on experimental treatments. As revealed in the orange-highlighted cells, the upshot is a fall in QALYs generated by around one-third. We conclude that if one focuses solely on the success rate effect and the shift away from experimental treatments, then, contrary to Rodríguez Loaiza et al., on one important criterion for fair priority setting, the reform is likely to have worsened the situation.

This negative conclusion regarding the potential effects of the reform depends, however, on ignoring the reform’s apparent deterrent effect on the number of claims for medication. As discussed in Table 4, compared to the pre-reform sample year, the post-reform sample year contains a reduction of around one-third in the share (and number) of writs of protection for medicines. If this reduction is due to the increased barriers to getting such a claim approved, then one must also consider the funds that the CCSS can now spend freely due to this drop. On this assumption, we again review our two indicators for the effects of the reform on resource allocation. The “deterred, rejected, and accepted claims” section of Table 5 shows that, taking deterred claims into account, the post-reform cumulative distribution over priority classes ordered from I through IV dominates the pre-reform distribution, meaning that on this criterion, the reform represents an improvement. Our indicator for QALYs generated per expenditure of 100 times GDP per capita “set aside” for dealing with possible claims tells the same story. It is given in the green-highlighted cells in the “deterred, rejected and accepted claims” section of Table 6. These reveal that when one takes account of all three apparent effects of the reform, the reform has had a net positive effect on the criterion we have focused on, increasing the estimated number of QALYs generated for this fixed, claim-related budget by 14%. In sum, our analysis estimates that the combined effect of (1) the shift away from granted claims to experimental medicines, (2) the increase in the acceptance rate of claims to medicines, and (3) the reduction in the number of claims for medication is a moderate improvement in the fairness of overall health resource allocation. Importantly, because claims are overwhelmingly for low-priority medications even post-reform, a principal route to this likely beneficial impact is the reform’s apparent deterrent effect on claims.

Limitations

While we address several lacunae in Rodríguez Loaiza et al.’s analysis, our arguments also inherit some key limitations of their data. One is that we use their priority classification, which, as they note, is only one reasonable way to classify cases. Another is that the estimates of individual QALY gain and cost-effectiveness of medicines are subject to a great deal of uncertainty. These estimates depend crucially on both the assumed comparison treatment (e.g. is the comparison “no treatment at all,” or is it some alternative medication?) and the country and health system context (e.g. the estimated cost per QALY of a treatment can be much higher in, say, a private health provider in the United States than in a public provider in a middle-income country). But due to the limitations of the available literature, the studies from which our estimates are derived cannot always match the Costa Rican context. Moreover, the cost-effectiveness estimates of experimental treatments are subject to great uncertainty, because the effects of these medicines are highly uncertain. In addition, in future work on this topic based on new data, it would be better to replace our indirect estimates of the expenditure occasioned by each accepted claim with more direct observations.

A third limitation is that while we can establish statistically significant and important post- and pre-reform differences, our methods cannot establish that these differences are caused by the reform. An important avenue for further research would be to investigate the causal mechanisms behind the observed changes.

A fourth limitation is that our analysis of the joint impact of these differences relies on many assumptions and that our estimates are therefore quite uncertain. Most prominent among them is the assumption (for which we have offered indirect empirical support) that money not spent on meeting claims is spent by the CCSS on high-priority health interventions with an indicative cost of 0.5 GDP per capita per QALY. Fortunately, as mentioned above, our qualitative findings are unaffected even if we make the far weaker assumption that the cost-effectiveness of CCSS general expenditure falls just about anywhere in the high- to medium-priority range.

Finally, our quantitative analysis does not consider a different aspect of fair priority setting, which is that there must be room for effective challenge to and revision of decisions. It therefore ignores the salutary effects that a culture of frequent challenge may have on the CCSS’s functioning and the legitimacy of its priority-setting process. While the volume of writs of protection for health remains substantial, it is worth examining whether the new process has created undue barriers for some patients to challenge the denial of medication by the CCSS. We also note that the estimated net effect may not be permanent. If the deterrent effect on claims of the reforms was in part due to uncertainty among claimants about their chances of success under the new regime, then the number of cases might start rising again once lawyers and potential claimants notice the good post-reform prospects of success for claims to medication of proven effectiveness.

Conclusion

We have investigated whether the cooperation between the Costa Rican constitutional court and the Cochrane Collaboration has helped reconcile health rights litigation and fair priority setting. Under this reform, a person advancing a claim for medication is typically referred to an independent forensic doctor, who evaluates the appropriateness of the prescribed medication using the Cochrane medical databases. We performed a novel analysis of data provided by Rodríguez Loaiza et al. on the apparent effects of this reform. We found that the reform coincided with three substantial and statistically significant changes: (1) among successful cases, an increase in the proportion of high-priority cases and a decrease in the proportion of experimental cases; (2) an increase in the overall success rate of cases; and (3) a decrease in the number of claims for medicine. We have also analyzed these changes’ joint impact. Under the assumption that the funds that are not spent on meeting court-approved claims are spent by the Costa Rican public sector on high-priority interventions, these three changes work in opposing directions. The reduction in successful claims to experimental medications represents an important improvement. Indeed, we estimate that it reduces the average cost per QALY for accepted claims by around one-third, from roughly 9 to roughly 6 times GDP per capita per QALY. However, as these numbers indicate, even post-reform, approved claims typically lead to extremely cost-ineffective spending. It follows that the increase in the acceptance rate of claims has a strongly negative effect on overall resource allocation, since it prompts more such low-priority spending. But for the same reason, the substantial drop in the number of claims to medication coinciding with the reform has been beneficial, since it permits public spending to flow toward high-priority interventions. We estimate that if ordinary public spending on health is indeed on such high-priority interventions, then the joint effect of all three changes is a moderate improvement in the fairness of overall health spending.

We conclude that the Cochrane Collaboration reform appears to have reduced the number of successful claims for unproven drugs and therefore probably represents a good first step toward reconciling fair priority setting with the right to health in Costa Rica. However, a large majority of successful claims are still for drugs that are not remotely cost-effective. It therefore seems advisable to further change the process of evaluation of such claims to take account of priority-setting criteria besides mere evidence of effectiveness, including cost-effectiveness and severity of disease.

Acknowledgments

Work on this paper was supported through the Bergen Centre for Ethics and Priority Setting’s research project “Decision Support for Universal Health Coverage,” funded by NORAD grant RAF-18/0009. It was presented at George Mason University and the London School of Economics. We thank Ole Frithjof Norheim for sharing his and his co-authors’ data, and Sigrid Morales Carrasco, Olman Rodríguez Loaiza, Arnaldur Stefansson, Bastian Steuwer, Daniel Wang, Bruce Wilson, two anonymous referees, the editors of this journal, and seminar participants for comments.