Health technology assessment and cancer imaging: who should be setting the agenda?

Editorial

Health technology assessment (HTA) has become an integral part of health policy generation and is actively supported by many government agencies throughout the world. When performed within the framework of evidence-based medicine (EBM), such evaluations have the potential to guide clinicians and patients regarding the appropriate use of new technologies and as to their safety. EBM is oriented towards optimising health outcomes for individual patients by facilitating better-informed clinical decisions. The improved outcomes that ought to result from effective application of EBM may however come at the cost of increased health care expenditure [1].

Analysis of the relationship between the cost of a procedure and its benefits can be evaluated in several ways but is broadly termed ‘cost-effectiveness analysis’. While clinical efficacy is the prime consideration for the consumer of health services, it is also an essential component of cost-effectiveness that is primarily relevant to the purchaser of health care. The level at which cost-effectiveness is considered to be achieved depends on arbitrary decisions related to the fiscal resources and perspective of the purchaser. When an individual is responsible for their own medical expenses, the cost of a procedure compared to expected outcomes can be fairly balanced. However, when health care is subsidised in part or completely by a third party, societal perspectives come into play and clinical efficacy may be judged to be less important than cost minimisation. This is more likely if fiscal resources are limited. Groups performing HTA reviews of new diagnostic and therapeutic procedures seek to address issues of safety, clinical and cost-effectiveness by employing EBM methodology, as well as providing guidance for resource allocation decisions. The potential conflict between the objectives of quality health care delivery regardless of cost and the rationing of health services based on notional relative values of cost-effectiveness requires that the process of HTA be rigorous, transparent and well understood by those who use this information. Furthermore, it is vital that benchmarks used to judge cost-effectiveness be prospectively defined and consistently applied in health policy. This has become particularly important since the major funding source for HTA agencies has become third-party funding bodies that have a vested interest in limiting health expenditure.

Despite a growing and clinically compelling body of evidence demonstrating the accuracy and impact of PET in oncology [2], recent HTA reviews sponsored by government agencies in Australia (http://www.health.gov.au:80/msac/reports.htm), Canada (http://www.ices.on.ca/) and Scotland (http://www.htbs.org/uk) to evaluate the justification for expansion of public funding of PET have been critical of the quality of evidence currently available. Since these reviews purport to be comprehensive assessments of the PET literature using principles of EBM and, as such, to provide doctors, their patients and those funding PET studies with the best possible advice on the role of PET, they have the potential to dissuade doctors and their patients from using this technology and to negatively influence health policy in other countries considering implementation of this technology. They have already had the direct effect of restricting access to funding of this technology in their countries of origin. My own involvement in the Australian PET review has convinced me that HTA, as currently performed, does not meet even basic standards of scientific rigour, does not deliver highest quality information and does not satisfy the tenets of EBM. Therefore, the reports serve neither the interests of good medical practice nor the laudable EBM goal of achieving better patient outcomes through use of relevant research information. I believe rather that they misrepresent the evidence to justify unpalatable funding decisions. They do this under the guise of exercising prudent caution in the face of inadequate scientific evidence.

Many of the PET experts who were listed as contributors to recent HTA reviews have published extensively both original articles and commentaries that have advocated wider clinical application of this modality and yet these opinions were clearly not reflected in the tone of the final conclusions of the reviews. For example, although I have been a strong advocate for wider clinical application of PET [3], I was a listed member of the Australian PET Review Supporting Committee that published the conclusion that ‘there is insufficient evidence at this time from which to draw definitive conclusions about the clinical effectiveness and cost-effectiveness of FDG PET’. The asynchrony between expert clinical opinion as expressed in the medical literature and its synthesis in HTA reviews has arisen in part from a fundamental difference in how quality of evidence is determined and the rules of engagement for clinical advisors to HTA. Expert opinion is derived from personal experience, focussed research and a critical, but often selective, reading of the peer-reviewed literature combined with assimilation of the opinions of other ‘experts’ as contained in reviews. The process of peer-review employed by medical journals to evaluate the quality of a paper involves a flexible range of criteria depending on the context and target of the communication. EBM constructs consider such a knowledge base to be bedevilled by bias [4] and have proposed hierarchies of evidence based on ‘comprehensive’ reviews of available data with quality filters regarding the acquisition methodology. Whereas the peer-review process is multifactorial including assessment of the clinical significance and relevance of the objectives, whether the study methodology is appropriate to answer the questions posed, and whether the conclusions are justified by the data presented, EBM as practised within HTA judges the quality of evidence on rigidly defined criteria of scientific orthodoxy applied to the data acquisition methods. Accordingly, papers that are influential in guiding clinical practice may not even be considered within HTA if their methodology is deemed to be inadequate, and the reasons for exclusion of the data may not be apparent. For example, papers by my group that have appeared in the Journal of Clinical Oncology [5] and Cancer [6] regarding the role of PET in oncology were not even listed in the ICES systemic review of PET in cancer. Due to the constraints imposed by an editorial of this type, a detailed discussion of the suitability of hierarchies of evidence for diagnostic imaging studies is not possible here bit it is recommended that interested readers refer to a recently published debate [7] regarding the applicability of randomised controlled trials (RCT) to the validation of diagnostic imaging tests since the lack of such RCT evidence was viewed as a major failing of the current data regarding PET in each of the HTAs cited.

The institutionalisation of EBM has seen a significant and well-orchestrated shift of medico-political power away from ‘expert opinion’ to self-appointed evaluators of scientific rigour who control EBM. Increasingly, HTA is being dominated by epidemiologists, statisticians, health economists and bureaucrats who have some knowledge of EBM but little or no clinical and scientific background. Empowered to define the rules of engagement with clinicians conscripted to, or willingly involved with HTA reviews, criticism from clinicians external to the process can be proclaimed ‘unscientific’. However, it is important to recognise that HTA has become an industry that has a strong motivation to please its major employer, the bodies that fund health care, and therefore may not represent the best interests of patients. As advocates for quality health care, it is my fervent belief that clinicians should be actively involved in critically evaluating EBM constructs and in promoting the need for flexibility in scientific methods in order to answer clinically important questions in the most practical way. Systemic reviews performed as part of HTA and that are self-proclaimed to be ‘Level 1’ evidence need to be critically appraised for quality in the same way that all scientific opinion should be open to scrutiny.