Data collection methods in prospective economic evaluations: how accurate are the results?

OBJECTIVES: Often in economic evaluations a division is made between those studies that have a high level of accuracy versus those that are easily generalized. This interstudy dichotomy is often translated into prospective, randomized controlled trials with high internal validity and observational and modeling studies with a high level of external validity. This article challenges this conventional view and examines intrastudy effects on validity.
METHOD: A review and summary of the literature was conducted in order to assess the impact that data collection strategies will have on internal validity. Two scenario models were created in order to gain a preliminary understanding of the magnitude of the problem.
RESULTS: Data collection strategies have an impact on the level of internal validity found in an economic evaluation. Comparisons of studies that are prospective in nature is misleading as data collection strategy can lead to different resource and cost estimates even when all other relevant factors are similar. It is possible to shift and improve the level of validity by combining different collection methods.
CONCLUSIONS: Instead of viewing internal and external validity as polar opposites, validity should be considered in terms of a continuum within a particular study. The use of proxies to collect resource utilization estimates, the reliance on patient self-reported data, and the method of collecting this type of data all impact the validity of study results. National guidelines for the economic evaluation of agents and devices should consider this issue in more depth, and existing evidence rankings should be adapted to be more appropriate to pharmacoeconomic studies.