Electronic Health Data for Postmarket Surveillance: A Vision Not Realized.

What has been learned about electronic health data as a primary data source for regulatory decisions regarding the harms of drugs? Observational studies with electronic health data for postmarket risk assessment can now be conducted in Europe and the US in patient populations numbering in the tens of millions compared with a few hundred patients in a typical clinical trial. With standard protocols, results can be obtained in a few months; however, extensive research published by scores of investigators has illuminated the many obstacles that prevent obtaining robust, reproducible results that are reliable enough to be a primary source for drug safety decisions involving the health and safety of millions of patients. The most widely used terminology for coding patient interactions with medical providers for payment has proved ill-suited to identifying the adverse effects of drugs. Directly conflicting results were reported in otherwise similar patient health databases, even using identical event definitions and research methods. Evaluation of some accepted statistical methods revealed systematic bias, while others appeared to be unreliable. When electronic health data studies detected no drug risk, there were no robust and accepted standards to judge whether the drug was unlikely to cause the adverse effect or whether the study was incapable of detecting it. Substantial investment and careful thinking is needed to improve the reliability of risk assessments based on electronic health data, and current limitations need to be fully understood.