Quality of clinical studies present in the package inserts of coagulation factors used in the treatment of hemophilia.

OBJECTIVE: To identify and analyze the quality of scientific evidence from clinical efficacy studies present in the package inserts of coagulation factors, used in the treatment of hemophilia A and B.
METHODS: Documentary study developed in two stages. The first stage consisted of identifying the medicine packages inserts electronically registered in the Brazilian Health Regulatory Agency, and analyzing the availability of the bibliographic references cited therein. This analysis was conducted in the PubMed, SciELO, Google Scholar, and Web of Science databases. The second step was the analysis of the methodological quality of the efficacy studies. Two trained researchers used the Cochrane Collaboration Risk of Bias version 5.1.0 tools for methodological quality analysis, and Review Manager 5.4 software to generate the risk of bias graph.
RESULTS: Of the 17 medicines listed, 7 had referenced package inserts. Of these, 10 studies were eligible for analysis of methodological quality. More than half of the analyzed studies did not control for selection, performance, and detection bias. A total of 100% controlled attrition and reporting biases, and 50% had a high risk of conflict of interest.
CONCLUSION: The biases present are significant and may have influenced the overestimation of the effects of the outcomes of each of the studies.

INTRODUCTION

Hemophilia is a disease characterized by qualitative or quantitative deficiencies of coagulation factors, with factor VIII (hemophilia A) and IX (hemophilia B) deficiencies being the most common.( Patients with hemophilia manifest in the short or long term, hemorrhages into joints, muscles, or internal organs. According to the World Federation of Hemophilia (WFH), Brazil has the third largest population of patients with hemophilia in the world, with about 11,856 records.(

The treatment of hemophilia is based on replacement of the deficient factor to obtain adequate hemostasis.( Treatment can be on demand (infusion of deficient coagulation factor concentrate during bleeding episodes) or prophylactic (regular infusion of factor concentrate at home). This prophylactic option was a great advance in the treatment of hemophilia, and its principles are the speed of infusion of the deficient factor, pain relief, reduction of hemophilic arthropathy, better clinical results and patients’ quality of life.(

For the treatment of hemophilia, factors of plasmatic origin and those of recombinant origin can be used. Factors of plasma origin are derived from human plasma and are currently considered safe due to purification techniques and viral inactivation. Concentrates of recombinant origin are highly purified, and made using molecular biology technology.( One of the complications of hemophilia treatment is that some patients produce antibodies that inhibit the factors used in the treatment (e.g. anti-FVIII or anti-FIX). For these patients, additional therapy is required to overcome the need for factors VIII and IX, generating thrombin through other mechanisms. Some of the options are desmopressin, increase in the usual dose of coagulation factors, recombinant activated factor VII, and complex concentrate activated prothrombin.(

Coagulation factors are expensive pharmaceutical products. In Brazil they are acquired directly by the Ministry of Health, and in 2019 alone, the federal government made available BRL 1.3 billion for the purchase of medicines for the treatment of hereditary hemorrhagic diseases for the Public Health System (SUS - Sistema Único de Saúde).(The costs of a medicine depend on several factors, such as the technology used in the development and production process, as well as its proven efficacy and magnitude of benefits found in phase III clinical trials. Thus, through the evaluation of all documentation made available by pharmaceutical companies to request a medicine registration, regulatory agencies assess whether this new product is effective and safe, and whether the cost-benefit ratio is positive, as it is necessary to rationalize the health system resources.(

After the medicine approval and registration, the industry is still responsible for preparing the package inserts. These are informational materials for patients and health professionals, with easy and quick access, being considered one of the most important sources of information about medications after medical prescription, and for this reason they must be reviewed and inspected by the regulatory agency.(

The gold standard design for measuring intervention effects is the Randomized Clinical Trial, since this research design has strict methodological criteria, and parameters that allow controlling the influence of various factors on the investigated outcome.( However, it is important to highlight that non-randomized, uncontrolled clinical studies, with no blinding, and other systematic errors have increased and are being used as a technical scientific basis for the approval of new treatments.( These methodological flaws, which are called biases, may overestimate the benefits of the tested intervention, which can compromise the reliable analysis of the results and give the industries margins to increase the costs of these products.(

Given the budget constraint of the Ministry of Health of Brazil, combined with the high cost of coagulation factors used in the treatment of hemophilia, it is urgent to conduct research for assessing the quality of scientific evidence of phase III clinical studies available in the package inserts of these medicines, as there are many studies in which the scientific bias has been masked by economic interests.(

OBJECTIVE

To identify and analyze the quality of scientific evidence from clinical efficacy studies present in the package inserts of medicines containing coagulation factors, used in the treatment of hemophilia A and B.

METHODS

This is a documental study developed in two stages:

Identification of the package inserts for health professionals and analysis of the availability of bibliographic references

To identify the references contained in the package inserts of coagulation factors available for distribution by the Ministry of Health, the inserts electronically registered in the Brazilian Health Regulatory Agency (ANVISA - Agência Nacional de Vigilância Sanitária) were consulted on June 20th, 2019 (http://portal.anvisa.gov.br/bulario-eletronico1). Table 1 shows the commercial presentations on the ANVISA website up to the consultation date.

Table 1

Coagulation factors distributed by the Ministry of Health for the treatment of hemophilia

Class	Commercial name
Factor VIII of plasma origin	8Y®; Advate®; Beriate P®; Biostate®; Fhandi®; Hemofil M®; Immunate®; Kogenate®; Octavi SDOptimum®; Optivate®
Factor VIII of recombinant origin	Hemo R8®
Factor VII of recombinant origin	Novoseven®
Factor IX of plasma origin	Factor IX Grifols®; Immunine®; Octanine F®; Replenine VF®
Partially activated prothrombin complex (factors II, IX, X, VII and factor VIII coagulant antigen)	Feiba®

The references described in the item “efficacy results” were analyzed in the package inserts for these medications. After identifying the references, the availability of full articles was analyzed in the PubMed, SciELO, Google Scholar, and Web of Science databases. Articles whose full texts were not available were requested from the industries that wrote the package inserts.

The articles found were categorized according to the methodological design used. Subsequently, all studies categorized as intervention studies were selected to identify their pre-clinical or clinical phase (phase I, II, III or IV).

Analysis of the methodological quality of clinical studies contained in the package inserts

For this stage, only phase III clinical studies were eligible for analysis. All those intervention studies in humans, whose phase was not described in the article, but which aimed to evaluate the efficacy of the products, were also included. Pre-clinical studies, phase I, II and IV clinical studies, review studies, and observational studies were excluded from the analysis.

Individual analysis of the methodological quality of eligible clinical trials was performed using the Cochrane Collaboration 5.1.0 risk and bias tool. This tool consists of seven domains: - random sequence generation; - allocation concealment; - blinding of participants and professionals; - blinding of outcome evaluators; - incomplete outcomes; - selective outcome reporting; and - other sources of bias - for this domain, the conflict of interest in the analyzed studies was evaluated.(

Two trained researchers performed the analysis and subsequently met to discuss the necessary adjustments as for possible disagreements. At the end of the analysis, a graph with the risk of bias was generated using Review Manager 5.4 software. The methodological components of the studies were categorized into low risk of bias, high risk of bias, or unclear risk of bias.(

Thus, the biases evaluated by the tool are: - selection bias (domains 1 and 2): systematic differences between the baseline characteristics of the compared groups; - performance bias (domain 3): systematic differences between groups in the care provided or in exposure to factors other than the interventions of interest; - detection bias (domain 4): systematic differences between groups during outcome analysis; - attrition bias (domain 5): systematic differences between groups in data loss and/or participants; - reporting bias (domain 6): systematic differences between reported and unreported outcomes; - other biases - conflict of interest bias - (domain 7): presence of a conflict of interest of any origin.(

RESULTS

After analyzing the package inserts of the 17 listed medicines, only 7 contained references. Of these, 47 referenced studies were identified. After searching the databases, 28 studies were accessed. Some companies responded and sent 5 studies requested by email (Figure 1). At the end of the searches, 33 complete articles were obtained, which were categorized according to their design (Table 2).

Figure 1

Availability of references cited in the package inserts of coagulation factors used in the treatment of hemophilia

Table 2

Categorization by methodological design of the studies present in the package inserts of each medicine

Medicine	Class	Total	Observational	Review	Intervention study
Advate®	Factor VIII of plasma origin	5	3	--	2
Biostate®	Factor VIII of plasma origin	2	2	--	--
Feiba®	Partially activated prothrombin complex	4	2	--	2
Hemo 8r®	Factor VIII of recombinant origin	1	--	1	--
Hemofil®	Factor VIII of plasma origin	3	--	3	--
Immunate®	Factor VIII of plasma origin	1	--	--	1
Novoseven®	Factor VII of recombinant origin	7	1	1	5
Optivate®	Factor VIII of plasma origin	2	2	--	--
Replenine VF®	Factor IX of recombinant origin	8	7	--	1
Total		33	17	5	11

Clinical studies described as phase III and all those that aimed to assess the efficacy and safety of medications were selected for the second stage of the present study. It is noteworthy that the study by Ewenstein et al.( mentioned in the package insert of the Replenine VF® product, was not eligible for the second stage of the study, as its objective was only to compare pharmacokinetics (Table 3).

Table 3

Profile of intervention studies present in the package inserts of coagulation factors used in the treatment of hemophilia

Medicine	Authors	Phase	Objective	Control Group
Advate®	Tarantino et al.,(22)	ND	Bioequivalence, pharmacokinetic, safety and efficacy	No
Advate®	Négrier et al.,(23)	ND	Efficacy and safety	No
Feiba®	Astermark et al.,(24)	ND	Efficacy	Yes
Feiba®	Sjamsoedin et al.,(25)	ND	Efficacy	Yes
Immunate®	Nemes et al.,(26)	III	Efficacy and safety	No
Novoseven®	Key et al.,(27)	III	Efficacy and safety	No
Novoseven®	Scharrer et al.,(28)	ND	Efficacy and safety	No
Novoseven®	Shapiro et al.,(29)	ND	Efficacy and safety	Yes
Novoseven®	Kavakli et al.,(30)	ND	Efficacy and safety	Yes
Novoseven®	Young et al.,(31)	ND	Efficacy and safety	Yes
Replenine VF®	Ewenstein et al.,(21)	ND	Pharmacokinetics	No

ND: no description.

It is important to highlight that of the 11 intervention studies analyzed, 10 were eligible for the analysis of methodological quality. Of these, 50% (n=5) conducted the study without a Control Group.

More than 50% of the analyzed studies did not control for the selection, performance, and detection bias, as they present a high risk of bias or uncertainties as for methodological quality. When analyzing by domain generation of random sequence (domain 1) and concealment of allocation (domain 2), 50% of the studies presented a high risk of bias for both domains. Regarding the blinding of participants and professionals (domain 3) and blinding of outcome evaluators (domain 4), a high risk of bias in 70% and 60% of the studies, respectively was found. As for the incomplete outcomes (domain 5) and selective outcome reporting (domain 6) domains, 100% of the studies had a low risk of bias. Finally, for domain 7 conflict of interest, 50% of the studies presented a high risk of bias (Figure 2).

Figure 2

Risk of bias summary: analysis of the authors judgments on each risk of bias item for each included study

DISCUSSION

The results of this study showed, in general, that clinical studies aimed at evaluating efficacy and safety referred to in the package inserts of medicines containing coagulation factors for the treatment of hemophilia, have methodological flaws. Among the domains evaluated, those with a high risk of bias in most studies were the blinding domains. Furthermore, in three domains, 50% of the studies were classified as high risk of bias (random sequence generation, allocation concealment, and conflict of interest). These weaknesses can overestimate efficacy effects and compromise the reliable analysis of results.

A large portion of the analyzed studies were not able to control the selection bias. This finding can be explained by the fact that 50% of the analyzed studies did not have a Control Group, and when they did, did not report on the performance and/or methods of randomization and allocation. These two steps are essential in a clinical trial to alleviate selection bias, thus providing baseline similarity, that is, the generation of similar groups in terms of their characteristics and baseline known and unknown risk factors. Similar groups in clinical trials contribute to the effect estimates obtained in the studies, being closer to the true result.( In contrast, it is important to recognize that due to the fact that hemophilia is a low-prevalence disease that directly impacts on patients’ health and quality of life, the use of Control Groups using pure placebo is limited. However, in an attempt to minimize biases, increase statistical power, and at the same time not violate ethical principles in human research, small groups could have been cross-randomized into intervention and usual care groups.( Therefore, it is important to be cautious when interpreting the results of the studies analyzed here, since most of them did not control for selection bias.

Most studies analyzed did not control for performance bias and detection bias. It is important to emphasize that blinding is only possible in studies with Control Groups, as the results of outcomes assessed in single-arm studies are only those of the applied intervention.( Of the five studies that used Control Groups, only two were with blinded outcome evaluators, and three with blinded participants and professionals. According to Buehler et al.,( and Kamper( it is not always possible to apply blinding, possibly because it is difficult to perform blinding in studies that assess the efficacy and safety of coagulation factors for the treatment of hemophilia, as they are injectable preparations. On the other hand, some alternatives could have been used to overcome this barrier, like those used in the study by Sjamsoedin et al.,( such as coding interventions and using dark colored bottles and opaque syringes. The absence of blinding in clinical trials directly implies the estimation of the effect of the evaluated outcome. Schulz et al.( evaluated the impact of methodological quality on the effect estimate and showed that studies that were not double-blind produced effect estimates 17% higher than those that were double-blind. In this context, it is clear that once again, the results reported in the studies contained in the coagulation factors package inserts may not be as encouraging, as the reported effect estimates may also have been influenced by the performance and detection bias.

Furthermore, the present study showed that all the analyzed studies controlled the attrition bias. A well-conducted study depends on adequate patient follow-up and the preservation of collected data. Data loss can affect the veracity of the effects of each outcome, and unexplained loss of patients can hide a possible failure in the protocol or even a safety issue in the treatment.( Most of the studies analyzed here performed adequate follow-up and all losses and exclusions were duly justified. It is noteworthy, therefore, that the attrition bias was controlled and, although the size of the treatment effect is questionable due to the presence of selection, performance, and detection biases, it can be judged with more confidence, that the evaluated treatments were safe for most of the analyzed studies.

Moreover, a positive point of the analyzed studies is that all of them controlled the reporting bias, as they presented a low risk of bias in the selective outcome report domain. Controlling reporting bias is essential so that there are no deviations in study protocols, such as the inversion of clinical outcomes (primary) by substitute (secondary) outcomes. This can occur intentionally, when, e.g., the authors assess multiple outcomes but report only those that had positive effects, leading to erroneous conclusions as for the efficacy of the studied intervention.( Therefore, it is notable that the authors reported the proposed primary and secondary outcomes, thus evidencing results that are of interest to the perspectives of health professionals and patients.

Furthermore, this research evidenced that half of the studies showed a conflict of interest for financing from companies interested in the approval of the medicine. Palma et al.,( bring in their study a reflection that the interest of the pharmaceutical industry in research, production, dissemination, and commercialization of medicines is a reality all over the world. A study conducted by Santos et al.( explained that the conflict of interest was present in most of the studies analyzed and that funding by the pharmaceutical industry was associated with conclusions favorable to the treatment being tested. That said, it is crucial to critically analyze the results of clinical trials where industry funding is available, as this study design is largely dependent on rigorous methods to control bias, and any deviation, whether intentional or not, can influence in an increase in the estimated effect of the evaluated outcome.

Before requesting the registration of a drug, the pharmaceutical industry must comply with current legislation and present to regulatory agencies documentation containing pre-clinical and clinical studies to prove efficacy and safety, in addition to administrative technical issues, such as certificates of good manufacturing practices. It is also responsibility of the industry to prepare the package insert.( The Collegiate Board of Directors Resolution (RDC) # 47 of 2009( regulates, standardizes, and assists in the preparation and updating of information on medicine package inserts in the country. This resolution provides that the reference, generic and similar package inserts, intended to health professionals, must describe the results of the studies used to prove the efficacy and registration of the drug, as well as bibliographical references of the cited studies.

This study brings worrying data as for the quality of technical information present in the package inserts for coagulation factors, showing that 10 of the 17 package inserts selected for health professionals do not refer to the described efficacy studies. The RDC # 47 of 2009,( which standardizes and assists in the preparation and updating of information on medicine package inserts for patients and health professionals, establishes that the described efficacy results must cite the bibliographic references used. Marques et al.( reinforce that in order not to negatively compromise the evidence-based clinical practice, a greater commitment of the industries in the adaptation to the current legislation for the production of package inserts, and greater rigor in the approval and inspection of these by the sanitary regulatory agencies is necessary. In this context, ANVISA needs to review and update the package inserts of products containing coagulation factors for the treatment of hemophilia in Brazil, so that health professionals have available quality and updated information, to enable them to seek the primary sources of the cited scientific evidence, allowing them to make a critical analysis of each of the studies used for the approval and use of the medicine.

This study has some limitations, such as the impossibility of accessing articles that are not freely accessible. However, to the best of our knowledge, this study brought important reflections on the quality of scientific evidence used to prove the efficacy and registration of medicines, technical quality in the production, updating and inspection of package inserts, and the possible influence of the pharmaceutical industry in this context. Therefore, this should be a topic for discussion and investigation by all actors involved in this process, including researchers, the pharmaceutical industry, the regulatory agency, health managers, and professionals.

Finally, it is important to emphasize that the authors are aware that the medicines approved for the treatment of hemophilia were a revolution in improving the patients’ quality of life and health, and for this reason the objective here was not to question innovative studies and proven effective pharmacological treatments, but rather to identify and explain the methodological gaps existing in the studies that support the approval for the use and commercialization of medicines.

CONCLUSION

The present study showed that the quality of scientific evidence available on the efficacy of medicines containing coagulation factors used in the treatment of hemophilia needs to be carefully analyzed and interpreted, as the biases present are significant from a methodological and clinical point of view. The results of this study suggest that the high frequencies of bias in these studies may have influenced the overestimation of outcome results in each of them. Furthermore, the available package inserts should be revised to adapt to current standards, with the aim of enhancing and improving the information intended for health professionals.