Is viscoelastic coagulation monitoring with ROTEM or TEG validated?

Recent years have seen increasing worldwide interest in the use of viscoelastic coagulation monitoring tests, performed using devices such as ROTEM and TEG. The use of such tests to guide haemostatic therapy may help reduce transfusion of allogeneic blood products in bleeding patients and is supported in European guidelines for managing trauma and severe perioperative bleeding. In addition, viscoelastic tests form the basis of numerous published treatment algorithms. However, some publications have stated that viscoelastic tests are not validated. A specific definition of the term validation is lacking and regulatory requirements of the US Food and Drug Administration (FDA) and European Medicines Agency (EMA) have been fulfilled by ROTEM and TEG assays. Viscoelastic tests have been used in pivotal clinical trials, and they are approved for use in most of the world's countries. Provided that locally approved indications are adhered to, the regulatory framework for clinicians to use viscoelastic tests in routine clinical practice is in place.

Introduction

Perioperative monitoring of blood coagulation is important for diagnosing the potential causes of bleeding and guiding haemostatic therapy [1]. As a result, recent years have seen increasing worldwide interest in the use of viscoelastic coagulation monitoring tests, performed using devices such as ROTEM® (Tem International, Munich, Germany) and TEG® (Haemonetics, Braintree, MA, USA) [2-7]. The annual number of published clinical trials involving ROTEM or TEG increased from eight in 2004 to 65 in 2014. Increased future use of viscoelastic tests is anticipated as new devices with increased automation (e.g. ROTEM sigma, TEG 6S) become available. These devices are easier to use and have the potential to increase reproducibility compared with the previous generation of devices, for reasons such as lack of need for accurate pipetting, decreased sensitivity to external vibrations, and electronic quality control before each measurement. The main drawbacks with ROTEM sigma and TEG 6S are the lack of peer-reviewed publications characterizing their performance, and the fact that few clinicians have access to these devices.

It is common for algorithms to be constructed as a means of guiding haemostatic therapy in bleeding patients, and the use of viscoelastic tests in preference to standard laboratory tests as a basis for treatment decisions has been advocated [8]. Such algorithms facilitate individualized goal-directed therapy, with intended improvements such as reduced transfusion of allogeneic blood products, reduced adverse outcomes, reduced mortality and increased cost-effectiveness. Evidence to support this approach exists in cardiac surgery [9,10], trauma [11-13], postpartum haemorrhage [14] and liver transplantation [15-17]. Notably, algorithms developed for TEG or ROTEM are not interchangeable between the two devices. This is because different assays are used with each device, with differences between reagents and their concentrations even for equivalent assays. In addition, it has been shown that when the same reagents are used, clot amplitude results are not consistent across the two devices [18]. Normal ranges and threshold values for intervention are specific to either TEG or ROTEM, necessitating specific algorithms for each device. TEG and ROTEM results are based on arbitrary, preset scales, meaning that they do not measure absolute physical properties of the blood clot such as shear modulus (G) [19].

The use of viscoelastic tests to characterize coagulopathy and guide haemostatic therapy is endorsed in guidelines for managing trauma, postpartum haemorrhage and severe perioperative bleeding [20-23]. A comprehensive UK NHS assessment of viscoelastic tests concluded that they are more effective than standard laboratory tests and cost saving [24]. In 2014, the UK National Institute for Health and Care Excellence (NICE) recommended use of ROTEM and TEG to monitor blood clotting during and after cardiac surgery [25]. In trauma and post-partum haemorrhage, it was recommended that ROTEM and TEG should only be used for research because of uncertainty as to how much benefit they provide in these settings [25]. However, more recent data have demonstrated the effectiveness of viscoelastic testing in trauma and post-partum haemorrhage [14,24,26].

Existing evidence, not only relating to treatment algorithms, shows potential for the use of ROTEM and TEG to reduce transfusion of allogeneic blood products in bleeding patients [9,24,27]. Viscoelastic coagulation tests have also been shown to be cost-saving in both cardiac surgery and trauma [24]. On the other hand, there is an absence of evidence that TEG or ROTEM improves morbidity or mortality in patients with severe bleeding [24,28]. This may be because the devices and assays themselves do not change patient care. Instead, improvements in morbidity and mortality are dependent on clinicians’ interpretation of viscoelastic test results and consequent treatment decisions. The availability of therapeutic agents will also affect outcomes. The design of treatment algorithms based on viscoelastic tests has much greater potential impact on clinical outcomes than the initial decision of whether to use viscoelastic tests.

There is some debate regarding the status of viscoelastic tests and some publications have stated that they are not validated [29-33]. In the most recent of these examples, the authors wrote: ‘However TEG continues to be a second-level hemostasis test due to the lack of its quality assurance procedures and that TEG is not validated, as far as international standards are concerned’ [32]. This text is unclear because no definition of the term ‘international standards’ is provided. However, such statements may lead to the reliability of viscoelastic tests being questioned and cause concern among clinicians applying these tests in clinical practice.

Regulatory requirements for viscoelastic coagulation monitoring tests

For market authorisation of in vitro diagnostic products such as ROTEM and TEG, the FDA require data showing the analytical performance characteristics including bias or inaccuracy, imprecision and the analytical specificity and sensitivity [34]. FDA regulatory requirements for in vitro diagnostic products make no mention of the term ‘validation’.

In vitro diagnostic medical devices (e.g. ROTEM, TEG) are not subject to pre-market authorization by a regulatory authority in Europe, but to a conformity assessment which, for the majority of devices, is carried out under the sole responsibility of the manufacturer [35]. Once certified, devices bear the CE marking which allows them to circulate freely in the EU/EFTA countries and Turkey. The regulations specify that the performance characteristics of in vitro diagnostic medical devices support the intended purpose, and that manufacturer-stated performance is achieved in relation to analytical performance (e.g. accuracy [trueness and precision], bias, sensitivity, specificity, reproducibility) and clinical performance (e.g. diagnostic sensitivity, diagnostic specificity, positive and negative predictive value). No specific requirements for validation are mentioned in the EU regulations for in vitro diagnostic medical devices [35].

Calibration is necessary to ensure the accuracy of devices providing quantitative information (e.g. measuring the concentration of a specific protein in plasma). It is also a consideration with the ROTEM and TEG devices but, because viscoelastic methods are semi-quantitative, formal calibration such as proficiency testing or inter-laboratory comparison is not a prerequisite. TEG 5000 devices are calibrated twice a year using biological controls [36], and this could be considered as an alternative assessment protocol in line with practices recommended by the Clinical and Laboratory Standards Institute [37]. ROTEM devices are calibrated during manufacture, and subsequent calibration procedures are not considered by the manufacturer to be required.

What is meant by ‘validation’?

‘Assay validation’ implies documented control of the test performance according to predefined criteria, relating for example to precision, linearity, accuracy, robustness, measurement limits. Such validation per se does not improve the assay quality, it simply attests the ‘quality check status’ (i.e. the assay has been quality checked). ‘Clinical validation’ of an assay is different, because it requires the assessment of relevance to clinical practice. Key considerations include comparability of the results with previous results, and evaluation of the effects of factors that may be encountered in clinical practice (e.g. variations in patient characteristics).

According to the US Food and Drug Administration (FDA), ‘Analytical method validation is the process of demonstrating that an analytical procedure is suitable for its intended purpose’ [38]. The FDA also states that ‘Validation data must be generated under a protocol approved by the sponsor following current good manufacturing practices with the description of methodology of each validation characteristic and predetermined and justified acceptance criteria, using qualified instrumentation’ [38]. The European Medicines Agency (EMA) has defined validation with similar wording to the FDA: ‘The objective of validation of an analytical procedure is to demonstrate that it is suitable for its intended purpose’ [39].

Although not included within the definition of validation, methods for viscoelastic coagulation monitoring must be shown to meet applicable international standards to be considered as validated. For example, ISO 13485 sets out the requirements for quality management systems relating to medical devices, and IEC 62304 specifies the software requirements. With comprehensive assessment of accuracy, precision, interference, reagent stability, and reference ranges as well as software validation, both ROTEM and TEG devices have been shown to meet all such applicable standards. The intention of quality control procedures is to ensure consistent, accurate device performance. The need for standardization of viscoelastic coagulation tests, with regular external quality assessment to ensure accurate results, has been highlighted in the literature [40-42]. Coefficients of variation require measurement using both intra and inter-laboratory samples. However, standardization is complicated by the fact that plasma would typically be used for this purpose, while in clinical practice the viscoelastic coagulation tests are performed using whole blood.

Have the viscoelastic tests been validated?

The ROTEM and TEG systems and their assays have been CE marked, ISO certified and FDA approved. Market authorization has therefore been granted in most of the world’s countries. To achieve this recognition, performance has been evaluated in accordance with the requirements outlined above. Both devices are routinely used in many centres to guide administration of haemostatic therapy, and both devices have been used in pivotal licensing trials [43-45]. Therefore, it is reasonable to state that the FDA and EMA definitions of validation (‘suitable for intended purpose’) have been met for both ROTEM and TEG. For the regulatory approval, performance characteristics of the devices were demonstrated, meaning that ‘assay validation’ has been achieved. Further evidence of assay validation for both ROTEM and TEG is available from a significant number of published studies, but it is beyond the scope of this publication to review these in detail.

Considering ‘clinical validation’, the reference method for both ROTEM and TEG is thrombelastography as introduced by Hartert in 1948 [46,47]. The TEG 5000 device uses methodology that is closely based on the apparatus developed by Hartert, and ROTEM assays have been shown to correlate with this method [48-51]. Correlations between viscoelastic test results and standard laboratory measurements have been reported, e.g. between FIBTEM MCF and plasma fibrinogen level [52] and between TEG maximum amplitude and platelet count [53]. However, ROTEM and TEG tests are conducted differently from all standard laboratory coagulation tests, meaning that differences are to be expected. For example, the FIBTEM assay and plasma fibrinogen concentration tests measure different physical properties with different SI units, and fibrinogen is not the only determinant of FIBTEM MCF [54,55]. Therefore, neither the FDA nor the EMA has decreed that any of the standard laboratory coagulation tests should be used as a reference method for ROTEM or TEG tests. Cut-off values for the management of coagulation in settings such as cardiovascular surgery, trauma and obstetric/post-partum haemorrhage have been established in numerous studies conducted with both ROTEM and TEG. Detailed consideration of these data is beyond the scope of this publication.

Conclusion

The use of ROTEM and TEG tests to diagnose coagulopathy and determine haemostatic treatment – commonly when implementing treatment algorithms – is increasing. Such use of these tests is accepted by European and American regulatory bodies, both in clinical trials and routine practice. Although the term ‘validation’ does not have a specific definition in relation to viscoelastic coagulation tests, relevant criteria within EMA and FDA documentation are fulfilled. There is evidence that viscoelastic coagulation tests can help reduce transfusion rates and that they are cost-effective. In addition, bleeding management guidelines support the use of viscoelastic tests. ROTEM and TEG devices/assays have been used in pivotal clinical trials and are approved for use in most of the world’s countries. Provided that locally approved indications are adhered to, the regulatory framework for clinicians to use viscoelastic tests in routine clinical practice is in place.