ISO 15189:2003 and Evidence Based Laboratory Medicine.

The relationship between quality, clinical effectiveness and evidence-based laboratory medicine

Evidence-based healthcare (EBHC) evolves from quality improvement initiatives and there is a significant overlap between the approaches and methodologies of evidence-based medicine (EBM) and quality management. The primary aims of both EBHC and quality management of health care services are to improve clinical effectiveness and patients' outcomes. EBM and evidence-based laboratory medicine (EBLM) are essential tools in the assessment of effectiveness, as high quality systematic clinical research is necessary for investigating the impact of any intervention on clinical outcome.

Effectiveness of service is also referred to in the new ISO 15189:2003 standard for medical laboratories (Clauses 4.7, 4.15.1). Clinical effectiveness of laboratory services refers to the best achievable outcome of service delivery in routine circumstances. It depends on the diagnostic performance of a test (i.e. efficacy), the applicability of this finding to local circumstances (i.e. efficiency or cost-effectiveness), and the standards, organisation and management of service (i.e. quality). Burnett defines quality as ‘fitness for purpose’ and also relates the evaluation of service to its ability to satisfy stated or implied needs (1). It is very important that this broader term of quality is used in the healthcare context. While quality assurance and quality assessment tools usually measure whether things are done well, EBM and EBLM challenge whether the principles of ‘fitness for purpose’ are met (2), and whether the laboratory responds to the needs of its users. For example, a laboratory may measure random glucose and blood gases on all healthy patients entering the day-care unit for minor surgery and provide this service at high quality (i.e. laboratory staff doing their work competently and to high professional standards). In terms of clinical effectiveness and ‘fitness for purpose’, however, this otherwise ‘high quality’ service is a waste of time and resources, and practice does not follow evidence-based guidelines which recommend no testing in this group of patients (3).

From the above it follows that quality management and accreditation of medical laboratories are part of the clinical effectiveness cycle (Figure 1) (4). Accreditation can achieve its prime aim (i.e. improving the quality and effectiveness of service) only if standards, as a ‘level of excellence or quality’ are defined and measured in an evidence-based manner.

Figure 1.

The clinical effectiveness cycle Reproduced with permission from SIGN (4)

Relationship between requirements of ISO 15189:2003 and EBLM

In the following sections clauses of ISO 15189:2003 (quoted in italics), where the principles of EBLM can be applied, will be discussed in more detail (5).

ISO 15189:2003 4. Management requirements

Health needs and service needs

The ‘needs’ of customers are mentioned nine times throughout the text of ISO 15189:2003 (Introduction, Clauses 4.1.2, 4.4.1.c, 5.5.1, 5.5.2, 5.8.8, 5.8.11, B5.4, B6.1). We highlight below those clauses that deal with this issue as a key requirement.

ISO 15189:2003, Introduction

Medical laboratory services are essential to patient care and therefore have to be available to meet the needs of all patients and the clinical personnel responsible for the care of those patients (5).

ISO 15189:2003, 4.1. Organization and management

4.1.2. Medical laboratory services, including appropriate interpretation and advisory services, shall be designed to meet the needs of patients and all clinical personnel responsible for patient care (5).

Before discussing the meaning of the term ‘needs’, we have to define who the ‘customers’ of laboratory services are. Customers are users of laboratory services, including patients, clinical staff and purchasers (Clause 5.1.4.c). According to this, the ‘needs of users’ have to be interpreted from different perspectives, and a distinction should be made between health needs of patients and service needs of health care staff and purchasers.

The ultimate health needs of patients, in terms of laboratory services, are to provide high quality, reliable diagnostic information which supports the correct screening, diagnosis, differential diagnosis, monitoring, prognosis or risk assessment of a given health condition. EBLM is instrumental in providing reliable and valid scientific data on the diagnostic efficacy of laboratory investigations, thus contributing to satisfying the needs of its prime users.

Service needs related to medical laboratories are linked to technical, organisational, financial or ethical considerations which often involve a consultation process and contracting between the laboratory and the users or buyers of its services. Service needs can also be related to the provision of technically reliable and accurate data (Clauses 5.5, 5.6) at fast turnaround times (Clause 5.8.11), and at the lowest possible costs (Clause 5.1.4.i). The impact of these needs on outcomes has to be addressed by suitable research and assessed in clinical audit cycles – both areas being within the scope of EBLM.

Evidence-based standards, criteria and indicators

ISO 15189:2003, 4.2. Quality management system

4.2.3. Policies and objectives of the quality management system shall be defined in a quality policy statement…This policy… shall include the … laboratory management’s statement of the laboratory’s standards of service; …and the laboratory’s commitment to good professional practice,… (5).

To achieve good professional practice, criteria and standards for best practice need to be valid. Valid criteria are based on evidence, measurable, and must respond to the needs of users (6). A conference of the Joint Commission in 1999 collected votes from 50 countries on accreditation issues and 81% of participants concluded that if evidence clearly supports best practice and clinical effectiveness, accreditation standards should adopt these findings (7).

Systematic reviews or evidence-based guideline recommendations provide useful information when developing criteria and practice standards (8, 9). If high quality secondary publications are not available, systematic searching of the primary literature and critical appraisal of the evidence are needed (9). In lack of scientific evidence, the views and formal consensus of professional groups should inform the process. It is very important that the sources of information underlying standards and criteria are explicitly stated. Evidence-based criteria and standards can be used in several other quality management activities investigating the clinical performance of laboratory service:

identification and control of nonconformities (Clause 4.9),

preventive action (Clause 4.11),

continual improvement (Clause 4.12),

internal audit (Clause 4.14),

clinical audit ( and

management review (Clause 4.15).

ISO 15189:2003, 4.12. Continual improvement

4.12.4. Laboratory management shall implement quality indicators for systematically monitoring and evaluating the laboratory’s contribution to patient care (5).

Measurement of current practice is based on criteria and measurable indicators (10) that can be classified according to whether the problem reviewed is related to structure, process or outcome of care (e.g. improvement of morbidity, mortality, patient satisfaction, decrease of turn-around time) (6). It is difficult to develop evidence-based outcome criteria in laboratory medicine for the lack of outcome studies, poor quality of primary studies, sources of bias (e.g. lead time and disease progression bias), poor transferability of research data due to heterogeneity of the investigated patient and disease spectrum, etc. Therefore surrogate outcome criteria are often used which are easier to measure (6). For example, HbA1c measurement is a proxy outcome for diabetes control and indirectly assesses the long-term clinical outcome of morbidity due to secondary complications of the disease.

Prioritization and monitoring

ISO 15189:2003, 4.14. Internal audits

4.14.2.…internal audits of all elements of the system, both managerial and technical, shall be conducted…The internal audit shall progressively address these elements and emphasize areas critically important to patient care (5).

The above statement emphasizes the need for prioritization of key areas of laboratory services relevant to patient care. As resources are limited in every laboratory, management should identify and prioritize areas, which have the greatest impact on effectiveness, in a systematic way. Research evidence can inform several steps in the prioritization process. When prioritizing, the following questions may need addressing (6, 8):

Is the area concerned of high cost, high volume, or increased risk to users?

Is there a serious quality problem, patient complaints or harm?

Is good evidence available to support diagnostic decisions?

Is there potential for change and improvement?

Is the area a national health priority?

Is the area important for the organisation?

Critical areas shall be audited in collaboration with clinical staff, if relevant. Multidisciplinary clinical audit, comparing actual performance with practice standards, is a powerful tool in setting targets for quality improvement and in bringing about change in service delivery. The term ‘clinical audit’ is not quoted as such in ISO 15189:2003, however, it is referred to indirectly both in Clause 4.14. and 4.15.

ISO 15189:2003, 4.15. Management review

4.15.1. Laboratory management shall review the laboratory’s quality management system and all of its medical services, including examination and advisory activities, to ensure their continuing suitability and effectiveness in support of patient care and to introduce any necessary changes or improvements.

4.15.3. The quality and appropriateness of the laboratory’s contribution to patient care shall…be monitored and evaluated objectively (5).

These sub-clauses highlight the responsibility of management in monitoring effectiveness and the employment of quality improvement cycles through the practice of clinical audit. It is particularly important that the requirement emphasizes ‘suitability’, ‘appropriateness’ (whether the service is provided to the right people, at the right time) and ‘quality’ of service in a broad sense. Application of research evidence helps in determining what suitable, appropriate and high quality is, in terms of effective diagnostic service, supporting the care of patients in the community.

ISO 15189:2003 5. Technical requirements

Do laboratory personnel have skills of EBLM?

ISO 15189:2003, 5.1. Personnel

5.1.4.The laboratory director or designees for each task should have appropriate training and background to be able to…a) provide advice…about the choice of tests,…and interpretation of laboratory data; …d) define, implement and monitor standards of performance and quality improvement of the medical laboratory service…(5).

EBLM supports diagnostic and therapeutic decisions by providing objective data for informed medical decisions. Do laboratory personnel meet the above requirements and do they have ‘appropriate training and background’ for applying evidence in practice (11)? An EC4 consultation document on the “Competence to be a consultant in laboratory medicine” emphasizes the need for “skills to search for and critically appraise the evidence and to apply evidence for optimising service provision” (12). The IFCC Committee on Evidence-Based Laboratory Medicine (C-EBLM) has recently carried out an international survey which demonstrated in many countries that formal training in EBLM is still lacking both at undergraduate and postgraduate level, and most curricula for specialist training do not cover EBLM (unpublished). In order to fill in these gaps, and to respond to the training needs of the IFCC community, the C-EBLM, together with experts of the Cochrane Collaboration, will organise a 4-day postgraduate course on EBLM in September 2005 (for details, see www.ifcc.org). The philosophy, opportunities, tools and resources for teaching EBLM have been reviewed by Price and Christenson recently (13). In the lack of good outcome studies, however, it remains to be demonstrated whether skills in EBM contribute to the improvement of the effectiveness of care (14, 15).

We mentioned above that EBLM is a decision support tool guiding both diagnosis and therapy. Diagnostic decisions are based on several factors. Testing depends on the prevalence and pre-test probability of the target condition, and information about the quality specifications (i.e. test information), and the discriminatory power and interpretation (i.e. post-test information) of the applied investigations. The potential role of EBLM in these different phases of the diagnostic process (i.e. pre-test, test and post-test phases), in relation to the relevant chapters of ISO 15189:2003, will be discussed in the following sections.

Evidence-based test ordering

ISO 15189:2003, 5.4. Pre-examination procedures

5.4.1. The …manner in which requests are communicated to the laboratory should be determined in discussion with the users…(5).

The pre-examination phase starts with the selection of the right test(s) for the right patient and at the right time. The fulfilment of the above requirement is one of the most critical areas of laboratory services, as both over- and under-utilization of diagnostic services occur and may cause harm to patients (16). The laboratory is responsible for giving “advice on choice of examinations and use of the services” (Clauses 4.7 and 5.1.4.a ). What should this advice be, where should it come from, and how can we transmit this information to clinical staff? According to the principles of EBLM, a diagnostic test should be requested only when an appropriate question is asked and when there is evidence that the result will provide an answer which will influence the clinical decision (17). Therefore, EBLM and accreditation in the pre-examination phase have the same goals, i.e. they both aim at improving test ordering patterns. Improved laboratory test selection combined with the enhanced presentation and interpretation of test results are cost-effective tools in changing requesting patterns and the use of diagnostic services by clinical staff (18-20). Laboratory management should be aware of the fact that changing test ordering behaviour is a difficult and endless task, needing continuous attention (20). Strategies that have been shown to work well include:

changes to request forms and reduced availability of tests on forms,

audit and personalized feedback to clinicians,

computer reminders

and financial incentives or disincentives (20, 21).

Interventions that worked less well include single training courses, lectures and CME. Feedback on costs or written materials on test ordering showed little or no effect. Implementation of any of these strategies needs to be tailored to local circumstances, and it is most likely that combination of different strategies will provide the largest impact.

Evidence-based analytical performance goals

ISO 15189:2003, 5.5. Examination procedures

5.5.1. The laboratory shall use examination procedures…which meet the needs of the users of laboratory services and are appropriate for the examinations.

5.5.4. Performance specifications for each procedure used in an examination shall relate to the intended use of that procedure (5).

The use and interpretation of laboratory investigations largely depend on the suitability of the methods of analysis for the intended application (Clause 5.5.4.). Test performance in the clinical setting is grossly influenced by the analytical quality of methods (Clause 5.6) (22) and the biological variation of the given parameter. The latter is also essential in the determination of analytical performance goals (23, 24; www.westgard.com/guest26.htm). The “evidence” used to establish analytical specifications of laboratory tests appears to be based largely on the consensus of experts, which scores a relatively low grade on any evidence scale. Therefore it seems appropriate to consider additional scientific approaches to strengthen the recommendations for defining biological variability and for setting analytical performance goals (23, 25).

Evidence-based information on diagnostic utility of tests

ISO 15189:2003, 5.7. Post-examination procedures

According to the terms and definitions of ISO 15189:2003(E) 3.9, post-examination procedures include “systematic review, formatting and interpretation, authorization for release, reporting and transmission of the results, and storage of samples of the examinations” (5).

The aim of EBLM in the post-test phase is to assist clinicians and patients both in the interpretation and in the clinical utilization of laboratory results. Accreditation standards also refer to the necessity of “interpretation of the results of examinations” (Clauses 4.7, 5.1.4.a, 5.8.3.j). How can we best deliver information on diagnostic utility of tests to healthcare professionals and patients?

When clinicians wish to make a diagnosis, they use tests as modifiers of disease probabilities in order to convert pre-test probability information to post-test probability estimate of a certain target condition. For making this judgement it is essential to know the diagnostic accuracy, i.e. the sensitivities and specificities, or the likelihood ratios of tests, which enable them to transform pre-test data into clinically meaningful information. It is the responsibility of laboratory professionals to present test results in clinically meaningful ways. Preliminary data suggest that interpretative comments on laboratory reports, provided they are written by well-trained and competent staff, could be one of the possible answers to this problem (26).

To help physicians to utilize laboratory services efficiently, the supporting evidence or guideline recommendations should be made accessible at the point of clinical decisions, preferably directly linked to patient data. Information technology can provide means to integrate decision support into patient care (27). Several initiatives and a number of evidence-based databases exist, including a systematic reviews database related to laboratory medicine recently released by the IFCC Committee on EBLM (www.ifcc.org).

Conclusions

EBLM gathers information from well-conducted research studies on the pre-test, test and post-test performance of laboratory investigations, and incorporates this evidence into the routine practices of medical laboratories. EBLM is an excellent method to enable better clinical decisions, and to integrate routine laboratory service with effectiveness, quality management, education and training in laboratories. By the process of constant questioning and reviewing current practice, and comparing it to the best available evidence for rational diagnosis and therapy of diseases, it is a practical tool for identifying deficiencies in our knowledge or service. This way the practice of EBLM initiates quality improvement cycles and generates new research ideas as well. EBLM, embedded into the culture of quality management in laboratories, is ‘best practice’ made explicit and accessible. Accreditation according to high professional standards, based on the best available research evidence, could be a powerful tool in putting evidence into the daily practice of medical laboratories.