Comparative analysis of quality assurance in health care delivery and higher medical education.

Quality assurance (QA) in higher medical education involves the development, sustenance, improvement, and evaluation of the standard of training of medical professionals. In health care delivery, QA focuses on guaranteeing and maintaining a high standard of the service provided in different health care systems. When the service delivered by the care provider is in accordance with what the recipients of health care expect, then quality in health care is considered to be present. There are several factors in higher medical education and health care that are responsible for the emergence of QA. These include externally imposed obligations requiring demonstration of public accountability and responsibility from educational institutions, as well as the need for activity-specific information by policy makers as an aid for important decision-making within educational institutions. In health care delivery on the other hand, the emergence of QA is linked to the need for containing rising health care costs in the face of limited resources and to guaranteeing high quality patient care in a changing health care environment where the power relationship between doctors and patients is shifting towards patients. Although medical education can be regarded as a distinct entity in the health care industry, it still remains an inherent part of the health care delivery system. As a result, different strategies aimed at guaranteeing and assuring high standards of health care and education in many countries tend to overlap. This paper reflects on whether quality assurance in health care delivery and medical education should be seen as separate entities.

Introduction

With all the changes being seen in the delivery of health care and professional training programs, the issue of guaranteeing and sustaining the quality of health care-related services remains a topic of ongoing discourse. To date, the term “quality” has been an important and often contested concept that defies a single befitting definition. Its interpretation has been influenced by several factors that include the value systems and deeply held assumptions of the various parties concerned, the timescale over which “quality” is being examined, and the purpose of the measure that is used to describe it.1,2

An often-cited definition of quality in health care is that of Donabedian3 where quality is described “as the degree of agreement between the care that is actually provided and previously stated criteria or demands”. This definition relates to whether an organization meets formal demands or criteria, and implies that good quality is provided if these formal requirements are met.3 The definition by Lohr et al4 on the other hand relates to the extent to which these desired results are achieved: “Quality being the degree to which health care, for individuals and for the population, increases the chance of desired outcomes and when the care that is provided is consistent with accepted medical knowledge and insights.”4 Presumably, the definitions of quality of care are influenced by the discipline and/or the times in which they arose. Older definitions tend to focus strongly on the technical aspects of care, whereas new definitions focus on the perspectives of the patient and society.

In medical education, the description of quality is best captured by the 2001 Institute Of Medicine report “Crossing the Quality Chasm: A New Health System for the 21st Century.” A series of recommendations were proposed in this report, aimed at improving quality of care and medical education in the United States. The report resulted in a paradigm shift in the description of the 20th century physician, who was typified as the doctor functioning in a solo practice, holding autonomy as a central value, and priding himself upon continuous learning and knowledge acquisition and laying claim to infallibility when confronting patients and colleagues. The new paradigm of the 21st century physician on the other hand, describes the doctor as one who understands teamwork and systems of care in which (s)he can provide leadership, group practice both virtual and real, that allows the support of information systems, the collection of evidence about care, and efforts for continuous quality improvement.5,6 Hence, in this new context, quality in higher medical education lies in striving for the endpoint of training where fallibility is replaced by an approach to multidisciplinary problem-solving and the acquisition of knowledge is associated with the commitment and understanding of the need for change.6,7

A closer look at how quality assurance (QA) is applied in higher medical education and health care delivery shows that there are similarities in the construct of these two concepts. Both share a common focus on the efficiency of the services that they provide as well as their various endeavors in securing a wide(r) market for the products that they deliver. In a reflection paper that examined the position of higher medical education in the continuum of health care and professional training, Kitto et al discovered four academic domains responsible for driving the educational process. These domains were defined as continuing education, knowledge translation, patient safety, and quality improvement. Despite being distinct fields of study and medical practice, these domains shared a collective goal of improving collaboration, patient care, and health care services.8 The authors recommended that the epistemological, political, and practical feasibility of integrating the four domains should be investigated in order to promote greater networking, information-sharing, and collaboration among health care professionals, as well as effective health care improvement strategies, all of which are elements reflecting QA. They argued that by promoting networking and information-sharing amongst these four domains, there was the potential for integrated and collaborative partnerships, which could result in the development and implementation of more holistic and effective interdisciplinary interventions, which are elements reflecting the 21st century physician.

Based on the abovementioned observations, it appears that the QA process in health care delivery and education can be improved significantly if there is a shared framework that can narrow down the focus of QA in the separate contexts and integrate them into a single construct. The aim of this paper is to explore whether there is a reasonable basis for this assumption and to examine for potential challenges in defining such an integrated QA system for health care and higher medical education.

What is QA in health care and higher medical education?

In the process of health care delivery, it is obvious that there is a need for quality in the kind of care (being) delivered and in the way health care professionals are trained to provide it. Equally, and at the same time, predefined standards in this process must be identified and set as requirements that need to be met and sustained. So what do we mean by the term “quality assurance”? Simply stated, QA can be considered as a systematic process of checking to see whether a product or service being developed or provided is meeting (or meets) specified requirements. Such a process, if well designed, would increase customer confidence and institution credibility, improve the efficiency in work processes, and enable institutions to compete better with others. Therefore, QA is a system that is driven by multiple events, which are in a state of constant motion and where efficiency and effectiveness constitute the core framework.9,10

In higher medical education, QA involves maintaining, enhancing, and comparing the standard of professional training of medical professionals. It aims at ensuring that individuals qualifying for licensure are safe and competent practitioners and have achieved the educational standard that is appropriate for the license being offered. Such individuals are also considered to be capable of coping with the changing patterns of health care delivery as well as disease patterns.11 The QA process in higher medical education also serves the purposes of demonstrating that the needs of key stakeholders are met (accountability), information is provided to the public and employers so that informed choices can be made about educational programs (information), and ensuring that there is a continuing and focused stimulus to enhance the learning experience based on feedback received and of professional, practice, service, and policy developments (quality improvement).

In health care delivery, QA focuses on guaranteeing that a high standard of health care is provided in different health care systems. When the service delivered by the care provider is in accordance with what the various recipients or customers think should be provided, quality in health care can be considered to be present.9,12–14

Therefore, a pragmatic way of interpreting the concept of QA would be to identify the way customers evaluate a particular service that is provided. The customer in this context would refer to anyone (or organization) who can make a judgment on the quality of service that has been provided or who has particular expectations about a service being provided. However, given that the expectations of customers are continually changing, the definition of quality would continue to shift, thereby making it a dynamic concept.13

Need for QA in health care delivery and higher medical education

There are a number of factors that have been found to determine the need for QA in higher medical education, and a number of these have arisen due to externally imposed conditions that require the demonstration of public accountability by educational institutions, eg, grants being awarded for research based on an institution’s educational reputation or potential for results. Other factors include the need for activity-specific information by policy makers that would aid important decision-making processes within educational institutions, eg, an institutional decision to continue with a particular research project based on a previous track record of remarkable scientific achievements. Furthermore, the desire by individual sectors (or organizations) to conduct their own progress reviews, assess the cause of problems identified or investigate how improvements could be made to existing procedures, are additional contributory factors for QA in higher medical education.9 However, the evaluative activity that often arises from these processes is generally considered to be politically motivated and aimed at serving the interests of particular stakeholders, who are in charge of reallocating resources based on revised priorities and market developments.15,16

In the context of health care delivery, the emergence of QA in hospital organizations is closely linked to the (increasing) need for improved quality of patient care and managing the costs involved in providing the service. It is also associated with the realization that the resources needed to deliver effective health care are limited, and that the contribution of patients to the management and decision-making process concerning their illness, is increasing. Consequently, the growing complexities involved in providing the sort of care that is valued and tailored to patient needs, demands a quality assurance process with a sharper focus on health care delivery.

Challenges facing reliable assessment of quality

The literature on quality in health care distinguishes between two main uses of measures of quality, which are often referred to as “indicators”. The first is the measurement for insight and learning in internal quality improvement and the second is measurement for judgment in external accountability systems.17–21 There are a number of indicators currently in use in health care and higher medical education today to assess the quality of service that they provide. For example, in higher medical education, these include mean student scores for admission, students’ achievements in later life, employment rates and mean starting salaries of graduates, publication citations per faculty, employer evaluations, employer reputational ratings, academic peer ratings, eg, by school deans, heads of department, residency directors, and rankings in international academic reference databases, all of which are based on single criteria and institutional self-study.22,23 In health care on the other hand, the quality of service is appraised using customer satisfaction ratings, excellence in specific fields of expertise, admission and discharge turnover rates, pay for performance, hospital rankings by independent organizations, public reporting, and net profits, for example.24,25

However, it is important to note that most of these methods of assessment have their merits and caveats. For example, reputational ratings as evaluation methods are based on the opinions of experts, alumni, or others. Their advantage is that the evaluators have professional knowledge of the academic environment and its standards. The pitfalls are the relative positive bias towards research institutions because their reputations are highly influenced by research results, the halo effect resulting from opinions of the general public, and the time lag between the outcome of institutional evaluation and the actual situation. Furthermore, one can expect that the reputation an institution has acquired is not likely to change as fast as the quality of service the institution provides. However, a great advantage of institutional self-study is the autonomy that it offers to institutions when decisions related to maintaining the standard of education are made and taken. Regrettably, this method is rather complex and time-consuming. Moreover, evaluators tend to give high scores to their own institutions, and focus more on the quality of products than on the quality of the process.9,26

Characteristics of effective QA in health care and higher medical education

While there are differences in the way QA measures have been defined or translated in health care and higher medical education, appraisal of quality of service provided remains identical in both contexts in being a process that continuously (re)designs, evaluates, and monitors the service or product being provided. Central to this process is the objective of achieving institutional focus for the evaluation of its services or products. For this, there would be a need to formalize a variety of activities as part of the functional hierarchy of the evaluative process. These evaluative activities can be found at different hierarchical levels within the health care or educational process, for example, at the levels of the individual, special task groups, permanent committees, boards, or specialized groups. Furthermore, QA systems should also have defined organizational structures within the institution that are expected to accommodate continuous improvement of quality. Therefore, it means that (pre)selection of relevant outcome (ie, performance) indicators should be identified in the early phase of the QA process, which can be used in evaluating the quality of services provided (Table 1).

Table 1

Important features of a quality assurance system

• Focus on the organization’s mission and the customer’s needs

• Approaches operations for improvement systematically, eg, the plan-do-check-act cycle that offers a scientific method for continuous process improvement

• Stimulates vigorous development of human resources

• Facilitates long-term thinking

• Ensures commitment by every participant in the process

Role of performance indicators in QA

A term that is often used in QA is “performance indicators”, which are empirical, quantitative, or qualitative data that point to an institution’s achievement of its goal. They are considered context-related and time-related, and have rapidly become integral parts of organizational management in every sphere of human activity, and to a limited but growing extent, in education.27 Often interchanged with the term “quality indicators”, the underlying purpose of performance indicators in all these areas is to serve as a support for decision-making when looking at the ways of improving organizational performance of a health care institution, for example, as well as the quality of service delivered. It is important to mention though that the attitudes towards comparative indicators vary between professional groups and between individual practitioners as a result of the knowledge and understanding, practice issues, and perceptions about the purposes to which the indicators may or may not be put to use28 (Table 2).

Table 2

Characteristics of performance indicators

• Valid for use at government and central institutional level

• Related to main goals for management of the service provided

• Indicated as relevant in international literature or discussions

• Results of validity study indicate no fundamental contradiction with other (lower) levels of aggregation

There are five core uses of performance indicators in the QA process, namely evaluation, monitoring, planning, implementation, and dialog. However, the main function of performance indicators is to identify the principal characteristics or components of successful performance, expressed in terms amenable to either quantitative measurement or reliable estimates of relative achievement. Performance indicators are also intended to provide a profile of performance levels attained by a particular organization, eg, hospital or medical school, at a particular time, against which to compare that of other organizations or the same organization at different times. They can thus be described as being of both contextual and temporary importance.

Performance indicators are operationalized by a set of variables which are specific in character and relevant only if they express the performer’s goals. The relationship between performance indicators and goals shows that in one situation, empirical data are considered to be management statistics or information, while another situation, they turn out to be variables. In order to avoid this ambiguity, any set of data that would be used for evaluating the quality of service an organization delivers would need to be translated into more concrete aspects, ie, operationalized. Therefore, the need to distinguish indicators forms the first step of the assessment process. Furthermore, while satisfying the above two preconditions, the indicator of choice should be able to assess the content of the theoretical characteristics (validity) as well as be quantifiable (measurability).29

Constraints in selection of performance indicators

Appropriate performance indicators should have considerable reliability of measurement and should be able to recognize intrinsic merit or worth. However, it is difficult to select an appropriate indicator that is relevant to the central functions of teaching, research and health care delivery. This is because of the imbalance between the statistical demand for broader aggregation (intended to minimize random variations in performance) and the conceptual demand for comparability that tends to favor smaller aggregation units required to maximize homogeneity.

Another problem is the expert judgment required in the process of interpretation, which demands that isolated decisions be made based on the merit of individual activities or the components included in each specified indicator. Therefore, decisions concerning overall performance of particular institutions or departments need to be made based on results from a range of performance indicators. Although teaching, health care delivery, and research are characteristic of institutions of higher education and/or teaching hospitals, the nature of these two functions and the balance of priorities between them may differ widely both within and between institutions, making the choice of appropriate performance indicators difficult. Taking the institutional context and priorities for achievement into account, the quality of institutional performance can also be defined in terms of its relative contribution to service improvement, ie, as a function of particular outcome characteristics rather than the specific quality of outcomes. However, it would probably be difficult, if not impossible, to develop a suitable reference scale without imposing unreasonable constraints of uniformity, as usually occurs in most standardized testing systems.

It is crucial that the quality of institutional performance is assessed in order to maintain the standard of service provided. At the same time, there should be consensus between the pressures for accountability through performance-based funding from governments on the one hand, and institutional demands for autonomy of action and self-determination on the other. In many countries, different methods, such as medical licensure, continuing higher medical education, and consumer satisfaction ratings, are used to accomplish this.26,30 Nonetheless, it is important to mention the strong and diverse concerns health care professionals have about the reliability of performance indicators in health care quality assurance, even when these indicators are only being used for internal quality improvement purposes. Performance indicators are often perceived to be flawed, difficult to interpret, and based on inaccurate data. There are fears that the data are used solely for managerial and cost-cutting purposes, that may impose significant constraints and control on health care professionals. There are also specific concerns that the use of patient experiences of measures of quality will draw attention to aspects of service organization rather than focus on clinical care28,31–36 (Table 3).

Table 3

Principles of quality assurance in healthcare and health care education

• The patient’s experience is central to health care and the health care learning process

• Professional integrity is respected, whilst the need for interprofessional learning and working and the sharing of professional experiences are recognized as essential

• Quality assurance is integral to the culture of learning in health care wherever it takes place

• Quality assurance encompasses self-evaluation, peer evaluation, and external evaluation

• Quality assurance processes are rigorous, fair, and transparent

• Criteria against which quality assurance judgments and outcomes are arrived at are rigorous, explicit, and acknowledged by all stakeholders

• All quality assurance processes are based on the best available evidence

• All quality assurance processes are effective, efficient, and, where appropriate, shared, avoiding duplication of effort

• Elements of quality assurance are interdependent and together support continuous improvement in health care education

• Judgments and outcomes from quality assurance processes contribute to enhancement of health care and health care education

Conclusion

In conclusion, this paper highlights how QA in higher medical education and health care delivery share similar elements in their construction. In both contexts, a common and shared objective is how to achieve efficiency in the delivery of services and secure a far-reaching market orientation for their respective products. There is also the need for a shared framework that would narrow down the focus of the QA system in these different contexts. With agreement secured across key stakeholders, the objective of such a framework would be to:

Secure more effective and responsive practitioners who, working in teams, will improve patient care

Clarify health care learning objectives and outcomes for employers and regulatory bodies

Improve the responsiveness of the health care curricula to employers in health care organizations, requirements of professional bodies, and expectations of consumers of health care

Enhance the learning experience of undergraduate and postgraduate trainees as well as of continuing professional development

Streamline existent quality assurance system with newer models as they emerge.

Bearing the above analysis in mind, a well defined institutional focus is necessary for achieving quality in higher medical education and health care delivery. This implies that the quality of services provided should be evaluated using predefined, reliable, and relevant indicators, and that clear organizational structures should be in place, which can accommodate continuous improvement and innovation of QA models. Hence, in order to achieve and sustain a high standard of health care delivery, more QA models that effectively integrate health care delivery and higher medical education are needed.