Planning feasible and efficient operational scenarios for a university hospital through multimethodology.

The COVID-19 pandemic required managerial and structural changes inside hospitals to address new admission demands, frequently reducing their care capacity for other diseases. In this regard, this study aims to support the recovery of hospital productivity in the post-pandemic context. The major challenge will be to make use of all the resources the institution has obtained (equipment, beds, temporarily hired human resources) and to increase production to meet the existing repressed demand. To support evidence-based decision-making at a major university hospital in Rio de Janeiro, hospital managers and operations research analysts designed an approach based on multiple methodologies. Besides multimethodology, one important novelty of this study is the application of a productivity frontier function to future scenario planning through the quantitative DEA methodology. Concept maps were used to structure the problem and emphasize stakeholders' perspectives. In sequence, data envelopment analysis (DEA) was applied, as it combines benchmarking best practices and assigns weights to inputs and outputs. To guarantee that the efficiency measurement considers all inputs and outputs before any inclusion of expert judgment, the scope was redirected to full dimensional efficient facet, if any, or to maximum efficient faces. The results indicate that production scenarios proposed by stakeholders based on the Ministry of Health parameters overestimate the viable production framework and that the scenario that maintains temporary human resource contracts is more compatible with quality in health provision, teaching, and research. These findings will serve as a basis for decision-making by the governmental agency that provided temporary contracts. The present methodology can be applied in different settings and scales.

Introduction

In March 2020, the World Health Organization (WHO) announced the newly identified coronavirus (COVID-19) outbreak as a pandemic. COVID-19 has had a worldwide impact, with a total of 258.2 million registered cases and 5.2 million deaths, as of December 2021 [1]. Given the severity of the disease, health systems needed to rapidly adjust to address the new demands for hospitals, beds, intensive care unit (ICU) structure, equipment, and human resources, along with health surveillance and, more recently, immunization [2]. Health system efforts varied among countries, from temporary hospitals in the United Kingdom to new hospital buildings in China and field hospitals in Central Park, New York [2,3].

In Brazil, approximately US$ 10 billion of federal funds were allocated to acquire extraordinary resources for the COVID-19 response and 51.8% of this amount were transferred to states and municipalities, which had the autonomy to decide where to invest [4]. In 2020, most of the resources were used to buy equipment (mainly mechanical respirators), oxygen, sedative drugs, and personal protective equipment for health staff and interns. Some hospitals were entirely transformed to receive only COVID-19 patients and field hospitals were implemented when the demand surpassed the available beds. Although social distancing and mask use were encouraged as preventive measures, the federal government failed to provide satisfactory communication to guide the population's behavior, denied current scientific developments, and delayed the purchase of vaccines, which also delayed the start of the immunization campaign to January/2021. These failures resulted in 22.1 million COVID-19 cases and 613.9 thousand deaths [1], along with the establishment of a parliamentary inquiry commission to investigate the government's actuation [5,6].

Despite uncoordinated actuation to address the COVID-19 pandemic [7], local initiatives led to the expansion of the supply of intensive care beds, mostly through public services [8]. In Rio de Janeiro, a joint effort of the government, universities, and private institutions was launched to increase the number of intensive care beds, purchase equipment, and hire health workers. Moreover, training human resources from diverse specialties to perform intensive care procedures (intubation and mechanical ventilation) was also a challenge. Owing to the high workloads, sick leaves to health professionals because of illness or burnout had to be managed. To address this problem, the government exceptionally allowed temporary hiring of personnel, in which public and private partnerships were established to secure funding.

Universities had a major role in this scenario, as teaching hospitals are centers of complex health care, with a high concentration of resources, providing simultaneous teaching and research activities [9]. In this context, Clementino Fraga Filho University Hospital (HUCFF) of the Federal University of Rio de Janeiro (UFRJ) was one of the leading institutions to treat COVID-19 patients [10]. HUCFF is a public teaching hospital in the north part of the city of Rio de Janeiro, responsible for outpatient and inpatient medium- and high-complexity procedures offered under the National Public Health System. The hospital has 265 hospital beds, 16 intensive care beds, and 8 operating rooms, with an average of 9000 admissions, 4700 surgeries, and 224,000 outpatient consultations annually. As a general hospital, it belongs to a university complex of 8 hospitals (along with maternity, gynecologic, pediatric, neurologic, pneumological, cardiologic, and psychiatric centers) that sum up 700 beds, but only HUCFF was a reference to treat COVID-19 patients.

After the pandemic joint effort, the installed capacity increased to 318 beds, 40 intensive care beds, 13 operating rooms, and 200 outpatient consulting rooms. Computerized tomography, nuclear magnetic resonance, and angiograph equipment units were also acquired. Most importantly, human resources were temporarily hired, including specialized technical staff for the operation of these beds, including 146 doctors, 171 nurses, 320 nursing assistants, and 226 from other health categories.

Given the immunization process of the population, the pressure for admission of severe cases of COVID-19 has been drastically reduced worldwide [11] and most of them are related to unvaccinated people [12,13]. By November 2021, 68% of the Rio de Janeiro state population had been fully vaccinated, which strongly reduced the COVID-19 admissions; by January 2022, 90% of all admitted patients had not been immunized. Even after the introduction of the highly contagious Omicron variant, the disease case-fatality rate (a proxy to evaluate the disease severity) in the city decayed from 8.7% to 0.2% [14]. Epidemiological analysis has suggested that the relationship between cases and deaths has considerably reduced and COVID-19 is becoming an endemic disease, with a severity pattern closer to that of seasonal influenza, at least in highly immunized populations [15].

In other words, it has been hypothesized that after achieving almost full vaccine coverage, COVID-19 admissions will be virtually zero, and hospitals’ acquired resources will be reverted to address the traditional epidemiological demand for chronic non-communicable diseases: cardiovascular, neoplastic, chronic respiratory disease and diabetes mellitus, which were mainly restrained during the health emergency state. Cardiovascular and neoplastic diseases had been responsible for 30% and 17%, respectively, of all Brazilian deaths before COVID-19 pandemic. After the pandemic announcement, most non-COVID diseases have been either neglected, or had their treatments interrupted, and healthcare systems were in turn stretched beyond their capacity to provide adequate care [16]. Cardiovascular deaths increased by 7% in Brazil due to lack of timely attendance [17] and substantial increases in the number of avoidable neoplastic deaths are to be expected in the next five years as a result of diagnostic and screening delays [18]. In Brazil, for instance, mammograms and colonoscopy, important tools for early diagnosis of breast and colorectal cancer, fell about 40% and 60%, respectively, in 2021 [[19], [20]]. Given this epidemiological profile, and the negative impact of COVID-19 on chronic non-communicable disease control, the Brazilian Ministry of Health (MoH) emphasizes the need to reduce mortality from these diseases through prevention and treatment, by means of regulation for all levels of care (primary, secondary and tertiary). In Brazil, regulation is accomplished through management contracts with each and all healthcare units and transparent monitoring of indicators, that are easily captured in the MoH information system [21]. The current policy is in line with United Nations Sustainable Development Goal 3 (SDG 3), proposed in 2015, regarding “Good Health and Well-being”. It is noteworthy that most of the 13 targets and their respective 28 indicators of SDG number 3 are mainly reached by non-hospital primary care (as reduction of maternal mortality, preventable deaths under five years of age, communicable infectious diseases, non-communicable chronic-degenerative diseases, and increase in universal access of sexual and reproductive care) or inter-sectorial actions (to handle substance abuse, road injuries, hazardous chemicals and pollution, tobacco control, vaccines and medicine development). Otherwise, given general hospital secondary and tertiary demand, mainly based on non-communicable diseases, management contracts between MoH and general hospitals, which is the case of HUCFF, predominantly address cardiovascular and neoplastic mortality rate control, in line with SDG 3.4, which intends to reduce these rates by one third, until 2030.

During the pandemic emergency, there was an abrupt decrease in hospital admissions other than COVID-19, and most hospitals had to stop surgeries, routine exams and outpatient visits. Concerning productivity, dynamic network DEA used to compare efficiency between Brazilian states and capitals during the pandemic showed major regional discrepancies. Efficient states and capitals have brought performance parameters for development of action plans to prevent the collapse of the health system [22]. The next challenge for hospitals will be the recovery of their production, taking advantage of the increased installed capacity to increase the usual service production and to address the repressed demand. The need for production recovery motivated the joint effort of hospital managers, stakeholders, and operations research (OR) analysts to evaluate future scenarios using a multimethodological approach.

First, the team developed qualitative concept maps to contextualize the problem, which highlighted the need to maintain temporary human health resource (HHR) contracts to guarantee health care, teaching, and research inside the university hospital. Second, OR analysts proposed the use of data envelopment analysis (DEA) to establish an efficient relationship between multi-output multi-input hospital healthcare components. Besides multimethodology, one important novelty of this study is the application of a productivity frontier function to future scenario planning through the quantitative DEA methodology, since DEA is mainly directed to evaluating past performances as part of the control function of management [23]. To enable this analysis, we developed a Goal Programming (GP) model applied to a special type of DEA frontier faces, known as full dimensional efficient facets (FDEFs) or, at least, maximum dimensional efficient faces (MDEFs), which present important properties regarding positivity of weights assigned to inputs/outputs. This study is intended to support hospital managers in the elaboration of future planning scenarios, providing DEA-based adjustments for stakeholders’ proposals to an efficient frontier. An immediate current demand is the modeling of efficient operation at any level following the acquisition of assets for pandemics, based on the HUCFF case study. This study is organized as follows. Section 1 introduces the study topic. Section 2 presents a brief review of the methods used: concept maps, goal programming (GP), and FDEF/MDEF, emphasizing health applications, along with the proposed quantitative model (GP-FDEF DEA). Section 3 presents the qualitative and quantitative methods applied to the empirical universe of the HUCFF case study. Section 4 discusses the contributions and limitations of the study.

Multimethodology and healthcare applications

In practical situations and real organizations, particularly in healthcare settings, problems are multifaceted, which hinders their satisfactory interpretation using a single quantitative or qualitative method. In this scenario, combining qualitative (soft approach) and quantitative (hard approach) methods for richer and more realistic representation of the problem can overcome this limitation. This approach is known as multimethodology [24], mixing methods [25], or hybrid models [26]. According to Ref. [24]; multimethodology is the “art” of using more than one methodology or parts of it in combination to consider the various problems embedded in complex real-world systems. In addition, multimethodology also addresses in the same intervention different paradigms that guide the different methodologies used, which increases the chances of implementation and validation of results by stakeholders [27].

Several multimethodological approaches can be applied in the healthcare sector, many of which support systematic government planning. This is the case with the workforce planning in the British National Health System, which combines qualitative and quantitative system dynamics approaches according to stakeholders’ perspectives and demands [28], and the efficiency analysis in Brazilian municipalities, which combines concept maps and DEA to support the National Public Health System [[29], [30]]. Multimethodology has already been applied to Brazilian university hospitals [31].

The present paper reports the results from the initial stage of a collaboration between HUCFF managers and the Production Engineering Department of the UFRJ. The study proposes an innovative approach that integrates the following methods: concept maps to support the contextualization of the problem and goal programming applied to maximum dimensional faces in data envelopment analysis. In order to implement this approach, we first formalize the qualitative context of the application, which normally remains only implicit in the modeling process. Next, we characterize the technology for producing services given by DEA faces based on vectors of healthcare production and resources. Finally, we propose and apply a combined DEA and GP methodology to obtain feasible and optimal targets.

Unlike the usual models for performance evaluation, we integrate two separate methods for the definition of the production technology and the search for an appropriate target on the frontier close to the scenarios proposed by the experts, which are the DEA full dimensional facets model and GP targets. In this way, it is possible to overcome two shortcomings of radial and non-radial DEA models concerning the optimal target, which are: the target is given by the farthest efficient point from the evaluated unit (DMU) and positioned on hyperplanes that often adopt null weights, neglecting several input and output variables. This is further discussed in subsections 2.3 and 2.4.

As we assume that the characterization of the frontier faces and the search for the target are two separate issues in DEA models, two objectives arise: to find faces with desirable characteristics (strictly positive multipliers) and to search for the closest targets on these faces according to an appropriate metric. These objectives are achieved by methods in the search for maximum dimensional faces and for finding closest targets through GP [32]. provide, for the first time, a well-defined efficiency measure based on the closest targets in the DEA. The authors also provide a literature review of methods to search for closer targets. Their method is based on the Russel output measure of technical efficiency and satisfies interesting properties from an economic perspective, such as unit invariance. Lins et al. [33] combine a mixed linear program and an Enhanced Russel Graph Measure (ERGM) to find the closest target on a full dimensional efficient facet.

In summary, the present multimethodology has a concrete application to health care management, exhibits desirable properties, and integrates three methods:

Conceptual maps to structure the problem qualitatively and facilitate metacognitive systemic contextualization, mutual understanding, and participative development of solutions.

The FDEF to yield strictly positive weights assigned to all variables that compose the performance indicator, monitoring all important components of hospital activities inside the planning horizon. FDEF also enables to obtain targets closer to the proposed scenarios, rather than farther away through GP.

GP to approach the productive frontier in a non-oriented and non-radial manner, considering priority demands and concerns from stakeholders, and making allowances for alternative decision-makers preferences.

Structuring the problem by concept maps

The interface between the theoretical approach and applied interventions can be managed by grasping and organizing the qualitative context. Structuring matters, issues, and situations compose one of the stages of modeling at the beginning of the decision-making process, when strategies to address complex problems are evaluated [34]. Thinking maps [35] and concept maps [36] are graphical tools used for knowledge representation that connect two concepts through verbs or linking words, generating a proposition. A graphical representation can be more effective than a text in communicating complex content because the mental processing of images can be less cognitively demanding than verbal processing of a text [37].

When making maps, each shape, line, arrow and image has a specific meaning. Concepts (nodes or cells) are related to each other through connecting phrases (lines), generating propositions [38]. The arrows show the direction of the relationship. In the end, the combination of propositions, also connected to each other through linking sentences, results in a visually appealing diagram that contains a lot of information about the problem to be solved. Other attributes also matter to include even more information, such as positioning of concepts, color and cell size.

The main advantages of using concept maps are: (i) information structuring in a network, similar to the way the brain organizes information, diverse from spoken and written communication; (ii) use of metaphorical sensory intelligence through patterns with spatial arrangement of concepts; (iii) identification of the boundaries of what does not yet exist and can be useful, not just what is present; (iv) joint construction of knowledge, constructivist teaching-learning [38,39]. The greatest challenges of their use are the need to absorb multiple, and sometimes contradictory, perspectives towards the analyzed problem and to turn this subjective information adequate for subsequent quantitative approaches. That is why it is so important to have a discussion facilitator, trained to develop consensus (when possible) and to depict relevant variables that will need analysis and interpretation, even if not used inside the quantitative model.

Health services provided at the hospital level are complex [40] and require a combination of methodologies capable of managing the diversity and dynamism of the system [[41], [42], [43]]. These issues involve decisions on how to better design, plan, and operate systems from the perspective of efficiency [44] and scarce resources [45]. Supporting methods for decision-making constitute the theoretical body of OR, considered as a conjugation of systemic and analytical paradigms [43], as the solution to these challenges cannot be obtained through mathematical modeling alone [24,35].

Visual mapping at the beginning of the planning process has been widely used in healthcare and public health settings to integrate inputs from multiple sources and to generate a framework that can immediately be used to guide action planning, program development, or evaluation and measurement [46]. For example [47], develop concept maps to operationalize performance indicators for addiction treatment centers aligned with the European Foundation for Quality Management, and [48] use concept maps to promote learning in community-based medical education activities.

Integrated quantitative approach DEA-GP

GP is a multi-criteria approach to decision-making that emerged in the 1960s after the failure of the regression model to address problems with several alternative solutions [23]. It consists of one approach to the more general multiple objective linear programming theory. The title ‘goal programming’ was first introduced in the book by Ref. [49].

Data Envelopment Analysis (DEA) was introduced in the journal literature by the highly influential paper of [50] based, however, in the work of [51]. According to Ref. [23]; GP and DEA have complementary features, as GP focuses on management ‘planning,’ whereas DEA focuses on problems in the ‘control’ and evaluation of activities. Despite the different original purposes [52], states that GP and DEA can be used together to generate planning scenarios and assist decision makers at different levels.

It is not a mere coincidence that both methods have the same authors. Both GP and DEA originated in actual applications that were successfully addressed, “what we have elsewhere referred to as ‘applications-driven’ theory” [23]. The ability to consider a variety of objectives is attractive when dealing with managerial decision-making problems [53]. In fact, GP and DEA were developed by the authors adopting strategies that consisted of focusing on practical problems and contemplating multiple objectives, then reduced, through linear expressions, to a single objective problem, to be optimized. This reduction is one way of dealing with multiple objective problems that arise from the approach committed to solve real world problems.

As to the multiple objective linear programming, its DEA counterpart counts on a few works that preserve the original independent multidimensionality, such as [54], who state that multi-objective linear programming (MOLP) can be used in a DEA context because they share the concept of Pareto efficiency. Both models focus on defining efficient/non-dominated faces and aim to identify efficient points in these faces while suggesting projections for inefficient points [55]. As to the formal specification of GP and DEA [23], provides a way to relate the two methods, showing that the ‘additive model’ of DEA has the same structure as a goal programming model in which only ‘one-sided deviations’ are permitted.

[56] published the first work integrating DEA and MOLP, proposing an interactive MOLP (IMOLP) to determine efficient output levels. The process uses DEA to generate an empirical production function to obtain a set of alternative efficient points, allowing DMUs to choose among them [57]. apply a multiple criteria DEA model to improve the classical DEA method to discriminate power and yield more reasonable weightings. The authors evaluate efficiencies under the framework of MOLP, including the classical definition of relative efficiency as one of the criteria; therefore, the DEA solution is contained in the set of MOLP.

[58] propose using multiple-objective linear-fractional programming approaches instead of the standard approach while solving DEA. In this way, more direct information can be obtained for the maximum efficiency of all others DMUs from a set of optimal weights for the observed DMU. The model is applied to simultaneously maximize the efficiency of all DMUs, subject to the constraints of the DEA multiplier model [59]. develop a procedure known as value efficiency analysis that uses a multi-objective model to determine the most preferred solution (MPS). The MPS builds an efficient frontier, which is the most preferred combination of inputs and outputs by the decision-maker, considering an existing or virtual (combination of existing) DMUs.

Multi-objective parameters are effective and necessary in healthcare planning activities [60]. propose an approach using DEA and MOLP to evaluate the efficiency of stroke care services in hospitals, demonstrating a way to transform the generalized DEA dual model to MOLP and locate the MPS for each DMU. Multi-objective parameters have also been used for public health resource allocation [61], human resource allocation [62], and to address medicinal drug distribution to healthcare facilities while simultaneously considering costs and client satisfaction [63].

Full and maximum dimensional efficient faces

DEA practitioners can use full dimensional efficient facets (FDEF) and maximum dimensional efficient faces (MDEF) to obtain references that support strategic decisions. Although the envelope model is dominant in classical DEA approaches, analyzing the multipliers (also known as weights) associated with the resources and results of the production process is essential, as they reveal the importance of these inputs and outputs. In the cases in the literature in which these weights are applied, we found that obtaining strictly positive weights assigned to all inputs and outputs is almost impossible, although there are excellent alternative solutions in dual multiplier models [64]. Zero weighting means that the maximum efficiency was obtained by neglecting some inputs or outputs, which is not reasonable because all resources and products should be considered to evaluate the overall performance [65]. The use of methods, such as weight restrictions, to address this problem is only a palliative solution that is difficult to model and can often generate infeasibility issues [66].

Lins et al. [33] propose that the faces used in the search for targets are only those of maximum dimension, preferably facets, that is, FDEFs, based on the results of Olesen and Petersen (2003, 2015). These authors conclude that such facets provide complete marginal substitution rates, that is, with all multiplier values strictly positive. Olesen & Petersen (2003) propose an algorithm for finding MDEF and, in particular, FDEFs. They use a full dimensional facet, which can be expressed as linear convex combinations of extreme efficient DMUs. A DMU is considered extreme efficient if it cannot be represented as a linear combination (with nonnegative coefficients) of the remaining DMUs [67]. However, datasets with full dimensional facet are uncommon and, when much, only maximum dimensional faces are found. Even so, it's important to perform datasets preprocessing, looking for extreme efficient DMUs that, if possible, belong to FDEFs. The FDEF guarantees that all variables are considered inside the model (weights different from zero); when not all variables can be considered, the MDEF guarantees the influence of the maximum possible number of variables.

The literature on applied FDEF to the healthcare sector is still scarce. Although weight restrictions based on expert judgment have been most studied [[42], [66], [68], [69], [70]], there are three main drawbacks to using this feature:

When imposing a bound value for an unacceptable weight in the optimization model, the weight will assume this exact value;

The supporting hyperplane is rotated such that it no longer corresponds to the face of the polyhedron, supported by a smaller number of DMUs;

Models with restrictions on the virtual weights often generate infeasible solutions. In contrast, full dimensional facets rest on the maximum number of DMUs; thus, they are defined by the maximum number of non-zero variables.

For these reasons, in this study, the efficiency measurement considers all the inputs and outputs without a priori expert judgment; only thereafter, stakeholders’ opinions are incorporated to validate results.

GP-FDEF DEA model: adjusting scenarios based on DEA technologies

We propose a model integrating GP with FDEF/MDEF that is applied to a frontier composed of FDEFs or MDEFs and has two objectives:

External scenario: Provide adjustments for a proposed scenario that is external to the production possibility set (PPS) and cannot reach the constant return to scale (CRS) frontier. The CRS assumption is necessary because the proposed scenarios aim at a significant expansion in the production of services through the acquisition of additional resources regarding the current production technology;

Internal scenario: Ensure an efficient operation for a scenario that is internal to the PPS, feasible, but inefficient, requiring projection onto the CRS frontier.

To achieve these objectives, we formulate a GP model assuming two instances: the first applies to external scenarios and the second to internal scenarios. The mathematical formulation is as follows.

Let be (m × n) matrix and (r × n) matrix composed of n observed extreme efficient DMUs that compose an FDEF/MDEF, where m is the number of inputs and r is the number of outputs.

Let be (m × 1) vector and (r × 1) vector defined for a desired scenario proposed by the managers.

Let be the weight assigned to the efficient DMU j.

is the ith input and the kth output of a feasible target that approaches a desired scenario based on the CRS technology. Differences are indicated by theinputs slacks () and the outputs slacks (). Thus, we can solve the GP model to find the optimal λ values that yield an efficient target in an FDEF/MDEF:

To account for the first instance, an external scenario, we minimize , where and are arbitrary vectors of weights assigned to the slack vectors , respectively. The restrictions aim to avoid increasing inputs and/or decreasing outputs when the proposed external scenario is projected onto the CRS frontier. However, this may be necessary to reach the frontier.

To account for the second instance, an internal scenario, we minimize , where and are arbitrary vectors of weights assigned to slack vectors , respectively. It is intended to guarantee that the internal scenario reaches the frontier. Note that all slacks must be understood as proportional changes in the scenario vector.

Therefore, we can obtain a feasible target for a scenario that is closer to the externally proposed one, minimizing any increase in inputs or decreasing outputs. We also obtain a target that is closer to an internal scenario. Alternatively, we can provide a delimiter that guarantees the achievement of the first objective followed by the pursuit of the second objective. This can be provided by a small number in (2), which corresponds to a mix of weighted and preemptive approaches in GP.

In the case of multiple FDEFs/MDEFs, we apply model (1) and obtain targets for each facet/face; thereafter, we choose the one that presents the closest distance to the original scenario according to the optimal value for the objective function. This is illustrated in the following example. The database in Table 1 consists of four DMUs (A, B, C, and D), two inputs: full time equivalent (FTE) physicians and FTE nurses, and one output (consultations). We consider two scenarios: one internal (Sc1) and another external (Sc2) to the CRS frontier and to the PPS.

Table 1

Dataset.

Variable	A	B	C	D	Sc1	Sc2
FTEa Physicians (I)	8000	10,000	15,000	7000	9000	16,000
FTEa Nurses (I)	12,000	9000	7500	16,500	16,000	10,000
Consultations (O)	30,000	30,000	30,000	30,000	32,000	45,000

Full time equivalent (weekly hours).

The results from the application of model (1) to the three-dimensional CRS problem can be observed in a two-dimensional frontier (Fig. 1 ), where the inputs are the ratios of FTE physicians to consultations and FTE nurses to consultations. Table 2 shows the results of the projection of the internal scenario on faces A-D, A-B, and B–C given by the vectors and distances. The target for scenario 1 (TS1) was chosen because it presented the shortest distance to the CRS frontier (Fig. 1). The same procedure holds for external scenario 2, yielding target TS2 (see Table 2).

Fig. 1

Frontier and proposed scenarios (Sc).

Table 2

Results from adjusting the desired scenario.

Variable	Projections of Sc1			Projections of Sc2
	Face A-D (TS1)	Face A-B (A)	Face B–C (B)	Face A-D (A)	Face A-B (B)	Face B–C (TS2)
FTE Physicians (I)	9000	9000	10,667	9000	10,667	16,000
FTE Nurses (I)	16,000	13,500	9600	13,500	9600	10,000
Consultations (O)	35,313	33,750	32,000	33,750	32,000	37,000
Distance (Objective function)	0.10	0.21	0.58	0.78	0.22	0.18

HUCFF case study

Understanding the problem: qualitative approach through concept maps

To obtain a feasible and efficient production scenario following the pandemic, the HUCFF board of health directors, supported by other health stakeholders (HSs), requested formal OR consultancy within the university. Interdisciplinary dialogue was driven by systematic meetings and concept maps were constructed to structure the problem, generate future production estimates and validate results.

The HSs team was composed of one representative of each of the following management sectors: medical, nursing, epidemiology, financing, informatics, teaching-research and human resources. The OR team was composed of three representatives from the Production Engineering Department of the university. There were weekly meetings with all members for two months, guided by the OR facilitator, responsible for explaining the multimethodological approach, making a sketch map during each meeting and validating the resulting complete map. The latter was especially important to establish priorities inside the group and to mitigate any lack of consensus (by compiling different points of view and voting, when necessary). The critical issues here are: how to increase resources and products maintaining satisfactory levels of efficiency, how to deal with the temporarily acquired human resources and how to plan health care provision without affecting the quality of teaching and research activities. After structuring the problem, the quantitative model was run and results were brought back to be validated, considering not only the quantitative figures, but also the context of qualitative perspectives.

Fig. 2 shows the operational framework of the hospital impacted by the pandemic (during 2020 and 2021), with a view to planning the production profile for the next year, 2022. It aims to present evidence-based justifications to the government's authority for the maintenance of HHRs acquired during the pandemic.

Fig. 2

Demand planning using expanded resources, 2020–2021.

Fig. 3 shows the concept map with both perspectives (HSs and OR) for the elements that influence the production capacity of a hospital. Based on the local information system, a set of 51 variables was first highlighted by the HSs to encompass the main dimensions of a university hospital: health care, teaching, and research. Each of them requires a specific structure (input) to guarantee quality in production (outputs). The dimensions also have connections: medical residents are inputs for health care provision and outputs for teaching; admissions are baseline (input) for research and output for health care; and research projects are inputs for teaching and output for research. These connections have already been modelled in a direct manner at university hospitals using the network DEA approach [31].

Fig. 3

HSs and OR analysts' perspective.

For HSs, there is consensus that health care adequate structure (inputs), process and results (outputs) provide the environment for teaching and research quality. According to stakeholders, health care provision is the main source of hospital financing, as teaching and research budget is directly paid to university academic departments and have no influence on routine hospital expenditures (human resources, equipment, medical devices). Moreover, the usual parameters of the Ministry of Health (MoH) adopted to plan health care provision do not consider the influence of teaching and research dimensions on costs and production profiles.

OR analysts can contribute to understanding the productive structure that relates inputs and outputs in the health care dimension. The first challenge was to define a feasible PPS based on the past information of the institution (pre-processed information), as there were fewer resources before the pandemic. The proposed methodology consists of the GP-FDEF DEA model (1) for two different scenarios: with and without the maintenance of the temporarily hired HHRs. It is noteworthy that intrahospital comparisons were chosen because there were no reliable HHR data published for other peer institutions (which would enable interhospital comparisons).

Preparing a quantitative model: variables, scenarios, and efficient production frontier

Two desired Scenarios were developed to reflect the opinions of managers from the perspective of hospital operation in terms of the monthly production vectors containing the same set of inputs and outputs. Scenarios 1 and 2 adopt actual values for the inputs, as observed before and after the pandemic, respectively, with the maintenance of temporary HHR contracts. For HHR workload, HSs discounted 20% of its value, considering retirements and activities not related to health care (teaching and research).

With regard to outputs, both scenarios considered the MoH parameters used to evaluate and fund the core activities developed in general hospitals (admissions, surgeries, outpatient consults) [71]. As university hospitals are responsible for public tertiary high-complexity procedures within the Brazilian Health System, admissions were divided into high and medium complexity. In the absence of case-mix index information that shifts and shapes the frontier [72], the high-complexity admission variable is based on a list of procedures paid separately by the MoH (such as transplants, cardiovascular surgery, and neurosurgery), surpassing the maximum expenditure. Estimates for MoH parameters consider, on a monthly basis: for admission, bed turnover rates of 2.8 and 40% admissions for high complexity procedures; for surgeries, three surgeries per room on working days; for outpatient consultations, 14 consults per consulting room on working days.

To develop an efficient production function to project the desired estimates provided by HSs (based on MoH parameters), we used a database containing the same set of input and output vectors for 120 months of operation (from January/2010 to December/2019), where each month was a DMU. Note that this timeframe with production characteristics is not related to epidemics, as expected for production after pandemic. After running the quantitative model, we expected to be able to answer the following questions: 1) How efficient are the MoH estimates for planning productivity after pandemics? and 2) How would the maintenance of HR temporary contracts affect the quality of health care, teaching, and research? These answers should be turned into assumptions for government agencies that provide temporary contracts and funding to the hospital.

The DEA variables for both proposed Scenarios are presented in Table 3 , with output estimates based on the MoH parameters. It was agreed that the dataset should emphasize healthcare provision variables, not only because they are the most relevant source of hospital financing, but also because they are directly related to the health care resources acquired during pandemics.

Table 3

Variables used in the DEA model and proposed scenarios by HSs (based on MoH parameters).

Types	Variables	Scenario 1: End of Contracts	Scenario 2: Contracts Maintained
Inputs	Beds (I)	265	318
	ICU beds (I)	16	40
	Surgery Rooms (I)	7	13
	FTE Physiciansa (I)	9253	11,581
	FTE Nursesa (I)	6272	10,376
	FTE Nursing Assistants (I)	20,816	28,496
Outputs	Admissionsb - MCc (O)	472	601
	Admissionsb - HCd (O)	315	401
	Surgeriesb (O)	462	858
	Outpatient Consultationsb (O)	29,413	29,413

Sum of FTE HHR (Human Health Resources), weekly hours.

Monthly production.

Medium Complexity.

High Complexity.

Results from the quantitative model: GP–FDEF DEA model

Table 4 presents descriptive statistics for the 120-month dataset, where the standard deviation (SD) is notably small compared to the mean for HHR (coefficient of variation less than 10%, which means low human resources turnover due to permanent contracts before the pandemic) and large for beds, surgery rooms, and outputs (coefficient of variation around 25%). The range between minimum and maximum observed monthly values is consistent with this variation. Variations for beds and surgery rooms were due to managerial decisions on maintenance works in the physical facilities of the hospital, periods of financing crisis and epidemiological pressures (as dengue and H1N1 outbreaks). Such situations impacted the output production. As there were no layoffs during these periods, HHRs were dislocated for other activities (teaching and research) and/or other hospitals of the university complex.

Table 4

Dataset: descriptive statistics and CRS benchmarks.

Variables	Descriptive Statistics
	Average	SD	Maximum	Minimum
Beds (I)	254	23	302	98
ICU beds (I)	26	6	36	16
Surgery Rooms (I)	9	2	13	0
FTE Physicians (I)	10,771	388	11,795	10,062
FTE Nurses (I)	6285	360	6848	5536
FTE Nursing Assistants (I)	31,319	2117	38,688	28,936
Admissions – MC (O)	486	99	803	66
Admissions – HC (O)	163	37	262	18
Surgeries (O)	384	92	611	3
Outpatient Consultations (O)	17,855	3211	29,413	3756

Table 5 displays the observed MDEF benchmarks used to build the CRS technology-DEA frontier. No facet defined by the strictly positive weights applied to every input and output was found, which is a requirement for it to be a FDEF. Therefore, we looked for MDEFs and found only one, supported by eight extreme-efficient DMUs (benchmarks) shown in Table 5. They are used to build the CRS technology-DEA frontier according to model (1). Regarding the peer group for the adjusted scenarios, the results were as follows: March 2017 (λ = 0.21), May 2019 (λ = 0.56), and July 2019 (λ = 0.06) for Scenario 1, and March 2010 (λ = 0.46) and March 2017 (λ = 0.73) for Scenario 2. Both scenarios emphasize slacks that provide a decrease in outputs ( and an increase in inputs (, indicating that the scenarios are located outside the CRS PPS.

Table 5

MDEF benchmarks in a CRS technology frontier.

Variables	Benchmarks
	March 2010	July 2011	March 2017	August 2018	May 2019	July 2019	August 2019	October 2019	Average	SD
Beds (I)	281	302	258	267	264	266	268	265	271	13
ICU beds (I)	28	29	28	17	16	16	16	16	21	6
Surgery Rooms (I)	13	10	10	10	8	10	10	11	10	1
FTE Physicians (I)	10,701	10,794	10,143	10,981	11,342	11,584	11,600	11,776	11,115	522
FTE Nurses (I)	6688	6784	5920	5888	6112	6400	6400	6464	6332	311
FTE Nursing Assistants (I)	32,088	38,624	30,088	29,704	30,088	30,472	30,472	30,536	31,509	2769
Admissions - MC (O)	694	803	554	588	576	599	610	634	632	76
Admissions - HC (O)	233	194	262	194	182	188	187	176	202	28
Surgeries (O)	564	539	461	437	414	484	512	515	491	48
Outpatient Consultations (O)	26,358	19,350	18,938	17,242	20,716	20,322	18,861	22,622	20,551	2644

Table 6 compares the planned (by MoH parameters) and forecasted (by projection onto the GP-FDEF frontier) values for each Scenario. The GP model for the MDEF technology, namely, the forecasted amounts for inputs and outputs in both scenarios, promotes a feasible and efficient solution in the CRS PPS and guarantees that most variables influence the model results. Efficiency, one of the seven pillars of quality, along with feasibility and steadiness of planning parameters, is a manner to ensure the quality of health care and its interrelated dimensions: teaching and research [73].

Table 6

Planned and forecasted amounts according to the GP-FDEF model.

Variables	Scenario 1 - End of Contracts		Scenario 2 - Contracts Maintained
	HSs Planned	GP-FDEF Adjusted	HSs Planned	GP-FDEF Adjusted
Beds (I)	265	220	318	318
ICU beds (I)	16	16	40	33
Surgery Rooms (I)	7	7	13	13
FTE Physicians (I)	9253	9253	11,581	12,342
FTE Nurses (I)	6272	5093	10,376	7408
FTE Nursing Assistants (I)	20,816	25,220	28,496	36,770
Admissions - MC (O)	472	479	601	725
Admissions - HC (O)	315	170	401	299
Surgeries (O)	462	361	858	597
Outpatient Consultations (O)	29,413	16,942	29,413	25,987

Numerical comparisons are shown in Fig. 4, Fig. 5 for Scenarios 1 and 2, respectively. As the maintenance of HHR contracts is a cut-off point for defining scenarios, even before the pandemic/Scenario 1, there was an excess of 39 nurses and a deficit of 110 nursing assistants. Without the maintenance of HHR contracts, a feasible production framework consists of approximately 650 admissions (26% high complexity), 360 surgeries, and 17,000 outpatient visits per month. Compared to the statistics in Table 4, all variables from Scenario 1 are below the MDEF reference average value and can fit in the internal scenario of a classical CRS PPS.

Fig. 4

Scenario 1: Stakeholder planning (MoH parameters) versus GP-FDEF forecast variables (beds sum ICU and others).

Fig. 5

Scenario 2: Stakeholder planning (MoH parameters) versus GP-FDEF forecast variables (beds sum ICU and others).

The values for Scenario 2 consider the maintenance of hired HHRs after the pandemic and define parameters to negotiate with the government authority. After the pandemic, a reduction in ICU beds from 40 to 33 is justified as the demand by COVID-19 patients for ICU beds decreases. An excess of nurses (99 of 171 hired during the pandemic) along with a deficit of 207 nursing assistants are still observed. All the resources, HHRs included, are compatible with a reasonable production of approximately 1000 admissions (27.5% high complexity), 600 surgeries, and 26,000 outpatient visits per month. Compared to the benchmarks from Table 4, these new parameters are higher than the reference average values pertaining to the proposed scenario that is external to the CRS PPS and prone to absorb new health care, teaching, and research needs when it reaches the CRS frontier.

Fig. 6 shows a comparison of the relative adjustments for both Scenarios to attain feasibility. On the resource side, both scenarios recommend 20–30% reduction in nurse workload and 20–30% increment in nursing assistant workload. Scenario 1 suggests a reduction in the number of beds (not in the ICU), whereas Scenario 2 suggests a reduction in ICU beds. In both Scenarios, the proposed surgery room was maintained. On the product side, medium-complexity admissions should be increased by 20% in scenario 2, but all other outputs should decrease in both scenarios. In other words, the HSs proposal based on MoH parameters overestimated the viable production framework, partly because certain production parameters of teaching hospitals differ from those of other general hospitals (e.g., bed turnover).

Fig. 6

Comparison of adjustments between scenarios.

Discussion & final comments

Post-pandemic health system reorganization is acknowledged to be a complex world problem, because all countries, states, cities, and hospitals have to manage new structures developed during the pandemic, when all efforts were directed to treating COVID-19 patients. Given the contemporary multidimensional vision of public health, sanitary authorities and health organizations struggle to deal with the many health-related, social, environmental, planning and sustainability consequences of the pandemic in order to ensure universal and equitable care for all and to avoid further epidemiological surprises [74]. As a starting point, a shift is required to resume health care focusing on cardiovascular diseases, cancer and other illnesses that could not be fully addressed during the pandemic. It is not known exactly how much COVID-19 has harmed or delayed the countries’ possibility to accomplish SDG 3, but some inequality is expected since low per capita income countries had more difficulty in organizing the system and promoting massive vaccination during the pandemic. In order to monitor the accomplishment of United Nation's SDG indicators by the WHO Member States, Pereira et al. [75] propose an approach, based on directional distance function, to monitor convergence of productivity, by examining if units get closer to the best practices or if the gap between the frontiers of the best and worst performers decreases over time. The authors use composite indicators that aggregate diverse SDG health goals, but different clusters of indicators can be chosen to monitor specific dimensions of health and well-being, such as the use of chronic non-communicable and social indicators, since there are lots of sociodemographic influences on chronic-degenerative disease prevalence [76]. Finally, there is an expectation on the part of hospital managers to expand access and improve performance to meet a repressed demand. Some studies have indicated that improving technical efficiency does not harm access to healthcare [77].

This study presents a dual contribution to the literature on hospital productivity after the pandemic. First, it provides an evidence-based application that helps HSs plan their future activities after the pandemic and negotiate the maintenance of HHR contracts with government authorities based on benchmarks. Secondly, it uses an innovative mathematical solution regarding the production frontier to support the decision in the acquisition of new resources on a MDEF based CRS PPS, which also ensures the participation of most variables within a maximum dimensional face.

In health applications, Hollingsworth's review (2008) on productivity demonstrates that much work undertaken and published in this area is of the ‘have software – will analyze’ nature (or have data – must analyze), with little thought to model specification, and no face validity [78]. In a more recent revision on DEA hospital applications [79], presents four clusters of purposes for publication: a) pure DEA efficiency analysis (sometimes, the first hospital performance analysis observed in a country, as in Ref. [80]; b) developments or applications of new methodologies (network DEA, fuzzy frontiers), as in Ref. [81]; c) specific management questions (ownership, production planning, quality, regulation), as in Ref. [82]; d) surveys on the effects of reforms (impact of health systems reforms, financing), as in de Castro [83]. The present paper mainly considers the third cluster, given the management questions related to the efficient utilization of new structural changes and the need to maintain the temporarily hired human resources. To answer these questions, an initial qualitative structuring is pivotal. The danger of the lack of qualitative contextualization is that misinterpreted or overlooked data on efficiency’ may lead to policy decisions based on potentially unreliable information, with potentially disastrous consequences. The qualitative approach that preceded the mathematical model was important for evaluating the big picture and depicting other variables and dimensions of care that influence healthcare quality, even if not included in the quantitative model. The usefulness of such qualitative approaches to understand relevant variables not included within models is discussed by Lins et al. (2021) and Jahara & Lins [41]. This was the case for the hospital mortality rate (HMR), with almost no oscillations over the 120 months analyzed (around 6.0%); thus, incapable of discriminating efficient DMUs. HMR is a well-known and the most frequently monitored indicator for hospital quality of care, although influenced by the severity of cases (usually higher in teaching hospitals) [84,85].

As indicated by the concept maps (Fig. 3), university hospital quality indicators may be represented by the health care dimension and also by teaching and research [86]. The teaching dimension depends on the healthcare structure, such as teaching dedication (number of residents per physician) and intensity (number of residents per bed) [85]. The main teaching predictors of health care efficiency at university hospitals are the size of the hospital, high teaching intensity, and low teaching dedication (Lobo et al., 2014). Research efficiency also relies on the healthcare structure, as it has been correlated with a high complexity volume [87]. Concept maps highlight flexible variables that can be used as inputs or outputs depending on the dimension being assessed. This is the case for residents (input for health care, output for teaching), admission (input for research, output for health care), and research projects (input for teaching, output for research). Addressing the multiple connections between variables and dimensions is important to evaluate the magnitude of a complex system in a university hospital. Considering these connections inside healthcare facilities by network DEA approaches would acknowledge the internal structure of the DMU and generate efficiency scores both for each DMU and for each subsector or dimension analyzed. It would also facilitate the planning of the system and its subsystems and the identification of target areas for improvement [71,88].

Although not fully explored in this study, new concept maps were used after the quantitative model was run to discuss the limitations of the study, validate the results, and propose future studies. The major limitation of the quantitative model is related to the fact that the reference frontier only considered intrahospital comparisons given the lack of information from other peers. A DEA-specific limitation is the absence of a FDEF, which is expected in most datasets. Interhospital comparisons could also contribute to increasing the probability of finding an FDEF. As previously mentioned, there was a lack of traditional quality variables inside the model, although the findings on trade-offs between quantity and quality have been quite controversial [77]. To overcome these problems, the authors intend to develop new models using information and data from other teaching and/or university hospitals to guarantee peer homogeneity. In addition, the quality, teaching, and research variables will be introduced. The idea is to use network DEA to examine the mutual relationship between these variables and dimensions.

In the HUCFF case study, the results showed that the desired production scenario proposed by stakeholders based on MoH parameters overestimated the viable production framework; moreover, the scenario that maintains temporary HHR contracts is more compatible with quality parameters in health provision, teaching, and research. These findings were turned into assumptions and new parameters were presented to the governmental agency that provided equipment and temporary contracts.

The observed difference between MoH and forecasted parameters is partly attributed to the specificities of university hospitals, where supply consumption and timing for procedures are higher for teaching activities [85,89]. MoH parameters only consider general hospital outputs and not teaching statuses. In addition to the fact that HHR uses contracted time to perform other activities (teaching and research), university hospitals health care activities alone can be 9–33% more costly than their counterparts [[90], [91], [92]]. Moreover, university hospitals are references to severe cases and comorbidities, performing high complexity procedures that are more expensive and time-consuming [42,71]. Based on the feasible forecasted production profile in Scenario 2, in which HHR contracts are maintained, new health care parameters proposed to the government agency should be bed turnover, 3.2; daily surgery per operating room, 2.0; proportion of high-complexity admissions, 27.5%.

In addition, Scenario 2 is well suited to accommodate the present number of residents, students, and medical teachers (approximately 200, 1,000, and 300, respectively), meaning 1.5 beds for each resident, 3.0 students per bed, and almost 1.0 medical teacher per bed. Teaching dedication equals 0.24 and teaching intensity equals 0.60 [[87], [89]]. Concerning research and its correlation with high-complexity procedures in university tertiary hospitals, the proposed parameter is 1.1 high-complexity admissions per medical teacher [42]. An alternative scenario without acquired HHRs is insufficient to achieve such teaching and research parameters.

Another contribution of the present research and consulting project is the systematic continuation of interdisciplinary activities. The development of OR sectors in hospitals has been proposed [[42], [78]]. Continued dialogue between HSs and OR analysts would solve several issues that arise on a daily basis inside the hospital, such as the organization lines for patients, improvement of logistic processes, and scheduling for surgery rooms among specialties. For each problem, the higher the multimethodological approach, the better the solution. For the specific issue considered in this study, meetings are still ongoing to gain more insights, deepen the methodology, and share mutual expertise among the involved disciplines. The authors believe that the application of multimethodological approaches and quantitative models can help other hospitals, cities, states, and countries develop new parameters for health care inside university hospitals.

Credit author statement

Lobo, Maria Stella de Castro: Conceptualization, Methodology, Validation, Writing - Original Draft, Project administration; Estellita Lins, Marcos Pereira: Conceptualization, Methodology, Formal analysis, Critical Review, Supervision; Rodrigues, Henrique de Castro: Conceptualization, Methodology, Resources, Investigation, Data Curation, Review & Editing; Soares, Gabriel Martins: Conceptualization, Methodology, Formal analysis, Review & Editing.

Declarations of competing interest

None.