Planning for Hospital IT Implementation: A New Look at the Business Case.

Compared to organizations in other industries, hospitals are slow to adopt information technology (IT). Those planning for system implementation must understand the barriers to IT adoption which, in healthcare, include the relatively high acquisition and maintenance costs of sophisticated administrative and clinical information systems. Understanding the overall business case is particularly important for hospital IT planners. This paper describes the literature that examines benefits from using health IT. In addition, we focus on a series of studies conducted in Florida that provide generalizable evidence regarding the overall business case associated with hospital adoption for information systems. These studies focus broadly on the improved financial, operational, and clinical performance associated with IT.

Introduction

Despite revolutionary changes to business practices in most industries, one major industry has noticeably lagged behind on the IT adoption curve. The healthcare industry, which makes up over 16% of the U.S.’s gross domestic product (GDP) has been criticized for being decades behind with respect to investments in IT (Chin, 2004; Raghupathi and Tan, 1999; Dornfest, 2000). Periodically, the information intensive nature of healthcare practice makes it a particularly good candidate for the benefits of computer automation. However, this same information intensive nature also makes designing and implementing information systems more challenging.

Hospitals have traditionally adopted IT for administrative functions (e.g. cost accounting, billing, etc.) more rapidly than for clinical or patient safety uses (Burke et al. 2002). This trend reflects the relative ease of establishing quantifiable returns on investments in administrative IT applications. Researchers and healthcare leaders alike have found it difficult to measure the economic value associated with less tangible benefits such as higher quality of care, improved patient safety, system-wide benefits, and enhanced competitive advantage. As a result, understanding the complete business case from IT becomes more challenging for those planning for IT implementations in hospitals.

Strategic plans for IT implementation in hospitals typically consider several barriers including resistance to training and use among physicians (Ball, 1992), concern over application incompatibility (McDonald, 1997; Dick and Andrew, 1996), the lack of strategic fit between IT and business strategy (Henderson and Thomas, 1992), and the lack of uniform data standards (Aspden et al. 2003). However, none of the barriers have been as important as financial constraints (Blair and Hilts, 2003; Erstad, 2003; Johnson, 2001) which are exasperated by a dearth of evidence that examines the overall business case associated with IT in healthcare settings. In non-healthcare settings, researchers have documented financial and operational performance increases associated with IT. But in healthcare, theoretical and anecdotal benefits of IT have been the primary forms of research. The theoretical foundation has been thoroughly discussed by medical experts and the prestigious Institute of Medicine (Bates and Gawande, 2003; Institute of Medicine, 2001). An overview of the empirical work on IT to-date is contained in Table 1.

Table 1

Abbreviated literature review of IT benefits.

Date	Author	IT studied	Sample/site/methods	Performance measure	Key finding
2004	(Barlow et al. 2004)	EHR	59-physician group with practices in nine locations treating 200,000 active patients	Financial Savings	The practice experienced an increase in net revenue of $952,000 compared with the prior year.
1999	(Bates et al. 1999a)	CPOE for lab tests	A large teaching hospital using a randomized control trial of lab entries (N = 77,609).	Lab orders—In the control group, 51% of ordered redundant tests were performed, whereas in the intervention group only 27% of ordered redundant tests were performed (P<0.001).	The estimated annual savings in laboratory charges was $35,000, which is less than the cost of the system. However, there were several improvements to the system that may yield a positive ROI
1998	(Bates et al. 1998)	CPOE	A stratified random sample of 6 medical and surgical units in a tertiary care hospital over a 6-month period	Medication errors—In the control units’ errors decreased 55%, from 10.7 events per 1000 patient-days to 4.86 events per 1000 (P = 0.01).	Physician computer order entry decreased the rate of nonintercepted serious medication errors by more than half, although this decrease was larger for potential ADEs than for errors that actually resulted in an ADE.
1999	(Bates et al. 1999b)	CPOE	Three medical units in a tertiary care hospital were studied for seven to ten-week periods in four different years.	Medication errors, excluding missed dose errors.	Large differences were seen for all main types of medication errors: dose errors, frequency errors, route errors, substitution errors, and allergies.
		HIT	A meta-analysis of prior studies into hospitals’ ROI from HIT		Higher investment in IT improves hospital business performance
2004	(Cooper, 2004)	EHR	A case study of a pediatric practice	15 business-related objectives	The EHR improved quality of patient care, office efficiency and patient safety. The physician felt the ROI was adequately demonstrated.
2001	(Dexter et al. 2001)	CDSS	A randomized trial of physicians using a computerized system processed on-line information for all 6371 patients admitted to a general-medicine service (for a total of 10,065 hospitalizations), generating preventive care reminders as appropriate.	Preventive measures that had not been ordered by the admitting physician.	A majority of hospitalized patients in this study were eligible for preventive measures, and computerized reminders significantly increased the rate of delivery of such therapies.
2004	(Ewing and Cusick, 2004)	EHR	A case study of one clinic.	Reduction in administrative costs	Within two years of implementing the system, the organization was able to significantly reduce the amount of paper filed in patient records, improve provider productivity and efficiency, reduce the potential for adverse drug events by automating prescriptions, and improve patient and provider satisfaction.
2008	(Housman et al. 2008)	IT	Analysis of secondary data on U.S. hospitals. A full dataset on 2,000 was attained.	The researchers created a IT Capital Index based on the number of applications adopted and their prevailing price.	Higher levels of IT investment were found to improve:
					1. Business performance;2. There is a minimum IT investment threshold required to receive any benefit;3. IT investments should be able to pay for themselves;4. Hospitals with higher levels of IT investment engage in cost-reducing activities more actively.
2006	(Kaushal et al. 2006)	CPOE	Cost and benefit estimates of a hospital CPOE system at Brigham and Women’s Hospital	Financial savings	Over ten years, the system saved BWH $28.5 million for cumulative net savings of $16.7 million and net operating budget savings of $9.5 million given the institutional 80% prospective reimbursement rate.
2007	(Meyer et al. 2007)	HIT	The study concerns 17 university hospitals within the Assistance Publique Hopitaux de Paris group	Production efficiency	In a general production function, hospital costs appear to be positively connected to the level of IT expenses, capital and labor.
2008	(Meyer and Degoulet, 2008)	HIT	Meta-analysis of HITs impact on efficiency and quality.	1. Studies’ abilities to measure ROI;2. Studies’ abilities to measure productivity;3. Studies’ abilities to measure quality.	Financial and accounting methods can provide interesting data on a specific IT project but are usually incomplete for revealing the global IT investment influence. Econometric methods tend to demonstrate the positive impact of IT on hospital productivity. Hospitals having higher levels of IT investment tend to deliver higher levels of clinical quality.
2004	(McMullin et al. 2004)	CDSS	A retrospective cohort study was performed using a pharmacy claims database. Clinicians using the CDSS were matched with a control group by pharmacy billed amount, number of patients treated, and number of new prescriptions filled during a 6- month baseline period in which neither group used the system.	Prescription Ordering Costs	Clinicians who received evidence-based messages had significantly lower pre- scription costs than those in the control group. The average cost per new pre- scription was 4.16 dollars lower (P = 0.02) in the intervention group, and the average cost for new and refilled prescriptions was 4.99 dollars lower (P = 0.01).
2000	(Teich et al. 2000)	CPOE	A time series analysis was performed at an urban academic medical center at which all adult inpatient orders are entered through a computerized system.	Use of a computerized guideline	Use of all of the recommendations studied increased significantly among the study cohort.
2003	(Wang et al. 2003)	EHR	The researchers performed a cost-benefit study to analyze the financial effects of electronic medical record systems in ambulatory primary care settings from the perspective of the health care organization.	Net financial benefit	The estimated net benefit from using an electronic medical record for a 5-year period was 86,400 U.S. dollars per provider.

When planning for IT implementations, resistance in hospitals typically comes from the misalignment of incentives (Scott, 2005). Many potential benefits of IT are intangible and/or do not contribute directly to the bottom line in readily quantifiable ways. Therefore, a complete understanding of the overall business case is necessary when properly planning for IT. This review is intended to assist hospital planners in understanding the business case associated with investments in IT.

This paper begins with a brief discussion of the literature that studied the benefits from IT use in hospitals. Many of these studies identified significant benefits from IT use; however, the existing studies frequently focused on individual IT applications (e.g. scheduling and billing software) in a unique setting (e.g. academic medical center or psychiatric facility). As a result, the utility of these studies to hospital administrators has been questioned (Chaudhry et al. 2006). To help the reader understand the challenges in conducting research across institutions, we will discuss the difficulties in measuring ‘IT sophistication’ at the hospital level and describe a method that was designed to facilitate comparison of performance among hospitals with differing levels of IT adoption. Lastly, we will describe the growing literature based on a relatively large sample of Florida hospitals that examined the financial, operational, and clinical benefits associated with IT adoption.

Positive Effect from IT

While other industries have reaped the benefits of their IT investments, the adoption and diffusion of information systems in the healthcare arena has been growing more slowly. Nevertheless, the literature that examines the benefits from the use of IT in healthcare has been growing rapidly. For example, Meyer et al. (2007) found that hospitals’ productivity increases with increased use of IT. Generally speaking, the goals of clinical IT systems are simple: get the right information for the right patient to the right provider at the right time. In assessing the positive aspects of IT adoption in healthcare, it is necessary to address the difference in IT usage within the health field. Adoption of IT in healthcare is typically in response to support clinical, administrative, or strategic organizational needs. Clinical benefits include improvements in the quality and safety of care; whereas, administrative benefits reflects increased efficiency and lower costs. Strategic benefits include those that assist the overall organization in attaining long term goals. It is important to note that the three sets of benefits are not mutually exclusive.

Among the many types of information systems used in healthcare, a few clinical systems have received the most attention. These include electronic medical record (EMR) systems, computerized physician order entry (CPOE) systems, and clinical decision support systems (CDSS).

Electronic medical records systems

EMR systems, initially referred to as ‘computerized patient records’ or ‘electronic patient charts,’ are designed to improve access to patient’s records, providing timely information to reduce errors and improve diagnosis. As computers became faster and smaller, the possibility of having EMR systems at every patient bedside has become more feasible. EMR systems provide immediate access to patient histories, allergies, prescriptions, current complaints, and a set of templates for treatment protocols and coding support for electronic billing. The benefits from EMR adoption are well documented. They include reduction in information duplication (Ewing and Cusick, 2004), lower drug interaction errors (Lipton et al. 2003), improved utilization of lab and radiology results (Wang et al. 2003; Ewing and Cusick, 2004), accurate coding and billing (Schmitt and Wofford, 2002; Menachemi and Brooks, 2006), and faster access to patient records (Sandrick, 1998; Wang et al. 2003).

Financial benefits to EMR use have also been well-documented. For example, the estimated net benefit from using an electronic medical record for a five-year period was $86, 400 per provider (Wang et al. 2003). Moreover, a pre- and post- analysis of EMR deployment in a large outpatient clinic found that the system was associated with direct reductions in spending and increases in revenue during the study period. Specifically they reported a first year savings of almost $1 million directly attributable to the EMR system. The overall savings was realized by reducing transcription expenses, increasing revenues due to improved billing coding and eliminating the need for extra physical space required to store large quantities of paper charts (Barlow et al. 2004). Lastly, a study of the economic effects of EMR implementation in a solo physician practice also yielded positive results (Cooper, 2004). In that study, the medical practice saw an increase in patient visits without adding new staff and was able to increase revenues by 271% resulting in an increase of physician profit of 102%.

Computerized physician order entry systems

CPOE systems are electronic versions of medication and test ordering systems. They replace handwritten notes and prescriptions and help streamline the processes around order entry. The use of CPOE in reducing errors during the medical testing and prescription ordering phase of care has been established with success (Bates et al. 1999a; Teich et al. 2000). So much so, that most advanced teaching hospitals employ such systems to help manage their patient loads. CPOE systems offer complete and accurate drug information, automatic dose calculations, and appropriate clinical decision support at the point of care. Decision support could included such items as checking for drug–drug interaction, allergy checking, and access to the latest evidence-based practice protocols (Mekhjian et al. 2002).

The benefits from CPOE have been well-documented (Kaushal et al. 2006). CPOE has been successfully implemented in long-term care facilities to reduce adverse events and prescribing errors (Kaushal and Bates, 2002; Rochon et al. 2005). The use of guidelines and dose selection functions present in CPOE systems have shown to significantly increase adherence to prescribing regimens that then translate into improved patient safety and lower costs (Teich et al. 2000).

In empirical studies, the implementation of CPOE systems resulted in a 55 percent reduction in serious medication-related errors (Bates et al. 1998) and in a similar study that included higher levels of support for clinical decisions, the results showed an 83 percent reduction in the overall rate of medication errors (Bates et al. 1999b). In summary, CPOE systems have demonstrated error reduction (Menachemi and Brooks, 2006; Teich et al. 1999), improved compliance to formularies (Lipton et al. 2003; Dexter et al. 2001), and standardization of order processing (Aspden, 2004).

Clinical decision support systems

CDSS are directly linked to evidence based care since the protocols and order sets are generated by reviewing previous literature. While evidence-based medicine has increasingly broad-based support in health care, it remains difficult to get physicians to actually practice it (Bates et al. 2003). Evidence based medicine resources are now more portable and easily accessible with CDSS available on personal digital assistants (Ray et al. 2006). The use of best practices based on levels of evidence from randomized clinical trials and extensive meta-analysis of the literature has made care protocols less variable.

CDSS have been providing benefits to adopters. Access to evidence based literature, including meta-analyses provide physicians with quality, accurate information that has been evaluated by experts. CDSS has been shown to reduce hospital length of stay (Chaiken, 2003), decrease prescription costs (McMullin et al. 2004) and decrease medication errors (Bates et al. 1998; Bates et al. 1999b).

Literature Gaps in Understanding and Demonstrating IT Benefits

For those planning for IT implementation in hospitals, the above literature provides a clear pattern of consistency. However, in a systematic review of the literature, Chaudhry et al. (2006) suggested that most of the studies on the benefits of IT have emanated from several select organizations with decades of commitment to IT and robust, homegrown systems. Moreover, those planning for IT, typically focus broadly on their organization’s overall IT capabilities in meeting organizational goals. Thus, the literature that focuses exclusively on a single IT application (e.g. EHR, CPOE, etc.), in a unique setting, may not provide enough of an understanding of the entire business case to be optimally useful.

Additional macro-level research is needed. Very few studies examining the benefits of IT have explored this relationship across institutions. Among the studies that included more than one organization was Rodger et al.’s (1999) study that examined whether management of information technology was associated with quality performance in ten healthcare facilities. They found that managers’ and customers’ perceptions of quality were both positively related to health information management. In another multi-hospital study, Hatcher found that the use of information systems in hospitals was related to employee morale, reductions in employee turnover, and goals being met (Hatcher, 1998).

Unlike the healthcare literature, the business literature contains several studies examining IT adoption across multiple organizations. For example, a study of firms representing multiple industries found that organizations with high IT capabilities tend to outperform their counterparts in a variety of financial performance metrics (Bharadwaj, 2000). Also in the business literature, firms with superior IT capability demonstrated better current and sustained financial performance when compared to average industry performance, even after adjusting for effects of prior firm performance (Santhanam and Hartono, 2003).

Collectively, the studies spanning more than one organization have been limited by relatively small sample sizes or a lack of focus on the healthcare industry. Moreover, existing hospital studies have been limited by the use of qualitative measures of performance or external rankings of IT sophistication (rather than direct measures).

Overcoming the Challenge: Measuring IT ‘Sophistication’ in Hospitals

One of the challenges in conducting IT research across multiple institutions is the lack of available data on IT utilization. Additionally, when such data exists, it is difficult to reliably operationalize a measure of IT sophistication in hospitals. In other words, what is the best way to measure the level of IT adoption by an organization? Early work by Nolan (1979) represents one of the first attempts at characterizing IT. His work, outside of the healthcare sector, attempted to measure “information systems maturity” through his “stages of electronic data processing” model. The notion of information system maturity was articulated, in the ideal, to mean the state of complete information resources development and integration (Nolan, 1979). Since then, numerous researchers have been interested in IT maturity or “sophistication” and have built on Nolan’s conceptual framework (Cheney and Dickson, 1982; Gremillion, 1984). More recently, researchers examined IT sophistication in small Canadian manufacturing businesses (Pare and Sicotte, 2001).

Measuring IT sophistication in healthcare has been more complex and traditionally onerous for several reasons. First, healthcare IT has been characterized as a series of stand-alone systems with little integration. Moreover, technology is evolving at such a fast pace that identifying an appropriate organizational measurement of IT adds to the challenge. As a result, existing work has focused on a rough gauge of IT adoption usually represented by total IT investment dollars (Devaraj and Kohli, 2000; Kimberly and Evanisko, 1981; Lee and Menon, 2000). The approach, while broader in scope, uses a proxy measure for IT capabilities with obvious limitations. Investment measures provide little insight into the IT capabilities that the organization investment may have achieved. For example, investments of 1 million dollars in IT for two different organizations may provide completely different capabilities for each organization. This makes IT measures based on investment data difficult to interpret. However, given the widespread availability of investment data, this proxy measure has been utilized in previous research.

Recent work by Burke and Menachemi (2004) has begun to develop a framework to capture IT capability from a hospital IT portfolio perspective. In their study, they used structural equation modeling to validate a series of constructs, operationalized as summated scales measuring the degree of hospital IT adoption in three clusters based on organizational function. The constructs were tested using a sample of 1,545 acute care hospitals located in the United States derived from the 1999 Dorenfest IHDS + Database (Version 2) and matched data on hospital characteristics obtained from the American Hospital Association’s Annual Survey. The results of their empirical work provided validation for the theoretical measures they developed and have been used elsewhere (Thouin et al. 2008). Data could now be collected to examine the relationship between hospital IT adoption and several organizational outcomes.

Examining Outcomes Associated with Hospital IT Adoption

This section describes how data was collected and analyzed by researchers who operationalized the measures of IT sophistication in hospitals described in the preceding section. The researchers then combined their IT data with several secondary sources of hospital performance data. The results of published studies examining the relationship between IT adoption and overall performance are then discussed.

Measures of IT sophistication

To collect the IT data needed, researchers developed and administered a questionnaire in the summer of 2003. The questionnaire was sent to all acute care hospitals (n = 199) located in Florida and was specifically addressed to the hospital’s Chief Information Officer (CIO) or equivalent employee. Questions on the survey collected data regarding IT adoption, governance, outsourcing, patient safety, and future IT adoption intentions. A total of 98 hospitals responded to the survey representing a respectable 49.5% response rate.

Using the data collected in the survey, four measures of IT sophistication were created to represent the total number of IT applications that each hospital had in one of several categories. The categories, which were derived using support from Rogers’ (1995) diffusion of innovation theory represent the main operational functions of a hospital. Specifically, the four measures represented clinical, administrative, strategic, and total IT adoption.

Each measure represented a summated scale where one point was awarded for each IT application that the hospital had in place. Table 2 includes a list of all IT applications organized by operational functions (clinical, administrative, or strategic). For example, clinical IT applications listed on the survey included such items as CPOE, pharmacy information systems, and EHR. The clinical summated scale could range from zero to 25. A clinical IT score of 25 indicated that the hospital had adopted all clinical IT applications that were studied.

Table 2

Information technology (IT) applications categorized by operational function.

Clinical IT	Administrative IT	Strategic IT
Computerized physician order entry	Patient scheduling	Outcome and quality management
Electronic medical record	Patient registration	Case-mix analysis
Pharmacy information system	Patient billing	Managed care software
Pharmacy dispensing	Customer relationship management or call center	Managed care contract management
Radiology information system	General ledger	Cost accounting
Laboratory information system	Accounts payable	Executive information system
Medical record imaging	Benefits administration	Flexible budgeting
Transcription	Payroll	Enterprise resource planning system
Nurse charting or care planning	Personnel administration	Nurse staffing system
Bar-coded medical management	Time and attendance	Business intelligence or decision support systems (e.g. data warehouse)
Clinical decision support system	Electronic mail
Clinical data repository	Two-way web-based applications
Clinical resource scheduling	Credit or collections
Chart tracking and locator	Electronic claims
Chart deficiency	Eligibility
Picture archiving and communication systems	Premium billing
Bioterrorism disease surveillance system	Supply chain management
Abstracting	Materials management
Critical care bedside	Encoder
Telemedicine system	Data repository
Emergency department medical system	Master patient (person) index
Medical/surgical bedside terminals
Operating room system
Order communication results
Scanning clinical documents
Total 25 items	Total 21 items	Total 10 items

The administrative summated scale ranged from zero to 21 and included such items as electronic billing, payroll, and human resources management. The strategic IT summated scale represented applications that are used for executive decision making. The specific items that made up this scale included nurse staffing information systems, managed-care contracting software and other executive information systems. The strategic IT summated scale ranged from zero to ten. Lastly, a final summated scale was calculated to represent total IT adoption (e.g. the sum of the first three scales) and could range from zero to 56. Each hospital was given a score on each of these four scales. Higher scores on each scale indicated that the hospital had adopted additional IT applications. For the interested reader, the validation study of this method has been previously published (Burke and Menachemi, 2004). These measures, representing the hospitals that responded to the Florida survey, have been analyzed in conjunction with other datasets on hospital performance as described in the next few sections. The compilation of research findings from a variety of published studies is summarized in Table 3.

Table 3

Significant relationships between IT Adoption and important organizational outcomes in hospitals: Results from a Florida-based study of 98 hospitals.

	Outcome measures	Measures of hospital IT adoption				Effect of outsourcing IT on outcome	Directions for future research
		Clinical IT adoption	Administrative IT adoption	Strategic IT adoption	Total IT adoption
Financial Performance	Net inpatient revenue	✓	✓	✓	✓	No effect1	Longitudinal studies are needed with a larger, nationally representative sample of hospitals.
	Net patient revenue	✓	✓	✓	✓	No effect1
	Hospital expenses	✓	✓	✓	✓	No effect1
	Total Expenses	✓	✓	✓	✓	No effect1
	Total margin	✓	✓	✓	✓	No effect1
	Operating margin	✓	✓	✓	✓	No effect1
	Cash flow ratio	✓	✓	✓	✓	No effect1
Operational Performance	Site visit scores by the Joint Commission on Accreditation of Healthcare Organizations (JCAHO)	✓			✓(weak relationship)	Unknown	Additional operational performance measures should be identified and studied.
Clinical Performance	AHRQ Inpatient Quality Indicators: A validated set of risk-adjusted outcomes measures used to compare hospitals on quality of care	✓✓✓✓✓✓	✓✓✓	✓✓✓✓✓✓	✓✓✓✓✓	Unknown	Longitudinal studies are needed with a larger, nationally representative sample of hospitals.
	AHRQ Patient Safety Indicators: A validated set of outcomes measures used to compare hospitals on medical error rates and other patient safety outcomes	✓✓✓	✓	✓✓✓✓✓✓	✓✓	Unknown

Note: AHRQ is the Federal Agency for Healthcare Research and Quality.

Outsouring had no effect; IT adoption was still related to positive performance.

Financial performance

This section summarizes published work, using the Florida data described above, that examined the relationship between IT adoption in hospitals and financial performance (Menachemi et al. 2006a). In order to examine financial performance in hospitals, secondary data were obtained from the Florida Agency for Health Care Administration (AHCA). AHCA is the governmental agency responsible for licensing hospitals and requires hospitals to submit annual financial reports. The reports are based on the Florida Hospital Uniform Reporting System and included a total of 27 worksheets of financial data.

Several widely used financial measures were extracted from the data. In particular, the researchers were interested in overall hospital financial performance and also operational financial performance. To measure overall financial performance, the following ratios were utilized: operating margin, total margin, and cash flow ratio. To measure operational performance, net patient revenue, net inpatient revenue, hospital expenses and total expenses were used. Performance ratios were divided by the number of beds at the given hospital and by the number of days in a given year. Thus, the measures represented per bed per day values of each financial metric.

Regression analyses were then used to examine the relationship between IT adoption (using each of the scales that were developed) and financial performance. In addition, each statistical model controlled for average patient severity at each hospital. Patient severity, or case-mix, is a description of how sick the average patient at each hospital is. This variable was included in the models because case-mix can affect financial performance and potentially IT adoption.

The results of the regression analyses suggest that a significant and positive relationship exists between scores on the IT sophistication scales and hospital performance for each of the financial performance measure tested. In addition, the researchers employed performance group analysis for the set of operational performance indicators so that a more complete understanding of the relationships could be discerned. Findings from the performance group analyses confirmed that a significant and positive relationship between increased levels of IT use and all measures of financial performance exist even after controlling for case-mix acuity and bed size.

Thus, regardless of the analysis or method employed, the results indicated that IT adoption was consistently related to improved financial outcomes, both overall and operationally. This relationship was present when examining IT collectively and for clinical IT, administrative IT, and strategic IT as individual measures. Lastly, while higher IT use was associated with a higher level of revenues, income or cash flow, higher IT use was also associated with ratios that are based on higher expenses.

The improved financial performance observed in the Menachemi et al. (2006a) study was most likely due to the improved efficiencies that are associated with the utilization of IT. However, in addition to improved revenues, IT use in hospital also seemed to increase expenses. The additional expenses are likely due to the relatively high acquisition and maintenance costs associated with the implementation of sophisticated information systems in hospitals.

The positive findings were consistent with previous healthcare literature that linked individual IT applications, in unique settings, to improved efficiencies and/or improved financial performance. However, in the Menachemi et al. (2006a) study, the positive association was observed across many organizations of varying characteristics. Overall, the findings are consistent with several studies both inside (Thouin et al. 2008) and outside of healthcare that linked IT utilization to improved financial performance in business settings.

When interpreting the above findings from hospitals, it is important to mention that not all studies in the business literature have found positive results from IT investments. Researchers were initially baffled that business sector IT adoption did not always yield positive organizational outcomes despite theoretical predictions to the contrary. This phenomenon, referred to as the “productivity paradox” was noted by Brynjolfsson (1993) who suggested that poor measurement of inputs and outputs contributed to those research findings. Additionally, it was suggested that lag in learning and adjustment, redistribution and dissipation of profit, and mismanagement of information and technology contributed to the occasional mixed findings in the literature (Brynjolfsson, 1993). Research also suggested that after automation in IT—intensive service industries, productivity often declined (Magrassi, 2000). To help explain these results, researchers suggested that misalignment between IT strategy and processes, between IT strategy and business strategy, and a gap between the IT function and the rest of the organization may contribute to those findings (Henderson and Venkatraman, 1999; Sohal et al. 2000; Ward and Peppard, 1996).

Thus, while the results from the hospital study are promising, the relationship between IT and financial performance is complex and will require further examination. Additional confirmatory studies of hospitals will be needed to truly understand the financial impacts of IT investments in healthcare.

Operational performance

Many IT applications are implemented in hospitals with the goal of improving operational performance. Operational performance includes improvements in worker productivity, resource allocations, or management efficiencies. Despite several ways of measuring operational performance in a given hospital, researchers have found it difficult to fairly compare operational performance from one institution to the next. The lack of consensus in the literature on how to best measure operational performance across hospitals is partly due to a lack of routinely collected data on such performance.

In an effort to strike a balance between available data and adequate measures of operational performance, researchers looked to the Joint Commission on the Accreditation of Healthcare Organizations (JCAHO). JCAHO is an independent agency responsible for auditing and certifying performance and quality ratings of hospitals and other healthcare facilities around the United States. The JCAHO hospital accreditation process includes a site-visit by auditors once every three years. Under federal rules, JCAHO accreditation is required for hospitals to participate in the government’s Medicare program. As such, JCAHO accreditation is eagerly sought by hospitals nationwide because Medicare payments represent a significant source of income to hospitals. When auditing, JCAHO teams evaluate how well a hospital meets more than 500 standards specified in the Accreditation Manual for Hospitals. This data is then aggregated into 46 “grid elements”, 16 “performance areas”, and overall performance score (on a 0 to 100 scale). Each hospital’s performance scores are then made available on JCAHO’s website (http://www.jcaho.org/) as Adobe Acrobat files.

The overall performance score is essentially a measure of operational performance. As a result, researchers using the Florida data examined whether hospital IT adoption was related to improved performance on the JCAHO accreditation site visit scores (Bhattacherjee et al. 2007). To do so; the most recent JCAHO performance score for the hospitals represented in the Florida data were merged with the variables representing hospital characteristics and IT adoption. Linear regression models were employed to examine the effect of IT adoption (as measured by the clinical IT, administrative IT, and strategic IT summated scales) on operational performance (measured as JCAHO performance score) while controlling for confounders.

Results of the analyses suggested that not all measures of IT were statistically related to operational performance. The clinical IT summated scale had a strong positive and significant effect; while the administrative and strategic IT scales had non-significant (but positive) effects. When combining all measures of IT adoption into the hospital wide IT summated scale, a weak but positive effect on operational performance was observed. These findings highlight the notion that not all IT applications have a similar effect on the operational performance of a hospital.

Previous studies examining the relationship between IT adoption and performance across hospitals had mixed results. Previous studies relied on proxy measures of adoption such as investment dollars in IT. The use of investment dollars in IT as a measure of IT sophistication provides little insight into the IT capabilities that the organization investment may have achieved. The findings from the Florida data suggest that it may be improper to group all IT applications together for assessing performance effects, because different types of technologies tend to have differential impacts on organizational performance (Bhattacherjee et al. 2007). In other words, aggregated measures of IT adoption (e.g. investment dollars spent) may mask the differential effects of these technologies and average out their effects, such that the overall effect may seem mixed or non-significant.

Clinical performance

Compared to the relative shortage of data on IT utilization by hospitals, data on hospital quality and patient safety is readily available. Measuring quality of care and patient safety at the hospital level has become more reliable with the development of a series of tools by the Federal Agency for Healthcare Research and Quality (AHRQ). Specifically, the AHRQ Inpatient Quality Indicators (IQIs) and Patient Safety Indicators (PSIs) are particularly useful when studying patient outcomes across hospitals.

The AHRQ IQIs and PSIs are an array of health care decision-making and research tools that can be used accurately to measure outcomes and utilization using widely available administrative data (Agency for Healthcare Research and Quality, 2006a). Hospitals routinely collect patient-level administrative data for reimbursement and licensure purposes. The IQIs and PSIs are sets of algorithms that are applied to routinely collected inpatient discharge data. The algorithms calculate various risk-adjusted rates that reflect quality of care, or adverse event rates, inside hospitals and can be used for research purposes.

The IQIs include risk-adjusted mortality rates for both procedures (e.g. abdominal aortic aneurysm repair, hip replacement, etc.), and certain medical conditions (e.g. acute myocardial infarction, stroke, etc.) so that hospitals can be compared. The IQIs also include risk-adjusted measures representing utilization of procedures for which there are questions of overuse, under use, or misuse.

The PSIs are similar to the IQIs except that they measure items such as infection rates and complications rates from certain types of procedures and conditions. The PSIs include measures that focus on provider-level situations which describe potentially preventable complications for patients who received their initial care and the complication of care within the same hospitalization.

Collectively, the IQIs and PSIs have been validated (Stanford-UCSF, 2005) and are frequently used by researchers to compare quality outcomes across hospitals (Weiner et al. 2006; Needleman et al. 2003; Miller et al. 2005; Encinosa and Bernard, 2005; Clement et al. 2007). For the interested reader, a complete definition of each IQI and PSI including the methodology describing each measures’ calculation is available elsewhere (Agency for Healthcare Research and Quality, 2006a; Agency for Healthcare Research and Quality, 2006b).

When examining the relationship between IT adoption and performance on the IQIs, intriguing results emerge. Hospitals that adopted a greater number of IT applications were significantly more likely to have desirable quality outcomes on seven IQI measures including risk-adjusted mortality from percutaneous transluminal coronary angioplasty, gastrointestinal hemorrhage, and acute myocardial infarction. An increase in clinical IT applications was also inversely correlated with utilization of incidental appendectomy; and an increase in the adoption of strategic IT applications was inversely correlated with risk-adjusted mortality from craniotomy, and laparoscopic cholecystectomy (Menachemi et al. 2008).

Similarly, when examining IT adoption and performance on the PSIs, several important trends were identified (Menachemi et al. 2007b). A total of eight PSI measures were related to at least one measure of IT adoption. For example, an increase in the total number of clinical IT applications was significantly inversely correlated with death in low-mortality conditions, risk-adjusted rates of decubitus ulcer, and risk-adjusted postoperative sepsis. Compared with administrative IT adoption, clinical IT adoption was related to more patient safety outcome measures. Additionally, hospitals with the most sophisticated and mature IT infrastructures performed significantly better on the largest number of PSIs including selected infections due to medical care, postoperative hip fracture, postoperative respiratory failure, postoperative sepsis, postoperative wound dehiscence, and accidental puncture or laceration.

These findings strongly suggest that hospitals that have adopted a relatively greater number of IT systems are providing notably better care. Adoption of IT was associated with desirable performance on many important hospital measures of both quality and patient safety. More importantly, while these findings confirm previous studies that examined an individual IT application in an individual facility, this study was not restricted to a single vendor or product.

It is important to note that the use of IT itself may not necessarily improve quality of care. Rather, IT in general and certain clinical IT applications in particular, enable clinicians to work more efficiently by improving access to information, reducing administrative tasks, improving the decision making process, and positively affecting satisfaction. Thus, clinicians are able to spend an increased amount of time on clinical duties and ultimately affect patient outcomes and the quality of care.

Conclusion and Directions for Future Research

The mission of most hospitals is to provide high quality care to their patients. Decisions regarding IT adoption will continue to be an important aspect for hospital managers to consider. As with any major investment, the financial impacts of IT adoption need to be considered. However, given the mission of most organizations, the effect of IT on the quality of patient care is also important.

In this review, we discussed a series of studies that took a macro-level approach at understanding the relationship between IT adoption and hospital performance. Researchers were able to link increased adoption of IT to improved financial, operational, and clinical outcomes. However, for a complete understanding of the complex relationship between organizational adoption of IT and subsequent performance improvements, additional studies will be needed.

A growing trend in the healthcare sectors is to outsource certain responsibilities and functions related to managing and implementing IT systems. The literature on the effects of outsourcing IT functions on performance in hospitals is relatively underdeveloped. Potential risks and benefits may be associated with outsourcing of IT by hospitals (Beardwood and Alleyne, 2004; Freed, 1993; Hoppszallern, 2003; Morrissey, 2003). Using the data described above, researchers were able to determine that outsourcing IT functions did not correlate with any of the financial outcomes described in this paper (see Table 3). This suggests that in most cases, IT outsourcing is not necessarily a cost lowering strategy, but instead, a cost neutral manner in which to accomplish an organizational strategy (Menachemi et al. 2007a). More research is needed to investigate the relationship between outsourcing IT and other important organizational outcomes.

The data stemming from the studies presented in this paper come from hospitals in one state. Additional studies from other geographic regions will be needed to confirm the first set of results from the Florida data. The Florida data was also merely a ‘snap shot’ in time regarding IT adoption and hospital performance. Future studies should look at both IT adoption and subsequent performance over lengthier time periods. Currently, there is a dearth of generalizable longitudinal studies that examine the benefits of IT in the healthcare literature. In a single health-related longitudinal study identified in the current literature, researchers found that the performance benefits of IT adoption in hospitals may be associated with a time-lag (Devaraj and Kohli, 2000). This finding suggests that the relationship between IT adoption and improved outcomes may not always be detectable in cross-sectional analyses.

When interpreting the results described in this paper, it is important to recognize the research limitations stemming from the Florida data. First, the Florida studies all had a non-experimental, retrospective, cross-sectional design suggesting that causality cannot be determined. Thus, the positive relationships identified should be considered mere associations. It is possible that some unobserved variable (e.g. effective management) is associated with both IT adoption and superior performance in hospitals and is affecting the results discussed. Second, despite a relatively high response rate of approximately 50% in the survey of Florida hospitals, any response rate less than 100% can potentially introduce response bias. If non-responding hospitals systematically differed from responding hospitals, the results of the studies may not generalize to the non-responding hospitals. However, given the non-sensitive nature of IT questionnaires, several studies failed to detect response bias in IT surveys of healthcare organizations (Hikmet and Chen, 2003; Menachemi et al. 2006b). Lastly, by design, studies that collect primary data with the use of survey instruments rely on the ability and willingness of participants to provide accurate responses. Thus, the Florida data may be subject to recall bias because it required individuals to rely on their potentially imperfect memory when answering questions.

Despite these limitations, it should also be mentioned that the relationships revealed in the Florida studies may underestimate the strength of association between hospital IT use and positive performance. This may be true because by design their survey methodology measured whether a given IT application was present at a given hospital, not how frequently it was used. The relationship between IT and improved outcomes may be even stronger had researchers examined actual use of IT, not just the availability of IT applications.

In conclusion, hospitals have found it difficult to systematically conduct return on investment analyses for many IT applications, particularly hospital-wide IT systems that improve efficiency but do not directly produce revenue. The data collected and analyzed from Florida provides provocative evidence regarding the ‘business case’ surrounding investments in IT by hospitals. Importantly, the evidence from the Florida data that links IT in various positive organizational outcomes did not focus on a specific IT vendor or product. Instead, the vendor agnostic approach of this body of literature examined the uniform effect of IT on a heterogeneous sample of hospitals. Thus, the generalizability of these trends is stronger than previous literature and hospital planners and other decision makers can now have a better understanding of the financial and non-financial impacts of IT investment decisions.