Description and pilot evaluation of the Metabolic Irregularities Narrowing down Device software: a case analysis of physician programming.

BACKGROUND: There is a gap between the abilities and the everyday applications of Computerized Decision Support Systems (CDSSs). This gap is further exacerbated by the different 'worlds' between the software designers and the clinician end-users. Software programmers often lack clinical experience whereas practicing physicians lack skills in design and engineering.
OBJECTIVE: Our primary objective was to evaluate the performance of Metabolic Irregularities Narrowing down Device (MIND) intelligent medical calculator and differential diagnosis software through end-user surveys and discuss the roles of CDSS in the inpatient setting.
SETTING: A tertiary care, teaching community hospital. STUDY PARTICIPANTS: Thirty-one responders answered the survey. Responders consisted of medical students, 24%; attending physicians, 16%, and residents, 60%.
RESULTS: About 62.5% of the responders reported that MIND has the ability to potentially improve the quality of care, 20.8% were sure that MIND improves the quality of care, and only 4.2% of the responders felt that it does not improve the quality of care. Ninety-six percent of the responders felt that MIND definitely serves or has the potential to serve as a useful tool for medical students, and only 4% of the responders felt otherwise. Thirty-five percent of the responders rated the differential diagnosis list as excellent, 56% as good, 4% as fair, and 4% as poor. DISCUSSION: MIND is a suggesting, interpreting, alerting, and diagnosing CDSS with good performance and end-user satisfaction. In the era of the electronic medical record, the ongoing development of efficient CDSS platforms should be carefully considered by practicing physicians and institutions.

Current literature and clinical practice continue to question whether the use of Computerized Decision Support Systems (CDSSs) can positively influence patient outcomes (1). The answer would seem to be a priori positive, because computers are everywhere and can process virtually unlimited amount of data rapidly, precisely, and cost-effectively. This concept has been challenged by the lack of consistent data to support improvement in performance parameters or patient care measures such as morbidity and mortality. The capacity for computerized decisions to supplement the average physician's thought process, including differential diagnosis and treatment plan at the point-of-care, is largely a function of our ability to create an excellent CDSS.

CDSSs are defined as ‘active knowledge systems, which use two or more items of patient data to generate case-specific advice’ (2). Based on this definition, Metabolic Irregularities Narrowing down Device (MIND) can be categorized as a CDSS, despite the fact that it was originally designed to be a functional component of a future CDSS. The challenge for physicians is how to create a seamless, comprehensive support system. CDSSs are difficult to study in randomized controlled trials (RCTs), given they are neither true diagnostic tests nor applied medical interventions. Furthermore, diverse CDSSs exist; differences in their context (3, 4), knowledge, and data sources (5); decision support methods; and work flow (6) compound the dilemma. Berlin, in an attempt to create CDSS taxonomy, identified these five categories with 26 further subaxes (5). This heterogeneity has been reflected in other studies (7). Moreover, many of the studies did not factor in the clustering of the sample size, which could potentially lead to false-negative results from the absence of statistical integrity (8).

CDSSs have been involved in every aspect of medicine. A few examples include optimization of antibiotic use (9), management of heart failure (10), anesthesia management of malignant crisis (11), prevention of venous thromboembolism (12), prediction of postoperative nausea and vomiting (13), distinguishing between bacterial and aseptic meningitis (7), and management of hypertension (14). RCTs evaluating the quality of decision support have been performed for many of these systems, but inconsistent results continue to exist because of lack of standardization and taxonomical differences (5), making subsequent analyses unpredictable and spotty (15). CDSSs sit at the tip of the ‘5S’ pyramid, higher than clinical trials, syntheses, synopses, and summaries (16), albeit a paradoxical gap between the growing field of evidence-based literature and the efficient use of data gathered at the point of care to provide decision support in real time to individual patients, the ultimate objective of a successful evidence-based approach.

MIND is a CDSS which integrates multiple different laboratory values and delivers a stratified differential diagnosis. MIND has built-in automaticity, including the ability to process electrolyte, acid–base, and demographic values and generate an array of real-time calculations such as the Glomerular Filtration Rate (GFR), Alveolar–arterial (A–a) gradient, acid–base analysis, and fractional excretion of sodium and potassium, and so on. It subsequently fits the available and calculated data into clinical algorithms and generates a stratified differential diagnosis, with a report.

In the present study, we describe the MIND software, synthesize a review based on the current literature and our pragmatic experience in programming, and present the results of a survey administered to the end-users of MIND.

Implementation

We developed MIND with Liberty BASIC v. 4.03 for Microsoft® Windows® (Shoptalk Systems, Framingham, MA, USA). MIND is a freeware and the authors report no financial conflicts of interest. MIND is available for download at http://www.softpedia.com/get/Others/Home-Education/MIND-medical-calculator-and-e-consultation.shtml. A user-friendly interface was designed to facilitate manual data entry (Fig. 1). ‘Save’ and ‘load’ functions were created to support storage of temporary files and more importantly to enable future incorporation of MIND into future CDSS platforms. Data entered at the point of care are processed rapidly and the results are delivered in real time via the calculations window (Fig. 2). A parallel engine uses the calculated values, along with the primary values to fit diagnostic algorithms and generate stratified differential diagnoses, based on the provided information (Fig. 3). An example of differential diagnoses is provided in Fig. 3. MIND factors a subject's age, sex, and Inspired Oxygen Fraction (FiO2) to calculate and diagnose, for example, an abnormal A–a gradient; if the data point to an underlying respiratory alkalosis on top of a metabolic acidosis, MIND will elevate the score for pulmonary embolism to the top of the already generated differential diagnosis for metabolic acidosis and respiratory alkalosis. It has the potential to enhance the CDSS platform by providing a machine-generated consultation (e-consultation) document (Table 1). The developers of MIND were practicing internal medicine house-officers with background in bioinformatics and software programming. MIND has undergone cycles of performance improvement based on feedback from internal medicine, emergency medicine, and anesthesiology attending physician house-officers. The electronic consultation document contains all the medical calculations, acid–base analyses, and differential diagnoses, based on the data entered. Also, it provides recommendations and may suggest additional tests.

Fig. 1

MIND main user interface. Data can be entered either manually via the user interface or loaded from a file through the data port. Critical laboratory values are annunciated by means of a notification bar on the left side of the screen; in this case, the red bar indicates a critical value for arterial blood gases.

Fig. 2

MIND's point-of-care real-time calculation results screen based on the data entered in Fig. 1.

Fig. 3

MIND's real-time differential diagnosis screen. Note the alert at the top, the differential diagnosis below, and the suggestion to test serum osmolality for further hyponatremia analysis. The rationale for the suspicion of each diagnosis is provided on the right side of the screen. MIND score is based on the level of suspicion for each diagnosis.

Table 1

Sample detailed analysis ‘e-consultation’, based on data from Figure 1

2. MEDICAL CALCULATIONS

Body Mass Index – BMI: 30.4218496 kg/square meters.

BMI is abnormal. Patient is * Obese.

Calculated serum osmolality: 260.468571

Measured serum osmolality: 300

Osmolar gap: 39.5314286 – Abnormal FENA: 0.42763647%

GFR (Crockroft-Gault formula): 41.4456522 ml/min – Normal GFR for s: 97–137 ml/min – GFR is below normal limits.

FENA: 0.42763647%

A–a gradient: −52.77. Normal A–a gradient for this patient age/fio2: 23.5. A–a gradient is Normal

3. ACID–BASE ANALYSIS/UNDERLYING DISORDER DETECTION

Anion Gap: 27.52. [Anion Gap calculation=Na+ – (Cl−+HCO3−)]

Delta Delta: 25.52. [Delta Delta calculation=AnionGap – 12+HCO3−]

>> NOT CONFIRMED. You provided. MIND calculations point to Metabolic acidosis.

ANION GAP DISORDER

>> No secondary metabolic acid base abnormality detected.

3.A. COMPENSATION ANALYSIS

>> Respiratory Acidosis may be hidden under the expected compensation for Metabolic acidosis.

* Estimated dCO2=1.8* dHCO3 +5. Anticipated paCO2 is: 9.8. Actual paCO2 is: 90.

3.B. SUMMARY

>> Primary Abnormality: Metabolic acidosis/Secondary/Underlying: Respiratory acidosis because compensation is:

Inappropriate

Anion Gap acidosis present.

METABOLIC ACIDOSIS ANALYSIS

>> ANION GAP Metabolic acidosis and abnormally high serum osmolality. Serum Osmolality was reported to be 300.

Anion gap: 27.52.

Diagnoses to consider in this setting:

Methanol poisoning

Antifreeze poisoning

Ethylene glycol poisoning

RESPIRATORY ACIDOSIS ANALYSIS

Diagnoses to consider in this setting

Pneumothorax

Large pleural effusion

Stroke in bulbar area of brain stem

**Morphine/Sedatives

Central sleep apnea

Obesity

***BMI is: 30.4218496 which can be consistent with and Sleep apnea as the potential causes of his Resp. acidosis.

COPD

ARDS

Chest wall disease, e.g., Polio, West Nile Virus, Kyphoscoliosis, Myasthenia gravis, muscular dystrophy, etc.)

Hypophosphatemia (causes depletion of ATP and drop in energy for the muscles)

Succynilcholine (paralysis for intubation)

** Note: Plasma phosphorus is: 2 mg/dl. This is consistent with HYPOPHOSPHATEMIA as the cause of this patient's respiratory acidosis.

HYPONATREMIA

************

Differential:

Mannitol because serum osmolality is 300mosm

MIND PROBABILITY STRATIFICATION

| 1. NOTICE: -,,,,,,,,, DANGEROUS MG levels, MIND score: 10

| 2. Hypophosphatemia: – Resp. acidosis, >Hypercalcemia,,, Detected, Resp. acidosis, MIND score: 6

| 3. Hypermagnesemia:,,,, Detected, MIND score: 5

| 4. Hypocalcemia: – MEASURE SERUM MG,,,, Decreased pH increases ion. calcium, MIND score: 5

| 5. Renal failure: – Hypervolemia, Abnormal GFR, Hypochloremia, Hypochloremia, Hypochloremia, MIND score: 5

| 6. **PROVIDE Ventilatory support**:,,,, SEVERE HYPERCARBIA, MIND score: 5

| 7. Metabolic acidosis -,,,, Detected, MIND score: 5

| 8. Respiratory acidosis -,,,, Detected, MIND score: 5

| 9. Anion Gap Metabolic acidosis: -,,,, +High. serum osmolality, MIND score: 5

| 10. Vomiting: – Hypophosphatemia, Hypochloremia, MIND score: 2

| 11. NG function: – Hypophosphatemia, Hypochloremia, MIND score: 2

| 12. Steroid medications: – Hypophosphatemia, Hypochloremia, MIND score: 2

| 13. Diuretic abuse: – Hypophosphatemia, Hypochloremia, MIND score: 2

| 14. Vit D def. – Hypophosphatemia, Low serum total calcium, MIND score: 2

| 15. Pseudohypoparathyroidism – Hypophosphatemia, Low serum total calcium, MIND score: 2

| 16. Addison's disease: – Hypermagnesemia, Hypochloremia, MIND score: 2

| 17. Sepsis: – Hypophosphatemia, Low serum total calcium, MIND score: 2

| 18. CHF: – Hypervolemia, Hypochloremia, MIND score: 2

| 19. Liver failure: – Hypervolemia, Hypoalbuminemia, MIND score: 2

| 20. Sleep apnea: -, Resp. acidosis, MIND score: 2

| 21. Obesity: – BMI >30, Resp. acidosis, MIND score: 2

| 22. COPD: – Resp. acidosis,, MIND score: 2

| 23. Antacid Abuse: – Hypophosphatemia, Hypermagnesemia, MIND score: 2

| 24. Diabetes: – Diabetes if glu is fasting, Hyperglycemia >Factitious hyponatremia, MIND score: 2

We implemented MIND in a community-based, academic tertiary care institution, and administered surveys to volunteer healthcare providers and users of MIND between October 2007 and December 2010. The healthcare providers practiced on internal medicine setting, which included the emergency department, internal medicine wards, step-down units, and the medical intensive care unit. The responders were exposed to MIND for at least 4 weeks, during their core internal medicine or medical ICU rotations.

On the survey, we asked the questions listed on Table 2 and received either binary responses or responses on the Likert scale, when this was appropriate.

Table 2

Online survey sought to obtain feedback from medical professionals who used MIND

Number	Questions
1	Is MIND easy to use?
2	Does MIND serve as a useful tool for medical students?
3	What is your position?
4	Rate MIND as a medical calculator:
5	Rate the stratified differential diagnosis provided by MIND:
6	Based on your experience, can MIND improve patient care?
7	Did MIND suggest actual diagnoses that were not included in your primary differential diagnosis list?
8	Will incorporating MIND into the computerized physician order system improve patient management?
9	Will incorporating MIND into everyday practice improve your knowledge of electrolyte and acid/base disorders?
10	Additional comments:

Results

Thirty-one responders answered the survey. Medical students made up 24% of the responders, attending physicians 16%, and the remaining responders were residents and visiting trainees. The survey results suggest that MIND has the potential to increase the quality of patient care. Among the responders, 62.5% reported that MIND has the ability to potentially improve the quality of care, 20.8% were sure that MIND improves the quality of care, and only 4.2% of the responses felt that it does not improve the quality of care (Table 3). Ninety-six percent of the responders felt that MIND definitely serves or has the potential to serve as a useful tool for medical students, and only 4% of the responders felt otherwise. Fifty-nine percent of the responders rated MIND as an excellent medical ‘calculator’; the remaining 41% rated it as ‘good’ based on the Likert scale. Ninety-six percent of the responders felt that MIND suggested diagnoses that were included in their primary differential diagnosis list and only 4% felt otherwise. Thirty-five percent of the responders rated the differential diagnosis list as excellent, 56% as good, 4% as fair, and 4% as poor. We present the Likert-based responses in Table 3.

Table 3

Survey responses on a Likert scale

Survey question	Likert scale answer
	Definitely yes (%)	Yes, has the potential (%)	Unsure (%)	Probably not (%)	Definitely not (%)
Is MIND EASY to use?	56.00	40.00	4.00	0.00	0.00
Does MIND serve as a useful tool for medical students?	52.00	44.00	0.00	4.00	0.00
Based on your experience, can MIND improve patient care?	20.80	62.50	12.50	0.00	4.20
Incorporating MIND into the computerized physician order system will improve patient management?	39.10	52.20	4.30	0.00	4.30
Will incorporating MIND to everyday practice improve your knowledge of electrolytes and acid/base disorders?	45.80	45.80	4.20	4.20	0.00

The vast and multidisciplinary spectrum of applications associated with CDSSs makes them a challenging area of research in medicine. We must recognize that all CDSSs are not the same, equal, or even equivalent. There is a functional taxonomy (5, 17) to these systems and their functions may vary (Fig. 4).

Fig. 4

Functionality of CDSSs.

MIND is a suggesting, interpreting, alerting, and diagnosing CDSS.

Discussion

Plasticity, customizability, and compatibility

The process of developing MIND, while being active physicians, enabled the authors to identify certain important parameters that could enhance the efficiency of future CDSSs. They came up with the new concept of ‘plasticity’, which addresses both the system itself (its ability to learn from its own data), as well as the relationship between the user, whether a single individual or the medical community as a whole, and the system. By providing the medical community with the opportunity and the mechanism for creating tailored ‘extensions’ or ‘apps’ with algorithms, patient care and screening alerts, and patient care order sets could all improve. Plasticity of the CDSS will promote enhanced diagnostic tools, superior instructional aids, and perhaps even more timely and pertinent educational opportunities for medical students at the point of care.

Flexibility and adaptability are both useful quality criteria for evaluation of future CDSSs (18). The newly coined term ‘plasticity’ describes the ability of the system to adapt and mature with experience. The two counterparts of plasticity, customizability and compatibility, are critical and underutilized components of current CDSSs. Customizability allows for micromanagement of the system, accommodating the needs and learning style of the individual user. Compatibility also targets the macromanagement of the system by the exchange of information within the medical community, hospital boards, service committees, and among task forces. Compatibility will allow a wide array of extensions or apps that can be preferentially added to the CDSS by the individual user, or group practices.

Automaticity and user-friendliness

Despite the heterogeneity of the currently marketed CDSSs, certain common characteristics exist that seem to influence the success of decision support systems. Some of them have been summarized as the ‘ten commandments’ of CDSS by Bates (Table 4) (19). Perhaps the most important one, though listed as number two in Bates’ commandments, is automaticity – the provision of real-time support at the point of care. A crucial element related to successful decision support system is the demand of time placed by the system on its user. A randomized study of a CDSS specific for the management of diabetes in general practice by Hatlevik and colleagues in Norway found no statistically significant change in patient outcomes after a total of 18 months of follow-up (20). In that system, information such as blood pressure and laboratory test results, were retrieved automatically from the electronic medical record. A striking 92% agreed that the CDSS was ‘too large’ (20). This example reinforces the sentiment that simple presentation of guidelines/algorithms results in improved adherence of physicians and clinical outcomes (20).

Table 4

The ten commandments of clinical decision support

1	Speed is everything
2	Anticipate needs and deliver in real time
3	Fit into the user's workflow
4	Little things can make a big difference
5	Recognize that physicians will strongly resist stopping
6	Changing direction is easier than stopping
7	Simple interventions work best
8	Ask for additional information only when you really need it
9	Monitor impact, get feedback, and respond
10	Manage and maintain your knowledge-based systems

From Ref. (19).

User-friendliness is a key feature for the success of CDSS, yet it is often overlooked. The successful decision support systems of the future will contain dynamic text, hyperlinks to more information, right-click functions, interactive ‘balloons’ that display in response to the hovering of a pointing device over a diagnosis or value, and lists with applicable guidelines.

Evidence-based support

The authors suggest that guidelines should be embedded in the decision-making process of the decisions support system and not presented visually, unless expressly requested – on demand – for patient-oriented education. Following the example of MIND, the authors recommend the application of algorithmic-based decision-point reminders. For example, MIND prompts ‘Serum Osm?’ next to the diagnosis of hyponatremia; this is an algorithmic point for the diagnosis of hyponatremia presented as a non-distracting, non-interrupting, mild reminder rather than a decision tree or a distracting alert. Furthermore, these reminders should be discreet and dynamic; clicking on them, upon decision or curiosity, shall lead to a review of the rationale and the level of evidence of this recommendation. These decision points need not demand immediate attention from the user, allowing the user to determine whether to pursue the additional path immediately or not at all; they should guide and help the physician to make his or her own decision by augmenting (10) – not replacing – the physician's judgment (21).

Educational mission

Knowledge-Based Systems (KBSs) are one of the most common types of CDSSs (22). Because of their complex design, foundational requirements, and cost involved, KBS-type CDSSs are considered a high-risk clinical information technology innovation (22). Despite this fact, they offer numerous bedside educational opportunities for medical students and residents. An extended differential diagnosis list, such as the one provided by MIND, was found to be helpful and resourceful for medical students and residents, based on the surveys. Some suggest that educational CDSSs provide more benefit to medical students and residents (23) than to attending physicians (24).

Academic detailing is an effective tool that has been underevaluated (25) and could potentially prove to be yet another benefit of a successful CDSS.

Limitations of MIND

The MIND CDSS is designed with the vision to function as an extension for future CDSSs. It uses a novel approach for the interpretation of electrolyte disturbances and their differential diagnoses. Despite the progress made to date, additional work remains. MIND does not provide adequate differential diagnosis stratification and it is not 100% automated, because integration with the electronic medical record has not yet been attempted. It has not yet been peer-reviewed and its application on patients has been limited. Survey responders noted that MIND is not fully evidence-based, because the information provided is mostly derived from summaries and not from studies, syntheses, and synopses, and the differential diagnosis score is additive based on the number of existent disorders. Based on qualitative survey responses, the generated differential diagnosis lists are sometimes quite extensive. Studies have shown that physicians are less likely to be influenced by differential diagnoses that appear lower than the top 10 (6).

Looking in to the future

In retrospect, after having developed MIND and reviewed the available literature, we have identified and proposed several key parameters for future development, including plasticity, timesaving orientation, and point-of-care targeted academic detailing and MCQ/CME implementation. CDSSs are ‘limited by the cumulative knowledge used to program their recommendations’ (7). The scientific method is insufficient to provide insights to their development, because it employs very narrow cross-sectional questions for the evaluation of the broader, dynamic, developing areas of medicine.

For CDSSs to provide better service, a deeper understanding of human decision making in general, and clinical decision making in particular, will be needed. Areas such as adaptive rationality (26), activity theory (27), and goal theories (28) could all provide good foundations toward achieving that.

Conclusions

In the era of the electronic medical record, the ongoing development of efficient universal CDSS platforms should be carefully considered. Review of the literature reveals that current CDSSs are far from perfect. The plethora of CDSSs on the market today contains an egotistical representation of its developers or its native institution. By abandoning the traditional approach, we examined the possibility of creating a CDSS extension, or an ‘app’. We identified several opportunities for improvement over traditional models and elevated the important need for a ‘universal language’ between CDSS platforms and their potential subprograms/extensions along with their knowledge-based summaries. The fact that providers of clinical information hold critical data hostage is intolerable (2). Several RCTs provided important insights for the identification of parameters that could improve patient outcome and physician performance, but these suggestions are rare.

By using MIND as an example of what could be accomplished through utilization of an open, intuitive, freeware CDSS extension, the authors stress the importance of coding patient and laboratory values into variables that can be potentially processed by third-party programs and other CDSS-like elements. At the same time, these variables can be used for the application of clinical algorithms; calculations of scores, ratios, and indexes; generation of stratified differential diagnoses; and suggestions of additional appropriate tests.

When the pillars of this new universal platform are set, incentives can be given to doctors and programmers to create apps and extensions with clinical algorithms, alerts, and reminders that will update their CDSSs themselves through the Internet, just like MIND did in the case study conducted for this initiative. If we, as members of a community dedicated to improving patient care and professional education, can reach a consensus to create such a platform or CDSS operating system, the possibilities can be endless. The key to the solution is plasticity: it will enable each doctor, committee, association, task force, and hospital to add, exchange, and improvise the evidence-based management system of the future.