Proposing Electronic Health Record Usability Requirements Based on Enriched ISO 9241 Metric Usability Model.

INTRODUCTION: System usability assessment is among the important aspects in assessing the quality of clinical information technology, especially when the end users of the system are concerned. This study aims at providing a comprehensive list of system usability.
METHODS: This research is a descriptive cross-sectional one conducted using Delphi technique in three phases in 2013. After experts' ideas were concluded, the final version of the questionnaire including 163 items in three phases was presented to 40 users of information systems in hospitals. The grading ranged from 0-4. Data analysis was conducted using SPSS software. Those requirements with a mean point of three or higher were finally confirmed.
RESULTS: The list of system usability requirements for electronic health record was designed and confirmed in nine areas including suitability for the task (24 items), self-descriptiveness (22 items), controllability (19 questions), conformity with user expectations (25 items), error tolerance (21 items), suitability for individualization (7 items), suitability for learning (19 items), visual clarity (18 items) and auditory presentation (8 items).
CONCLUSION: A relatively comprehensive model including useful requirements for using EHR was presented which can increase functionality, effectiveness and users' satisfaction. Thus, it is suggested that the present model be adopted by system designers and healthcare system institutions to assess those systems.

INTRODUCTION

Today, most hospitals and health institutes need to buy health information systems and parts of these systems (1). Increased care quality, decreased expenses and errors (2), improved efficiency and increased patient security (3) are among advantages of using these systems. However, it seems that these systems are not widely used in health care centers and are not accepted by users (4). Studies have shown that failure in these systems is 30-50 percent (5). System usability problems such as delay in running orders and function processes which cause users’ dissatisfaction (6) and have negative effects on acceptance and effectiveness of the system are among existing obstacles in accepting this technology (7).

Information system usability is the extent to which a system can be used by specific users while efficiency, effectiveness and user’s satisfaction is guaranteed to achieve specific aims (8). Not only does usability increase user’s speed and precision, but it also guarantees user’s security (9). Nielson (1997) stated that characteristics including ease of learning, efficiency of using, capability to remind, preserving and prevention of errors and satisfaction are vital factor in system usability (10). Roger and Sharp (2012) mentioned that the aims of system usability include being effective, being efficient, being secure, being suitably productive and being easy to learn and maintain (11). In physicians’ point of view, usability problems result in increased education time and decreased productivity while working with medical electronic record (5); therefore, it is necessary to pay much attention to system usability in order to reach the highest level of productivity in information technology (12).

Evaluation of system usability is considered one of the important dimensions in evaluating clinical information technology quality, especially for final users of the system (13) and helps to recognize the strengths and weaknesses of the system (14) and assures that system functions are in accordance with users’ tasks (15). In fact, the aim of system usability’ evaluation is to discover, understand, decrease and prevent problems of system usability (16) and it is considered as having a significant role in user-centered system design (15). There are several methods for evaluating software usability among which questionnaire is an economical one especially in big hospitals and organizations (17). Different questionnaires are used with the aim of evaluating usability of health-care information systems (18) Software Usability Measurement Inventory (SUMI) Questionnaire, Questionnaire for User Interface Satisfaction (QUIS) and International Organization for Standardization (ISO) 9241 Part 10 are among well-known questionnaires (14, 15, 17-20). Since the aim of user-centered design is to ensure good system usability and to make sure that users are able to do theirs tasks effectively, efficiently and with high levels of satisfaction (6), this research tried to make a comprehensive list of system usability requirements using end-users’ feedback. It is hoped that using such requirements in system design makes system usage as user-friendly as possible for users.

METHODS

This research takes an applied descriptive cross-sectional design conducted using Delphi technique in 2013. Three independent steps were taken in order to conduct this research: in the first step, library studies and wide online search were carried out using databases such as Google scholar, Science direct and Pubmed. The result of this step was a semi produced guideline and a questionnaire (8, 21-23). Keywords including Hospital Information System (HIS), Electronic Health Record, Electronic Patient Record (EPR), and system usability and information system evaluation were used for this purpose. The guideline included introducing the study, its aims and applications of electronic health record for users. In the semi-produced questionnaire, system usability requirements for electronic health record were designed in nine areas including suitability for the task, self-descriptiveness, controllability, conformity with user expectations, error tolerance, suitability for individualization, suitability for learning, visual clarity and auditory presentation. An option was added to the end of suggested requirements so that in case experts believed that a requirement needed amendment or change, they could suggest deleting, adding or compounding of requirements. In the second step, the semi-produced guideline was sent to five experts via email. These people were at least M.A graduates in medicine and had 10 years of experience in this field. After analyzing the contents of the first draft of the questionnaire concluding the presented opinions received through the first step, the final questionnaire was designed. It consisted of demographic information and 163 closed-ended questions including suitability for the task (24 questions), self-descriptiveness (22 questions), controllability (19 questions), conformity with user expectations (25 questions), error tolerance (21 questions), suitability for individualization (7 questions), suitability for learning (19 questions), visual clarity (18 questions) and auditory presentation (8 questions) and an open-ended question. In the third step, closed-ended questions were designed based on the requirements resulted from a library study and concluding experts’ opinion of the second step. Five-point Likert scale was used for ranking the items from completely agree to completely disagree and an open-ended question was also added to the end of questionnaire for collecting other requirements considered by experts. Validity of the questionnaire was determined based on content validity and experts’ judgments and test-retest method was applied to determine the reliability of the questionnaire; therefore, 10 experts were provided with the questionnaire and then were provided with the same questionnaire again after a week. The correlation coefficient was calculated as 0.99 for the questionnaire. Then, eligible users in research universe hospitals were provided with the questionnaire and experts were randomly provided with the final questionnaire which had been designed using Delphi technique in two sessions. Experts participating in this step included 40 individuals working in hospital information system who were interested in taking part in the study and had at least 10 years of experience in medical field and 5 years of experience working with HIS in nursing, pharmacy, laboratory, medical documents, finance, and nutrition units and outpatient clinics having M.A or higher degrees. 38 out of 48 questionnaires were filled out. After the questionnaires were collected, they were analyzed using SPSS software (18th version). Answers were given points and 0, 1, 2, 3 and 4 points were given to answers of completely disagree, disagree, no opinion, agree, and completely agree, respectively. Then, mean point of each requirement was calculated. Requirements with final mean point of 3 or higher were finally confirmed, those with final mean point of less than 2 were deleted and those with final mean point of 2 to less than 3 were offered to experts for further analysis in the second phase of Delphi technique until agreement was reached and the items were either confirmed or deleted.

RESULTS

In this study, most experts participating in this study were female participants (71.1%) and just 28.9% of them were male participants. Mean age of participants was 36.1± 3.1 and the minimum and maximum ages of participants were 31 and 44 years, respectively. Based on Table 1, most participants had B.A degrees and most of them were employed contractually (39.5%) and their job experience mean was 11.39±2.27 years and the job experience mean for experts in HIS was 5.81±0.45 years.

Table 1.

Demographic information of experts

variables	number	percentage
gender	MaleFemaletotal	112738	28.971.1100
Level of education	B.AM.A.PH.D.total	343138	89.57.92.6100
Kind of employment	FormalProvisionalContractualOthertotal	8915638	21.123.739.515.8100

In the first phase of Delphi technique, all raised requirements related to subjects of suitability for the tasks (24 items), self-descriptiveness (22 items), controllability (19 items), conformity with user expectations (25 items), error tolerance (21 items), suitability for individualization (7 items), suitability for learning (19 items), visual clarity (18 items) which had been offered for poll obtained mean points of 3 or higher and were finally confirmed. In this step, 6 out of 8 requirements in the subject of “auditory representation” which had been offered for poll obtained mean score of 3 or higher and were finally confirmed and 2 requirements obtained mean points of 2 to less than 3 which had been offered for poll in the second phase of Delphi technique and were finally confirmed. The final list of system usability requirements was designed in nine subjects with 163 requirements.

DISCUSSION

Due to the fact that most presented definitions for information system usability have focused on system users, the present study tried to present a comprehensive list of usability requirements of EHR by gaining experts’ opinions. In this study, usability requirements of EHR were determined in nine subjects including suitability for the task, self-descriptiveness, controllability, conformity with user expectations, error tolerance, suitability for individualization, suitability for learning, visual clarity and auditory presentation; in ISO9241questionnaire, part 10, system usability was presented in seven subjects including suitability for the task, self-descriptiveness, controllability, conformity with user expectations, error tolerance, suitability for individualization, suitability for learning and included 75 items (8). Visual clarity and auditory presentation were added to this study, making it different as compared with ISO9241questionnaire, part 10. These nine subjects can be used in designing EHR or in the form of a questionnaire to evaluate such systems.

In the present study, 24 requirements related to suitability for the tasks that had been offered to experts for poll were finally confirmed. In ISO9241 questionnaire, part 10, 15 requirements were determined for the subject of suitability for the tasks (8) which lacks the following requirements presented in the current study. They include “decreasing time of doing tasks by working with software”, “breaking long sequence of tasks to smaller parts”, “organizing information on the screen reasonably”, “finding needed information on the screen easily”, “defining professional language and terminology in the first step of the task”,“supporting different kinds of entering information” and “supporting different methods of entering information”; moreover, requirement of “easy recoverability of information of the specific field” was stated under the subject of suitability for the tasks in this study while it has been placed in the subject of self-descriptiveness in ISO9241 questionnaire, part 10. In this study, the highest mean point of requirements of suitability for the tasks was calculated for the requirements of “easily finding needed information on the screen”, “sensible order of fields on the screen” (3.78), “decreasing time of doing tasks by working with software” (3.76) and “easy recoverability of information of a specific field” (3.71). In ISO9241 questionnaire, part 10, suitability for the task has been stated as one of the seven important principals and “easy recoverability of information of a especial field” has been mentioned there, too (8). Ravden and Johanson (1989) referred to the subjects of suitability for the tasks and “easily finding needed information on the screen” in their general booklet of evaluation information systems usability(22). In another study, Ash et al. (2004) and Scheleyer et al. (2007) stated that displaying information on a few of the screens of a computer would result in problems for users (24, 25). In addition, Thyvalikakath et al. (2008) believed that the information related to a special task must be displayed at the same time (26). As separating the clinical information and displaying needed information of users in few screens would finally result in making low-quality decisions, designers of EHR software must design the system in a way that needed information of users be displayed in a page and information system be suitable for the tasks of users.

In this study, 22 requirements of self-descriptiveness that had been offered to experts for poll were finally confirmed. In ISO9241 questionnaire, part 10, 12 requirements were determined for the subject of self-descriptiveness (8). Requirements including “the graphic symbols and related similes of application related to user work definition”, “leading user in using shortcuts”, “simple and short messages and instructions on the screen”, “related messages on the screen”, “accurate and informative messages”, “existence of subject and heading in each screen”, “existence of default values in software”, “informing the user at the time of ending operation”, “presenting suitable feedback about completed tasks”, “suitable help in order to correct errors”, “suitability of the content of help and educational training videos with tasks and applications” of the existing system had not been mentioned in ISO9241 questionnaire, part 10. In the current study, the highest mean point was calculated for the requirements of “understandable messages on the screen” (3.76), “simple and short messages and instructions on the screen” (3.73), “suitable help in order to correct errors” (3.73) and “informing the user at the time of ending operation” (3.71). In ISO9241questionnaire, part 10, self-descriptiveness is one of the principals of system usability and requirements of “immediate understanding of messages displayed on the screen for the user”, “existence of no ambiguity in expressions and definition used in the software” and “possibility of easily distinguishing the difference of system’s messages from each other” have been reported by users (8). Ravden and Johanson (1989) mentioned system learner feedback and requirements such as “simple messages and instructions on the screen” and “informing the user at the time of ending operation” in their general booklet of evaluation of usability among users’ interaction (22). Nielson (1993) considered using common dialogue, user language and visible and retrievable instruction as the rules of usability (27). Moreover, Tognazzini (2003) mentioned “using simplicity and capability to read” as the rules of usability (28). In addition, Darbyshire (2000) mentioned” using directly understandable diagrams and shapes”, “presenting help if it was needed” and “access to reminders and promoters on the screen” as indicators of user-friendliness of computer information system for physicians (29). As ambiguity in understanding expressions, definitions, shapes and displayed messages in electronic health record would result in wasting users’ time and doing tasks incorrectly, designers are recommended to pay attention to requirements of self-descriptiveness in EHR and design the system in a way that can be easily understood by users. In this study 19 requirements of Controllability that had been offered to experts for poll were finally confirmed. In ISO9241 questionnaire, part 10, 11 requirements have been presented for the subject of controllability (8) including “direct access to special screens in consecutive screens by the user”, “possibility of entering different parts of the system in case the user is in need”, “reversing function for the user”, “possibility of re-doing the canceled function for the user”, “access to shortcuts in necessary occasions”, “possibility of changing produced information by computer for the user”, “observing steps of task completion either forward or backward” and “pre-registered repetitive information on the screen” had not been mentioned. In this study, the highest mean point was observed for “easy movement among screens” (3.76), “directly returning to the main menu from each screen” (3.65), “easy switching among different levels of menu” (3.63) and “existence of enough help in software” (3.63). In ISO9241 questionnaire, part 10, system controllability for the user was stated as one of the seven principals of system usability and requirements of “easy movement among screens”, “directly returning to the main menu from each screen”, “easy switching among different levels of menu” and “existence of enough help in the software” were present there (8). Ravden and Johanson (1989) mentioned flexibility and controllability and requirements of “easy access to the main menu from every part of the system” and “returning to previous step easily” in their general booklet of evaluation of usability among users’ interaction (22). In addition, Tognazzini (2003) considered status inquiry as one of the rules of usability (28). Controlling has been mentioned as one of the criteria of usability in SUMI (30). As the users of information system must be allowed to control paths and steps and easily transmit among different levels of software and work with system while feeling dominated on software function; therefore, designers of HER systems must observe requirements of system controllability while designing systems. In this study 18 requirements of Conformity with user expectations that had been offered to experts for poll were finally confirmed. In ISO9241 questionnaire, part 10, eight requirements have been presented for the subject of Conformity with user expectations (8). In this study, requirements of “existence of standard procedure for doing similar and related functions”, “similar icons, symbols and other visual information all over the program”, “similar function of similar operational keys all over program”, “constancy in using different colors all over the system”, “similar method of displaying in all screens of the system”, “displaying similar information items in the same form in each displayed place”, “similar format of entering special kinds of information all over the program”, “similar method of entering information all over the program”,” similar method of selecting options all over the program”, “similar method of system respond to special function of user at all times”, “similar functions for moving the cursor all over the program”, “the cursor being appeared in the same default place on similar screens” have been confirmed by users, while in ISO questionnaire, requirement of “constancy in designing the software” is emphasized. In the present study, the highest mean point was calculated for the requirements of “conforming the format of displaying information to method of entering information in the system” (3.68), “similar format of entering special kinds of information all over the program” (3.65), “conforming the colors used in designing the software to the general understanding of colors” (3.65) and “conforming abbreviations, acronyms and other numeral alphabetic information with the usual method” (3.65). In ISO9241 questionnaire, part 10, “conformity with user expectations” has been stated as one of the seven principals of system usability (8). Ravden and Johanson (1989) mentioned the criterion of “conformity with user expectations” and requirements of “conforming format of displaying information to method of entering information in the system”, “similar method of entering information all over the program”, “conforming the colors used in designing the software to the general understanding of colors and conforming abbreviations”, “acronyms and other numeral alphabetic information with the usual method” in their general booklet of evaluation of usability among users’ interaction (22). Moreover, Dix et al. (2004) mentioned “capability of predicting and being familiar” as related issues (31). Tognazzini (2003) stated prediction as one of the principles of system usability (28). It is necessary for software designers to design EHR in a way that system function is expected for users and conformation and consistency of system function are considered in all parts. In this study, 21 requirements of error tolerance that had been offered to experts for poll were finally confirmed. In ISO9241 questionnaire, part 10, 15 requirements were presented for the subject of error tolerance (8) among which the requirements including “preventing illegal user functions”, “possibility of testing possible functions without processing and making problems”, “assuring users of amending all errors recognized before processing”,“providing canceling options for the user (such as undo option) in order to cancel errors”, “informing the user if information entered is more than the space available and understandable error messages” had not been mentioned. In the present study, the highest mean point was calculated for the requirements of “understandable error messages” (3.73), “returning to previous function easily if a mistake is made” (3. 71), “assuring users of amending all errors recognized before processing” (3.71). In questionnaire standard ISO9241, part 10, error tolerance has been stated as one of the seven principals of system usability (8). Ravden and Johanson (1989) have mentioned criteria of “preventing and amending errors” and requirements of “assuring user of amending all errors recognized before processing” in their general booklet of evaluation of usability among users’ interaction (22). Moreover, Dix et al. (2004), Nielson (1993) and Tognazzini (2003) have mentioned recoverability, preventing from error/wrong messages and preserving users’ function as the rules of usability, respectively (26, 27, 31). Regarding ease and speed of returning to the previous step and whether the function has important effects on users in preventing them from wasting their time to amend errors and as a result in making accurate data, designers of EHR software are recommended to easily provide the requirements of error tolerance such as possibility to amend errors and capability to return to the system.

In this study, 7 requirements of suitability for individualization that had been offered to experts for poll were finally confirmed. In ISO9241 questionnaire, part 10, six requirements have been presented for the subject of suitability for individualization (8). However, the requirement of “possibility of selecting the method of entering information in the system” has not been mentioned in ISO9241 questionnaire. In this study, the highest mean point was calculated for “software’s compatibility with users’ level of skill and knowledge” (3.52). In questionnaire of standard ISO 9241, part 10, suitability for individualization has been stated as one of the seven principals of system usability and “requirement of software’s compatibility with users’ level of skill and knowledge” has been mentioned too (8). In addition, Dix et al. (2004) and Tognazzini (2003) have referred to capability to customize and system autonomy as the rules of usability, respectively (28, 31). Due to the fact that system users must individualize the system based on their need, it is necessary for designers and developers of EHR to consider and design the capabilities related to software individualization such as “possibility of changing forms, screens and menus in accordance with user’s taste”, “software’s compatibility with users’ level of skill and knowledge”, “possibility of regulating the volume of displayed information on each screen”, “possibility of changing the name of orders, subjects and functions in proportion to user’s dictionaries”, “possibility of regulating parameters of input tools in accordance with user’s needs” and “possibility of regulating response time of software in accordance with user’s working speed”. In this study, 19 requirements of suitability for learning that had been offered to experts for poll were finally confirmed. In ISO9241 questionnaire, part 10, eight requirements have been presented for the subject of suitability for learning (8). In ISO9241 questionnaire, requirement of “easily re-learning of working with system after a long interval” has been mentioned that is not presented in the present study. Moreover, in present study, requirements of “using simple English Language or user’s preferred language in software”, “learning the system without making mistake for the user”, “reminding names in software easily”, “possibility of asking for help in each part of the system”, “direct access to help and no need to look for unnecessary information”, “using help to look for information about other parts of the system”, “explaining requested information by the user”, “presenting comprehensive and accurate description in printed help”, “finding needed part in printed help easily”, “conformity of all kinds of help and user support to their task”, “keeping and updating all kinds of user guide”, “running and preserving the software without getting help from software providers” were included which had not been mentioned in ISO questionnaire. In the present study, the highest mean point was calculated for requirements of “becoming skillful in using the software by presented explanations” (3.71) and “using software correctly without asking colleagues for help” (3.71). In questionnaire of standard ISO 9241, part 10, suitability for learning has been stated as one of the seven principals of system usability and requirement of “software’s compatibility with users’ level of skill and knowledge” have been used (8). Shneiderman et al. (1995) have stated four factors including screen, terminology and information system, learning and system usability in recognizing strengths and weaknesses of the system for users’ interaction with information systems (32). In addition, Dix et al. (2004), Tognazzini (2003), and Nielson (1993) have stated that suitability for learning is one of the rules of system usability (27, 28, 31). Moreover, capability to learn and easily learning the information system have been mentioned in SUMI and QUIS, respectively (30, 33). A suitable information system must make it possible for users to learn the system by themselves and without much effort. Also, developers and designers must regard these characteristics as the criteria of system usability at the time of designing. In this study, 18 requirements of Visual Clarity that had been offered to experts for poll were finally confirmed. This area has not been covered standard ISO9241 questionnaire, part 10. In the present study, the highest mean point was calculated for “visually clear active window in software” (3.73), “easily watching and reading information on the screen” (3.73), “clarity of pictures and characters on the screen” (3.71) “nicely-ordered display of the screens” (3.71). Darbyshire (2000) has mentioned clarity of screens as one of the indexes of user-friendliness of the computer information system (29). Ravden and Johanson (1989) have studied users’ interaction with criteria of visual clarity and requirements of “visually clear active window in software” , “easily watching and reading information on the screen” , “clarity of pictures and characters on the screen” and “nicely-ordered display of the screens “in their general booklet of evaluation of usability (22). Designers of EHR are recommended to consider these requirements in order to increase visual clarity in any software and make clear and the information on the screens well-organized so that they can be easily read. In this study, 8 requirements of auditory presentation that had been offered experts for poll were finally confirmed. They had not been mentioned in questionnaire of standard ISO9241, part 10 (8). In the present study, the highest mean point was calculated for “easily stopping auditory presentation” (3.36) and “capability to present auditory items and display information on the screen at the same time” (3.36). It is recommended that voice be used in order to transmit the information from EHR to users and designers and developers of electronic health records consider it as one of the criteria of usability.

CONCLUSIONS

In the present study, a pattern was offered to users for poll to help designing and evaluating usability of EHR in nine subjects including Suitability for the task, Self-descriptiveness, Controllability, Conformity with user expectations, error tolerance, Suitability for individualization, Suitability for learning, Visual Clarity and Auditory Presentation. In this pattern of confirmed requirements, subject of Suitability for the task makes it possible for users to do their routine tasks effectively and efficiently using EHR. Confirmed requirements in the subject of Self-descriptiveness help to understand expressions, definitions, shapes and messages displayed in EHR and prevent wasting users’ time and doing incorrect tasks. Confirmed requirements in the subject of Controllability make it possible for users to control paths and steps and easily transmit among the software levels and give them a feel of domination on the software. Confirmed requirements in the subject of Conformity with user expectations make system functions expectable and make all software parts consistent. Confirmed requirements in the subject of error tolerance make it possible for users to amend errors and mistakes and return to system easily and securely. Confirmed requirements in the subject of Suitability for individualization make it possible to make necessary changes in some parameters of the system in order to do related activities for users and allow users to personalize the system based on their working needs. Confirmed requirements in the subject of Suitability for learning make it possible for users to learn how to work with the system without using much effort. Confirmed requirements in the subject of Visual Clarity make the information on the screen clear, well-organized and readable and confirmed requirements in the subject of Auditory Presentation make it possible for users to receive auditory information in the form of voice from information system. Therefore, considering that being user-friendly and paying attention to usability requirements of information system are as effective as functional requirements of these systems in accepting the system with maximum efficiency, effectiveness and satisfaction by users, it is recommended that the pattern presented in this research is emphasized by designers of systems and health-care institutes that would buy this system in order to evaluate it..