Pernille Heyckendorff Lilholt1, Morten Hasselstrøm Jensen2, Ole K Hejlesen3. 1. Department of Health Science and Technology, Aalborg University, Aalborg 9220, Denmark. Electronic address: phl@hst.aau.dk. 2. Department of Health Science and Technology, Aalborg University, Aalborg 9220, Denmark. 3. Department of Health Science and Technology, Aalborg University, Aalborg 9220, Denmark; Department of Computer Science, University of Tromsø, Tromsø 9019, Norway.
Abstract
OBJECTIVE: The aim was to evaluate the usability of the design of the telehealth system, named Telekit, developed for the Danish TeleCare North Trial, early into the design process in order to assess potential problems and limitations which could hinder its successful implementation. METHODS: Five experts, including one who pilot-tested the Telekit system, individually evaluated its usability and its compliance with Jakob Nielsen's ten usability heuristics for interaction design. Usability problems were categorised according to Rolf Molich's severity classification. RESULTS: The five experts identified a total of 152 problems in the Telekit system, each identifying 22-40 problems. 86 (57%) out of the 152 problems were identified only once. All heuristics were used, but the three most frequently used were: "Match between system and the real world" (32%), "Consistency and standards" (13%) and "Aesthetic and minimalist design" (13%). The most widely used classifications were: "Improvement" (40%) and "Minor problem" (43%). CONCLUSION: Heuristic evaluation was an effective method for uncovering and identifying problems with the system. The consistent finding of particular usability problems confirms that the development of a telehealth system should pay particular attention to user aspects. The most serious problem was the inability of the system to inform users of how to perform measurements correctly and to "speak the users' language". The problems found in the heuristic evaluation have led to several significant changes in the telehealth system. We suggest that heuristic evaluation always be followed by user tests to evaluate the design of telehealth systems.
OBJECTIVE: The aim was to evaluate the usability of the design of the telehealth system, named Telekit, developed for the Danish TeleCare North Trial, early into the design process in order to assess potential problems and limitations which could hinder its successful implementation. METHODS: Five experts, including one who pilot-tested the Telekit system, individually evaluated its usability and its compliance with Jakob Nielsen's ten usability heuristics for interaction design. Usability problems were categorised according to Rolf Molich's severity classification. RESULTS: The five experts identified a total of 152 problems in the Telekit system, each identifying 22-40 problems. 86 (57%) out of the 152 problems were identified only once. All heuristics were used, but the three most frequently used were: "Match between system and the real world" (32%), "Consistency and standards" (13%) and "Aesthetic and minimalist design" (13%). The most widely used classifications were: "Improvement" (40%) and "Minor problem" (43%). CONCLUSION: Heuristic evaluation was an effective method for uncovering and identifying problems with the system. The consistent finding of particular usability problems confirms that the development of a telehealth system should pay particular attention to user aspects. The most serious problem was the inability of the system to inform users of how to perform measurements correctly and to "speak the users' language". The problems found in the heuristic evaluation have led to several significant changes in the telehealth system. We suggest that heuristic evaluation always be followed by user tests to evaluate the design of telehealth systems.