User requirements for exploring a resource inventory for clinical research.

The CTSA Inventory of Resources Explorer facilitates searching and finding relevant biomedical resources in this rich, federated inventory. We used efficient and non-traditional formal usability methods to define requirements and to design the Explorer, which may be extended to similar web-based tools.

Introduction

Clinical research requires the use of diverse and often non-obvious resources. Semantic and technological advances in clinical and translational medicine have produced many available resource inventories with query interfaces, including the neuroscience information framework (NIF), Bioportal biomedical ontology; the federated National Centers for Biomedical Computing (NCBC) Biositemaps; the local and regional Rochester Resource Inventory and Atlanta Clinical and Translational Science Institute (ACTSI) and the Clinical Research Network (CRN). The goal of such resource inventory browsers is to help diverse user groups efficiently retrieve relevant information that will lead to productive outcomes. Consequently, usability-based designs are critical for these tools. In our NIH-funded Clinical and Translational Resource Explorer project (http://biositemaps.ncbcs.org/cirwp/index.html), we aim to build this user-centeredness into the Explorer tool. To do so, we integrate heuristic evaluations, task-oriented use case driven design, and rapid prototyping. We argue that these methods can build usability into inventory query Benefits resulting from this method include the creation of user requirements early and the development of a prototype with baseline usability that can then be tested with users formally for deeper usability improvements.

We present here our set of user-centered objectives and requirements, which we have not seen synthesized previously in one place in research literature. These systematized objectives and requirements can guide designing for user-centeredness in similar tools. We also present our rapidly developed and easy-to-use and fully functional prototype and explain its baseline usability features. Our next phase of systematic user performance testing is underway. Our work advances the goal of developing optimal human-computer interaction models for this class of clinical research tools.

Background

The Clinical and Translational Resource Explorer that we present in this article is a web-based front-end to an inventory of over 800 resources from 38 institutions with Clinical and Translational Science Awards (CTSA) funding. The inventory and the Explorer have been developed through a highly collaborative effort of 10 institutions and 40 cross-disciplinary specialists.

An underlying information model defines the set of biomedical resources properties, as follows:

In this model, Resource Type, Area of Research, and Research Activities follow standardized annotations from the Biomedical Resource Ontology (BRO) [1]. In the BRO, for example, the top level Resource Type categories include: software, material resources, service resources, information resources, funding resources, people resources, or training resources. Multiple sub-levels make up each type and are mapped to resources.

Preceding our interface design and development work, two earlier prototypes were proofs of concept and open for public use without usability assessments. Our third version of our Explorer interface specifically focuses on user-centeredness.

Established information retrieval (IR) principles and user models in the research literature have guided our user-centered designing [3-4]. For example, research establishes that faceted search facilitates users immediately seeing important meta-information about inventory contents to evoke more informed querying. This and other query design choices are described below in Results and Discussion. From a user perspective, however, when complicated queries are needed, advanced search designs are better.

Unlike facet searches, advanced searches let users select one or more pre-defined items within a category and, if desired, across categories in one “query statement.” Free-text searching is also a prime user need. It lets users define and enter self-defined terms to retrieve results relevant to their purposes. We integrate all three methods into a design that clearly separates and gives users easy access to interface mechanisms for the three distinct purposes. This combination accords with IR research findings that tools should combine facets, keyword searching, and advanced search capabilities [5-6]. We also draw on IR usability test findings that emphasize the importance of keeping hierarchical relationships visible to users; providing flexible sorting and filtering; and allowing search expansion and refinement [5]. Assessment methods leading to these findings, however, are time-consuming.

We were attuned to these findings but keenly aware that we could not conduct similar, formal user performance testing until we had a prototype with sufficient usability to support actual task flows. All too often, user testing results in relatively inconsequential findings if the tested tool does not have sufficient user-centered support [7]. We also wanted our eventual user performance testing to provide results that were not “low hanging” usability problems but rather deeper semantic problems, e.g. those tied to satisfying users’ expectations for and understanding of search terms, navigation, and cumulative filtering. Toward this end, we incorporated user-centered guidance in design and development. Supported by research that shows the effectiveness of heuristic evaluations (HE) when combined with other usability methods, we combined HEs with task-oriented use cases to inform iterative design and prototyping and solicited informal user feedback on prototypes [8]. Heuristic evaluations are low cost inspections of user interfaces to find and fix problems related to clarity, consistency, ease of operations for access, use and navigation

Method

Assessments through heuristic evaluations

We conducted heuristic evaluations of the two early prototypes that preceded our work on the version 3 Explorer prototype. We also heuristically evaluated the NIF search tool as a positive model as it has undergone previous usability testing that informed its design [9]. Our heuristic evaluation instrument was developed at the National Center for Integrative Biomedical Informatics (NCIBI) and has been applied with success to improve web-based NCIBI tools for biomedical informatics [10] Two of us applied the instrument reliably to the three tools, rating problems on a scale of 0 (no problem) to 5 (severe).

We analyzed ratings by grouping usability issues across tools into serious problems (4–5 scores) and less serious – cosmetic and minor – problems (1–3 scores). From findings we composed usability guidelines for inventory exploration tools.

Selection of interface designs based on findings, use case design and review sessions

Our starting point was with a semi-functional early prototype for version 3 that one of us, as the developer of earlier versions, created as a new approach that might be more user-friendly. This prototype included facet searching. To modify this prototype design, as needed, we sought to maintain the positives from comparative heuristic evaluations while avoiding the shortcomings. From HE findings, we defined usability-based tool objectives and drew from and added to various task- and goal-based use cases that large CTSA groups had composed to characterize diverse users and their approaches to inventory exploring. We were also guided by our first hand knowledge of the demands and constraints of use contexts. We worked from ten hypothetical use cases. Due to space limitations we give only two examples here:

A researcher is studying physiology and metabolism following the intake of a nutritional supplement. She already makes use of a calorimeter at her home institution, but is not aware of a doubly labeled water technology to quantitate oxidation that is available at another institution- useful for various applications within the study of metabolism and physiology.

An investigator runs clinical trials involving genetic engineering. She needs services and resources to extract DNA from participant blood samples and to conduct screening.

We held a series of weekly 1–2 hour design sessions in which we employed other usability methods. We walked through several use cases with the initial prototype, evaluated support against our tool objectives, and updated the prototype accordingly. At times we constructed two or more prototype designs compared them to determine optimal support for users’ needs. As design options/prototypes evolved, we iteratively conducted comparative assessments of IR websites for additional ideas. Prototypes changed often, and our favored approaches were subjected to repeated and continued use case walkthroughs. When satisfied, we documented design options and the rationales for our selected design, along with the user requirements. We also enumerated high level functional requirements from our user requirements document in order to prioritize development activities.

Iterative priority setting, prototyping and user feedback

Next we held several cross-team meetings with colleagues in the larger inventory development project. In these meetings, we jointly discussed the impacts of our requirements on back-end processes and data and defined and prioritized tasks for front and back-end development. Concurrent with the interface development, we informally gathered user feedback on the evolving prototype from four users. We aimed to determine what improvements were necessary to achieve baseline usability. Baseline usability could encourage initial use and, importantly for our purposes, would provide a stable, user-friendly version for subsequent systematic user performance testing.

Users who did not take part in the Explorer design process were drawn from CTSA programs at a variety of institutions, all of whom were familiar with the resources and institutions participating in the inventory. Each user participated in a one hour hands-on session and carried out pre-defined use cases with the Explorer, giving stream-of-thought feedback as they progressed. Users’ feedback guided further design considerations and user requirements.

Results and Discussion

User-centered designs based on HEs

The following URLs show displays of the interfaces that we evaluated heuristically:

Top usability successes and problems we uncovered are captured in our objectives for a usable query tool for clinical researchers listed below. They coincide with findings in the IR literature from more extensive usability methodologies, and include the following:

Flexible search and search refinement: For diverse user needs, the tool should (1) provide quick and simple queries that let users progressively filter by one search term at a time and (2) allow searches on multiple criteria. Users should be able to refine searches, keep oriented, and go back as needed to prior queries.

Ease of access: The tool should enable users to get help, find out more about the search tool and resources, and see examples to get started quickly.

Transparency and accuracy. The tool should have clean metadata, consistent and understandable terms denoting pre-defined categories and their values, no duplicates, complete, and accurate entries in records, the latest tool update, and resource counts.

Navigation. Organization of displays should help users navigate quickly to find relevant information, including link-outs; help them see what resources are available; and allow them to organize results.

Save and share: The tools should let users save, print, and/or share results in a variety of formats.

Data access and social computing: The tools should establish a permissions system for public and proprietary resources. Through web log tracking, it should indicate resources accessed by other people.

Interface designs based on findings

In our designs sessions, we considered numerous design choices based on use cases, focusing on each objective and overlaps between objectives. We continuously refined our objectives and user requirements accordingly. For example, for flexible search and browse we intended to offer three search options: free text search, facets, and advanced search.

Walking through use cases with the initial prototype reinforced the importance of hybrid searches with facets and free text as an easy means for progressively narrowing down on user-defined terms and structured information. We found that advanced search with check boxes rather than drop down lists kept the information context visible. However, flattening the BRO hierarchies for ready access to facets and advanced search led to the need to limit the display of the vast BRO. We chose to display only the first two levels. We also saw that users would need to understand the hierarchical relationships so created a hierarchy browser view (to be implemented in a later version).

We went through similar deliberations for the other objectives. Our final decisions include the following user-centered highlights, some for later versions:

Ready access to help and orienting materials.

Multi-mode search: facets, keyword, advanced.

Simple means (to users) for refining a search or choosing to start a new search. Mechanisms for keeping users oriented (e.g. breadcrumb search paths, counts of resources).

Cues showing users the fields searched, synonyms, and the search terms in the results.

Ease of navigating the BRO categories for facets and advanced searches. Option for interactively selecting BRO facets from a hierarchy view.

Results in tables with sortable columns, links to resources, contact names, and records.

Geographical ways to locate resources.

Mechanisms for saving/exporting.

Front and back-end mechanisms for social aspects – to be built out over time.

We found that outcomes of use case walkthroughs and iterative prototyping revealed the same demands that more extensive assessments in the IR literature do. These convergent findings are not surprising. Rather, they suggest that enough is known about usability in IR to make it possible to uncover important findings through discount inspections and rapid prototyping.

Figures 1a and 1b at the end of the article show our current prototype with annotations. The Facet view (Figure 4a) displays pre-defined high level categories of properties pertaining to the inventory items. Under each category are it constituent values, or facet values. For example, for the category Institution, the facets are the names of universities. A click on a facet retrieves results with that property. For BRO properties, the tool retrieves results for the category and its sublevels. When users progressively click facets, results accordingly get narrowed down (an implicit AND in Boolean logic). Users can remove filter terms and broaden results again by interacting with the breadcrumb display of all terms clicked so far. On this display free text search is available, and users can do a hybrid search with facets and free text.

1a.

Faceted & free text search. A combination of both pictured in this query (biositemaps.ncbcs.org/cirwp/search.html)

1b.

Advanced search. Accessed through a “Go” button on the facet search screen, it has check boxes with implicit logic. The BRO hierarchies (not shown here) are flattened into comparable (alphabetically ordered) lists of properties.

The Advanced search screen (Figure 4b) displays all terms in each category (facet values). Users select as many items as they want from a category for results containing any of these traits (an implicit OR in Boolean logic). Users may click one or more items across categories, as well, which retrieves only results that share traits of the selected top level categorical items (an implicit AND in Boolean logic).

User feedback

Subsequent real user walkthroughs of sample use cases show that the current prototype fosters productive outcomes. Looking for microarray analysis services, a user called up the Explorer, clicked the top level BRO class “Services” (see Figure 4a), and retrieved 296 resources. Next typing “microarray” in the search box, the user narrowed the list to 12 relevant resources. The user quickly sorted the results table by Institution and found services near the user’s home base. With one more click, the user linked out to an email address of the contact person for the resource.

User feedback has strengthened existing requirements and added new ones. For example, users reinforced the need for better hierarchical relationship cues; better record accuracy and completeness; and more intuitive search logic in Advanced Search

Conclusion

Our study has produced task-based objectives and user requirements for tools that support users in exploring a rich resource inventory for biomedical research. We have developed a user-friendly prototype that will now enable us to conduct next-stage user testing. We expect to get past low-hanging problems and address the more difficult issues, such as those specified above by initial informal users. These assessments will help advance HCI models for biomedical resource inventories

Resource name	Technical support
Organization	Documentation available
Center or Institute	Resource sharable
Research program	Contact person data
Description	Resource Type
URL	Area of Research
Keywords	Research Activities