Presenting an Evaluation Model for the Cancer Registry Software.

OBJECTIVES: As cancer is increasingly growing, cancer registry is of great importance as the main core of cancer control programs, and many different software has been designed for this purpose. Therefore, establishing a comprehensive evaluation model is essential to evaluate and compare a wide range of such software.
METHODS: In this study, the criteria of the cancer registry software have been determined by studying the documents and two functional software of this field. The evaluation tool was a checklist and in order to validate the model, this checklist was presented to experts in the form of a questionnaire. To analyze the results of validation, an agreed coefficient of %75 was determined in order to apply changes. Finally, when the model was approved, the final version of the evaluation model for the cancer registry software was presented.
RESULTS: The evaluation model of this study contains tool and method of evaluation. The evaluation tool is a checklist including the general and specific criteria of the cancer registry software along with their sub-criteria. The evaluation method of this study was chosen as a criteria-based evaluation method based on the findings.
CONCLUSION: The model of this study encompasses various dimensions of cancer registry software and a proper method for evaluating it. The strong point of this evaluation model is the separation between general criteria and the specific ones, while trying to fulfill the comprehensiveness of the criteria. Since this model has been validated, it can be used as a standard to evaluate the cancer registry software.

1. INTRODUCTION

Nowadays, cancer is one of the diseases threatening human life seriously (1). It is increasingly growing in a way that it is one of the eight causes of death around the world (2). In many countries, an increasing growth has been reported for this disease (3-5). According to the studies conducted in Asia by 2020, the rate of mortality from cancer will increase by one and a half times (6-7). In Iran, the third cause of mortality is cancer and it is predicted to be growing. Therefore, the implementation of cancer control programs is essential (8-9). The main purpose of cancer control is to mitigate the prevalence and mortality rate of cancer. Cancer control programs are possible through the implementation of evidence-based systematic interventions to prevention, early diagnosis, and treatment (10). However, qualified data is essential to implement cancer control program, to find causes of cancer and preventing it, and specify the objectives and priorities of the activities. Therefore, cancer registry is extremely important to fulfill the mentioned objectives (10-11).

Cancer registry is a systematic process of gathering, storing, analyzing, and reporting relevant data to cancer (12) which is used to plan cancer control programs.

As the evaluation in the field of health care is of vital importance, many studies have been done in this field such as, Huang’s research titled “a systematic investigation: a review on approval of medical software”. This study discusses the necessity of evaluating medical software and also emphasizes that it is necessary to apply accurate validation and evaluation methods in terms of both software production and the produced item to ensure the quality (15). Burkel et al., in a study titled “the evaluation of clinical information system, what can be evaluated and what cannot?” have emphasized the importance of evaluating these clinical information systems due to their notable use in health care, treatment and their remarkable effects on the outcome of patient care, supporting the decisions, and the quality of care (16).

Rafrez in a study titled “Case Completeness and data quality in central cancer registries and their relevance to cancer control” remarked the quality and comprehensiveness of cancer registry data. He has also considered providing solutions to obtain and evaluate the quality of cancer registry data as essential (17).

Due to the importance of cancer registry in cancer prevention and control; and improving patient care, the necessity of evaluating the cancer registry software, and lack of an evaluation model for such software in Iran, the researchers seek to provide a comprehensive evaluation model to evaluate cancer registry software.

2. MATERIAL AND METHODS

The goal of present study is to provide a proper model to evaluate the cancer registry software. For this reason, data collection has been done through the in-depth study of texts including articles, documents, standards, and valid models. So, the characteristics and criteria of the software were divided into two general and specific criteria. To determine the general criteria, validated qualitative models were used. The specific criteria are different depending on the type of the software. In order to determine the specific criteria, the important and widely used software of Canreg and Abstract Plus in the field of cancer registry are studied in this research through the in-depth study of the texts. The evaluation tool includes four general and specific criteria. Finally, the established checklist was provided to be validated in the form of a questionnaire filled by 8 experts including 3 in the field of health information management and medical informatics and 5 oncologists. To decide on the sub-criteria for each of the specific criterion, the agreed coefficient of 75% was considered. That is, if more than 75% of the experts consider the presence of a sub-criterion as necessary, that sub-criterion will remain in the list; otherwise, according to the experts, it will be modified or removed from the list. Analyzing the collected data was done through descriptive statistics. Finally, the final version of evaluating tool of the cancer registry software was confirmed by the experts..

3. RESULTS

Four main groups were considered for general criteria including:

The security of the software.

The maintenance and development of the software.

The interoperability of software.

The user-friendliness of the software.

Furthermore, based on the findings, four main groups were considered for specific criteria of the cancer registry software including:

The facilities of admission.

The ability to record, store, and search the data.

Reporting.

Supporting clinical decisions.

Following tables show the general and specific criteria; and their sub-criteria (Tables 1-8).

According to the studies and feedback of experts and professionals, the evaluation tool was implemented in the form of a checklist including the specific and general criteria of the cancer registry software and their sub-criteria are presented under each related criterion. The evaluation method of this study is the Criteria-based method which is considered as an appropriate evaluation method to cover the objectives of the present study, as this method allows comparison with its deductive approach. The comparison of different criteria with this method is objectively possible, so that the result of the evaluation will be more tangible.

The first general criterion, security, and its sub-criteria got the 100% agreement of the experts (Table 1). The second general criterion, the maintenance and development of the software, and its sub-criteria got the agreement of all the experts based on the agreed coefficient, and just the 4th sub-criterion got one disagreement and got the acceptance of 87.5% (Table 2). All the sub-criteria of the interoperability criterion of the software, were confirmed (Table 3). For the last general criterion, the user-friendliness of the software, the 6th sub-criterion got the acceptance of 75% and the agreement of all experts (Table 4). Totally, all of its sub-criteria were agreed on by all the experts.

For the specific criteria, the first criterion, the facilities of admission, was accepted by all the experts and professionals and just its 4th sub-criterion got the acceptance of 87.5% (Table 5). All the sub-criteria of the second specific criterion, the ability to record, store, and search the data, except the eleventh sub-criterion were confirmed by the coefficient of 75% and the rest were confirmed by 100% (Table 6). The last two specific criteria, reporting and the ability to support the clinical decisions were confirmed by all the experts and professionals (Table 7 and 8). Generally, based on the agreed coefficient of 75%, none of the sub-criteria were removed and all remained in the final checklist.

Finally, the evaluation model of the cancer registry software consists of 4 parts including the general and specific criteria of the cancer registry software. The evaluation tool and evaluation method were presented after validation.

4. DISCUSSION

For Using software in the field of health care has caused many progresses in various areas including offering service and improving patient care. However, if the software is not of a high quality and does not function properly, there is a possibility of making incorrect decisions and causing heavy economic losses to the organization. It can also cause serious damage to the individual’s health and public health system (15). So, the software of health care requires accurate quality assurance processes (16). As a result, different quality models have been provided to evaluate the quality of the software in general, such as the quality models by McCall, Boehm, Dromey, FURPS, ISO-9126 (18). Each of these models has a series of qualitative characteristics or factors that can be considered as the general criteria of the software. Organizations active in the field of health care have presented various standards and criteria for different software in this domain. For example, ISO-9126-1 which is an international quality model, considers six main criteria along with their sub-criteria includes function, reliability, usability, efficiency, maintainability and portability (19).

In ISO-9126-1 security is a sub-criterion of the function but in the present study, security is considered as a main criterion. The third volume of the cancer registry standards mentions protective policies of cancer registry which are considered as the security sub-criteria in this research. Some of them are determining classified access levels, determining functional roles and their access levels (20).

Another general criterion is the maintenance and development criterion of the software. The standard of IEEE 1219 defines the software maintenance as applying changes in a software product after delivery to correct errors, improve efficiency, adapt to the changed environment (21). In this study, two capabilities of maintenance and development of the software are provided in the form of a criterion. Regarding the definition of software maintenance, software must be able to evolve and expand according to the changes and users’ needs. These two concepts have a stronger connotation when they are used together and the sub-criteria they cover will not be repeated. Another criterion to consider for the evaluation of all software such as the cancer registry is interoperability with other software. HIMSS defines interoperability in the health care sector as communication, data exchange and using the exchanged data. The data exchange can be done among centers such as hospitals, laboratories and pharmacies (22).

Another characteristic of software is being user-friendly. In cancer registry software, the proper designing of user interface can attract users and also facilitate understanding, learning, working, and controlling the software thereby preventing errors and harming the patients in this study. Therefore, in this research, the user-friendly feature is considered as a separate criterion along with some sub-criteria. Finally, four main general criteria have been considered for the cancer registry software which are as follows:

The security of the software.

The ability to maintain and develop the software.

The interoperability.

The user-friendliness of software.

To determine the required specific criteria of the cancer registry software, the widely used cancer registry software of Abstract Plus and CanReg in the field of cancer registry were reviewed carefully. Moreover, in this research, other studies involving criteria related to the cancer registry software in the field of health care were reviewed (23-28). One of these studies is the report published by the American Medical Association (2010) on the evaluation criteria of the service management software. In this report, capabilities such as the ability to make appointments for the patient and reminding the patient of his appointment, the timing of resources, the ability to record and maintain demographic information and information about patient insurance are mentioned (26). Furthermore, the CCHCIT has provided four main specific criteria of documentation, management, reporting and supporting therapeutic decision making to evaluate outpatient services and has defined a sub-criterion for each of them (27). The software of Canreg and Abstract Plus has the ability to document, search the data and report (23-24). The key steps of registry have been included in this study and have been considered in specialized criteria such as: The facilities of admission; The ability to record, store, and search the data; Reporting; Supporting clinical decision-making.

Overall, in this study, we have tried to categorize the specific criteria in a way that all the necessary sub-criteria will be covered. Finally, the categorization of the specific criteria was done in such a way that it can cover all the necessary sub-criteria. This classification includes four categories:

The facilities of admission.

The ability to record, maintain and search the data.

Reporting.

Supporting clinical decision making.

Regarding the evaluation method, the studies showed that a large part of the evaluation studies and research in the field of information systems focus on the criteria-based evaluation method. One of the advantages of this evaluation method is implementing standards and ensuring system quality, and according to the growing need for standards and instructions, using this method, in comparison of other methods, is increasingly growing (29-31). However, the criteria-based evaluation method is a deductive approach (32). As a result, it facilitates comparing different software on the market and makes the evaluation more objective and more tangible. Therefore, the criteria-based evaluation method was chosen as a proper method for this study.

5. CONCLUSION

The presented model in this study has been developed by extensive review of resources and documents and the study of cancer registry software. The aim of this model is to support cancer registry processes with the maximum integrity in covering the general and specific criteria in order to achieve the goals of cancer registry. One of the advantages of this model is to distinguish between the general and specific criteria facilitating the evaluation process, in addition this model was validated and approved by the associated experts. Thus, the designers and programmers can use it as a basis to design, implement, and produce cancer registry software. Companies producing the cancer registry software can use this model to improve their software. Using this model to build and upgrade the cancer registry software and selecting the proper software can pave the way for the interoperability of such software.