Reliability of trauma coding with ICD-10.

PURPOSE: The reliability of trauma coding is essential in establishing the reliable trauma data and adopting efficient control and monitoring policies. The present study aimed to determine the reliability of trauma coding in educational hospitals affiliated to Shahid Beheshti University of Medical Sciences, Iran.
METHODS: In this descriptive cross-sectional study, 591 coded medical records with a trauma diagnosis in 2018 were selected and recoded by two coders. The reliability of trauma coding was calculated using Cohen's kappa. The data were recorded in a checklist, in which the validity of the content had been confirmed by experts.
RESULTS: The reliability of the coding related to the nature of trauma in research units was 0.75-0.77, indicating moderate reliability. Also, the reliability of the coding of external causes of trauma was 0.57-0.58, suggesting poor reliability.
CONCLUSION: The reliability of trauma coding both in terms of the nature of trauma and the external causes of trauma does not have a good status in the research units. This can be due to the complex coding of trauma, poor documentation of the cases, and not studying the entire case. Therefore, holding training courses for coders, offering training on the accurate documentation to other service providers, and periodically auditing the medical coding are recommended.

Introduction

Trauma is a pervasive phenomenon and has become a major global health concern, with negative effects on the individual, family and society., The global burden of trauma is very high, with injuries constituting 10% of all the deaths and >15% of disability-adjusted life years., Millions of deaths with trauma represent only a small fraction of the tens of millions of injured people, who are hospitalized or visit a general practitioner. Trauma and injuries occur frequently, which are associated with significant mortality. The worst consequence of physical injuries is loss of life. Non-fatal injuries impose heavy personal and social costs. To effectively manage and prevent trauma, it is important to identify the factors that may cause and affect it. The first step in developing countries, a harm-prevention strategy is injury monitoring. To this end, the availability of high-quality data on injury, assessment of the current risk of injury, identification of potential risk factors and establishment of a preventive intervention of injury are crucial. Therefore, accurate information is required on the mechanism of harm to inform prevention programs, and information retrieval standards should be established.

Coding of clinical data is integral to the compatibility and retrieval of standard medical information, and clinical reports depend primarily on coded data. Maintaining the desired quality of coding is significant for the accurate analysis and interpretation of clinically important data. According to Avila-Weil and Regan, the reliability, accuracy and completeness of the code are the most important components of coding quality. Coding reliability refers to obtaining the same results upon repeating the coding activity. In other words, the coding reliability is an agreement between different people in coding a diagnosis (external reliability), or an individual coding the same diagnosis at different time (internal reliability). Adopting a consistent encoding process that results in the reliable coded data is crucial to using these data because users will trust the data when they are convinced that the data encoding process is reliable. The reliability of the data indicates the stability of the data since unreliable information reduces the usefulness of the system.

Various studies on the reliability of medical coding have shown different levels of reliability. For instance, Peng et al. reported the International Statistical Classification of Diseases and Related Health Problems – Tenth Revision (ICD-10) coding agreement and reliability of ICD diagnosis codes in emergency department records was 86.5% and 82.2% at 3 and 4 digits levels, respectively, and reliability was 0.86 and 0.82, respectively, between auditors and hospital coders. The main causes of coding discrepancies between coders were the use of codes at different levels of specificity and the utilization of unspecified coding in practice. Daniels also examined the reliability of the coding related to a children's hospital and stated that the reliability of the data is poor. He cited limitations in the ICD, in which ambiguous medical records documentation and inadequate coder training are the reasons for the lack of code reliability.

In a trauma registry system, the data can be used to assess injury management, injury protocols and hospital statistics, only if they are reliable. An unreliable registry can mislead the hospital statistics at both regional and national levels. Considering the importance of trauma prevention and the accurate and reliable data obtained from high-quality trauma code and its external causes, the present study investigated the reliability of trauma coding as a component of trauma coding quality with ICD-10.

Methods

This descriptive cross-sectional study was conducted in 2018 to evaluate the reliability of the coding of trauma patients' records with ICD-10. The research population comprised all coded medical records with the final diagnosis of trauma in 2018 in 8 educational hospitals affiliated to Shahid Beheshti University of Medical Sciences (Iran). To collect the data, a researcher-made checklist was employed to evaluate the reliability of the coding related to the nature of trauma (main condition) and the coding of external causes of trauma (external causes). The validity of the checklist content had been confirmed by experts. To determine the sample size, each educational and medical center was considered as a class.

Using stratified sampling methods and calculating the sample size through Cochran's formula in infinite population and considering d = 0.05 and z = 95% confidence interval, 591 coded medical records were selected as the sample size.

In the equation, “n” denotes the sample size, “z” represents the value of the normal variable (1.96), “p” is the disagreement rate, “q” is 1-p and “d” is the significant level (0.05).

All components of medical diagnoses with the coding assigned by the coder working in the research units were recorded in the checklist. The two coders then recoded the diagnosis of the nature of trauma and the external causes of trauma at different times and places. Both coders were health information technology experts with >10 years of experience in the coding unit. After collecting the data via the checklist, they were inputted to SPSS version 20 and analyzed by Cohen's kappa test, which is often utilized to test the reliability. The importance of reliability is that it indicates the degree to which the data collected in the study accurately represents the measured variable. Cohen's kappa is a strong and standard statistical value and the correlation coefficient can vary from -1 to 1, where 1 indicates complete agreement between individuals. Cohen's kappa value < 0 is impossible in practice. The rating provided by McHugh was adopted to judge the status of the reliability coefficient using Cohen's kappa (Table 1).

Table 1

Interpretation of Cohen's kappa values based on McHugh's rating.

Value of kappa	Level of agreement	Percentage of reliable data
0–0.20	None	0%–4%
0.21–0.39	Minimal	4%–15%
0.40–0.59	Weak	15%–35%
0.60–0.79	Moderate	35%–63%
0.80–0.90	Strong	64%–81%
Above 0.90	Almost perfect	82%–100%

Results

The calculation of Cohen's kappa coefficient to examine the reliability of the coding related to the nature of trauma is presented in Fig. 1, Fig. 2. The agreement between Coder 1 and the primary coder on the coding of the nature of trauma in the research units was 0.75, suggesting average reliability (Table 1). The agreement between Coder 2 and the primary coder on the coding of the nature of trauma in the research units was 0.77, which indicated average reliability (Table 1).

Fig. 1

Agreement between Coder 1 and the primary coder on the reliability of the nature of trauma coding in the research units.

Fig. 2

Agreement between Coder 2 and the primary coder on the reliability of the nature of trauma coding in the research units.

In the next step, the agreement between Coders 1 and 2 and the primary coder about the external causes of trauma coding was examined (Fig. 3, Fig. 4). The agreement of Coder 1 with the primary coder on the coding of external causes of trauma in the research units was 0.57 which suggested poor reliability (Table 1). The agreement of Coder 2 with the primary coder on the coding of external causes of trauma in the research units was 0.58 which indicates poor reliability (Table 1).

Fig. 3

Agreement between Coder 1 and the primary coder on the external causes of trauma coding in the research units.

Fig. 4

Agreement between Coder 2 and the primary coder on the coding of external causes of trauma in the research units.

Discussion

Today, medical coding and its quality are becoming increasingly important due to their relationship with the quality of medical care. The reliability is one of the dimensions of coding quality. The usefulness of classified and coded medical data essentially depends on the uniform coding of similar entities, independent of the coder or coding time. Coding trauma cases is difficult due to the existence of multiple and complex coding rules. In this regard, Curtis et al. stated that accurate coding and evaluation of clinical information coding in trauma patients are integral to trauma management. Coding for trauma patients is generally difficult because providing and documenting medical care are complex processes. Care programs for such patients may comprise multiple types of care during unpredictable hospitalization, which leads to problems in coordinating different care and treatment services. The low quality of trauma patient codes is probably due to the complex nature of trauma, extensive patient records, imperfect documentation, multiple traumas and the use of multiple codes.

The results of the present study revealed that the reliability of the nature of trauma coding in the research units varies from 0.75 to 0.77, indicating the average reliability. As for the external causes of trauma coding in the research units, the findings showed that the reliability is 0.57–0.58, suggesting the poor reliability (Table 1). After investigating the reliability of cause of death coding, Antini et al. reported the reliability of 76.4% for all death and 80.6% for the main coding of death. Misset et al. also evaluated the reliability of coding related to an intensive care unit and stated that the agreement was 34% between two coders and only 18% among three coders. Furthermore, Daniels studied the reliability of the coding related to a children's hospital and concluded that the reliability of coding in this hospital is poor.

Little research has been conducted on the reliability of coding the external causes of trauma. Scott et al. reported the reliability of 0.94 and 0.97 for individual activity at the time of the trauma and the location of the trauma, respectively. Neil et al. stated that the reliability of the external cause of trauma coding was 0.80, intention (intentional or unintentional trauma) was 0.58, and location of the trauma was 0.44.

In the research units, no accurate and high-quality medical records, no cohesion and coherence in the documents and exited contradictions in the documentation caused the most errors and confusion of the coder. For example, radius trunk fractures were reported in the medical history sheet, but distal radius fractures were recorded in others sheet; the diagnoses were not recorded in the admission and discharge summery sheet. Moreover, the coders coded only by studying the medical history sheet, whereas other sheets and the operation report sheet were not examined. The number of multiple coding, strict rules of coding trauma, and lack of training of coders were the other factors affecting the reliability of coding. Coding rules are forgotten due to the absence of training or ineffective training, which affects the quality of the code. Failure to use the cover of the ICD-10 (Tabular list), read the entire medical records (by the coder), and complete the diagnosis in the admission and discharge summery sheet (by the physician) are other factors leading to the poor reliability of trauma coding. Also, the location of trauma, person's activity during trauma, and whether the trauma is intentional or unintentional has not been recorded in most cases.

According to Annest et al., the accurate medical records require to contain sufficient detail about the condition of the injury. However, evidence suggests that healthcare providers may not record these details because they see no reason to do so. In addition, sometimes they are unable to determine the details of the situation. For instance, an unconscious person may not be able to provide any information about how the injury has occurred. Stausberg et al. argue that some well-established problems, such as ambiguities and inconsistencies in the ICD, as well as the complexity of coding rules, raise concerns of the current reliability of coding diagnoses with ICD. In a study of Peng et al., on the reliability of emergency coding, it was stated that the most common difference between coders was that they assigned different codes to the same condition (23.6%). The issue of coding properties is another common cause of discrepancy because one encoder may be assigned to more specific codes. Stausberg et al. believed that the low reliability may have two causes: insufficient training, inadequate standardization of the coder and the coding scenario; weaknesses in the classification system adopted for coding. In the latter case, the low reliability indicates the poor quality of a coding system and needs fundamental revision. A research also shows that differences in the interpretation of coding rules lead to differences in coding reliability. Using ICD-based administrative data without knowing its regulations may result in misinterpretation. The findings thus highlight the need for standardization of documents.

The issue of coding specificity is another major reason for differences in coding, which can lead to ambiguities in the medical records reported by the coder. Providing an accurate definition of coding, as well as continuous training of coders, are essential for ensuring optimal coding reliability. Thus, the lack of adequate and specialized training for employees should be considered as an important factor.

Studies demonstrate a clear link between poor medical documentation and poor coding reliability. The reasons for the unreliability of coding include: lack of experienced and specialized coding staff, insufficient training of the current coding staff (medical and non-medical), poor medical documentation, lack of training and/or poor usability of the office data software, inherent limitations of the ICD-10 coding system in relation to concepts/terms, the insignificance of coding, and lack of complete regular auditing of administrative data. Therefore, the employment of trained coders at the specialized level and presenting courses on data quality are beneficial interventions for guaranteeing high coding reliability. Studies on coding reliability in other countries report different results. In some of them, a degree of coding reliability has been set, while in some others, the results have been expressed moderate reliability.,, These results are in line with the results of the present study.

The findings of the present study showed that the reliability of trauma coding both in terms of the nature of trauma and the external causes of trauma does not have a good status in the research units. The main reasons for these results are the use of general and unclear codes, not reading the entire medical records, and not using the cover of the ICD-10 (Tabular list). The unreliable code can limit the health research and planning projects as it will complicate the statistical analysis of information. Accordingly, periodical evaluating and auditing the codes, training the coders, making the quantitative and qualitative analysis of medical records, and keeping accurate and complete documentation of medical records are recommended.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical statement

Not applicable.

Declaration of competing interest

The authors have no conflicts of interest relevant to this article to disclose.