Describing agreement in the main condition coding field using Canadian ICD-11 inpatient data.

INTRODUCTION: Countries use varying coding standards, which impact international coded data comparability. The 'main condition' (MC) field is coded within the Discharge Abstract Database as "reason for admission" or "largest resource use".
OBJECTIVE: We offer a preliminary analysis on the frequency of and contributing factors to MC definition agreements within an inpatient Canadian dataset.
METHODS: Six professional coders performed a chart review between August 2016 and June 2017 on 3,000 randomly selected inpatient charts from three acute care hospitals in Calgary, Alberta. Coders classified the MC as "reason for admission", "largest resource use" or "both". Patients were admitted between 1st January and 30th June 2015 and met the inclusion criteria if they were ≥18 years, had an Alberta personal health care number, and had an inpatient visit for any service outside of obstetrics. Agreement between the two MC definitions was stratified by length of stay (LOS), emergency department admission, hospital of origin, discharge location, age, sex, procedures, and comorbidities. Chi-square analysis and frequency of inconsistencies were reported.
RESULTS: Only 34 (1.51%) of the 2,250 patient charts had disagreeing MC definitions. Age, emergency visit on admit, LOS, hospital, and discharge location were associated with MC agreement. Chronic conditions were seen more often in MC definition agreements, and acute conditions seen within those disagreeing.
CONCLUSION: There was a small proportion of cases in which the condition bringing the patient to hospital was not also the condition occupying the largest resources. Within disagreements, further research using a larger sample size is needed to explore the presence of MC in a secondary/tertiary condition, the association between patient complexity and disagreeing MC definitions, and the nature of the conditions seen in the inconsistent MC definitions.

Introduction

Inpatient chart documentation is used to generate coded data worldwide. Healthcare administrators use inpatient coded data to claim physician services to insurance providers, to inform health policies and to improve the delivery of health services [1]. Researchers use coded data to collect essential data on healthcare systems worldwide, including data on resource use, disease prevalence, and quality of care. Diagnoses in coded health data are classified using the International Classification of Disease (ICD), developed by the World Health Organization (WHO). Having an international coding system allows for the comparison of mortality and morbidity data across countries, which can inform research initiatives, medical advancements, and resource allocation decisions. The ICD was first established in 1893, and was entrusted to the WHO in 1948. Since then, the WHO has released 6 revisions of the ICD. Each revision improves on its predecessor, by offering more detailed coding options for each disease (i.e. Diabetes Type 1 versus Diabetes Type 2), and thus improving data quality. However, countries customize ICD versions to include condition descriptions that best suit their country. They will include more or less layers of detail according to their country’s disease prevalence, coding needs and country-specific secondary use of ICD-coded data. Variations in ICD include ICD-10-AM (Australia), ICD-10-CA (Canada), ICD-10-GM (Germany), ICD-10-CM (United States), and ICD-10-TM (Thailand). Though the WHO has attempted to regulate ICD-10 modifications, the ICD variations have been found to affect international data comparability, and jeopardize international data usability [2]. The WHO has recognized this limitation to ICD variation, and has thus created the newest ICD version, ICD-11 (not yet released for use) [3]. Countries are encouraged to use the most recent version of ICD, in order to maintain international coding standards and re-establish data comparisons.

The ‘main condition’ (MC) field is heavily affected by ICD version modifications. Currently, the WHO has established 2 ways in which the MC is coded: ‘reason for admission’ (RA) or ‘largest resource use’ (RU). Inpatient visits in Canada are coded using the International Classification of Diseases-10 Canadian coding standards (ICD-10-CA). Therefore, in Canada, MC is coded as the diagnosis most responsible for a patient’s stay in hospital which is neither RA or RU. If more than one such condition exists, then the condition responsible for the greatest resource use is chosen (RU) [4]. Comparatively, some countries such as USA are selecting their MC based on reason for admission (RA) [5].

Conditions occupying the largest amount of resources are often those that are difficult to diagnose, or appear later throughout the patient’s stay due to complex and changing health status. Similarly, conditions that are the reason for admission often do not align with the largest resource use as they are often a symptom of a more complex condition that is later discovered [6]. For example, suppose a diabetic patient was admitted with a transient ischemic attack (TIA). The TIA was treated with a non-invasive procedure, and the patient was ready for discharge within 48 hours. However, during the hospital stay, the patient developed hospital-acquired pneumonia which led to pulmonary arrest and resuscitation. This required admittance to the intensive care unit, several weeks of antibiotic treatment, use of ventilator supported breathing, and rehabilitation therapy. Using the ‘reason for admission’ MC definition, the main condition would be TIA, with pneumonia and diabetes as secondary conditions. However, given the ‘largest resource use’ definition, the MC would be classified as pneumonia, with TIA and diabetes as secondary conditions. The difference between these definitions can have large implications, predominantly on secondary use data. If the data collected under the ‘resource use’ definition is used for estimating the incidence of TIA or the TIA disease burden in the population, then the result is an outcome-based subset of TIA cases where analyzing the MC only provides the cohort of TIA admissions for which there were no complications that consumed greater resources. Not only does this affect research, where variability in case selection can impact studies, but it also affects healthcare administration where different definitions impact financial planning and prevention measures [5].

As a WHO Collaborating Centre, our research team focuses on field-testing for new ICD versions. This study is part of a large-scale ICD-11 field trial to test the appropriateness, quality, and performance of the ICD-11. There is a dearth of recent literature on the discrepancy between MC definitions, and its impact on data coding. However, the study idea originated from the findings of Quan et al. [7], which emphasized the importance of ensuring unified MC definitions worldwide. This recommendation was supported by the World Health Organization’s Morbidity Reference Group, which proposed an amendment to international coding MC definitions, whereby if a condition was discovered later in the patient’s stay that occupied the most resources, that condition should thus be the MC [8]. Previous to Quan’s publication, another study led by Ghali et al. [9] proposed that when defining the MC for research use, conditions that arise after admission should be excluded. Ultimately, the WHO has decided that the new ICD-11 will migrate toward unity, discouraging country-specific versions, and standardizing the MC definition to the following: ‘the reason for admission, after assessment at the end of the stay’ [3].

With a new coding system incoming, we sought to renew our understanding of the frequency of disagreeing MC definitions within our coded data. Further, roll-out of a new ICD version can take several years (ICD-9 is still used in many countries), and several researchers will continue to use and compare data from ICD-9 or ICD-10 for years to come [2]. Therefore, investigation into factors contributing to MC definition discrepancies will caution researchers using data from older ICD versions. Consequently, we aim to understand the effects of the discrepancy between RU and RA definitions. We explore the frequency of agreeing and disagreeing MC definitions within a 2250 inpatient Canadian cohort, and describe contributing factors to MC definition discrepancies.

Methods

Chart selection

In this retrospective study, six professional clinical coders applied ICD-11 codes on 3,045 randomly selected inpatient charts from three acute care hospitals in Calgary, Alberta, between August 2016 to June 2017. Patients were admitted between 1st January and 30th June 2015 and met the inclusion criteria if they were ≥18 years of age, had an Alberta personal health care number, and had an inpatient visit for any service outside of obstetrics.

Coding

The clinical coders had varying inpatient coding experience and years of practice. All Canadian coders are certified and trained under the Canadian Institue for Health Information (CIHI) and follow the Canadian Coding Standards when coding [4]. CIHI is a standardized, secure, data holder providing pubically-accessible de-identified data on healthcare systems and the health of Canadians. It also is responsible for setting coding standards for all professional coders across Canada [10]. Additionally, all 6 clinical coders were trained for this study by a University of Calgary research team, CIHI and WHO experts [11, 12]. The University of Calgary is a WHO Collaborating Centre for Classification, Terminologies, and Standards, and is therefore responsible for piloting new ICD versions. Hence, the ICD-11 coding definitions were used for this portion of the study, though the ICD-11 classification system was not yet available for international use at the time. The database was also dually coded with ICD-10-CA for the purpose of abstracting demographic, baseline characteristic, and Charlson comorbidity information. Inter-rater reliability (IRR) was conducted on 60 charts by 2 coders, with a final Kappa score of 0.88 reached on main condition codes.

For our study, we required coders to input RA or RU for each patient’s MC. Coders captured the MC within the primary diagnosis coding field (DxCode1) for each patient visit. Two DxCode1 columns were provided for this exercise, so that coders could designate whether the MC was RA or RU. Coders were asked to enter the MC into the appropriate column (‘reason for admission’ or ‘largest resource use’), which allowed for both columns to be selected at once when applicable.

Coders followed Canadian coding standards when selecting the MC, which state the following: Coders are asked to choose the MC by looking at the problem list and selecting the first problem listed (RA). However, if an intervention was performed, then the diagnosis responsible for that intervention becomes the MC (RU). Coders use the physician’s discharge summary and problem list to identify MC. If no definitive diagnosis is listed in these documents, coders are asked to list the main symptom during the patient’s stay as the MC [4]. This practice reflects the decision-making process that coders in Canada are expected to make for each chart.

Canadian coders are expected to fill a specific quota of charts per day, and therefore spend a select amount of time on each chart [6]. Additionally, the only comorbidities that require coding are those that required treatment beyond maintenance of the pre-existing condition [4]. Since coders are not medically trained, certain pre-existing conditions (comorbidities present prior to the admission) that were being routinely treated with medications might be missed, though they were specified as comorbidities in the problem list. However, to increase data granularity in our study, coders were encouraged to code all problems detected in the problem list, regardless of monitoring or treatment. There were no time or code restrictions.

Data sets used

Since 2006, the system used by Calgary healthcare providers for documentation in the inpatient electronic medical record is Sunrise Clinical ManagerTM (SCM)- EMR. The clinical coders were thus given access to both SCM-EMR and the paper copies of the chart to complete the chart review, as a hybrid charting system was in use. Clinical coders were required to adhere to national standard coding practices, which include prioritizing the electronic discharge summary document and coding only what is documented by the most responsible physician [4].

Study variables

A list of possible associated variables was compiled using expertise from clinicians and researchers. They were selected based on the authors’ hypothesis that patients with higher complexity could be more likely to have a disagreeing MC definition, due to a complex patient’s susceptibility to complications in hospital, which would thus consume a greater amount of resources. The United Nations’ standard age classification was used to determine age banding [13]. Length of stay (LOS) parameters were drawn from the Canadian Institute for Health Informatics (CIHI). According to CIHI, the 2018 average length of stay for Albertan patients was 7.8 days [14]. Therefore, length of stay was dichotomized into <7 or ≥7; presumably, the longer the LOS, the more complex the admission. The number of Charlson comorbidities and consequent complexity of each patient was included. Discharge location/status was included to assess patient complexity, as a sicker patient could have died, or could have been discharged to a facility with continued care or discharged home with homecare. Assessment of whether or not a patient had a procedure while in hospital was also included, due to the possibility of a procedure indicating higher complexity in patient health status, or increasing the patient’s LOS. Additionally, given the coding standards for a diagnosis requiring a procedure taking precedence as MC, the distinction between procedural and non-procedural admissions was important. Admittance to hospital through the Emergency Department was included to identify those patients admitted for an elective versus urgent/emergent reason. Lastly, the hospital to which the patient was admitted was included to explore site variation. The three hospitals varied in patient acuity and services. Hospital A is a tertiary academic centre with more cardiac and trauma cases and a high level of patient acuity. Hospital C offers more psychiatric services than the others, while Hospital B offers more surgical procedures.

Statistical analysis

A series of descriptive analyses were performed on the study data. Agreement between the two MC definitions was stratified using the study variables. Agreement between MC definitions was determined if the same diagnosis was recorded under the ‘reason for admission’ column and the ‘resource use’ column for the primary diagnosis. The codes of the ICD–11 are alphanumeric and cover the range from 1A00.00 to ZZ9Z.ZZ. The ICD-11 coding scheme begins with the chapter number or letter; there are 28 chapters, each pertaining to a different bodily system or disease origin. The subsequent digits within the stem code add further specifications. The digit after the period (i.e. the “ZZ” in the code ZZ9Z.ZZ above) is the extension code which offers detail to the stem code. For example, CA22.Z is the code for chronic obstructive pulmonary disease, unspecified. However, CA22.0 is the code for chronic obstructive pulmonary disease with acute exacerbation, unspecified. For certain analyses, this degree of granularity was not necessary, therefore, extension codes were at times truncated. ICD-11 uses code clusters which contain multiple stem codes. For example, if a patient was admitted to hospital in a diabetic coma and the patient had Type 2 diabetes mellitus, the code would be written as follows: 5A23/5A11; where 5A23 is a diabetic coma and 5A11 is Type 2 diabetes mellitus. Given that only one stem code should be used at a time for the Main Condition categorization, code clusters were truncated to the first code listed, as per WHO recommendations [3].

Pearson’s Chi-Square tests were performed to assess the differences in the associated variables between the agreement cases only, due to the small sample size of disagreement cases. A pre-set alpha of <0.05 was used for statistical significance. Analyses were performed using Stata 14.0 (Stata Corp, College Station, TX, USA).

Results

Study cohort

A final number of 2,250 patient charts was used for this study, with reasons for exclusion outlined in Figure 1. While the study was underway, the WHO released a reference guide that provided recommendations to coders. A series of charts (n = 716) were excluded from the study from before the guide release due to a number of errors in the ICD-11 system detected by the guide.

Figure 1: Flow diagram of final cohort used for chart review

There were slightly more females (51.0%) than males. Ages ranged between 18 and 106 years upon admission, with the highest proportion of patients being ≥65 years of age (45.6%). The majority of patients admitted to hospital were urgent or emergent cases (63.1%), and were discharged home without homecare (74.9%). The largest proportion of patients came from Hospital B (44.9%), closely followed by Hospital C (39.9%). Almost a quarter of patients had at least 1 comorbidity (24.2%). Most patients had at least 1 procedure in hospital (57.3%), and stayed less than 7 days (66.7%). There was no missing data for any of the variables displayed in Table 1.

Table 1: Characteristics of 2,250 patient sample

Variable	Frequency (%) (n =2,250)
Sex
male	49.02 (1,103)
female	50.98 (1,147)
Age
<25	4.62 (104)
25–44	16.27 (366)
45–64	33.47 (753)
≥65	45.64(1,027)
Emergency Department on Admission
yes	63.07 (1,419)
no	36.93 (831)
Discharge Status/Location
Acute inpatient facility	5.07 (114)
Long term care	3.56 (80)
Other (e.g. hospice, addiction treatment centre)	1.24 (28)
Home with homecare	11.65 (262)
Home without homecare	74.89 (1,685)
Against medical advice	1.47 (33)
Died	2.13 (48)
Number of Comorbidites
0	28.49 (641)
1	24.18 (544)
2	19.69 (443)
3	12.76 (287)
4+	14.9 (335)
Hospital+
A (80016)	15.82 (356)
B (80020)	44.85 (1,009)
C (80148)	39.93 (885)
Procedure
yes	57.25 (1,288)
no	42.76 (962)
Los
<7	66.66 (1500)
≥7	33.33 (750)

+FMC: Foothills Medical Centre; PLC: Peter Lougheed Centre; RGH: Rockyview General Hospital.

Table 2 and Table 3 display the agreement patterns seen within the MC definitions. Out of the 2,250 patient cohort, only a small proportion of patients had disagreeing MC definitions (1.5%). Therefore, further statistical analysis on those with disagreeing definitions was not performed. A series of chi-square analyses were performed to assess for significance of agreements within independent variables. Age, Emergency Department on admission, discharge location/status, hospital of origin, and LOS were all statistically significantly associated with having MC definition agreement. Comparatively, sex, number of comorbidities, and procedure were not statistically significant.

Table 2: Description of main condition definition agreement and disagreement

Main condition definition	Frequency (%) (n = 2,250)
Agreement	92.04 (2,071)
Disagreement	6.45 (145)
Missing	1.51 (34)

Table 3: Proportion of main condition agreements stratified by variable

Variable (p-value)	Frequency (%) (n = 2,071)
Sex (0.612)
male	48.87 (1,012)
female	51.13 (1,059)
Age* (0.012)
<25	4.64 (96)
25–44	16.27 (337)
45–64	33.56 (695)
≥65	45.53 (943)
Emergency Department on Admission* (<0.001)
yes	62.00 (1,284)
no	38.00 (787)
Discharge Status/Location* (0.029)
Acute inpatient facility	4.59 (95)
Long term care	4.59 (95)
Other (e.g. hospice, addiction treatment centre)	1.35 (28)
Home with homecare	12.12 (251)
Home without homecare	74.26 (1,538)
Against medical advice	1.30 (27)
Died	1.30 (37)
Number of Comorbidities (0.204)
0	28.63 (593)
1	24.43 (506)
2	19.36 (401)
3	12.94 (268)
4 +	14.64 (303)
Hospital*+ (0.009)
A (80016)	16.42 (340)
B (80020)	44.08 (913)
C (80148)	39.50 (818)
Procedure (0.099)
yes	57.75 (1,196)
no	42.25 (875)
Los* (0.004)
<7	67.50 (1,398)
≥7	32.50 (673)

*indicates statistically significant association for chi-square analysis using an alpha of 0.05.

+FMC: Foothills Medical Centre; PLC: Peter Lougheed Centre; RGH: Rockyview General Hospital.

Descriptive analyses were performed on the MC definition disagreement cohort. Within the 145 disagreements, 133 (91.7%) only had ‘resource use’ selected, and not ‘reason for admission’ when defining MC. The remaining 12 (8.3%) only had ‘reason for admission’ selected when defining MC. The top 10 conditions within each of these definitions were thus displayed in Table 4 and Table 5, in efforts to better understand the nature of the conditions that occupy the largest resource use yet are not the reason for admission. More chronic conditions appeared in the ‘resource use’ cohort, while more acute conditions appeared in the ‘reason for admission’ cohort. There was limited ability to further analyze these conditions, as none of the top 10 conditions for ‘reason for admission’ were repeatedly seen in the data.

Table 4: Top 10 disagreeing conditions by ‘reason for admission’

Reason for admission
2A82.2	Hairy-cell leukaemia
2A83.1	Plasma cell myeloma
6A20.Z	Schizophrenia, unspecified
6A91.4	Recurrent depressive disorder, current episode severe, with psychotic symptoms
BD3Y	Other specified chronic arterial occlusive disease
DB10.01	Acute appendicitis with localised peritonitis
GB2Z	Abnormal uterine or vaginal bleeding, unspecified
GB60.Z	Acute kidney failure, stage unspecified/GC32 Acute tubular necrosis
ME03	Dysphagia
ME24.Y	Other specified clinical manifestations of the digestive system

Table 5: Top 10 disagreeing conditions by ‘resource use’

Resource use
6B43	Adjustment disorder
QB9B	Palliative care
BA21	Orthostatic hypotension
CA22.Z	Chronic obstructive pulmonary disease, unspecified
CA40.Z	Pneumonia, organism unspecified
2F35	Benign neoplasm of urinary organs
3A9Z	Anaemias or other erythrocyte disorders, unspecified
6D10.Z	Personality disorder, severity unspecified
BA80.0	Coronary atherosclerosis of native coronary artery
CA22.0	Chronic obstructive pulmonary disease with acute exacerbation, unspecified

Discussion

Major findings

Different definitions of MC can impact morbidity research internationally, leading to inappropriate resource use and inaccurate case identification. Using an ICD-11 coding model, we aimed to provide an updated analysis on the frequency of MC definition disagreement within select Canadian patient charts, as well as aiming to identify associated variables with MC definition disagreements. There was a very small proportion of cases in which the condition that brought the patient to hospital was not also the condition occupying the largest amount of resources. This finding is dependent on what the coders were able to identify from the physician documentation, therefore, it is representative of the current coding standards within Canada, which requires coders to select the most responsible diagnosis (as seen on the problem list), unless an intervention/procedure was done [4]. Canadian coders are trained and accustomed to make the distinction between RA and RU within physician documentation and delegating the MC code. Therefore, within our Canadian context and coder training, this finding is indicative of the low frequency with which patients’ admitting diagnoses are not also the largest resource use. Researchers using a similar healthcare and coder training system as ours, therefore, can proceed with data comparability, regardless of differing MC definitions. However, caution should be used when applying this finding to geographical locations with a different healthcare and coder training system. Apart from different coder training standards, this small proportion of MC definition disagreement could have other explanations, including the high level of physician training in Canada.

Several Canadian medical schools rank within the top 20 medical schools worldwide [15]. Due to our country’s rigorous clinical diagnostic processes and high quality physician training, it is possible that physicians have methods to consistently diagnose the patient’s underlying issue. Therefore, they can see beyond the patient’s presenting condition and diagnose the underlying condition, which eventually consumes the highest resource use during the hospital stay. Thus, the ‘reason for admission’ and ‘resource use’ MC definitions would be coded with high agreement. To test this hypothesis, it would be appropriate to conduct this study in a variety of countries with different healthcare systems and physician training programs [16]. Nonetheless, this study finding is still relevant for researchers at the national and possibly at the international scale.

Contributors to MC disagreement

The disagreements seen in the data may be explained by the the coder’s dependency on physician documentation. A study focusing on coder perceptions in data quality reported that the largest barrier to high quality coding was poor physician documentation [17]. Though our physician training and practice may be of high caliber, that does not discount the possibility of poor physician documentation quality. Worldwide, medical and residency programs provide minimal training for high quality physician documentation [18]. Identification of the RU and RA depends on documentation, and it can be challenging to determine which condition is most responsible for either of the MC definitions when physicians enter scarce or no information in the problem list/discharge summary. Clinical coders must often rely on their training and subjective judgement for determining RU, based on their understanding of healthcare service costs. However, a possible alternative to ensuring higher agreement between RA and RU is to determine RA upon discharge rather than on admission, so as to attempt to capture the underlying diagnosis that brought the patient to hospital. This could increase the likelihood that the underlying diagnosis resulted in the largest resource use, and therefore qualified as the RA and RU. This is precisely what the WHO’s ICD-11 MC definition will be. Additionally, the ICD-11 encourages coders to add “extension codes” to indicate different types of discharge diagnoses (i.e. those occupying largest resources, but not deemed the reason for admission) [3].

Our study demonstrates the high prevalence of agreement within different MC definitions within our Canadian context. Additionally, the association between complexity of a patient’s hospital stay and MC definition disagreement has been shown to be statistically insignficant in this study’s findings. Though there were statistically significant associations between certain variables and MC agreement, the original hypothesis of patient complexity being associated with disagreements did not prove to be valid. Though number of comorbidities and undergoing a procedure are both common indicators of health status complexity [19], LOS and admission through the ER are both often used as indicators of complex health status as well [20]. However, LOS and ER were both found to be statistically significant in association with having MC definition agreement. The small sample size hindered the possibility to assess for statistical significance in those with MC definition disagreements. It is therefore difficult to draw a conclusion on whether or not patient complexity contributes to disagreement in MC definition. That said, the importance of coding either RA or RU is not dismissed with these study findings. Each provide valuable information; for example, the RA is an important indicator when establishing preliminary care pathways for the patient, while RU is essential for cost weighting of healthcare services. Consequently, the ICD-11 offers the opportunity to code RU if it is not the same as the RA, since the MC definition in ICD-11 will be RA [3].

The top 10 MC condition disagreements found in those with the ‘resource use’ MC definition were typically chronic conditions (i.e. palliative care or orthostatic hypotension). On the contrary, those found in the ‘reason for admission’ top 10 condition disagreements were typically more acute conditions (i.e. acute appendicitis or abnormal uterine bleeding). This is an important finding that requires further exploration. A possible explanation is that these chronic conditions are often quite obscure, and the disease origin, course, and treatment is often multifaceted and presents more fully with time in hospital. For example, someone with orthostatic hypotension (low blood pressure when standing) could have a cardiac, circulatory, respiratory, or neurological etiology. Therefore, it is possible that the origin of this condition (e.g. peripheral vascular disease relating to poor circulation) is in fact the ‘reason for admission’, but the condition itself (orthostatic hypotension) required the greatest amount of resources to make a differential diagnosis (contacting neurologists, cardiologists, respiratory therapists, conducting physical and laboratory examinations). Therefore, orthostatic hypotension would classify as the greatest resource use, and peripheral vascular disease would be the reason for admission. On the contrary, the patient with acute appendicitis most likely came into the hospital with symptoms of acute appendicitis, and the subsequent appendectomy most likely occupied the greatest amount of resources. Therefore, appendicitis would likely have been both the reason for admission and the largest resource use.

Limitations

There are limitations to this work. First, only the Main Condition (DxCode1 in the coded database) was analyzed. In the event that a condition was entered into DxCode2 and ended up being classified as a MC, it would not have been captured. Nonetheless, it should be noted that if a comorbidity (DxCode2) was classified as the largest resource consumer, this would still be detected as a disagreement, and the proportion of disagreeing MC definitions found in this study would remain the same, as would the implications of the findings. Second, the small sample size limited data analysis and rigor. Third, Canada’s coding standards are some of the highest worldwide, with Canada being 1 of only 3 countries that offers a University degree in coder training [21]. Therefore, the coding practices required for proper identification of MCs may not be applicable to other countries. All three of these limitations lead to decreased generalizability of study results. This calls for future work to replicate this study using a larger dataset where at least the first two diagnostic codes are assessed. This should also be done in a variety of countries with different coding standards.

Lastly, though this study data was specifically created for our research purposes, some data were missing, which decreased our sample size by one third. Additionally, the possibility of misclassification bias did exist with regards to the opportunity coders had to code as many comorbidities and procedures as possible, without a time or code limit. Due to this lack of restrictions for coders, coders might have identified a patient as having a certain comorbidity, when in reality they did not, thus misclassifying that patient. Therefore, when comparing the number of comorbidities between agreeing and disagreeing MC definitions, this hinders the reproducibility of these study results within a real-life context, where coders have more restrictions.

Implications for future work

The study results remain promising with regard to the usability of the MC definition in coded data. Replication of this study with a larger sample size in countries with diverse coding practices is recommended. However, results from this analysis demonstrate that the MC definition disagreements have a small frequency, and there is limited evidence for association between MC definition agreements and clinical or demographic variables. Researchers using data from similar coding and healthcare systems as Canada should expect to see a small proportion of MC definition disagreements and therefore feel comfortable using ICD-coded data with different MC definitions. However, the issue of data incomparability across countries, particularly those with minimal coder or physician training, still remains. Therefore, the WHO’s decision to standardize the MC definition within ICD-11 is appropriate and needed.