Comparisons of multi-morbidity in family practice--issues and biases.

BACKGROUND: As the population ages, practice and policy need to be guided by accurate estimates of chronic disease burden in primary care.
OBJECTIVE: To produce a preliminary set of methodological considerations for cross-sectional and retrospective cohort studies of multi-morbidity in primary care using three studies as examples. Prevalence rate results from the three studies were re-estimated using identical age-sex groups.
METHODS: We compared the methods and results of three separate studies in primary care: (i) patients in the Saguenay region of Quebec, Canada (2005); (ii) a substudy of the BEACH (Bettering the Evaluation and Care of Health) programme in Australia (2008); and (iii) the DELPHI (Deliver Primary Health Care Information) project in South-western Ontario, Canada (2009). Areas where the methods of multi-morbidity studies may differ were identified. The percentage of patients with two or more chronic conditions was compared by age-sex groups.
RESULTS: Multi-morbidity prevalence varied by as much as 61%, where reported prevalence was 95% among females aged 45-64 in the Saguenay study, 46% in the BEACH substudy and 34% in the DELPHI study. Several aspects of the methods and study designs were identified as differing among the studies, including the sampling of frequent attenders, sampling period, source of data, and both the definition and count of chronic conditions.
CONCLUSIONS: Understanding the differences among the methods used to produce prevalence data on multi-morbidity in primary care can help explain the varying results. Standardization of methods would allow for more valid inter-study comparisons.

Introduction

In family practice, the care of patients with multiple health problems is a growing concern. New approaches to care must be developed to cope with what most professionals consider will be a tsunami of demands, given the aging population.[1]

Family medicine research has been woefully inadequate in providing basic data on multi-morbidity that mimic the kinds of data readily available for common single chronic and infectious diseases, i.e. studies on prevalence, characterization, risk factors (or determinants) and prognosis.[2,3]

The purpose of this study is to assist researchers in their endeavours to launch prevalence studies of multi-morbidity and to help readers interpret and compare results from these studies. The need for such assistance became clear to the authors during the discussion after a presentation at the 2009 North American Primary Care Research Group.[4] In an attempt to put one study into context, the presenter briefly compared her results with those of several other studies. The authors of those compared studies happened to be in the room and explained, during the discussion period, which methods used in their studies may have led to the contrasting results. We quickly realized that (i) the ‘Methods’ sections of our articles could have been more detailed had the word limits not applied, leaving the reader unable to fairly interpret the results when comparing one study with another and (ii) standards for calculating and reporting multi-morbidity were lacking.

Therefore, our comparative analysis produced the following: (i) a preliminary set of methodological considerations for cross-sectional and retrospective cohort studies of multi-morbidity in primary care; (ii) a description of the methods across the three studies as examples; (iii) re-estimates of prevalence rates conducted using identical age–sex groups; and (iv) a detailed assessment of the methods thought to be pertinent to the rates.

Methods

This study reviewed the methods and results of three separate studies of multi-morbidity in primary care herein referred to as (i) the Saguenay study, from the Saguenay region of Quebec, Canada;[5] (ii) a BEACH substudy, from the Bettering the Evaluation and Care of Health programme in Australia;[6,7] and (iii) the DELPHI study, from the Deliver Primary Health Care Information Project in South-western Ontario, Canada.[4,8] The three studies were selected based on the participation of the authors in the discussion after the presentation at the 2009 North American Primary Care Research Group[4] regarding their varying results.

For the purposes of this study, the raw data used in each of the three prior studies were extracted and the rates of multi-morbidity were compared for the realigned six age–sex groups: 18–44 years, 45–64 years and >65 years. These data were not adjusted for the cluster design of their studies (clustering of patients by physician or practice) or for representation of their respective population profiles.

In addition, the authors reviewed the three studies and emailed for clarification of the methods used to produce the published results. Points of difference among the methods were found in an iterative process, through review of the studies, by discussion at meetings and through teleconferences and email. The following areas were discussed: (i) research design; (ii) population and sampling; (iii) data collection, including definition of chronic active problems and list of chronic conditions; and (iv) the definition of multi-morbidity. All three studies had ethical approval by their respective institutions.

Results

Preliminary set of methodological considerations

Table 1 presents the list of methodological considerations that arose from the detailed discussions of the authors. The methodological considerations appear on the left and the possible variations appear on the right. The judgement of which variations are stronger than others is presented in the Discussion.

T

Preliminary set of methodological considerations for cross-sectional and retrospective cohort studies in multi-morbidity (Methods Crystals for MM)

Major categories	Considerations	Details	Variations
1. Design	Research design	–	Cross-sectional study, retrospective cohort study
2. Population and sampling	Location	–	Country and/or region
	Sampling	Sampling method	e.g. Two-stage cluster sampling of FPs and then patients within them
	Primary care setting(s)	Sampling frame	e.g. All in region, affiliated with an academic institution and other subgroupings
		Selection method	Random, non-random
		Sample size	# Settings recruited or represented.
	Family practitioners (FPs)	Sampling frame	e.g. all FPs in the region or only those meeting specific criteria
		Selection method	Random, non-random
		Sample size	Number of FPs recruited
	Patients	Sampling frame	All citizens, total patient list and visiting patients
		Selection method	Random, non-random and consecutive
		Sample size	Number of patients contributing data
	Rationale for sample size	–	e.g. Based on the power of the test, when every patient had been approached or other considerations.
3. Data and definition	Data collection	Source of data	Whole medical record, specific time period in record, provider documentation at visit, provider knowledge of patient, patient self-report
		Method of data collection	Chart audit, electronic medical record data extract, questionnaire, physician form or other sources.
	Coding	Morbidity coding	No coding, nomenclature ICPC-Plus, coding with ordering principle ICPC, or other coding methods
	Time	Time period of data source	Length of medical record, specific time period in medical record or at encounter
		Length of recruitment period for patients for each FP	Number of days, weeks, months?
		Dates of data collection	When did the researchers collect the data?
		Morbidity time focus	Current conditions under medical management, within a specific time period or lifetime
	Definitions	Definition of multi-morbidity	Two or more conditions, three or more conditions or other definition
		Definition of chronic conditions	How were conditions identified as chronic and was there a specific list of conditions used?
		Operational definition of the count of chronic conditions	Did the FP or nurse use judgement to code evolving diagnoses as one condition? Could double counting occur?
4. Outcomes	Results	Outcomes reported	Prevalence of single morbidity, multiple morbidity, other health or health system outcomes (e.g. self-reported health, utilization of services)
		Confounders controlled	e.g. Socio-economic status, mental health problems and other confounders studied must be specified
		Results presented	e.g. Prevalence of multi-morbidity

Table 2 presents the methods of the three studies of the authors in terms of the methodological considerations outlined above as a test of the preliminary list. During the completion of this table, the authors identified methodological details that were not described in their publications or were described elsewhere, as in the case of the BEACH substudy[6,7] and the DELPHI study.[4,8]

T

Populating the preliminary set of methodological considerations “Methods Crystals for MM” with three examples

Major categories	Considerations	Details	Saguenay study	BEACH substudy	DELPHI study
–	–	–	Fortin et al. 20055	Britt et al. 20086; Knox et al. 20087	Stewart et al. 20094,8
1. Design	Research design:	–	Retrospective cohort study	Cross-sectional study	Retrospective cohort study
2. Population and sampling	Location:		Regional: Saguenay region, Quebec, Canada	National, Australia	Regional, South-western Ontario, Canada
	Sampling	Sampling method	Two-stage sampling: first of FPs, and then of their patients.	Two-stage sampling: first of FPs, and then of their patients.	Two-stage sampling: first of FPs, and then of their patients.
	Primary care setting(s)	Sampling frame	Not applicable (N/A), as study did not sample by practice.	N/A, as study did not sample by practice.	N/A, as study did not sample by practice.
		Selection method	N/A, as study did not sample by practice.	N/A, as study did not sample by practice.	N/A, as study did not sample by practice.
		Sample size	N/A	N/A	N/A
	Family practitioners (FPs)	Sampling frame	All FPs in region with a general practice in a doctors’ office or an institution, with accessible medical records, for adult patients of all ages.	All practicing FPs in Australia.	All FPs in south-western Ontario not using electronic medical records in 2005.
		Selection method	Contacted all FPs in sampling population for recruitment.	Random.	Non-random; FPs who agreed to participate in the DELPHI project.
		Sample size	21	305	25
	Patients	Sampling frame	Visiting patients ≥18 years old.	Visiting patients of all ages.	Visiting patients of all ages.
		Selection method	Consecutive.	Consecutive, 30 of the 100 patients per FP in the BEACH Programme9	Random selection of one patient per day, and patients followed up prospectively.
		Sample size	980	9156	2998
	Rationale for sample size	–	Not stated.	Not stated.	Recruitment ended after ~10% of all patients in FP’s practice were selected.
3. Data and Definition	Data collection:	Source of data	Whole medical record reviewed by a trained nurse.	Whole medical record, FP knowledge, provider documentation at visit, patient self-report.	Coding by FP at each visit.
		Method of data collection	Chart audit.	FP form.	Electronic medical record.
	Coding:	Morbidity coding	No coding.	Coding with nomenclature, classified to the International Classification of Primary Care (ICPC-2).10	Coding with ordering principle ICPC-2-R.11
	Time:	Time period of data source	Length of medical record.	Length of medical record.	Two years of medical record.
		Length of recruitment period for patients for each FP	2–3 weeks.	Several days.	6–12 months.
		Dates of data collection	December 2002–July 2003.	July–November 2005.	March 2006–February 2008.
		Morbidity time focus	Lifetime	Conditions currently under medical management	Conditions currently under medical management
	Definitions:	Definition of multi-morbidity	Two or more chronic conditions.	Condition(s) in two or more morbidity domains of the Cumulative Illness Rating Scale (CIRS).12	Two or more chronic conditions.
		Definition of chronic conditions	Conditions identified by chart auditors as meeting the World Health Organization’s (2002) definition of chronic conditions: ‘health problems that require ongoing management over a period of years or decades’.13	A selected list of 22 common cardiovascular, psychological, respiratory, musculoskeletal, endocrine problems and malignant neoplasms; classified in ICPC-210 and then mapped to eight domains in the CIRS and a separate domain for malignant neoplasms.	List of 98 ICPC-2-R12 codes, 85 defined as chronic conditions by Lamberts and Okkes11 and 13 codes added by DELPHI study investigators.
		Operational definition of the count of chronic conditions	Nurse judgement to document evolving or similar diagnoses as one condition.	FP judgement to document evolving or similar diagnoses as one condition.	FP to document each encounter, whereby evolving or similar diagnoses could be counted as several conditions.
4. Outcomes	Results	Outcomes reported	Prevalence of multi-morbidity.	Prevalence of multi-morbidity.	Prevalence of multi-morbidity.
		Confounders controlled	None. The region’s residents were compared with the rest of Canada by socio-economic status, age, educational level, median household income and unemployment rate.	Results adjusted by age and sex to the general practice patient population in 2005 and weighted to adjust for visit frequency. Study population compared with the overall Australian population, and a national population prevalence calculated.	Age and sex of all patients compared with those of the remaining people of Canada.
		Results presented	Prevalence of multi-morbidity by age and sex.	Prevalence of multi-morbidity by condition, age and sex.	Prevalence of multi-morbidity by age and sex.

BEACH, Bettering the Evaluation and Care of Health programme in Australia (2008); and DELPHI, Deliver Primary Health Care Information.

Re-estimated prevalence rates

Figure 1 shows the re-estimated prevalence of multi-morbidity for each of the three studies, stratified by age and sex. Prevalence in the Saguenay study was significantly higher than that found in the other studies in all age–sex groups. Among the age groups, multi-morbidity prevalence varied by as much as 61%, where the reported prevalence among women aged 45–64 was 95% in the Saguenay study, 46% in the BEACH substudy and 34% in the DELPHI study. For men and women aged 18–44, prevalence did not differ between the BEACH and DELPHI studies. However, for both men and women aged 44–64 and >65, the prevalence varied significantly among the three studies.

F

Results of thirty-six post-hoc multi-morbidity tests by age and sex were significantly different among all three studies (Pearson chi-square analysis).

Discussion

One interpretation of the results seen in Figure 1 is that they are real; the rates of multi-morbidity in the Saguenay region of Quebec are much higher than those in the region of South-western Ontario, with the rates in Australia, for the most part, falling in between. However, the authors suspect that an alternative interpretation holds: differences in method may play an important part and hence the emphasis, in this study, on parsing the details of the methods and weighing their importance to the rates found.

There are at least three methodological issues raised by this comparison.

The first issue that researchers should consider is the extent to which the studies are sampled differently. Sample size will make a difference to the rates of multi- morbidity, with larger samples, all other methods being equal, resulting in more reliable estimates. The BEACH substudy had both a larger sample and was of a national nature compared with the Saguenay and DELPHI studies, which represented small geographic regions. With regard to recruitment of patients within each practice, all three studies in the current comparison sampled visitors to the family practice. The time period set aside for recruitment of patients in each practice varied from 2–3 weeks in the Saguenay study to several days in the BEACH substudy and 6–12 months in the DELPHI study. Regardless of this time period, all studies were more likely to have sampled frequent attenders.

Another aspect of sampling was that only the DELPHI study selected a subset of consecutive patients at random, to control the workload of coding by the family physician (FP) and his or her staff. For the BEACH substudy, 30 consecutive patients were selected for recording from each FP’s 100 patients contributing data to the larger BEACH Programme. In Saguenay, the study group was selected after limiting the time period to days or weeks. We believe that the random sample versus a time-limited consecutive sample was not likely to affect the rate of multi-morbidity.

The second methodological issue is the completeness and accuracy of the source of data and the period of time over which data were considered for each patient. The three studies used different sources of data: Saguenay used the medical record; the BEACH substudy used the FP report, patient report and medical record; and the DELPHI study used morbidity managed at the initial and subsequent encounters coded by FPs.

Incompleteness may have affected each of the studies in slightly different ways. When care was delivered by the FP but not written in the chart, there was inscription bias, leaving paper charts incomplete regarding psychological problems, social problems and some physical problems. This type of error could reflect the level of commitment of the FPs, their workload that day or recording fatigue. All diagnoses may not have been recorded. As an example, the prevalence of obesity in the Saguenay study was ~4%, which was much lower than the prevalence in the community at large. If FPs were not actively managing obesity as an entity, they did not write it in their record.[14] Furthermore, the likelihood of incompleteness may be actually higher in studies of multi-morbidity than those of single morbidity because of the complexity of the patients’ constellation of problems. Furthermore, the number of visits or encounters may have affected the completeness of the data abstracted.

Another aspect of incompleteness arose from the time period over which the patient record was examined. The Saguenay study used the entire record, which could have extended back over decades, to ascertain the number of morbidities. Similarly, the substudy of BEACH included FP’s knowledge of the entire patient history and the patient’s knowledge. In contrast, the DELPHI study used only encounters that occurred during a specific 2-year period. For identification in the DELPHI study, the FP had to have been actively managing and coding the chronic conditions during encounters over the specified 2 years. This important difference from the other two studies, in which the entire record including the problem list was used, leads to the possibility of an underestimate of multi- morbidity in the DELPHI study.

Several other aspects of inaccuracy could have arisen. For example, a tentative diagnosis may have evolved over time and may or may not have been counted as multiple conditions. In the DELPHI study, such diagnoses were most likely counted multiple times, leading to a possibility of an overestimate. In the Saguenay, such tentative diagnoses could have been counted many times or not, depending on the rigour of the reviewer in following and interpreting such illness trajectories. In the BEACH substudy, the problem could only be ticked (and therefore counted) once.

The previous three paragraphs indicate that although all three studies were based on real-life practice, which was an asset in itself, the pressures of the FP’s workload could have minimized the completeness in each of the three studies through different mechanisms.

To summarize the issues of completeness and accuracy, each methodology had strengths and limitations. Recall bias was a potential problem in the BEACH substudy, because FPs were asked to provide the morbidity data. To minimize recall bias, the study protocol asked FPs to ask the patient and to review the medical chart as they completed the data form; this review may have been conducted to a greater or lesser extent. Nonetheless, on the positive side, the FP had just seen the patient that day and re-familiarized him- or herself with the patient’s problems. Additionally, on the plus side was the fact that they were led to respond to specific conditions, not record problems in free text. Incompleteness of data may have been an issue in the Saguenay study if some chronic condition data had not yet been recorded in the medical chart in a place where reviewers were looking or perhaps not yet charted at all. Incompleteness of data may have been a problem in the DELPHI study also because FPs were asked to code all problems presenting for care at each encounter and they may have left out some conditions of multi-morbid patients due to coding fatigue; perhaps the first and second conditions may have been coded, but the third, fourth and fifth conditions may have been missed. Moreover, the patient may have ongoing problems that the FP did not manage at these encounters (e.g. because the patient was under specialist care for those problems).

On the third issue, we noticed that the greatest difference among the studies was the definition and the count of the chronic conditions. There were two dimensions of this issue: (i) the definition of a chronic condition; and (ii) how they were counted. The BEACH substudy captured chronic conditions out of a list of 18 specific disease entities, plus broader diagnostic group options such as ‘other cardiovascular’, ‘other psychological problem’, ‘other arthritis’, ‘cerebrovascular disease’, ‘peripheral vascular disease’ and ‘malignant neoplasm’.[7] The DELPHI project defined a chronic condition as any of a list of International Classification of Primary Care (ICPC-2-R)[11] rubrics adapted from Lamberts and Okkes.[15] The Saguenay study definition was open to all diagnoses; although they used a Quebec survey list[16] to categorize conditions, they added any conditions that did not appear on that survey list and extended their codebook accordingly.

With regard to counting, the BEACH substudy clustered 22 diagnoses into 8 of the CIRS domains (plus another domain for malignant neoplasms)[12]; therefore, the maximum number of chronic domains could not exceed nine. The conditions included those determined by the Australian Government as National Health Priority Areas,[17] selected on the basis of chronicity[18] and management frequency in Australian general practice.[9] The DELPHI study counted any of the 98 individual chronic conditions on the adapted Lamberts and Okkes list,[15] which meant that the maximum could not exceed 98. The Saguenay study counted all conditions and their unique codes actually tallied to 33 separate conditions. As an example, if a patient had hypertension, peripheral vascular disease and other cardiovascular disease that would have counted as one disease domain in the BEACH substudy (vascular) and three conditions in the Saguenay and DELPHI studies. Similarly, osteoarthritis and low back pain (not related) would have been counted as one in the BEACH substudy and as two in the Saguenay and DELPHI studies. When comparing the prevalence in the sample of multi-morbidity, counting each morbidity type listed as one (rather than grouping the presence of one or multiple diseases into domains), the published results of the BEACH substudy[6] are far closer to those of the Saguenay estimates than the DELPHI estimates.

To highlight the important methods issue, sampling of visiting patients may overestimate multi-morbidity but one can adjust for attendance rates.[14] The sources of data used in these three studies overall may have underestimated multi-morbidity due to incompleteness of recording; however, some aspects of the data source for one study may have led to an overestimate. The largest difference among the three studies, likely to impact the rates of multi-morbidity, was the definition and counting of chronic conditions. Therefore, significant methodological issues may affect the results of studies of the prevalence of multi-morbidity, including sampling issues, source of data and definition and counting issues. We recommend therefore that international guideposts be developed for researchers to follow in an effort to create internationally comparable methods to study multi-morbidity.

Conclusions

Our goal is to improve methods of studying multi-morbidity. One option would be, instead of obtaining data from existing records, researchers could consider creating a data collection specifically designed for such studies. Such a tool could be used for billing purposes, so FPs could be reimbursed for the complexity level of patients. Such a tool for research and reimbursement will become more important as the population ages.

However, in the absence of such a universal tool, what can researchers do to improve the status quo? Many of the methodological details reviewed by this comparison would not be found in a ‘Methods’ section of typical articles, providing the reader of such studies with limited ability to interpret the results or with the necessity to contact the author for further details. Therefore, we recommend that writers and readers insist on detailed descriptions of the type of sampling, the completeness and accuracy of the source of data and the definition of the chronic conditions. We offer Table 1 as a guide and call it ‘Methods Crystals for MM’.

Finally, the difference in definitions of chronic conditions was the most likely candidate to explain the large differences in rates of multi-morbidity among the three studies. This is a testable hypothesis. We recommend conducting further comparisons among multi-morbidity data using agreed-upon standards for the definition of chronic conditions and the way to count multi-morbidity, in order to assess the impact of these methodological variations and enhance our understanding of multi-morbidity.

Declaration

Funding: Martin Fortin is funded by the Chaire de recherche appliquée des IRSC sur les services et politiques de santé en maladies chroniques en soins de première ligne/IRSC, ISPS, FCRSS, Centre de Santé et de services sociaux de Chicoutimi. The Saguenay project was funded by the Fonds de la Recherche en Santé du Québec. In the year of this sub-study BEACH was funded by the Australian Institute for Health and Welfare, the Australian Government Department of Veterans’ Affairs, National Prescribing Service Limited, Office of the Australian Safety Commission, AstraZeneca, Roche Products, Janssen-Cilag: Merck Sharp & Dohme, Pfizer Australia. The DELPHI project was funded by the Canada Foundation for Innovation, the Primary Health Care Transition Fund, and the Enhancing Quality Management in Primary Care Initiative of the Ministry of Health and Long Term Care of Ontario. The views expressed here are those of the authors and do not necessarily reflect the views of the Ontario Ministry of Health and Long Term Care. Dr. Stewart is funded by the Dr. Brian W. Gilbert Canada Research Chair in Primary Health Care Research.

Ethical approval: none.

Conflict of interest: none.