Evidence for validity of a national physician and patient-reported, cross-sectional survey in China and UK: the Disease Specific Programme.

OBJECTIVE: Diabetes represents a significant challenge for Chinese healthcare providers. Healthcare decision-making is generally based on many data sources, including randomised controlled and real-world studies; however, good-quality data from Chinese diabetes patients are scarce. We performed an initial validation to assess the representativeness of one source of real-world data-the Diabetes Adelphi Disease Specific Programme (DSP) in China.
SETTING: China, UK. PARTICIPANTS: The Chinese DSP included 2060 patients with previously diagnosed type 2 diabetes mellitus (T2DM) sampled by 200 physicians. The reference Chinese population comprised 238 639 patients with previously diagnosed T2DM. The UK DSP contained 1481 patients with T2DM sampled by 125 physicians; the reference UK population comprised 289 patients with diabetes. PRIMARY AND SECONDARY OUTCOMES: The primary outcome was comparison of unweighted China DSP and reference data for sex, body mass index (BMI), blood pressure (BP), patients achieving glycosylated haemoglobin (HbA1c)<7%, total cholesterol, coronary heart disease and dyslipidaemia. The secondary outcome was comparison of weighted UK DSP and reference data for BMI, BP, mean HbA1c, total cholesterol, smoking and insulin status.
RESULTS: Comparison of unweighted China DSP and reference data revealed statistical equivalence for BMI, systolic BP, proportion of patients achieving HbA1c <7%, total cholesterol, coronary heart disease and dyslipidaemia. Sex, age, diabetes duration, diastolic BP and mean HbA1c level were not equivalent, although differences were generally small. Weighting of data did not substantially affect the results. A similar pattern was observed for UK data.
CONCLUSIONS: This study provides evidence that the methodology used for the China and UK parts of the Diabetes DSP produces representative samples that are comparable with other independent sources of patient treatment outcomes data, which may ultimately inform public health decision-making. Although this method could be used in other countries, the current validation applies to UK and China. Further research is required for other countries. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

The Adelphi Real World Disease Specific Programme (DSP) is a valuable source of information on patients with type 2 diabetes in China, a region where reliable and up-to-date information is lacking.

This analysis has demonstrated, by comparison with a large, reference population-based, cross-sectional survey, that the DSP population is representative of patients with type 2 diabetes in China.

The representativeness of the DSP population was further supported by comparison of the UK Diabetes DSP with diabetes data gathered in the Health Survey for England.

Limitations of the study include the selection of patients included in the DSP samples, which depends on the physician's diagnostic skills, and the potential for over-representation of patients with more severe disease than the general population.

Patient-level data were available for the DSP and Health Survey for England populations but not the Chinese reference population, for which only aggregate data were available; as a result, possible design bias could not be addressed in the Chinese reference population.

Introduction

The International Diabetes Federation estimated the national prevalence of diabetes to be 9.32% for China in 2014,1 a significant increase from the <1% prevalence reported in 1980.2 This translated into an estimated 96 million individuals with diabetes and 1.2 million diabetes-related deaths in 2014.1 This represents a significant challenge; public health planners making formulary and reimbursement decisions must decide how to meet changing priorities by efficiently allocating funding and ensuring appropriate access to medicines. To date, treatment guidelines have largely been based on evidence from non-Asian populations, although an increasing number of randomised clinical trials are now in progress in China.3

Currently, decisions regarding the availability and reimbursement of medicines are made at the government, regional, local and hospital level in China. Decision-makers have differing evidentiary requirements and varied data are required to support the value of specific interventions. Data need to be current, treatment-specific, valid at a patient level, relevant and obtained from a representative sample. Outcomes data are also required, including safety and drug surveillance information, and efficacy, cost and resource-use data. Such data may be scarce and not readily available.

Although considered the gold standard for questions relating to efficacy and safety, data from randomised controlled studies are often unrepresentative of the population in which the intervention will be used because of strict inclusion criteria. Renal and cardiovascular complications may lead to the exclusion of many patients with diabetes from randomised clinical studies.4 For example, only half of Finnish patients with diabetes beginning treatment with statins for diabetic dyslipidaemia would have qualified for inclusion in the Heart Protection Study and the Collaborative Atorvastatin Diabetes Study.5 Epidemiological or ‘real-world’ studies provide information on larger, more representative populations but are generally not accessible to patient-level interrogation.

One source of real-world data for Chinese patients with diabetes is the Adelphi Real World Disease Specific Programme (DSP) for diabetes. DSPs, which are cross-sectional surveys generating data from real-world clinical practice, collect current patient demographic data and treatment practices, in addition to resource-use and quality-of-life data, in specific therapy areas and meet the majority of the criteria described above.6 DSPs have been conducted for a variety of therapy areas, including diabetes, in countries with varying healthcare systems and following societal changes, such as seen in China.

One important consideration of DSP validation is determining the representativeness of the data compared with wider populations. With that aim, we compared DSP diabetes data for China against a reference Chinese data source. To support this analysis, we performed a similar validation of the DSP in a developed western market with a contrasting socioeconomic and healthcare system to China, to demonstrate the adaptability of the data collection methodology. We selected the UK for this confirmatory validation because the availability of a reliable reference data source made it possible to assess the representativeness of DSP diabetes data for the UK compared with the wider UK population. We hypothesised that, viewed together, these analyses would provide evidence for the representativeness of the DSP as a source of real-world evidence for patients with diabetes in China.

Methods

Disease Specific Programmes

DSPs are large, multinational surveys of clinical practice that describe current disease management, disease-burden impact and associated treatment effects (patient-reported, clinical and physician-reported). The survey method is designed to adapt to any country, culture or disease area, with rapid implementation facilitating collection of up-to-date data. DSPs collect qualitative and quantitative data from four key sources of information: physician interviews, physician workload questionnaires, patient record forms completed by the doctor and questionnaires completed by the same patients. Physicians are selected for participation based on their eligibility to participate in the DSP in terms of specialty, location (hospital or office), whether they are personally responsible for treatment decisions and how many patients they see in a typical week. Candidate physicians who meet these criteria are invited to participate in the DSP; those who agree to participate are reimbursed for their time according to national reimbursement rates in their country. Patients are recruited only once and have no further follow-up as each DSP is a point-in-time survey. DSPs are repeated every 1 or 2 years, depending on the disease area, introduction of new treatments and how often guidelines are updated. The stages of DSP development are summarised in online supplementary figure S1; full details of DSP methodology have been published previously.6

The stages of a Disease Specific Programme (DSP)

The Diabetes DSPs selected for this analysis were conducted in China in 2012 and in the UK in 2013; these versions were chosen to match the time of data collection for the reference data sources. Geographically representative primary care physicians and specialists (hospital physicians only in China) were asked to sample the next 10 patients presenting with type 2 diabetes mellitus (T2DM), aged >18 years and currently taking antidiabetic drugs. Additional criteria that applied specifically to the present analysis, over and above those used to recruit patients for the DSP and in order to match the criteria of the comparator study, were that patients were not to be presenting for the first time with T2DM and insulin monotherapy was not allowed. Physicians completed a patient record form for these patients and gave them a patient self-completion form. Physicians recorded information on patient demographics, clinical characteristics (including glycaemic control and hypoglycaemic events in the Diabetes DSP), medications administered and resource use. The questions used in this analysis in China and the UK are shown in online supplementary appendices 1 and 2, respectively. Although not used in this analysis, the voluntary patient self-completion forms collected information about how diabetes affected the patient's everyday life, together with their opinions and understanding of their medications and glycaemic control. Patients could complete any or none of the questions and were instructed to complete the form without help from their healthcare practitioner.

The DSP is research involving survey procedures and as such does not require ethics committee approval. Patients provide informed consent for use of their anonymised and aggregated data for research and publication in scientific journals. This is achieved by means of a check box on the front page of the patient-completed survey. Data are collected in such a way that patients and physicians cannot be identified directly, with all data being aggregated and de-identified before receipt by Adelphi Real World. DSPs are performed in accordance with the European Pharmaceutical Marketing Research Association (EphMRA) guidelines7 and the US Health Insurance Portability and Accountability Act 1996.8

Comparator data

The Chinese reference data set was based on a multicentre, cross-sectional survey of outpatients with T2DM in 606 hospitals across China between April and June 2011.9 The first seven patients entering the facilities and meeting the following criteria were included: diagnosed with T2DM; one or more previous outpatient medical record pertaining to diabetes; aged >18 years; and treated with oral antidiabetic agents, either alone, with insulin or with glucagon-like peptide-1 agonists.

The UK Diabetes DSP data were compared against the Health Survey for England (HSFE) 2011 (10 617 patients).10 This is a cross-sectional, country-wide health survey that monitors health trends across the general population, estimating proportions of people with specified health conditions and the prevalence of risk factors and combinations of risk factors associated with these conditions. The 2011 HSFE had a special focus on cardiovascular disease, hypertension and diabetes, and included diabetes-related variables, for example, age at diagnosis and insulin treatment, in addition to standard demographic data and information on other disease areas. The HSFE included patients taking antidiabetic medication and those managing their condition with insulin alone or with diet and exercise alone; we applied exclusion criteria to identify those with T2DM who were treated with an antidiabetic medication.

To ensure comparability of the UK DSP and HSFE populations, patient characteristics were matched as closely as possible. This required exclusion of patients aged ≤18 years; diagnosed before age 35 years and treated with insulin (a proxy for type 1 diabetes in the absence of an explicit indicator in this database); and those not receiving endocrine drug treatment (a broad term used in the HSFE coding system11 and used in the current analysis to exclude patients managing their condition with diet and exercise). Pregnant women were also excluded.

Statistical analysis

Clinical and demographic characteristics common to the China DSP and Chinese reference population and the UK DSP and the HSFE data were compared to assess the validity of the UK and China DSPs. In order to compare the DSP and reference populations, weighting was needed to correct for imbalances between the groups. In the DSPs, patients consulting more frequently have a greater chance of selection and for a given frequency of visiting, a patient's chance of being sampled is a function of the total number of patients managed by the doctor, that is, the more patients a doctor manages, the less likely it is that any individual will be sampled. Inverse probability weighting was used to account for this, incorporating the frequency of visits made by the patient in the last 12 months (adjusted to 12 months if the patient had been managed for <12 months) and the total number of patients managed by each doctor (in conjunction with the number of patients sampled by each doctor). A random sample of patients with T2DM would include very few patients diagnosed on the same day as the study sampling, whereas the DSP population contained a relatively high number of patients attending for initial diagnosis because patients were sampled on days when they consulted their doctor. To better approximate a random sample, as well as matching the inclusion criteria for the comparator study in China, patients diagnosed on the day of the sample were excluded from the weighted analysis. The HSFE 2011 used a clustered stratified multistage sampling design. Similar to the weighting applied to the DSP, weights were applied to the HSFE data, according to guidance issued by the HSFE, to account for selection and non-response bias.12 Missing data were assumed to be missing at random and were not imputed.

Standard tests, such as the t-test and χ2 test, assume a null hypothesis that the two comparator groups are the same. Only if the p value is <0.05 can that hypothesis be rejected and a significant difference be claimed. A p value ≥0.05 does not allow the claim that there is no difference. Therefore, standard tests were not appropriate in this analysis, where the aim was to show no difference and tests for ‘equivalence’ were required. Variables common to each pair of data sets were compared using two one-sided tests aimed at testing for equivalence.13 Two means are considered equivalent if they occur within a predefined ‘distance’ or tolerance of each other. A sensible tolerance is the minimum important difference (MID). If the MID is unknown, assuming an MID of 25–50% of the overall SD is considered reasonable.14 In the present analysis, an MID of 25% was assumed for all variables. For proportions of patients (eg, proportion with hypertension), an MID of 25% of that proportion was used.

Results

China

The Chinese DSP included 2060 patients with T2DM sampled by 200 physicians. A total of 398 patients were receiving insulin only and were excluded in line with the reference population; the Chinese unweighted DSP population therefore included 1662 patients with T2DM. Table 1 shows clinical and demographic variables collected in both surveys. Patients' mean body mass index (BMI) was on the upper limit of normal at 24.3 kg/m2. Mean diastolic blood pressure (DBP) and systolic blood pressure (SBP) were high (83.2 and 132.7 mm Hg, respectively), mean glycosylated haemoglobin (HbA1c) level was high (7.4%) and the proportion of patients with HbA1c <7% was low (33%). Weighting of data to account for DSP design bias led to the exclusion of 79 patients but did not substantially change any disease characteristics other than the proportion of patients with comorbidities.

Table 1

Comparison of the Chinese Diabetes DSP and Chinese reference populations

	Chinese reference population (n=238 639)	DSP unweighted		p Value*	DSP weighted†		p Value*
Variable		n=1662	Missing		n=1583	Missing
Male, %	52.2	47.7	0	1.0000	46.7	0	1.000
Mean age, years (SD)	58.7 (11.7)	56.1 (11.3)	0	0.1423	56.4 (11.1)	0	0.0963
Mean time since diagnosis, years (SD)	5.6 (5.3)	3.3 (3.6)	34	1.0000	3.5 (3.5)	34	1.000
Mean BMI, kg/m2 (SD)	24.4 (3.2)	24.3 (3.1)	6	0.0000	24.5 (3.2)	6	0.0000
Mean total cholesterol level, mg/dL (SD)	182.9 (57.2)	185.4 (39.6)	399	0.0000	186.9 (41.5)	379	0.0000
Mean HbA1c level, % (SD)	7.9 (1.7)	7.4 (1.0)	73	1.0000	7.3 (1.0)	71	0.9990
HbA1c <7%, %	31.8	32.8	73	0.0000	33.3	71	0.0000
Mean diastolic BP, mm Hg (SD)	81.0 (11.1)	83.2 (8.6)	54	0.0962	83.1 (8.1)	51	0.1439
Mean systolic BP, mm Hg (SD)	131.9 (15.1)	132.7 (12.3)	54	0.0000	133.4 (11.8)	51	0.0000
Comorbidities, %
Coronary heart disease	10.9	9.2	0	0.0000	8.8	0	0.0000
Dyslipidaemia	19.7	22.8	0	0.0000	25.6	0	0.0193

Values in bold are p>0.05, that is, evidence is not strong enough to show equivalence.

*p Value for comparison with Chinese reference population.

†Weighted to account for design bias in the DSP.

BMI, body mass index; BP, blood pressure; DSP, Disease Specific Programme; HbA1c, glycosylated haemoglobin.

The reference Chinese population comprised 238 639 patients with T2DM. Patients had a mean age of 58.7 years, mean BMI of 24.4 kg/m2 and mean diabetes duration of 5.6 years (table 1). Mean DBP and SBP were high (81.0 and 131.9 mm Hg, respectively); mean total cholesterol level was normal at 183 mg/dL, although mean HbA1c level was high (7.9%) and the proportion of patients with HbA1c <7% was low (32%).

Comparison of the unweighted China DSP and Chinese reference populations revealed statistical equivalence for BMI, SBP and the comorbidities coronary heart disease and dyslipidaemia (table 1). Variables for which there was not enough evidence for equivalence were sex, age, duration of diabetes, DBP and HbA1c level, although the proportion of patients achieving an HbA1c level <7% was equivalent in both populations. Weighting of the DSP data did not substantially affect these differences.

United Kingdom

The UK DSP contained 1481 patients with T2DM sampled by 125 physicians; 1213 patients were eligible for inclusion in the analysis of unweighted data. In total, 268 patients were not eligible because they had type 1 diabetes (n=244), were diagnosed before the age of 35 years and treated with insulin only (n=19) or were pregnant (n=5). Weighting resulted in exclusion of another 41 patients who were diagnosed with diabetes on the day of the survey; the weighted analysis population thus comprised 1172 patients.

Overall, 8610 adults and 2007 children were interviewed for the 2011 HSFE; 2206 were excluded as they were under the age of 18 years, 7878 did not have a diagnosis of diabetes and 244 were not receiving an endocrine agent and were also excluded. The UK reference population therefore comprised 289 patients. Variables collected in both surveys are given in table 2. Total cholesterol levels were normal in both groups, as was DBP. SBP and BMI were high in both groups.

Table 2

Comparison of UK Diabetes DSP and UK reference HSFE populations

Variable	HSFE (n=289)	DSP unweighted		p Value	DSP weighted*		p Value
		n=1213	Missing		n=1172	Missing
Male, %	58.2	58.4	0	0.0001	57.7	0	0.0003
Mean age, years (SD)	63.9 (13.7)	61.5 (12.7)	32	0.1747	61.8 (12.5)	12	0.117
Mean age at diagnosis, years (SD)	54.1 (14.1)	54.6 (11.5)	98	0.001	54.7 (11.5)	98	0.0027
Mean time since diagnosis, years (SD)	9.7 (9.1)	6.9 (6.2)	87	0.989	7.3 (6.1)	87	0.925
Current smoker, %	14.3	16.1	32	0.002	15.7	32	0.0018
Insulin treated, %	21.0	26.3	0	0.0281	25.5	0	0.018
Mean BMI (SD), kg/m2	32.3 (6.3)	31.6 (6.6)	46	0.0241	31.5 (6.7)	46	0.0374
Mean total cholesterol level, mg/dL (SD)	172.1 (51.1)	178.3 (55.9)	121	0.0619	176.7 (54.7)	101	0.0349
Mean HbA1c level, % (SD)	8.0 (1.6)	7.8 (1.7)	17	0.0754	7.8 (1.7)	13	0.063
Mean diastolic BP, mm Hg (SD)	71.1 (11.3)	76.8 (9.6)	216	1.000	76.7 (9.5)	206	1.000
Mean systolic BP, mm Hg (SD)	132.6 (16.5)	134.9 (15.4)	216	0.070	135.1 (15.6)	206	0.109

Values in bold are p>0.05, that is, evidence is not strong enough to show equivalence.

*Weighted to account for design bias in the DSP.

BMI, body mass index; BP, blood pressure; DSP, Disease Specific Programme; HbA1c, glycosylated haemoglobin; HSFE, Health Survey for England.

Comparison of the unweighted UK DSP and HSFE populations revealed statistical equivalence in sex, age at diagnosis, BMI and total cholesterol level (table 2). Some exceptions, where equivalence could not be demonstrated, were observed. DSP patients were younger, had a shorter time since diagnosis, lower HbA1c level and higher SBP and DBP than HSFE patients. Weighting of the UK DSP population did not substantially affect these differences, with the exception of total cholesterol level, which became statistically equivalent after weighting.

Discussion

Public health planners in China face a diabetes epidemic and must make treatment recommendations complicated by a paucity of good-quality data obtained in relevant populations in a timely manner. While collection and reporting of data is improving, data are scarce that satisfy all the criteria required to meet the needs of Chinese decision-makers. DSPs offer one solution to this issue, being sufficiently up to date, collected rapidly and frequently and containing information on a breadth of clinical, demographic and outcome variables that can inform public health decision-making when used with other supporting data.

The current analysis was undertaken to demonstrate the representativeness of DSP data compared with the Chinese T2DM population. Comparison of the China DSP and reference Chinese populations identified equivalence in many variables common to both studies. Some areas of non-equivalence were observed: time since diagnosis of diabetes was longer in patients in the reference population and DBP values were non-equivalent, although the difference was small (<2 mm Hg) and its clinical relevance questionable. Mean HbA1c level was non-equivalent, although the proportion of patients achieving HbA1c <7% was equivalent. This suggests that the reference data set may have contained more patients with high values, which would have a greater impact on mean HbA1c than on the proportion with HbA1c <7%. The between-group difference is within the bounds of natural variation, as reflected by the SDs and the mean baseline HbA1c levels of 7.0–8.3% reported in other observational and phase IV studies or surveys.15–17

The comparison of UK DSP and HSFE diabetes data, which was performed to substantiate the findings of the Chinese comparison, also provided evidence for the representativeness of the DSP, with equivalence in many of variables. Although equivalence was not demonstrated for some, including patient age and time since diagnosis, this may reflect different characteristics of presenting patients in the DSP versus the randomly selected HSFE group. HbA1c and SBP were non-equivalent, although between-group differences were small (<0.2% and 2.5 mm Hg, respectively), within the bounds of natural variation as reflected by the SDs and of questionable clinical relevance.18 19

In line with other observational and real-world studies, several limitations of the data sources should be considered. The primary limitations of the DSP relate to selection and diagnosis of patients. Physicians were asked to include the next 10 presenting patients with T2DM to reduce selection bias. The integrity of this process depends on the physician's diagnostic skills as no formal entry criteria were specified. In addition, this process favours patients presenting more frequently, as more frequent consultation increases the likelihood of selection. Although we corrected for this using a weighting process, the possibility cannot be excluded that patients with more severe disease or complications might be over-represented in the DSP population. No patient-level data were available for the Chinese reference source and aggregated evidence was used instead. As a result, design bias could not be corrected through weighting. This means that even if the DSP data were perfectly corrected to be representative of the Chinese diabetes population, some differences would still be observed. We noted differences between the DSP and reference populations that were not corrected either by looking only at the prevalent population or by weighting; further research is required to identify potential reasons for such differences. The HSFE also has limitations, including oversampling in underpopulated areas, lack of response and differences in how study visits and procedures were performed. These limitations were addressed using a complex weighting strategy. Potential bias relating to non-response as a result of ill health was not, however, accounted for in the HSFE. These limitations must be taken into consideration before generalising the findings to other populations.

The strengths of the DSP approach should be considered. Although the DSPs are exploratory studies that complement rather than replace larger studies, advantages include the ability to rapidly perform studies in relatively small populations that nonetheless provide insights into diseases, attitudes and outcomes that might otherwise be difficult to obtain in such a timely manner and at in-depth patient level. A consistent methodology is used for DSPs across countries and economic environments, enabling cross-country comparisons. This may not be possible using registries or databases designed to be specific for a particular country or region. DSPs can also include elements related to patient-reported outcomes and impact on usual activities, providing insights into aspects not routinely assessed in randomised clinical trials.

Observational studies, such as performed by Ji et al,9 provide important epidemiological information in rapidly changing healthcare environments. This reference population comprised patients from 606 hospitals representing every region of mainland China other than Tibet and was designed to represent all regions of the Chinese mainland. Other strengths include the population size and census-style data. All patients were consulting a physician, similar to the DSP, and physicians sampled consecutive presenting patients in both studies. The strength of the HSFE lies in its epidemiological robustness in terms of general population coverage and collection of comprehensive clinical data and patient-reported symptom and burden outcomes from a representative cross section of England, repeated annually with a consistent methodology.10 Both sources complement the DSPs as a result of consistency of overlapping variables; the depth of information provided by the DSPs on a smaller number of patients also complements the larger but less detailed data presented by Ji et al9 and the HSFE.10

Ultimately, no single source provides all the data needed by every stakeholder. The China DSP and reference study are complementary and valid as variables common to both studies are consistent. Importantly, the independence of each study allows validation of the other. The HSFE and UK DSP have also been shown to be complementary, each validating the other. The DSPs can therefore be used to complement data from clinical trials performed in well-defined but potentially unrepresentative populations to provide an update on data otherwise obtained from large-scale but costly and time-consuming epidemiological studies. Based on the results of the present analysis and considering the limitations discussed above, the Chinese and UK DSP data could be considered appropriate for inclusion in submissions for health technology assessments. While the DSPs are a useful additional tool for modelling and health technology authority requirements, further validation is required to determine whether data from countries other than China and the UK can be extrapolated to larger populations.

In conclusion, the present analysis indicates that the China Diabetes DSP, with appropriate weighting applied, is an epidemiologically valuable source of information on patients with T2DM that is representative of the wider diabetes population in China, as indicated by comparability of data collected in the DSP and a reference population-based cross-sectional survey. Comparison of the UK Diabetes DSP and a reference UK diabetes population derived from the HSFE provides further support for this approach in the diabetes setting. Together, these findings highlight the need for good-quality data collected using standardised collection methodologies and suggest that data generated using the DSP methodology may complement other data sources of information on patients with T2DM by filling a need for up-to-date patient treatment outcome data, which may ultimately inform public health decision-making in China.