The competence of village clinicians in the diagnosis and treatment of heart disease in rural China: A nationally representative assessment.

BACKGROUND: While strengthening primary care quality is key to China's health system reforms, evidence to guide this work has been limited, particularly for rural areas. This study provides the first nationally-representative assessment of village doctors' competence in diagnosing and managing presumptive heart disease.
METHODS: A cross-sectional study of village clinics was conducted across five provinces. We presented standardized clinical vignettes to evaluate clinicians' competence in diagnosing and managing unstable angina. Enumerators accompanying mock patients documented the interaction, including questions, physical examinations, diagnoses, and management options provided by the doctor. We measured diagnostic process competence as adherence to "recommended" questions and examinations based on national clinical practice guidelines, diagnostic competence according to whether clinicians provided a correct diagnosis, and management as correct medication and/or referral. Management was assessed twice: following clinicians' own diagnoses determined through questioning and examinations, and after enumerators provided doctors with the correct diagnosis.
FINDINGS: Clinicians completed 26% (95% CI 24% to 28%) of recommended diagnostic questions and examinations; 20% (14% to 27%) arrived at a correct diagnosis. Rates of correct management were 43% (35% to 51%) following clinicians' own diagnosis and 51% (43% to 59%) after being given the correct diagnosis. When given the correct diagnosis and only asked to provide treatment, clinicians prescribed 82% fewer potentially harmful medications than in treatments based on their own (potentially incorrect) diagnosis.
INTERPRETATION: The ability of village doctors to diagnose a textbook case of unstable angina is limited. Deficits in diagnostic competence led to low rates of correct management.

Evidence before this study

Many of China's healthcare reforms over the past decade have strengthened its primary care system. However, based on searches of PubMed and Web of Science for peer-reviewed studies in English and Mandarin, few studies have examined primary care quality, particularly in China's village clinics, where patients often first contact the system. Several recent studies using regional samples have shown substantial deficits in clinicians’ competence and practice, yet nationally representative evidence on the quality of rural primary care is lacking.

Added value of this study

This study provides the first nationally representative assessment of village doctors’ competence in diagnosing and managing presumptive heart disease. In clinical vignettes designed to evaluate doctor competence of diagnosis and management of unstable angina, deficits were widespread. On average across all vignettes, doctors completed only 39% of questions and examinations deemed essential in official national clinical practice guidelines and 20% of doctors arrived at a correct diagnosis. Overall, providers correctly managed 43% of vignette cases depicting textbook symptoms of unstable angina. Our results suggest that deficits in management are due to diagnostic rather than therapeutic errors: while 35% prescribed potentially harmful medications when making their own diagnosis, only 6% prescribed these drugs when given the diagnosis.

Implications of all the available evidence

Our study affirms the importance of improving quality of care in rural China. Although China has increased its healthcare spending and improved rural access to care, village doctor competence is still inadequate to address patient needs. Policy efforts underway to strengthen primary care in China's rural areas is urgently needed.

Alt-text: Unlabelled box

Introduction

Strong primary care systems are the foundation of high-quality, integrated healthcare systems and are vital in low- and middle-income countries (LMICs) [1]. This is especially true in China, which [2], like many other LMICs, faces an increasing burden of noncommunicable diseases (NCDs) while continuing to face important burdens of some infectious diseases [3], [4], [5]. This “dual burden” increases the necessity of a high-quality primary care system to efficiently and effectively triage and manage patients.

Over the past two decades, China has expanded coverage under public health insurance schemes and increased governmental investments in the public health system [2,6,7]. More recently, China has focused on improving quality and efficiency, largely through strengthening the primary care system [7], [8], [9]. There is limited evidence, however, on the quality of primary care or its determinants in China's rural areas. While a few studies have examined care quality in certain regions of rural China [10], [11], [12], no nationally representative studies have done so. To inform their efforts, policymakers need a more comprehensive understanding of primary care quality and the barriers to its improvement.

This study provides nationally representative evidence on the quality of care in China's rural areas. A representative sample of village doctors were administered clinical vignettes to assess their ability to properly diagnose and manage a case of unstable angina. In China, the burden of NCDs such as angina has been increasing rapidly due to changing lifestyles and population aging [3]. NCDs caused more than 9•2 million premature deaths in 2016 [3,13]. A 2013 nationally representative survey found a total diabetes prevalence of 11%, with more than 60% of cases unaware they had diabetes [14]. In rural areas, 71% of persons with diabetes were unaware, and only 25% were receiving treatment.

To address the NCD health burden, primary care providers must be able to correctly diagnose and manage afflicted patients. In rural areas, the country's approximately 622,000 village doctors are often the first point of contact for patients experiencing symptoms of heart disease [9,15]. For patients to receive timely care, therefore, requires that village doctors are able to diagnose potentially serious conditions and triage patients to upper level facilities when appropriate. Village-level providers, however, often have limited formal medical education. Traditionally, the primary source of revenue for village doctors has been drug sales, raising concerns that they face perverse incentives misaligned with appropriate treatment [8]. With more recent public health initiatives, a larger proportion of village doctor compensation comprises government subsidies attached to increased public health duties, including the monitoring and management of NCD patients [9]. Some have expressed concern, however, that increased public health duties have reduced village doctor time available for training and the provision of curative care services [16].

To assess their competence, we presented doctors with a hypothetical disease case in the form of a standardised clinical vignette. We used quantitative data collected from the vignette to assess rural village doctors’ competence, defined in this study as doctors’ diagnostic process quality, correct diagnosis, and correct management of a presumptive case of unstable angina [17].

Methods

Sample and participants

Data were collected as part of the 2016 wave of the China Rural Development Survey (CRDS), a national survey of rural households and infrastructure conducted by the Chinese Academy of Sciences and Peking University. The survey's sample includes 101 villages from 25 rural counties across five provinces. To ensure a nationally-representative sample, villages were selected through a stratified random sampling procedure. In the first stage, one province was randomly selected from each of China's five major agricultural and ecological zones: Shaanxi province in the arid north west; Sichuan province in the poor, mountainous south west; Jiangsu province in the low lying south eastern coastal region; Hebei province in the flat northern plains region; and Jilin province in the far north east temperate region. Within each province, counties were divided into five strata based on per capita income, and one county was randomly selected each stratum [18]. Two townships were then randomly selected from each county, and two villages were selected from each township, for a total of 50 townships and 101 villages (one village split into two during the study). In villages with three or fewer clinics, all clinics were surveyed. If the village contained more than three clinics, three clinics were randomly selected for inclusion in the survey. Sample selection was conducted by researchers at the Center for Chinese Agricultural Policy using Stata (version 8). As data collection for this study was incorporated into a long-running multipurpose survey, the sample size was determined by the objectives and constraints of the original survey. Data were collected using structured questionnaires on printed paper and subsequently shipped to a central location where they were scanned and double-entered. Data were analyzed using Stata (version 15).

Fig. 1 shows the determination of the final sample of village doctors used in the analysis. Of the 101 villages, 92 had village clinics. Eighty-nine villages had between one and three clinics, and three had more than three clinics. Following the sampling procedures yielded 120 village clinics in total. We included all clinicians in the 115 clinics with between one and three clinicians and randomly selected three clinicians in the five clinics with more than three clinicians. In total, 159 clinicians were included. Twelve clinicians were unavailable to complete the survey and an additional three observations were excluded, yielding a final analysis sample of 144 village doctors across 114 clinics.

Fig. 1

Selection of Village Doctors

Note: 1Sampling rules for village clinics: when there are more than three village clinics in one village, select three clinics to survey. The priority of selecting clinics:1) select those which have been surveyed in previous waves; 2) select those which diagnose diseases mainly based on western medicine; 3) select those which undertake public health services; If there are still more than three clinics after applying above criterion, randomly select three clinics. 2Sampling rules for village doctors: when there are more than three village clinicians in one clinic, select three clinicians to survey. The priority of selecting clinicians: 1) select those have been surveyed in previous waves; 2) select those diagnose disease mainly based on western medicine; 3) select those undertake public health services; If there are still more than three doctors after applying above criterion, randomly select three clinicians per clinic.

Surveys and vignettes

Village doctor and facility surveys

Enumerators administered both doctor and facility surveys (Table 1) in person in the selected clinics. Doctor surveys collected doctors’ age, gender, years of experience, highest general and medical education level attained, qualifications, and salary. Facility surveys collected information on the annual revenue, number of doctors employed, patient volume, stock of medicines, presence of medical instruments (including those relevant for the disease case – electrocardiographs (ECGs), stethoscopes, sphygmomanometers, and thermometers), and other characteristics.

Table 1

Village Clinician Characteristics by Qualification.

	Full Sample (N=144)		Clinicians with Village Doctor Qualification or No Qualification (N=102)		Clinicians with High Medical Qualification (N=42)
	Mean (sd) / n(%)	95% CI	Mean (sd) / n(%)	95% CI	Mean (sd) / n(%)	95% CI
Age of village doctors (years)	49•63 (11•04)	(47•81 to 51•45)	50•81 (11•45)	(48•57 to 53•06)	46•76 (9•51)	(43•80 to 49•73)
Male village doctors (0/1)	109 (76%)	(68% to 82%)	82 (80%)	(72% to 87%)	27 (64%)	(49% to 77%)
Practicing experience (years)	28•00 (12•45)	(25•94 to 30•06)	28•76 (13•14)	(26•17 to 31•36)	26•17 (10•50)	(22•89 to 29•44)
Highest education of village doctors
Junior high school or less (0/1)	20 (14%)	(9% to 20%)	17 (17%)	(11% to 25%)	3 (7%)	(2% to 19%)
Academic/vocational high school (0/1)	97 (67%)	(59% to 74%)	68 (67%)	(57% to 75%)	29 (69%)	(54% to 81%)
Junior College degree or higher (0/1)	27 (19%)	(13% to 26%)	17 (17%)	(11% to 25%)	10 (24%)	(13% to 39%)
Highest medical education of village doctors
No medical education (0/1)	26 (18%)	(13% to 25%)	24 (24%)	(16% to 33%)	2 (5%)	(1% to 16%)
Vocational medical high school (0/1)	90 (63%)	(54% to 70%)	60 (59%)	(49% to 68%)	30 (71%)	(56% to 83%)
Junior medical college or higher (0/1)	28 (19%)	(14% to 27%)	18 (18%)	(11% to 26%)	10 (24%)	(13% to 39%)
Type of medical education of village doctors
No Medical Education (0/1)	26 (18%)	(13% to 25%)	24 (24%)	(16% to 33%)	2 (5%)	(1% to 16%)
Chinese (0/1)	9 (6%)	(3% to 11%)	4 (4%)	(2% to 10%)	5 (12%)	(5% to 25%)
Western (0/1)	62 (43%)	(35% to 51%)	40 (39%)	(30% to 49%)	22 (52%)	(38% to 67%)
Chinese & Western (0/1)	45 (31%)	(24% to 39%)	34 (33%)	(25% to 43%)	11 (26%)	(15% to 41%)
Public Health (0/1)	2 (1%)	(0% to 5%)	0 (0%)	(0% to 4%)	2 (5%)	(1% to 16%)
Receives basic wage from the government (0/1)	34 (24%)	(17% to 31%)	17 (17%)	(11% to 25%)	17 (40%)	(27% to 56%)
Total annual income (yuan)	23,718•12 (14,239•33)	(21,372•56 to 26,063•69)	22,154•99 (13,014•93)	(19,598•62 to 24,711•37)	27,514•31 (16,403•27)	(22,402•69 to 32,625•93)
Weekly working days (days)	6•29 (1•70)	(6•01 to 6•58)	6•17 (1•85)	(5•80 to 6•54)	6•57 (1•24)	(6•19 to 6•96)

Data are n (%) for binary variables and mean (SD) for continuous variables. Village doctor qualification is not a regular certification for clinicians. Clinicians with village doctor qualification are permitted by local health authorities to work only in village clinics. Having high medical qualification is defined as having registered as a medical practitioner or an associate medical practitioner.

Standardized clinical vignettes

We used standardized clinical vignettes to measure doctors’ ability to correctly diagnose and manage a patient presenting with a presumptive case of unstable angina. Unstable angina was chosen as the test case for three reasons. First, heart disease comprises an increasing proportion of China's disease burden in rural as well as urban areas [19]. Second, while diagnosis is not particularly complex, it requires adequate competence of subtleties of the disease presentation. Third, correct management of the disease - ultimately the most important outcome - is still possible without a specific diagnosis if providers recognize the need for higher-level care and triage appropriately. With the assistance of cardiologists from the Peking University Health Science Center, we adapted the vignette script to the Chinese context from earlier studies in India [20]. This script, provided in the supplementary material, includes disease symptoms, medical history, and patient background.

The standardized clinical vignettes were presented to doctors by enumerators who were trained with the goal of standardizing the implementation. The role of enumerators in conducting vignettes was determined at the time of training before the launch of the survey. Measures were taken to ensure that the vignettes were implemented in a standard way, including all instructions and response to clinician questioning and exam requests being fully scripts, as well as several days of material training and pair practice. Enumerators were then dispatched in each province by the survey team.

During the survey, two enumerators presented the vignette to doctors. One enumerator assumed the role of the “mock patient” and the other “facilitator” stated instructions to the doctor, documented the interaction, and provided additional information that the patient might not know but that the doctor would determine if she/he actively solicited it, such as the results of tests or examinations. To begin the vignette, the facilitator informed the doctor that a 45-year-old male patient was visiting the clinic. The mock patient then told the doctor about his problem with an opening statement (“Doctor, I've had chest pains recently”) (Supplemental text A8). Doctors were then asked to proceed as they would with a real patient and were told that the patient would answer any questions asked and comply with any instructions given. During the interaction, the facilitator documented the doctor's questions, diagnostic examination requests, stated diagnosis, treatment prescribed (drugs or patient education/instructions), and whether the provider referred the patient to another provider.

Doctors choose management strategies based on both their diagnostic and therapeutic competence. To determine how deficits in diagnostic competence or in competence of appropriate treatment of angina patients are independently contributing to mismanagement, following the completion of the initial vignette, doctors were asked directly to state how they would manage a patient with unstable angina. Facilitators then documented the treatment prescribed (drugs or patient education/instructions), and whether the provider referred the patient to another provider.

Quality measures

We evaluated the doctors’ competence on three domains of the clinical encounter: diagnostic process quality, diagnosis given, and case management.

We assessed diagnostic process quality by documenting the questions that doctors asked and the physical examinations they conducted in comparison with standard checklists stated in the Chinese national practice guidelines [21]. For patients who present stating typical symptoms of unstable angina, these checklists contain 16 “recommended” and seven “essential” questions that doctors should ask, along with six “recommended” and one “essential” physical examinations that doctors should perform to make proper diagnoses (Supplemental Table A1 and Table A2). We scored the number and percentage of recommended and essential questions that doctors asked, along with the number and percentage of recommended and essential physical examinations that doctors performed.

We categorized diagnoses as correct if they named unstable angina, angina, or acute coronary syndrome; partially correct if they named coronary artery disease, cardiovascular disease, or a number of other cardiac ailments; or incorrect if the doctor made no mention of the correct or partially correct diagnoses (Supplemental Table A6). If the doctor did not provide any diagnosis, the vignette patient was instructed to ask for a diagnosis at the end of the interaction.

We categorized case management as “correct” if (a) the doctor referred the patient to an upper level provider and no drugs were prescribed, or (b) the doctor referred the patient to an upper level provider and the doctor prescribed a correct medication. We considered case management “partially correct” if the doctor (a) did not refer the patient to a higher level provider but prescribed a correct medication, or (b) did refer the patient to a higher level provider, but prescribe any unnecessary drugs or potentially harmful medication, which we defined operationally as an antibiotic, hormone, or analgesic (Supplemental Table A7). We considered case management “incorrect” if the doctor did not refer patient to a higher-level provider and prescribed any unnecessary drugs, or potentially harmful medication, or no drugs at all.

Statistical methods

Descriptive statistics were computed for village doctor and facility (clinic) characteristics and each of the diagnostic and disease management quality measures described above. For disease management, we tested for differences between management decisions following doctors’ own diagnosis and management decisions after doctors were provided with the true diagnosis of unstable angina using two-tailed t-tests for continuous outcomes and tests of proportions for binary outcomes.

We explored how observable characteristics of village doctors and clinics are related to quality measures using multivariate regressions. We report average marginal effects from logistic regression where the dependent variable is binary and generalized linear models with a binomial distribution and logit link where the dependent variable is a proportion. Village doctor characteristics included as independent variables are doctor gender, experience, general and medical education, medical qualification, and total annual income. Clinic characteristics included are the number of clinicians working in the facility, annual patient volume, varieties of medicine in stock, the total value of medical instruments, and the population in the facility's catchment area for public health services. All regressions additionally control for province fixed effects. In a small subset of models, inclusion of province fixed effects caused separation; in these cases we estimate Firth logits. Standard errors in all regressions are adjusted for clustering at the village level.

All analysis were conducted using Stata version 14.1.

Role of the funding source

The funder had no role in study design, data collection, and analysis, decision to publish, or preparation of the manuscript. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.

Ethical approval

Ethics approval was not required for this study as it consisted only survey data collection and data were anonymized so that providers could not be readily identified.

Results

Village doctor characteristics

Participating doctors had low levels of both general and medical education (Table 1). While 19% had a junior college degree or higher, 67% had completed only academic or vocational high school and 14% had only junior high school level or below. Similarly, while 19% had completed junior medical college or higher, 62% had completed vocational medical high school, and 18% had no medical education. Most were trained in Western medicine (43%) or a combination of Chinese and Western medicine (31%). About 10% had the highest “Medical practitioner” qualification, 19% had the next highest “Associate medical practitioner” qualification, 66% had the “Village Doctor” qualification, and 5% reported having no formal qualification.”

Clinic characteristics

We surveyed doctors in 114 total clinics across the five sample provinces (Table 2). About one to two doctors staffed each clinic, supported by one to two non-clinical staff. Clinics had a mean volume of 4716 patients per year (around 12•9 patients per day). Clinics held an average of 213 different varieties of medicine within their inventories including 115 varieties of Western medicine and 35 varieties of Chinese patent medicine. Notably, some clinics only held either Western medicine or Chinese patent medicine. Of the four medical instruments needed for the standard recommended examinations used to diagnose unstable angina – electrocardiographs (ECGs), stethoscopes, sphygmomanometers, and thermometers – nearly every clinic had stethoscopes, sphygmomanometers, and thermometers, but only 13% had access to ECGs on site.

Table 2

Clinic characteristics (N=114).

	Mean (sd) / n(%)	95% CI
Annual net income (yuan)	36,308•88 (37,089•09)	(29,364•30 to 43,253•46)
Number of clinicians	1•44 (1•02)	(1•25 to 1•63)
Number of non-clinicians	0•21 (0•67)	(0•09 to 0•34)
Annual patient volume (visits)	4715•59 (10,142•55)	(2816•49 to 6614•69)
Varieties of medicine in inventory	213•42 (181•12)	(179•51 to 247•33)
Varieties of western medicine	114•57 (95•28)	(96•57 to 132•58)
Varieties of Chinese patent medicine	34•51 (36•23)	(27•66 to 41•36)
Total value of medical instruments in clinic (yuan)	15,079•45 (30,142•96)	(9435•46 to 20,723•43)
Population in village clinician's catchment area for public health services	1426•66 (1274•88)	(1186•85 to 1666•46)
Presence of medical instruments that might be used to diagnose unstable angina
Electrocardiograph (0/1)	15 (13%)	(8% to 21%)
Stethoscope (0/1)	113 (100%)	(97% to 100%)
Sphygmomanometer (0/1)	113 (100%)	(97% to 100%)
Thermometer (0/1)	113 (100%)	(97% to 100%)

Note. Data are n (%) for binary variables and mean (SD) for continuous variables. Wilson CIs are calculated for proportions. The number of varieties of medicine in inventory is more than the sum of varieties of western medicine and Chinese patent medicine because there were Chinese herbal medicines in some clinics.

Village doctor competence

Diagnostic questions and examinations

On average, doctors completed only 26% of the full recommended checklist, including an average of 25% of recommended questions and 27% of recommended examinations. Doctors completed 39% of the smaller subset of essential questions and examinations items, including an average of 34% of the essential questions, and 78% of providers recommended or completed the single essential exam (ECG).

Although the average is low, we do find substantial variation across providers in checklist completion. For instance, while 60 out of 151 (40%) of providers completed less than 20% of checklist items, 38 (25%) completed more than 30%. The inter-quartile range (IQR) is 17 percentage points for all recommended checklist items and 25 percentage points for essential items.

Diagnosis

All doctors either gave a diagnosis to the patient at the end of the vignette or were asked for one, but most diagnoses were inaccurate. As seen in Table 3, only 20% of doctors correctly diagnosed the patient with unstable angina, angina, or acute coronary syndromes. The remainder either gave a partially correct diagnosis (37%), which included mentions of cardiovascular or cardiac heart disease, heart problems, cardiac insufficiency, myocardial infarction, or myocardial ischemia; gave a wrong diagnosis (31%); or were unable to give any diagnosis (12%).

Table 3

Village clinicians’ competence in process quality, diagnosis, and management of unstable Angina.

Recommended examinations	Mean (sd) / n(%)	95% CI
Number of village clinicians (N=144)
Questions
Number of questions asked in total	7•78 (4•31)	(7•08 to 8•49)
Recommended questions (16 in total)
Number of recommended questions asked and recommended exams performed	4•03 (2•32)	(3•65 to 4•42)
Proportion of recommended questions asked	0•25 (0•14)	(0•23 to 0•28)
Essential questions (7 in total)
Number of essential questions asked	2•35 (1•58)	(2•09 to 2•61)
Proportion of essential questions asked	0•34 (0•23)	(0•30 to 0•37)
Exams
Number of exams performed in total	3•17 (2•02)	(2•83 to 3•50)
Recommended examinations (6 in total)
Number of recommended exams performed	1•62 (1•03)	(1•45 to 1•79)
Proportion of recommended exams performed	0•27 (0•17)	(0•24 to 0•30)
Essential examinations (1 in total)
Number of essential exams performed	0•78 (0•42)	(0•71 to 0•85)
Proportion of essential exams performed	0•78 (0•42)	(0•71 to 0•85)
Questions & Examinations
Recommended questions & examinations (22 in total)
Number of recommended questions asked and recommended exams performed	5•65 (2•81)	(5•19 to 6•12)
Proportion of recommended questions asked and exams performed	0•26 (0•13)	(0•24 to 0•28)
Essential questions & examinations (8 in total)
Number of essential questions asked and essential exams performed	3•12 (1•69)	(2•85 to 3•40)
Proportion of essential questions asked and exams performed	0•39 (0•21)	(0•36 to 0•43)
Diagnosis
Gave any diagnosis (0/1)	127 (88%)	(82% to 92%)
Correct diagnosis (0/1)	29 (20%)	(14% to 27%)
Partially correct diagnosis (0/1)	53 (37%)	(29% to 45%)
Incorrect diagnosis (0/1)	45 (31%)	(24% to 39%)

Note. Data are n (%) for binary variables and mean (SD) for continuous variables. Wilson CIs are calculated for proportions.

Management

Despite low rates of correct diagnosis, correct management of patients was higher as doctors may correctly refer patients even if they are unable to make a correct diagnosis (Table 4). Following the doctor's own diagnosis (i.e. the possibly incorrect diagnosis arrived at by the doctor following questioning and examinations), 43% correctly managed the case, while 23% provided partially correct management and the remaining 35% incorrectly managed the case.

Table 4

Village doctor management of unstable Angina.

	Disease ManagementAfter Clinician's Own Diagnosis (N=142)(1)		Disease ManagementAfter True Diagnosis Given (N=142)(2)		Difference (2) - (1)	P-value
	Mean (sd) / n(%)	95% CI	Mean (sd) / n(%)	95% CI
By Management
Correct management	61 (43%)	(35% to 51%)	72 (51%)	(43% to 59%)	8% (-4% to 19%)	0•19
Referral with no drugs	56 (39%)	(32% to 48%)	59 (42%)	(34% to 50%)	2% (-9% to 14%)	0•72
Referral with correct drugs but no harmful or unnecessary drugs	5 (4%)	(2% to 8%)	13 (9%)	(5% to 15%)	6% (0% to 11%)	0•051
Partially correct management	32 (23%)	(16% to 30%)	49 (35%)	(27% to 43%)	12% (2% to 22%)	0•026
Referral with correct drugs and harmful or unnecessary drugs	0 (0%)	(0% to 3%)	0 (0%)	(0% to 3%)	••	••
Referral with harmful drugs	10 (7%)	(4% to 12%)	2 (1%)	(0% to 5%)	-6% (-10% to -1%)	0•018
Referral with unnecessary drugs	13 (9%)	(5% to 15%)	21 (15%)	(10% to 22%)	6% (-2% to 13%)	0•14
No referral with correct drugs but no harmful or unnecessary drugs	9 (6%)	(3% to 12%)	26 (18%)	(13% to 25%)	12% (4% to 19%)	0•0021
No referral with correct drugs and harmful or unnecessary drugs	0 (0%)	(0% to 3%)	0 (0%)	(0% to 3%)	••	••
Incorrect management	49 (35%)	(27% to 43%)	21 (15%)	(10% to 22%)	-20% (-29% to -10%)	0•0001
No referral with no drugs	8 (6%)	(3% to 11%)	3 (2%)	(1% to 6%)	-4% (-8% to 1%)	0•12
No referral with harmful drugs	17 (12%)	(8% to 18%)	3 (2%)	(1% to 6%)	-10% (-16% to -4%)	0•0012
No referral with unnecessary drugs	24 (17%)	(12% to 24%)	15 (11%)	(7% to 17%)	-6% (-14% to 2%)	0•12
By Components
Referral (0/1)	84 (59%)	(51% to 67%)	95 (67%)	(59% to 74%)	8% (-3% to 19%)	0•18
Gave any medication (0/1)	78 (55%)	(47% to 63%)	80 (56%)	(48% to 64%)	1% (-10% to 13%)	0•81
Number of drugs prescribed	2•49 (1•37)	(2•18 to 2•80)	1•95 (1•08)	(1•71 to 2•19)	-0•54 (-0•92 to -0•15)	0•0069
Number of western drugs prescribed	1•36 (1•34)	(1•06 to 1•66)	0•91 (0•86)	(0•72 to 1•10)	-0•45 (-0•80 to -0•09)	0•013
Number of Chinese patent drugs prescribed	1•12 (0•81)	(0•93 to 1•30)	0•76 (0•89)	(0•56 to 0•96)	-0•35 (-0•62 to -0•09)	0•0099
Number of herbal drugs prescribed	0•01 (0•11)	(-0•01 to 0•04)	0•00 (0•00)	(0•00 to 0•00)	-0•01 (-0•04 to 0•01)	0•31
Correct medication, conditional on any prescription (0/1)	14 (18%)	(11% to 28%)	39 (49%)	(38% to 60%)	31% (17% to 45%)	<0•0001
Unnecessary medication, conditional (0/1)	37 (47%)	(37% to 58%)	36 (45%)	(35% to 56%)	-2% (-18% to 13%)	0•76
Unnecessary western medication (0/1)	13 (17%)	(10% to 26%)	9 (11%)	(6% to 20%)	-5% (-16% to 5%)	0•33
Unnecessary Chinese patent medication (0/1)	35 (45%)	(34% to 56%)	24 (30%)	(21% to 41%)	-15% (-30% to 0%)	0•053
Unnecessary herbal medication (0/1)	0 (0%)	(0% to 5%)	0 (0%)	(0% to 5%)	••	••
Harmful medication, conditional (0/1)	27 (35%)	(25% to 46%)	5 (6%)	(3% to 14%)	-28% (-40% to -17%)	<0•0001
Antibiotics prescribed (0/1)	15 (19%)	(12% to 29%)	3 (4%)	(1% to 10%)	-15% (-25% to -6%)	0•0022
Hormone prescribed (0/1)	2 (3%)	(1% to 9%)	0 (0%)	(0% to 5%)	-3% (-6% to 1%)	0•15
Analgesic prescribed (0/1)	14 (18%)	(11% to 28%)	3 (4%)	(1% to 10%)	-14% (-24% to -5%)	0•0040

Data are n (%) for binary variables and mean (SD) for continuous variables. Wilson CIs are calculated for proportions. A significance test is performed to compare the differences between the cases: continuous variables are compared using two-sample t-test and binary variables are compared using two-sample test of proportions. Two village doctors were excluded: one with missing medication data and the other clinician only prescribes traditional Chinese medicine; Correct medication refers to prescribing one of the following: 1) Antiplatelet drugs, 2) Nitrate esters, 3) Statins, 4) ß-blockers, 5) Angiotensin converting enzyme, and 6) Angiotensin converting enzyme inhibitors; A village doctor may prescribe more than one of the three potentially harmful medicines (antibiotics, hormone, or analgesic) in one prescription; No village doctors were found to prescribe both harmful and unnecessary drugs.

Examining the management components individually, doctors referred the case to upper level facilities 59% of the time, though 16% referred while also prescribing harmful or unnecessary drugs. Overall, village doctors prescribed drugs 55% of the time. Only 18% of drug prescriptions included drugs deemed appropriate for a patient with unstable angina and 35% included drugs deemed potentially harmful (antibiotics, hormones, or analgesics).

Doctors chose the preceding management strategies based on both their diagnostic and therapeutic competence. To determine how deficits in diagnostic competence or in competence of appropriate treatment of angina patients are independently contributing to mismanagement, doctors were asked directly to state how they would manage a patient with unstable angina. The results of this “treatment-only vignette” are shown in Table 4. Compared to management following the doctor's own diagnosis, rates of correct management when doctors were given the correct diagnosis were 8 percentage points higher (p-value 0•19), partially correct management was 12 percentage points higher (p-value 0•026) and incorrect management was 20 percentage points lower (p-value <0•001). This increase was not primarily due to increased rates of referral, but rather to changes in drugs prescriptions. While, conditional on prescribing any drugs, only 18% of doctors gave correct medication based on their own diagnosis, 49% gave correct medication when told the correct diagnosis of unstable angina (p-value <0•001). There was also a notable decline in prescriptions of unnecessary and harmful medication. Although the decline in the number of clinicians prescribing any unnecessary drugs was not statistically significant, there was a 15 percentage point reduction in the number of prescriptions containing unnecessary Chinese patent drugs (p-value 0•053). There was also a statistically significant reduction in the number of prescriptions containing potentially harmful medications from 35% to 6%, a 83% reduction (p-value <0•001). This reduction was driven by near elimination of prescriptions containing antibiotics (from 19% to 4%, p-value 0•002) and analgesic medications (from 18% to 4%, p-value 0•004).

Correlates of competence

We find measures of competence to be associated with a number of observed doctor and clinic characteristics. For diagnostic quality, the proportion of recommended questions and exams completed is positively correlated with doctor medical qualification, doctor income, and the number of clinicians working in the clinic; while being negatively correlated with the years of experience of the doctor and the population living in the clinic's catchment area for public health services (Table 5). Whether doctors arrived at a correct or partially correct diagnosis, however, is only associated with the annual income of the village doctor at a 5% level of significance.

Table 5

Correlates of diagnostic competence.

	Correct or Partially Correct Diagnosis (0/1)	Proportion of Recommended Items	Proportion of Essential items
Village Doctor Characteristics
Male (0/1)	0•03 (-0•15 to 0•20)	0•01 (-0•04 to 0•06)	-0•04 (-0•12 to 0•04)
Practicing experience years > median (0/1)	0•01 (-0•17 to 0•19)	-0•05 (-0•09 to -0•01)	-0•06 (-0•13 to 0•01)
Junior college degree or higher (0/1)	0•04 (-0•20 to 0•27)	0•03 (-0•02 to 0•08)	0•08 (-0•00 to 0•17)
High medical qualification (0/1)	0•10 (-0•08 to 0•29)	0•04 (-0•00 to 0•09)	0•05 (-0•02 to 0•12)
Total annual income of village doctor (log-transformed)	0•19 (0•05 to 0•34)	0•04 (0•01 to 0•07)	0•03 (-0•02 to 0•09)
Village Clinic Characteristics
Number of clinicians	0•01 (-0•04 to 0•06)	0•02 (-0•00 to 0•03)	0•02 (-0•00 to 0•04)
Annual patient volume (log-transformed)	-0•04 (-0•13 to 0•05)	-0•01 (-0•03 to 0•02)	-0•00 (-0•04 to 0•03)
Varieties of medicine in inventory (log-transformed)	0•05 (-0•05 to 0•16)	0•01 (-0•01 to 0•04)	0•02 (-0•03 to 0•07)
Total value of medical instruments (log-transformed)	-0•03 (-0•14 to 0•08)	0•01 (-0•02 to 0•04)	0•01 (-0•03 to 0•06)
Population in village clinician's catchment area for public health services (in thousands)	-0•00 (-0•06 to 0•05)	-0•03 (-0•05 to -0•01)	-0•05 (-0•07 to -0•02)
Observations	138	138	138

Data are average marginal effect (95% CI). The standard errors are clustered at the village level. Provincial fixed-effects are controlled in all regressions. Having high medical qualification is defined as having registered as a medical practitioner or an associate medical practitioner.

Following doctors’ own diagnoses, doctors with higher medical qualifications (i.e. those having medical practitioner or associate medical practitioner qualifications) were more likely to correctly manage the vignette patient by 19 percentage points (Table 6). Annual patient volume was also positively associated with correct management, while the total annual income of the doctor and the number of clinicians in the facility were negatively associated with correct management.

Table 6

Correlates of disease management.

	Correct or Partially Correct Management (0/1)	Referral (0/1)	Any Medication (0/1)	Unnecessary/Harmful Medication (0/1)
A. Under "Own" Diagnosis
Village Doctor Characteristics
Male (0/1)	-0•03 (-0•23 to 0•17)	-0•02 (-0•21 to 0•17)	-0•21 (-0•42 to -0•00)	-0•05 (-0•32 to 0•23)
Practicing experience years > median (0/1)	0•06 (-0•12 to 0•23)	0•04 (-0•14 to 0•22)	0•08 (-0•12 to 0•29)	0•06 (-0•15 to 0•28)
Junior college degree or higher (0/1)	0•09 (-0•15 to 0•33)	0•08 (-0•16 to 0•32)	0•03 (-0•21 to 0•28)	-0•06 (-0•25 to 0•12)
Higher medical qualification (0/1)	0•19 (0•02 to 0•37)	0•09 (-0•09 to 0•27)	0•01 (-0•19 to 0•21)	-0•15 (-0•31 to 0•01)
Total annual income of village doctor (log-transformed)	-0•15 (-0•29 to -0•00)	-0•12 (-0•27 to 0•04)	0•26 (0•12 to 0•40)	0•14 (-0•06 to 0•33)
Village Clinic Characteristics
Number of clinicians	-0•07 (-0•12 to -0•01)	-0•04 (-0•11 to 0•03)	0•02 (-0•06 to 0•09)	0•05 (0•00 to 0•10)
Annual patient volume (log-transformed)	0•10 (0•01 to 0•19)	0•03 (-0•10 to 0•16)	-0•10 (-0•19 to -0•01)	-0•12 (-0•24 to -0•01)
Varieties of medicine in inventory (log-transformed)	-0•01 (-0•12 to 0•10)	-0•02 (-0•14 to 0•11)	0•06 (-0•06 to 0•17)	-0•09 (-0•24 to 0•06)
Total value of medical instruments (log-transformed)	-0•05 (-0•15 to 0•04)	-0•04 (-0•15 to 0•07)	0•02 (-0•09 to 0•13)	0•09 (-0•01 to 0•19)
Population in village clinician's catchment area for public health services (in thousands)	-0•01 (-0•06 to 0•05)	-0•01 (-0•08 to 0•06)	0•05 (-0•02 to 0•13)	-0•09 (-0•18 to 0•00)
Observations	136	136	136	76
B. Under "True" Diagnosis
Village Doctor Characteristics
Male (0/1)	-0•05 (-0•24 to 0•14)	0•05 (-0•14 to 0•25)	-0•05 (-0•26 to 0•15)	-0•02 (-0•27 to 0•23)
Practicing experience years > median (0/1)	0•08 (-0•05 to 0•21)	0•13 (-0•04 to 0•30)	-0•08 (-0•28 to 0•12)	0•07 (-0•16 to 0•31)
Junior college degree or higher (0/1)	0•04 (-0•18 to 0•26)	0•01 (-0•20 to 0•22)	0•12 (-0•11 to 0•35)	0•03 (-0•19 to 0•26)
Higher medical qualification (0/1)	0•25 (0•08 to 0•42)	0•09 (-0•07 to 0•25)	-0•03 (-0•22 to 0•16)	-0•34 (-0•61 to -0•08)
Total annual income of village doctor (log-transformed)	-0•04 (-0•14 to 0•06)	-0•06 (-0•19 to 0•07)	0•05 (-0•12 to 0•22)	-0•02 (-0•18 to 0•13)
Village Clinic Characteristics
Number of clinicians	-0•04 (-0•10 to 0•01)	-0•01 (-0•11 to 0•08)	-0•01 (-0•12 to 0•09)	0•08 (-0•01 to 0•17)
Annual patient volume (log-transformed)	0•10 (0•02 to 0•18)	0•03 (-0•08 to 0•14)	0•00 (-0•10 to 0•11)	-0•03 (-0•14 to 0•07)
Varieties of medicine in inventory (log-transformed)	-0•02 (-0•10 to 0•06)	-0•07 (-0•17 to 0•04)	0•05 (-0•08 to 0•19)	-0•12 (-0•27 to 0•02)
Total value of medical instruments (log-transformed)	-0•10 (-0•17 to -0•03)	-0•02 (-0•13 to 0•08)	-0•02 (-0•13 to 0•10)	0•17 (0•04 to 0•29)
Population in village clinician's catchment area for public health services (in thousands)	0•01 (-0•03 to 0•06)	-0•01 (-0•10 to 0•08)	0•02 (-0•05 to 0•09)	-0•14 (-0•21 to -0•07)
Observations	136	136	136	76

Data are average marginal effect (95% CI). Standard errors are clustered at the village level. Provincial fixed-effects are controlled in the regressions. Having higher medical qualification is defined as having registered as a medical practitioner or an associate medical practitioner.

Although we find no correlates of referral to be statistically significant, a number of characteristics are significantly correlated with drug prescription. Male village doctors and those in clinics with higher patient volume were less likely to prescribe any drugs and those with higher incomes were more likely to prescribe. Conditional on making any prescription, unnecessary and potentially harmful medications were prescribed less often by those with higher medical qualifications, working in clinics with higher patient volumes and a larger population living in the clinic's catchment area for public health services. Unnecessary and potentially harmful drugs were more likely to be prescribed in clinics with more clinicians and those with a higher estimated total value of medical equipment.

Correlates of management outcomes after village doctors were told the true diagnosis are presented in Panel B of Table 6. Compared to management following doctors’ own diagnoses, higher medical qualification and annual patient volume at the clinic continue to be significantly associated with correct or partially correct management, while the value of medical instruments in the clinic is associated with incorrect management. No characteristics are correlated with referral or making any drug prescription. Conditional on prescribing drugs, doctors with higher medical qualifications were 34 percentage points less likely to prescribe unnecessary or potentially harmful drugs. Prescriptions containing unnecessary and harmful were more likely in clinics with more clinicians, fewer medicines in inventory, higher total value of medical equipment, and smaller public health service populations.

Discussion

Our study indicates that village doctors in rural China have limited ability to correctly diagnose, treat, and manage unstable angina. On average, doctors asked and performed 26% of the recommended questions and examinations and 39% of the questions and examinations deemed essential for correct diagnosis. Possibly due to this low adherence on diagnostic process, only 20% provided a correct diagnosis. Despite low rates of correct diagnosis, most providers indicated that they would refer patients to upper levels of care where patients are more likely to be correctly diagnosed and treated. While the proportion of cases ultimately correctly managed by doctors was higher than those correctly diagnosed our results suggest that about 40% of patients presenting to village clinics with a textbook case of unstable angina would not be referred and 35% would be prescribed potentially harmful medication.

In a sense, diagnostic ability is only important because of how it contributes to better patient management. The fact that many more doctors indicated that they would refer than are able to make a correct diagnosis suggests that many doctors are aware of their lack of diagnostic skill and compensate by referring patients. This compensation is only partial, however. Although the case of angina studied here should always be referred, this may not be needed for patients with other symptoms and over-referral of such patients imposes unnecessary costs on the health system and patients. Possibly more important, we found that resolving diagnostic uncertainty by telling doctors the correct diagnosis of unstable angina led to a statistically significant increase in correct drug prescription and reduction in the prescription of potentially harmful drugs, despite having little effect on referral rates. It is unclear why this occurs in hypothetical vignettes, but one hypothesis is that when faced with diagnostic uncertainty (due in part to deficits in diagnostic skill), clinicians have an incentive to prescribe drugs covering a range potential diagnoses. Previous research has suggested that diagnostic uncertainty may be an important driver of over-prescription of antibiotics in some settings[22,23] Our results here suggest that this incentive may lead to overtreatment more broadly. If providers face this type of incentive to over-prescribe drugs, improving doctors’ diagnostic ability may be important to avoid unnecessary and/or harmful treatment – independently of whether patients are referred and ultimately correctly diagnosed and treated.

The low level of competence that we found is consistent with findings of regional studies with comparable methodology performed in China and elsewhere [24], [25], [26]. Deficits in competence as measured with vignettes are particularly concerning as the care that patients actually receive are most likely worse. Studies have consistently shown large `know-do’ gaps, or provider underperformance relative to their competence of appropriate care [10,24,26,27]. Although know-do gaps may be due to other factors such as lack of access to equipment or medication, prior studies consistently indicate that weak or misaligned incentives facing providers is a primary cause as significant gaps exist even in well-stocked settings [22].

China's difficulty in recruiting primary providers for rural areas may contribute to low provider performance. Public sector salaries are insufficient to attract highly trained and skilled providers to serve in rural areas. More than three quarters of doctors in our sample had less than post-secondary general education and about 20% had no medical education of any kind. Although local governments have recently increased subsidies to rural providers, the compensation offered by these programs remains too low to incentivize highly-skilled doctors to locate in rural areas [28].

Strengths and limitations

To our knowledge, this study is the first nationally representative assessment of rural doctor competence and care quality in China. One limitation is that providers’ ability to diagnose and treat unstable angina may not be representative of other conditions. Second, the cost of surveying providers across a wide geographic area constrained our sample size and consequently the precision of our estimates. Third, our study assessed doctors’ competence in response to a vignette without additional assessment of doctors’ actual practice. A provider's competence as measured in vignettes acts as the upper bound on how providers may perform in treating actual patients. Providers are restricted by their competence and skills and, aside from guessing correctly, cannot perform better than what their training allows. As mentioned, previous studies have documented how the competence know-do gap contributes to the provision of low quality care in developing settings [26]. Finally, we are only able to present the quality of care provided in a representative sample of rural providers rather than a representative assessment of the quality of care received by patients. The latter requires taking into account how patients seek care from different providers and levels of care.

Policy implications

Our study affirms the importance of improving the quality of care in China's rural areas. It suggests that although China has increased healthcare utilization and expenditure in its rural regions, doctor competence is still inadequate to address the patient needs.

China could improve the quality of its rural primary care by recruiting more highly qualified providers to serve in rural areas or by improving the performance of existing providers. Policy efforts along these lines are underway [29]. Recruiting or subsidizing providers for rural areas may prove costly, however, if providers require substantial compensation to live outside of urban areas. A more feasible option may be programs requiring a period of rural service in return for public educational subsidies [30]. Interventions to improve existing providers’ competence may include training programs. A recent randomized trial in India, for instance, found that a multi-topic training for informal providers increased rates of correct case management by 8% [31]. Nevertheless, training alone is unlikely to substantially improve provider performance [32]. More effective strategies combine training with other approaches such as supervision and other management techniques [32]. As China upgrades its digital infrastructure, mobile technologies and diagnostic machines may enable access to urban providers or provide diagnostic and management support to rural doctors [33]. Our results suggest that strengthening the diagnostic ability of rural clinicians, whether through new tools, training, or recruitment, is important not just because it can increase the probability that patients are ultimately correctly treated, but because doing so may produce meaningful reductions in overtreatment with unnecessary and potentially harmful drugs. To date, the policy initiatives to reduce over-prescription, focusing primarily on addressing perverse incentives tied to drug sales, have had mixed results [8].

Conclusion

Using a nationally representative sample of village doctors, this study shows that rural providers have low levels of competence to diagnose and manage patients presenting with symptoms of angina. Although China has expanded rural primary care access, quality deficits remain. Further research is needed to identify cost-effective policy interventions to improve quality of care in rural China.

Contributors

WG, XZ and HY had full access to all of the data in the study and takes responsibility for the integrity and the accuracy of the analysis. HY and SS designed the study. HY and XZ collected the data. WG, XZ, and YC cleaned and analysed the data under the guidance of SS, KU, and HY. WG drafted the manuscript. SS, HY and WG interpreted the data. KU and SS made critical revisions on the manuscript. All authors read and authorized the final manuscript before submission.

Data sharing

A full replication package for this study, including the dataset and analysis codes, is publicly available at https://doi.org/10.7910/DVN/GICUXP.

Declaration of Competing Interest

We declare we have no competing interests.