The competence of village clinicians in the diagnosis and management of childhood epilepsy in Southwestern China and its determinants: A cross-sectional study.

BACKGROUND: Due to lack of neurologists in low- and middle-income countries, communities of patients living with epilepsy are calling for task-shifting of diagnosis and management from physicians to paramedical providers in the primary health care systems to narrow the huge treatment gap. Evidence to guide this work has been limited. This study assesses the competence of village clinicians (VC)- mostly paramedical providers- in the diagnosis and management of a presumptive case of childhood epilepsy and its determinants.
METHODS: A cross-sectional study was conducted in rural areas of a province in Southwestern China from July 2017 to January 2018. We randomly selected 370 VCs who practiced Western medicine and assumed the main responsibility of providing medical services in his/her clinic. A standardized clinical vignette based on national clinical practice guidelines was used to evaluate clinicians' competence in three domains: number and proportion of recommended (and essential) checklist (questions, examinations, and tests) completed, correctness of diagnosis, and correctness of case management.
FINDINGS: Though VCs completed 14•3% (IQR 9•5%-19•1%) of the recommended checklist, 63•2% (234/370, 95%CI 58•2%-68•0%) provided a correct diagnosis. Only 1•6% of VCs (6/370, 95%CI 0•7%-3•5%) gave correct management with both correct medication and referral, however 90•3% (334/370, 95%CI 86•8%-92•9%) provided partially correct management by referring patients to upper-level health facilities (89•5%, 331/370, 95%CI 85•9%-92•2%) or prescribing anti-epileptic drugs (AEDs) correctly (0•8%, 3/370, 95%CI 0•3%-2•4%). Around 1/4 VCs referred patients to Township Health Centers which usually were not staffed with pediatric neurologists. Fewer provided helpful medical advice to patients for daily management. The heuristic process was found to be negatively associated with the proportion of the recommended checklist that VCs completed, which is positively associated with correctness of diagnosis.
INTERPRETATION: Most VCs could diagnose and refer childhood epilepsy patients correctly; however, they lacked competence when it came to assuming the responsibility of primary care providers, referring efficiently, refilling AEDs, as well as supervising and instructing daily management of patients. FUNDING: HY received the funding for this study from the "Health and Hope Fund" of the Business Development Center of the RCSC (Beijing) and UCB (Belgium). UCB provided support in the form of a salary for author DET.

Evidence before this study

The treatment gap (TG) of epilepsy ranges from 50% to 90% in low- and middle-income countries (LMICs). One of main causes is a lack of neurologists. To address this issue, the World Health Organization (WHO) called for a task-shifting of diagnosis and treatment from physicians to paramedical providers. However, based on searches of PubMed, Web of Science, Google Scholar and CNKI for peer-reviewed studies in English and Mandarin, evidence on the competence of paramedical providers to diagnose and treat patients with epilepsy has been very limited to guide this task-shift. In rural China, the TG is as high as 63% and low competence of primary care provider is considered one of main constraints to narrow this gap.

Added value of this study

This study provides the first assessment of village clinicians’ competence in diagnosing and managing presumptive childhood epilepsy in Southwestern China. It finds that 63% of village clinicians made correct diagnoses; although, they only performed an average of 14% of the recommended checklist (questions, examination, and tests). Approximately 91% of village clinicians referred patients to upper-level health facilities. It is concerning to note that a 26% referral rate might be of low efficiency because patients were referred to Township Health Centers which were usually not staffed with pediatric neurologists. Only around 2% of village clinicians prescribed anti-epileptic drugs (AED) correctly. Fewer village clinicians provided medical advice on daily care to patients. These findings help us understand challenges in task-shifting of diagnosing and managing childhood epilepsy patients from physicians to paramedical providers in LMICs. The method developed by this study can also be used in other countries to assess competences to diagnose and manage childhood epilepsy patients. Furthermore, our study also found that the heuristic process reduced the percentage of recommended checklist completed in the diagnostic process which was statistically significantly associated with correctness of diagnosis.

Implications of all the available evidence

Our study informs policy makers of what village clinicians knew and did not know in the diagnosis and management of childhood epilepsy patients in rural Southwestern China. In particular, village clinicians require education and assistance to provide AEDs correctly and offer medical advice on daily care of childhood epilepsy patients. Policy efforts to strengthen primary care in China's rural areas are urgently recommended.

Alt-text: Unlabelled box

Introduction

Epilepsy is a disorder of the brain characterized by an enduring predisposition to generate epileptic seizures and by the neurobiological, cognitive, psychological and social consequences of this condition [1]. It accounts for 0•5% of the world's disease burden [2]. Epilepsy directly affects an estimated 50–70 million people worldwide and involves an additional 500 million people as family members and caregivers of people living with epilepsy (PwE) [3,4] PwE have an increased risk of premature death, loss of work productivity, and increased healthcare needs and expenditure [5,6]. Nearly 90% of the global burden of epilepsy occurs in the populations of low- and middle-income countries (LMICs) [7].

Seizures can be treated. About 70% of PwE achieve seizure-free lives with medical treatment [8]. However, the treatment gap (TG), defined as proportion of PwE who require treatment and do not receive it, in LMICs ranges from 50% to 90% [9]. Rural populations have TGs nearly twice as high as populations from urban locations [9]. A lack of neurologists has been identified as one of the main causes of these disparities [9,10] As a result, in LMIC, up to 91% of neurological care is provided by paramedical providers or non-physician clinicians with varying levels of education [10].

Narrowing the TG in LMICs is a great challenge. The World Health Organization (WHO) suggested a three-step model to improve quality care for PwE. The first step called for primary care to identify, diagnose PwE, and provide basic medical treatment [4]. The tasks of epilepsy diagnosis and management are expected to shift from physicians to paramedical providers in the primary health care system [4,7] The paramedical providers are numerous and can “truly decentralize” the distribution of core professional services in LMICs [11,12] Several studies in India [13], China [14], and Zambia [15] suggested that the TG could be minimized by educating paramedical workers on how to identify, diagnose and manage PwE.

Despite these studies, there has been little research done on what knowledge primary health care workers have and what should be strengthened when it comes to diagnosing and managing PwE in LMICs. A systematic review reported that the frequency of misdiagnosis of epilepsy in high-income countries range from 2% to 71% [16]. Misdiagnosis leads to mismanagement of both PwE and people without epilepsy [16]. However, evidence on misdiagnosis in LMICs is very limited, and anecdotal evidence indicates that paramedical providers sometimes used practices that are harmful to the patient [17]. The harmful treatment might be as a result of low competence, unsupportive environments, inappropriate incentives, or a combination of these from the perspective of providers. Thus, the first step to narrow the TG is to understand what health workers in LMICs know and what they don't know in diagnosis and management of PwE.

It is also important to explore the factors that are associated with misdiagnosis in clinical decision-making. Research showed that sociological and psychological factors embedded in clinicians’ reasoning play a role in the diagnostic process. One of the classic views was that misdiagnoses are likely to occur when individual clinicians invoke heuristics, because using heuristics requires less effort [18]. Heuristics are a rule of thumb that enable individuals to make judgements and decisions without resorting to any complex measurements or reasoning [19]. However, there were a number of cases that proved that simple heuristics are more accurate than standard statistical methods that have the same or more information [20]. Heuristics prove particularly helpful in situations in which the actor has little knowledge and limited capacity to acting rationally [21,22].

In rural China, VCs are usually the first contact of PwE in the health care system. China was estimated to have nearly 10 million PwE in 2015 [23]. In rural areas, the prevalence was 7•0 per 1000 population, and, in 2001, the TG was 63% [2]. China has been working to build a primary-care model in which they train local primary care providers to make an initial diagnosis, treatment and follow-up of PwE [2,14] However, a recent study found that about 35% of patients with active epilepsy were unaware of their own disease in the areas covered by the model. The gap might be explained by low competence in diagnosing epilepsy of local primary care providers [24]. China's rural primary health care system consists of three tiers. In the first tier, VCs are at the frontline and provide health care services on common clinical conditions and public health services to rural population in its catchment. Most VCs in village clinics are non-physician clinicians, or paramedical providers. The majority of rural patients were first seen in village clinics particularly in remote areas [25].

The objectives of our study were to (i) assess competence of VCs in rural Southwestern China to diagnose and manage presumptive childhood epilepsy; and, (ii) investigate the association between clinician characteristics, clinic characteristics and clinician's competence. The correlation analysis focused on testing two hypotheses. First, we tested whether the heuristic decision-maker is associated with less diagnostic efforts and subsequently poorer diagnostic process quality [26]. Second, we tested which reasoning process–heuristic and/or analytical—was statistically significantly correlated with correct diagnosis and correct case management.

Methods

Study design

A cross-sectional design was used. The data used in this study is from a concurrent randomized controlled trial [27] and collected by two rounds of visits. The first round collected basic information on VCs and clinics in July 2017. The second round was conducted in January 2018 and consisted of two parts. In the first part, a heuristic decision-making survey was administered to VCs. In the second part, a vignette was used to collect information on VCs’ competence in diagnosing and managing childhood epilepsy. By comparing VCs’ performance in vignette with the Chinese National Practice Guidelines, we assessed the competence of VCs to diagnose and manage PwE. Different factors associated with the competence were further examined.

Setting

VCs were randomly selected from counties where Han-population accounted for 80% of population in three prefectures of the Yunnan province in China. The per capita GDP in Yunnan was 5068 US dollars in 2017, 42% lower than the national level (8777 US dollars) [28]. Yunnan had a rural population of 6.5 million people in 30 counties within the three prefectures [29].

Sampling and eligibility criteria

Fig. 1 is a flowchart that shows the process of determining village clinicians. A total of 330 village clinics were randomly selected from 1320 rural clinics in 10 counties chosen in the first round (July 2017). In each clinic, one VC who practiced Western medicine and assumed the main responsibility of providing medical services was selected. If there were more than one such VC in a clinic, we randomly selected one. In the second round (January 2018), we revisited the same clinics and VCs. Five clinics were newly selected to replace the clinics that were not followed successfully. Another 40 village clinics were newly selected using the same sampling procedure as a supplement of the sample. In summary, we visited 370 village clinics and VCs in the second round of survey, yielding a final analysis sample of 370 VCs.

Fig. 1

SROBE flowchart.

Variables

The assessment of competence of VCs was carried out based on three aspects of the clinical practice: diagnostic process quality, diagnosis accuracy, and case management [30]. The assessment standards are in alignment with Chinese National Practice Guidelines [31,32]

The diagnostic process quality was measured by the number and proportion of the recommended checklist completed by VCs, and the number and proportion of the essential checklist completed by VCs. The checklist included questions that VCs should ask and examinations and laboratory tests that VCs should virtually perform. We reported it both by types and in total. The checklists by items are available in Supplementary Tables ST1 and ST2.

Diagnosis was classified by three categories: correct, partially correct, or incorrect. Correct diagnoses included tonic-clonic seizure, epilepsy, or epileptic seizure. Partially correct diagnoses included neurologic disorder, convulsion, or Chinese slang word (yangdianfeng) for seizure. Incorrect diagnoses include any other diagnosis not on the list of correct and partially correct diagnoses including misdiagnosis of the cause, or no diagnosis given.

Management was also classified into one of three categories: correct, partially correct, or incorrect. Correct management was defined as referring patients to a teaching hospital or a higher-level provider, and prescribing any of the correct anti-epilepsy drugs (AEDs) only. Partially correct management was defined as referring patients to a teaching hospital or a higher-level provider, or prescribing any of the correct AEDs only. Incorrect management included any guidance not on the list of correct or partially correct management.

The medication was classified into one of the categories: necessary (or correct), unnecessary, potentially harmful, or their combinations. Necessary drugs included valproic acid, carbamazepine, phenobarbital, phenytoin, or benzodiazepine that are available in primary health care system. Potentially harmful drugs included drugs that may result in seizures. Unnecessary drugs included medicines not mentioned in correct or potentially harmful category, e.g., sedatives and flunarizine.

We considered clinic-level factors and clinician-level factors in the statistical analyzes. The clinic-level factors include number of doctors, outpatient visits in the previous month, whether the clinics medical instrument index ranked top 1/5, distance to Township Health Center (THC), and whether it is in a village with clinically diagnosed childhood epilepsy patients. The clinician-level factors are gender, age, non-Han ethnicity, junior college or higher education, physician (medical practitioner or associate medical practitioner), receiving any trainings on epilepsy in the previous two years, familiarity with terminology of clinical pathway, and heuristic decision-maker. According to Shah and Oppenheimer [26], examining fewer alternatives is a way to reduce efforts by heuristic decision-makers. Thus, the variable of a heuristic decision-maker is defined to be equal to 1 if a VC gave only one diagnosis about the presented symptom in the heuristic decision-making survey. Otherwise, it will be equal to 0.

Surveys and vignette

Village clinician and clinic surveys

We administered separate surveys to VCs and facilities in July 2017. The VC survey collected VC's age, gender, ethnicity, educational level, qualification(s), training experience with epilepsy, and familiarity with clinical pathway terminology. Facility surveys collected information on the number of doctors employed, patient volume, a checklist of medical instruments, distance to THC, and information on whether any childhood epilepsy patients were diagnosed in the village where the clinic was located. The same survey was completed by newly selected clinics and VCs in January 2018. Survey forms were presented in Supplementary Forms SF1-SF3.

Heuristic decision-making survey

In the second round of visits, we collected information on VCs’ heuristic decision-making about signs and symptoms of potential childhood epilepsy right before the vignette. Specifically, the enumerator presented a hypothetical case, asking the VC to assume she/he was being visited by a 35-year-old father with his 11-year-old son. The father told the clinician that his son fell down on the ground and was twitching convulsively last night. Then, the VC was asked to report her/his diagnosis based on the symptoms, including what types of disease she/he perceived this to be and the likelihood of each type of disease. No further information on the patient and disease was provided to the clinician at this stage. The heuristic decision-making survey form was presented in Supplementary Forms SF4.

Standardized clinical vignettes

We used a standardized clinical vignette to measure VCs’ competence to correctly diagnose and manage a patient presenting a presumptive case of childhood epilepsy. With the assistance of pediatric neurologists (ZW, YH, JQ) and local clinicians, we developed a culturally acceptable vignette script. This script included disease symptoms, medical history, and patient background. It was presented in Supplementary Forms SF5.

At the end of the visit of January 2018, two enumerators in each team presented the vignette to VCs. a total of 32 enumerators (16 teams) had been trained in classroom and in the field for seven days before the survey to present the standardized clinical vignette to VCs. One enumerator in a team assumed the role of a “mock patient” and the other assumed the role of “facilitator” and read the instructions to the clinician, documented the interaction, and provided additional information that the patient might not know but would be helpful to the clinician and his/her diagnosis if he/she actively solicited it, e.g., the results of tests or examinations. To begin the vignette, the facilitator informed the VC that a 35-year-old man was visiting the clinic with his 11-year-old. The “mock patient” (in our case, the father of the “mock patient”) then told the clinician about his son's problem with an opening statement - “Doctor, my son fell down to the ground last night and was twitching convulsively.” The VC was then asked to proceed as if with a real patient and were told that the patient would answer any questions asked and comply with any instructions given. If the VC did not provide any diagnosis, the “mock patient” was instructed to ask for a diagnosis at the end of the interaction. During the interaction, the facilitator documented the VC's questions, diagnostic examination and tests requested, stated diagnosis, treatment prescribed (drugs or advices), and potential patient referral(s).

Study size

A total of 370 village clinicians were enrolled in this study. The sample size was determined by a concurrent randomized control trial. The information on the study is accessible at https://www.isrctn.com/ISRCTN14264710.

Statistical methods

The data was collected using Survey Monkey by enumerators in the field, and cleaned using Stata 15.1 by the research team in the office. All analyzes were done using Stata 15.1. The median and interquartile range (IQR, the 25th and 75th percentile values) of the continuous variables, numbers and 95% Wilson CI of the binary variables were calculated. We used linear univariate regressions for continuous dependent variables and logistical univariate regressions for binary dependent variables to compare the differences between physicians and non-physicians in basic characteristics and outcomes (Supplementary Table ST3).

We used multivariate regressions to examine (i) association between proportion of recommended checklist completed and heuristic decision-maker; controlling for clinician-level factors and clinic-level factors; (ii) association between correct or partially correct diagnosis and heuristic decision-maker, as well as proportion of recommended checklist completed; controlling for clinic- and clinician-level factors; and (iii) association between correct or partially correct management and heuristic decision-maker, proportion of recommended checklist completed, and correct or partially correct diagnosis controlling for clinic- and clinician-level factors. A linear multivariate regression was used to estimate the association with continuous outcome, and we used a logistic multivariate regression to estimate the association with binary outcome. We also controlled for county fixed effects. We did not take the experimental design into account although the data were from a randomized controlled trial because no statistically significant effect of the intervention was found on the primary outcomes (Supplementary Table ST4). The standard errors were adjusted at county level. The marginal effect and 95% CI were reported in the tables for both univariate and multivariate regressions. The marginal effect in the regression of proportion of recommended checklist completed can be interpreted to mean that the change in the outcome is related to the change in a covariate. The marginal effect in the regressions of correct diagnosis or management can be interpreted as the predicted probability of the outcome changes with a change in a covariate.

Ethics

Full ethical approval for this survey was obtained from the Peking University Institutional Review Board on April 26, 2017 (IRB00001052-17033). The Board approved the verbal consent procedure and verbal consent was obtained from local health departments and participants at the start of the survey. We collected verbal consents for three main reasons: the research presented minimal risk of harm to subjects, involved no procedures for which written consent was normally required outside of the research context, and verbal consents were more culturally acceptable than written consents in the region.

Role of the funding source

HY received funding for this study from the “Health and Hope Fund” of the Business Development Center of the RCSC (Beijing) and UCB (Belgium). UCB provided support in the form of a salary for author DET. Neither UCB nor the RCSC had any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Results

Characteristics of village clinicians and clinics

Table 1 presents the summary statistics of village clinic and VCs. Out of 370 VCs, 235(63•5%, 95% CI 58•4% to 68•3%) were male, 39 (10•5%, 95% CI 7•8% to14•1%) were non-Han ethnicity, and the median of age was 44 years old (IQR 38 to 50). The education level and medical background of the VCs was low. Specifically, only 26•2% (97/370, 95% CI 22•0% to 30•9%) completed junior college or higher education. Only 17•8% (66/370, 95% CI 14•3% to 22•1%) were aware of the terminology of clinical pathway, and 18•4% (68/370, 95% CI 14•8% to 22•6%) had received certain extent training on epilepsy in the past two years. In the heuristic decision-making survey, 77 VCs (20•8%) gave one diagnosis, 245 VCs (66•2%) gave two diagnoses, 40 VCs (10•8%) gave three diagnoses, and eight VCs (2•2%) gave four or more diseases with certain possibility. That is, 77 VCs (20•8%, 95% CI 17•0% to 25•2%) were heuristic decision-makers.

Table 1

The characteristics of village clinicians and clinics.

	Total		Physicians c		Non-physicians
Village clinician characteristics
Number of village clinicians	370		29		341
Binary variables	n (%)	95% CI	n (%)	95% CI	n(%)	95% CI
1. Male, 1=yes	235 (63•5)	(58•4, 68•3)	16 (55•2)	(37•5, 71•6)	219(64•2)	(59•0, 69•1)
2. Non-Han ethnicity, 1=yes	39 (10•5)	(7•8, 14•1)	2 (6•9)	(1•9, 22•0)	37(10•8)	(8•0, 14•6)
3. Junior college or higher education a, 1=yes	97 (26•2)	(22•0, 30•9)	15 (51•7)	(34•4, 68•6)	82(24•0)	(19•8, 28•9)
4. Receiving any trainings on epilepsy in the previous two years, 1=yes	68 (18•4)	(14•8, 22•6)	6 (20•7)	(9•8, 38•4)	62(18•2)	(14•5, 22•6)
5. Familiarity with terminology of clinical pathway, 1=yes	66 (17•8)	(14•3, 22•1)	9 (31•0)	(17•3, 49•2)	57(16•7)	(13•1, 21•0)
6. Heuristic decision-maker, 1=yes	77 (20•8)	(17•0, 25•2)	3 (10•3)	(3•6, 26•4)	74(21•7)	(17•7, 26•4)
Continuous variables	Median	IQRb	Median	IQR	Median	IQR
7. Age, year	44	(38, 50)	41	(39, 47)	44	(38, 50)
Clinic characteristics
Number of clinics	370		29		341
Binary variables	n (%)	95% CI	n (%)	95% CI	n(%)	95% CI
8. Top 1/5 of index of medical instruments, 1=yes	74 (20•0)	(16•2, 24•4)	8 (27•6)	(14•7, 45•7)	66(19•4)	(15•5, 23•9)
9. Village with clinically diagnosed childhood epilepsy patient, 1=yes	81 (22•0)	(18•0, 26•5)	4 (13•8)	(5•5, 30•6)	77(22•6)	(18•5, 27•4)
Continuous variables	Median	IQR	Median	IQR	Median	IQR
10. Number of doctors in each clinic, person	3	(2, 3)	3	(2, 4)	3	(2, 3)
11. Out-patient visits in the previous month, encounter	300	(100, 600)	346	(130, 850)	287	(90, 583)
Distance to Township Health Center, km	10	(5, 17)	5	(3, 10)	10	(6, 17)

It includes correspondence education.

IQR(interquartile range) is a measure of statistical dispersion and the numbers in the parenthesis are the values of 25th and 75th percentiles respectively.

Physicians include village clinicians having “Medical Practitioners” qualification or “Associate Medical Practitioners” qualification. Non-Physicians include village clinicians having “Village Doctor” qualification, no formal qualification, “Nurse” qualification, or “Pharmacist” qualification.

The median of the number of VCs in each clinic was three (IQR 2 to 3) persons, the median of the distance to THC was 10 (IQR 5 to 17) kilometres, and 22•0% (81/369, 95%CI 18•0% to 26•5%) of clinics were in villages with clinically diagnosed childhood epilepsy patients. In the month before the survey, the median of the out-patient visits in each clinic was 300 (IQR 100 to 600).

Among 370 VCs, only 7•8% (29/370) had the highest “medical practitioner” qualification or the next highest “associate medical practitioner” qualification (physicians hereafter), the rest (non-physicians hereafter) had only “Village Doctor” qualification (82•3%, 307/370), no formal qualification (5•9%, 22/370), “Nurse” or “Pharmacist” qualifications (3•2%, 12/370). When we look at the characteristics by VCs’ qualification, we find that 51•7% (15/29, 95% CI 34•4% to 68•6%) of physicians had junior college or higher-level education in comparison with 24•0% (82/341, 95%CI 19•8% to 28•9%) among non-physicians. As showed in Supplementary Table ST3, the difference between physicians and non-physicians in education was statistically significant (95% CI 0•08 to 0•46).

Village clinician competence

Diagnostic process quality

VCs completed three (IQR 2 to 4) recommended items in a total of 21 items on the checklist, which accounted for 14•3% (IQR 9•5% to 19•1%) of the recommended checklists (Table 2). From this checklist, VCs asked an average of two (IQR 1 to 3) out of 15 recommended questions, which accounts for 13•3% (IQR 6•7% to 20•0%) of recommended questions. They performed a median of zero (IQR 0 to 1) out of six recommended examinations and tests, which accounts for 0•0% (IQR 0•0% to 16•7%) of recommended examinations and tests.

Table 2

Quality of village clinicians’ diagnostic process of childhood epilepsy a.

	Total		Physicians		Non-physicians
	Median	IQR	Median	IQR	Median	IQR
Number of village clinicians	370		29		341
Recommended checklists (questions, examinations, and tests) (21 in total)
1. Number of recommended checklists completed	3	(2, 4)	4	(2, 6)	3	(1, 4)
2. Proportion of recommended checklists completed, %	14•3	(9•5, 19•1)	19•1	(9•5, 28•6)	14•3	(4•8, 19•1)
Essential checklists (questions, examinations, and tests) (9 in total)
3. Number of essential checklists completed	1	(1, 2)	2	(1, 3)	1	(1, 2)
4. Proportion of essential checklists completed, %	11•1	(11•1, 22•2)	22•2	(11•1, 33•3)	11•1	(11•1, 22•2)
Recommended questions (15 in total)
5. Number of recommended questions asked	2	(1, 3)	3	(2, 4)	2	(1, 3)
6. Proportion of recommended questions asked, %	13•3	(6•7, 20•0)	20•0	(13•3, 26•7)	13•3	(6•7, 20•0)
Essential questions (7 in total)
7. Number of essential questions asked	1	(0, 2)	1	(1, 3)	1	(0, 2)
8. Proportion of essential questions asked, %	14•3	(0•0, 28•6)	14•3	(14•3, 42•9)	14•3	(0•0, 28•6)
Recommended examinations and tests (6 in total)
9. Number of recommended examinations and tests performed	0	(0, 1)	1	(0, 2)	0	(0, 1)
10. Proportion of recommended examinations and tests performed, %	0•0	(0•0, 16•7)	16•7	(0•0, 33•3)	0•0	(0•0, 16•7)
Essential examinations and tests (2 in total)
11. Number of essential examinations and tests performed	0	(0, 0)	0	(0, 1)	0	(0, 0)
12. Proportion of essential examinations and tests performed, %	0•0	(0•0, 0•0)	0•0	(0•0, 50•0)	0•0	(0•0, 0•0)

The checklist of recommended and essential questions is listed in Supplementary Table ST1, the checklist of recommended and essential examinations, and tests is listed in Supplementary Table ST2.

VCs completed 11•1% (IQR 11•1% to 22•2%) of a smaller subset of the essential checklist with 14•3% (IQR 0•0% to 28•6%) of the essential questions and 0.0% (IQR 0•0% to 0•0%) of essential examinations and tests.

The physicians performed better than non-physicians in 10 of 12 outcome variables on diagnostic process except number of essential questions asked (95% CI -0•04 to 0•78) and proportion of essential questions asked (95% CI -0•52 to 11•13) (Supplementary Table ST4).

Diagnosis

Nearly two thirds of VCs made correct or partially correct diagnosis. As seen in Table 3, 63•2% (234/370, 95% CI 58•2% to 68•0%) made correct diagnosis, 4•1% (15/370, 95% CI 2•5% to 6•6%) made partially correct diagnosis, and 32•7% (121/370, 95% CI 28•1% to 37•6%) provided a wrong diagnosis. The correctness of diagnosis was not statistically significantly different between physicians and non-physicians (Supplementary Table ST4).

Table 3

Quality of village clinicians’ diagnosis of childhood epilepsy.

	Total		Physicians		Non-physicians
	n (%)	95% CI	n (%)	95% CI	n (%)	95% CI
Number of village clinicians	370		29		341
1. Correct diagnosis a, 1=yes	234 (63•2)	(58•2, 68•0)	22 (75•9)	(57•9, 87•8)	212 (62•2)	(56•9, 67•2)
2. Partially correct diagnosis b, 1=yes	15 (4•1)	(2•5, 6•6)	0 (0•0)	(0•0, 11•7)	15 (4•4)	(2•7, 7•1)
3. Incorrect diagnosis c, 1=yes	121 (32•7)	(28•1, 37•6)	7 (24•1)	(12•2, 42•1)	114 (33•4)	(28•6, 38•6)

Correct diagnosis includes tonic-clonic seizure, epilepsy, or epileptic seizure.

Partially correct diagnosis includes neurologic disorder, convulsion, or Chinese slang word synonym (yangdianfeng) for seizure.

Incorrect diagnosis includes hysteria, psychological problems (disorder), misdiagnosis of the cause such as rickets caused by calcium deficiency, any other diagnosis not on the list of correct and partially correct diagnoses, or no diagnosis given.

Case management

Although the proportion of correct diagnosis was relatively high, the number of VCs recommending correct management was not as high (Table 4). Specifically, 1•6% (6/370, 95%CI 0•7% to 3•5%) of VCs provided correct management to PwE, and 90•3% (334/370, 95% CI 86•8% to 92•9%) provided a partially correct management. Of those who provided a partially correct management, 0•8% (3/334, 95% CI 0•3% to 2•4%) correctly prescribed AEDs and 89•5% (331/334, 95% CI 85•9% to 92•2%) referred PwE to upper-level health facilities. Of those referrals, 74•0% (245/331) referred to county or upper-level health facilities, and 26•0% (86/331) referred to THCs. Only 8•1% of VCs (30/370, 95% CI 5•7% to 11•3%) provided incorrect management.

Table 4

Quality of village clinicians’ case management of childhood epilepsy.

	Total		Physicians		Non-physicians
	n (%)	95% CI	n (%)	95% CI	n (%)	95% CI
Management a
Number of village clinicians	370		29		341
1. Correct management a, 1=yes	6 (1•6)	(0•7, 3•5)	1 (3•4)	(0•6, 17•2)	5 (1•5)	(0•6, 3•4)
2. Partially correct management b, 1=yes	334 (90•3)	(86•8, 92•9)	27 (93•1)	(78•0, 98•1)	307 (90•0)	(86•4, 92•8)
3. # Correct medication only, 1=yes	3 (0•8)	(0•3, 2•4)	0 (0•0)	(0•0, 11•7)	3 (0•9)	(0•3, 2•6)
4. # Referral only, 1=yes	331 (89•5)	(85•9, 92•2)	27 (93•1)	(78•0, 98•1)	304 (89•1)	(85•4, 92•0)
5. Incorrect management c, 1=yes	30 (8•1)	(5•7, 11•3)	1 (3•4)	(0•6, 17•2)	29 (8•5)	(6•0, 11•9)
Medication
6. Giving any drug, 1=yes	46 (12•4)	(9•5, 16•2)	2 (6•9)	(1•9, 22•0)	44 (12•9)	(9•8, 16•9)
Conditional on giving any drugs
Number of village clinicians	46		2		44
7. # Prescribed correct drug only d, 1=yes	8 (17•4)	(9•1, 30•7)	1 (50•0)	(9•5, 90•5)	7 (15•9)	(7•9, 29•4)
8. # Prescribed unnecessary drug only e, 1=yes	32 (69•6)	(55•2, 80•9)	1 (50•0)	(9•5, 90•5)	31 (70•5)	(55•8, 81•8)
9. # Prescribed potentially harmful drug only f, 1=yes	4 (8•7)	(3•4, 20•3)	0 (0•0)	(0•0, 65•8)	4 (9•1)	(3•6, 21•2)
10. # Prescribed both correct and unnecessary drugs together, 1=yes	1 (2•2)	(0•4, 11•3)	0 (0•0)	(0•0, 65•8)	1 (2•3)	(0•4, 11•8)
11. # Prescribed both correct and potentially harmful drugs together, 1=yes	0 (0•0)	(0•0, 7•7)	0 (0•0)	(0•0, 65•8)	0 (0•0)	(0•0, 8•0)
12. # Prescribed both unnecessary and potential harmful drugs together, 1=yes	1(2•2)	(0•4, 11•3)	0(0•0)	(0•0, 65•8)	1(2•3)	(0•4, 11•8)
13. # Prescribed correct, unnecessary, and harmful drug together, 1=yes	0(0•0)	(0•0, 7•7)	0(0•0)	(0•0, 65•8)	0(0•0)	(0•0, 8•0)

Correct management was defined as referring patient to a teaching hospital or a higher-level provider, and prescribed any of the correct anti-epilepsy drugs (AEDs) only.

Partially correct management was defined as referring patient to a teaching hospital or a higher-level provider, or prescribed one of the correct AEDs only.

Incorrect management included any guidance not on the list of correct or partially correct management.

Correct drugs included valproic acid, carbamazepine, phenobarbital, phenytoin, and benzodiazepine.

Unnecessary drugs included medicines that not mentioned in correct or potentially harmful category, e.g., sedatives and flunarizine.

Potentially harmful drugs included drugs that may result in seizures.

Of 370 VCs, 46 (12•4%, 95% CI 9•5% to 16•2%) prescribed AEDs. In terms of prescription accuracy, 17•4% (8/46, 95% CI 9•1% to 30•7%) of prescriptions included correct drugs only, 69•6% (32/46, 95%CI 55•2% to 80•9%) included unnecessary drugs only, 8•7% (4/46, 95% CI 3•4% to 20•3%) included only potentially harmful drugs, 2•2% (1/46, 95% CI 0•4%-11•3%) included both correct drugs and unnecessary drugs, and 2.2% (1/46, 95% CI 0•4% to 11•3%) included both unnecessary and potentially harmful drugs. Similarly, no statistically significant differences were found in management between physicians and non-physicians (Supplementary Table ST4).

Further analysis on the medical advice also indicates a limited capability of VCs in the management of childhood epilepsy patients (Table 5). Of the seven medical practices that are helpful with daily management of childhood epilepsy patients, only about 2% of VCs mentioned even one practice. There is no statistically significant difference between physicians and non-physicians (Supplementary Table ST4).

Table 5

Village clinicians' medical advices to patients with childhood epilepsy.

	Total		Physicians		Non-physicians
	n (%)	95% CI	n (%)	95% CI	n (%)	95% CI
Number of clinicians	370		29		341
1. Counseling the specialist regularly, 1=yes	1 (0•3)	(0•0, 1•5)	0 (0•0)	(0•0, 11•7)	1 (0•3)	(0•1, 1•6)
2. Giving advices on taking drugs regularly, 1=yes	2 (0•5)	(0•1, 1•9)	0 (0•0)	(0•0, 11•7)	2 (0•6)	(0•2, 2•1)
3. Pay attention to daily routines, 1=yes	6 (1•6)	(0•7, 3•5)	1 (3•4)	(0•6, 17•2)	5 (1•5)	(0•6, 3•4)
4. Giving advices on mental health care, 1=yes	2 (0•5)	(0•1, 1•9)	1 (3•4)	(0•6, 17•2)	1 (0•3)	(0•1, 1•6)
5. Giving advices on caring for children to caregivers, 1=yes	7 (1•9)	(0•9, 3•9)	1 (3•4)	(0•6, 17•2)	6 (1•8)	(0•8, 3•8)
6. Suggesting restrictions on some risky exercises, 1=yes	2 (0•5)	(0•1, 1•9)	0 (0•0)	(0•0, 11•7)	2 (0•6)	(0•2, 2•1)
7. Giving advices on children's safety such as avoiding falling, 1=yes	5 (1•4)	(0•6, 3•1)	1 (3•4)	(0•6, 17•2)	4 (1•2)	(0•5, 3•0)

Factors associated with VC competence

Table 6 provides the results of multivariate analysis showing that a heuristic decision-maker is significantly associated with reduced proportion of recommended checklist completed by 3•77 (95% CI -5•94 to -1•60) percentage points. The results also suggest that a one year increase in age is significantly associated with a 0•06 (95% CI -0•14 to 0•01) percentage points decrease in the proportion of recommended checklist questions completed.

Table 6

Factors that are associated with village clinicians’ competence a.

Variables	Proportion of recommended checklist completed	Correct or partially correct diagnosis	Correct or partially correct management
1. Heuristic decision-maker, 1=yes	-3•77	0•03	-0•02
	(-5•939, -1•601)	(-0•066, 0•120)	(-0•073, 0•028)
2. Proportion of recommended checklist completed		0•01	0•00
		(0•001, 0•013)	(-0•001, 0•003)
3. Correct or partially correct diagnosis, 1=yes			0•09
			(0•050, 0•133)
Village clinician characteristics
4. Male, 1=yes	0•08	0•01	-0•04
	(-1•907, 2•065)	(-0•088, 0•115)	(-0•058 - -0•019)
5. Minority, 1=yes	0•24	-0•07	-0•05
	(-3•177, 3•660)	(-0•213, 0•076)	(-0•139, 0•033)
6. Age, year	-0•06	-0•00	-0•00
	(-0•135, 0•009)	(-0•006, 0•003)	(-0•003, 0•001)
7. Junior college or higher education, 1=yes	2•15	0•12	-0•03
	(-1•945, 6•243)	(0•043, 0•204)	(-0•084, 0•024)
8. Physicians, 1=yes	5•21	0•02	0•02
	(-1•338, 11•749)	(-0•218, 0•253)	(-0•027, 0•067)
9. Receiving any trainings on epilepsy in the previous two years, 1=yes	-1•22	0•04	0•02
	(-4•042, 1•608)	(-0•047, 0•118)	(-0•015, 0•053)
10. Familiarity with the terminology of clinical pathway, 1=yes	-0•67	0•09	0•03
	(-4•927, 3•585)	(0•001, 0•175)	(0•006, 0•059)
Clinic characteristics
11. Number of doctors in the clinic, person	0•38	0•02	0•01
	(-0•906, 1•675)	(-0•053, 0•083)	(-0•006, 0•026)
12. Log form of outpatient visits in the previous month, encounter	0•11	0•01	-0•01
	(-0•484, 0•703)	(-0•018, 0•043)	(-0•015, 0•003)
13. Top 1/5 of index of medical instruments, 1=yes	1•09	0•07	-0•01
	(-1•454, 3•638)	(-0•066, 0•198)	(-0•071, 0•042)
14. Distance to Township Health Center, km	-0•08	0•00	0•00
	(-0•194, 0•044)	(-0•004, 0•005)	(0•000, 0•003)
15. Village with clinically diagnosed childhood epilepsy patient, 1=yes	1•33	0•04	-0•05
	(-1•073, 3•734)	(-0•084, 0•162)	(-0•126, 0•017)
County fixed effect	YES	YES	YES
Constant	13•94	NA	NA
	(9•552, 18•333)
Number of observations	369	369	369
R-squared/Pseudo R-squared a	0•164	0•061	0•214

Marginal effect and 95% CI were reported.

However, heuristic decision-making was not statistically significantly associated with correct or partially correct diagnosis. Instead, the proportion of recommended checklist completed was significantly associated with the probability of correct or partially correct diagnosis although the marginal effect size was small (0•01, 95% CI 0•001 to 0•013). VCs with junior college or higher-level education would be more likely to give correct or partially correct diagnosis by 12 (95% CI: 4•3–20•4) percentage points. VCs who were familiar with the terminology of the clinical pathway would be more likely to give correct or partially correct diagnosis by 9 (95% CI: 0•1–17•5) percentage points.

Neither heuristic decision-making nor proportion of recommended checklist completed was found to be statistically significantly associated with correct or partially correct disease management. However, correct or partially correct diagnosis is associated with an increase of 9 (95% CI 5•0 to 13•3) percentage points in correct or partially correct management. Male clinicians were 4 (95% CI -5•8 to -1•9) percentage points less likely to give correct or partially correct management compared to female clinicians. VCs who are familiar with the terminology of clinical pathway were 3 (95% CI: 0•6–5•9) percentage points more likely to give correct or partially correct management than those who are not. Although the coefficient of distance to THC was statistically significant, the effect size was negligible (< 1 percentage points).

Discussion

This study presented four findings on the VCs’ competence in the diagnosis and management of childhood epilepsy patients. First, the quality of the diagnostic process was poor among VCs in rural Southwestern China. Second, VCs performed well in giving correct or partially correct diagnosis. Third, although few VCs managed the case correctly, a large percentage managed the case partially correctly by referring the patients to upper-level health facilities. It is noticeable that around 1/4 of patients were referred to THCs where pediatric neurologists are usually not available. It remains unclear whether these PwE would be adequately followed-up on. Finally, few VCs provided prescriptions to treat patients or gave useful medical advice to patients. Worst of all, when a prescription was given, the quality of prescription was low. That implies VCs lack the skills to satisfactorily manage children living with epilepsy.

VCs limited competence in prescription and medical instruction might undermine their role in primary health care system in rural China. China has initiated the family doctor program since 2016 and aimed to cover the population by 2020. VCs would assume the responsibility of family doctors in rural areas. As that in other LMICs, they are expected to refer PwE in their catchment to hospitals with pediatric neurologists, provide a correct drugs refill when patients have been diagnosed, supervise and assist daily management. Our study indicates that there is a long way to go, in particular, in rational drug use and instructions in daily management.

In the correlation analysis, although we find that heuristics reduced VC's efforts to collect information for analytical reasoning, they did not cause more diagnostic errors or management errors. This challenges the classic view that cognitive heuristics imply greater errors in diagnosis [18]. Nevertheless, there is also no evidence to indicate that cognitive heuristics could function well in rural primary care systems—a context where fast and frugal heuristics can be used successfully by recent studies [20]. The possible reason might be that we did not account for whether VCs diagnoses made by the heuristic process were correct or not in this study. Around 75% (278/370) of VCs gave correct or partially correct diagnoses through the heuristic process. Among them, 78% (219/278) also gave correct or partially correct diagnosis in the vignette. Fig. 2 showed that VCs who made correct or partially correct diagnoses by the heuristic process were more likely to make correct or partially correct diagnosis in the vignette than those who did not. This is particularly true when clinicians completed less than five items on the recommended checklist. This result means that heuristics could improve VCs’ clinical decision-making only when the process was based on correct experience or practices.

Fig. 2

Percent of correct or partially correct diagnosis in vignette by number of recommended checklist completed and correctness of diagnoses made by heuristic process.

We also find that analytical reasoning effectively improved the correctness of diagnosis. In particular, Fig. 2 shows that when VCs asked more than 5 items on the recommended checklist, even VCs who did not make correct or partially correct diagnosis by heuristic decision process, their diagnosis in the vignette was not significantly different from those who made correct or partially correct diagnosis by heuristic decision process (84•9% vs. 66•7%, p = 0•2631).

The associations between the factors that represent the professional literacy of VCs and VCs’ competence are mixed. We did not find a significant association between VCs competence and their qualification. However, the education level of VCs was found to be significantly associated with their competence in diagnosis. In particular, VCs familiarity of the terminology of clinical pathway was associated with their competence in both diagnosis and management. However, the statistically insignificant role of training experiences on the diagnosis and management of epilepsy suggests that the content of the training might not be tailored to the needs of VCs, or VCs did not effectively practice and apply the training. In addition, the statistically insignificant association between working in a village with the presence of childhood epilepsy patients and clinician's competence indicates that these VCs did not learn from the ongoing treatment options of these patients.

Our study is the first study to use vignettes to assess paramedical providers’ competence in diagnosing and managing childhood epilepsy patient in LMICs. It provides evidence of the challenges in task-shifting of diagnosing and managing PwE from physicians to paramedical providers. The method developed by this study is available and can also be used in other countries to assess health care workers’ ability to diagnose and manage PwE. In addition, our findings contribute to the current debate on the role of heuristics in clinical decision-making.

Our study has several limitations. First, we want to make clear that this study cannot represent the overall quality of primary care provided by VCs. It only measures the competence of village clinicians to diagnose and manage a specific disease i.e. childhood epilepsy. Meanwhile, the study sample was not nationally representative although we took a rigorous sampling strategy. Second, even for childhood epilepsy, this study only measures VCs’ competence and may provide insights on some of contributing factors of the TG. Indeed, in addition to VCs’ competence, the TG is also determined by incentives, supporting resources etc. Finally, the relationship between outcomes and factors may not be causal although a number of confounding factors in this relationship were controlled for.

Conclusion

The results of this study are encouraging. Many paramedical providers in the primary care system could make roughly accurate (correct or partially correct) diagnoses of childhood epilepsy, and most referred patients to upper-level health facilities in rural Southwestern China. However, a more fundamental concern is of limited competence of VCs in the treatment and daily life management of childhood epilepsy patients. The lack of physicians and neurologists in remote areas will be a long-term problem across the world. Thus, in order to narrow the TG, it is important to educate and assist paramedical providers in the primary care system to treat and follow up on their patients correctly. There are three ways to go. First, as suggested by Li et al. [33], China could consider tailoring continuing training for the primary health-care workforce through online platforms. Second, a telemedicine system based on clinical pathway could assist village clinicians to prescribe appropriate AED refills on a regular basis and provide correct advice on daily care of PwE. There is growing evidence that telemedicine might be the promising way to do so in the future [34,35] Finally, China should educate and encourage more young clinicians with higher education to work in rural primary care system in order to replace aging VCs.

Contributors

HY had full access to all study data and had final responsibility for the decision to submit for publication. HY, SS, and DET designed the study. HY, SS, and HX developed the survey forms, ZW, JQ, YH, HY, HX, and HL developed the assessment tools of knowledge. HY and HX collected the data. HX and HL cleaned the data. HY analysed the data, drafted the manuscript, and interpreted the data. ZW, JQ, HS and DET made critical revisions on the manuscript. All authors read and approved the final manuscript.

Data sharing statement

Replication files are available on the Harvard Dataverse: https://doi.org/10.7910/DVN/PQSZR0.

Declaration of Interests

DET is employed by and received a salary from UCB. This does not alter our adherence to the Lancet Regional Health policies on sharing data and materials. There are no patents, products in development or marketed products to declare.