Development and validation of a questionnaire to evaluate satisfaction of the patient with diabetes at the primary care level.

Aims: To develop and validate a new instrument to measure satisfaction with integral care (doctor-nurse) of the patient with type 2 diabetes mellitus, considering expectations-experiences together for the primary level of care.
Methods: The instrument was constructed with questions regarding integral care to measure the satisfaction of the diabetes patient and was classified into four domains. The validity of the content was done through a panel of experts, apparent validity through a focus group, the validity of the construct through analysis of the main components and confirmatory factorial analysis, instrument reliability with internal consistency, determined by Cronbach alpha and temporal stability (test-retest).
Results: The reliability of the questionnaire was 0.942. The intraclass correlation coefficient was 0.849. Validity of the construct showed acceptable goodness-of-fit and factorial structure with four factors: communication, empathy, technical care, care continuity, and 24 items for each domain, giving a Kayser-Meyer-Olkin index above 0.80 and a total variance above 73. Conclusions: The instrument is reliable and is also valid in terms of up into construct and content to evaluate satisfaction. Practice Implications: In addition, these results allow to have elements for the design of strategies aimed at improving the relationship of health personnel with the patient. Copyright:

Introduction

Satisfaction is an important outcome of health care, and its assessment has been encouraged.[12] Talking about satisfaction is important since it is an essential indicator of the quality of care where users’ perspectives can express their experiences and comfort their expectations.[34]

With this they can solve many of the deficiencies in care, especially in the first level of care; since the malfunction of this has a direct impact on the quality of the other levels of care[567] and is a key point to obtain positive benefits for the population.[89] Satisfied patients are more likely to follow a plan of care and make better use of health services. Satisfaction in type 2 diabetic patients has been associated with good blood glucose results, mainly due to HbA1c.[1011]

A recently proposed instrument that measures satisfaction with health care (i.e., Helping Alliance Questionnaire or HCSQ), with adequate theoretical support, but lacks a qualitative approach that may minimize the gap between real experience and evaluated reality, in spite of being aimed at the elderly, restraining a general approach.[12] The HCQS[131415] evaluates just the doctor–patient relationship, from the perspective of the patient in primary care, but does not include the healthcare team approach.

Several assessment tools for patient satisfaction that have been developed, are enriched by their psychometric properties,[16] but they are mainly useful in hospital settings, rather than in primary care units,[17] and few instruments evaluate the satisfaction of patients with type 2 diabetes.[1819]

Since most patients with type 2 diabetes are being treated and cared for by primary care providers, the authors have considered that it is important to develop and validate an instrument that allows measuring satisfaction with integral care of the patient with type 2 diabetes in the primary care setting, including evaluation by the health care staff, as primary care physicians and nurses, considering altogether the expectations-experience dimensions.

Methods

Aim

To develop and validate a new instrument to measure satisfaction with integral care (doctor-nurse) of the patient with type 2 diabetes mellitus, considering expectations-experiences together for the primary level of care

Study design

Cross-sectional psychometric methodology in the construction and validation of a measurement instrument.

Participants

At random, 320 patients with type 2 diabetes mellitus were selected, registered at the Family Medical Unit 11, of the Mexican Institute of Social Security in the state of Aguascalientes in Mexico. That they were receiving care with the comprehensive care model (doctor–nurse), at the first level of care.

Data collection tools

Construction of the instrument

A panel comprising of nine health professional experts developed different items centered on four main dimensions: communication satisfaction, empathy, technical care, and care continuity,[1] taking into account health-care, health-promotion, prevention, rehabilitation, and treatment. All these components are essential for the control of diabetes and independent of medical treatment, they have a major impact on individual quality of life improvement in type 2 diabetes patients, as well as on their family and the community quality of life.

Validation

For content validity, the Delphi technique was used[20] in two rounds. Each expert received the objectives of the study, the instrument, through their respective electronic mail, and they were asked to evaluate each item of the questionnaire based on an expert judge template.[2021] with the indicators for each item ranging from a value of 1 (does not meet the criteria) to 4 (high level of compliance with the criteria). There were four categories to evaluate: sufficiency, coherence, relevance, and clarity. With this procedure, the items of the instrument were reformulated according to the observations of the experts, and they were sent a second round for a new evaluation and to reach a consensus. A focus group was created, consisting of nine experts in the area of satisfaction, who worked with the instrument created by the Delphi technique.[7]

For apparent validity, a focus group was created with 10% of patients with diabetes, randomly selected, trying to have a homogenous sample according to age and gender. They were asked to analyze the elements of each item, reformulating them until a consensus was reached[7]

For construct validity, 93 patients were selected for each of the domains (total of items plus one), and then an exploratory factorial analysis was carried out for the main components. Items were then reduced (5 n) to conclude with the confirmatory factorial analysis. For criteria validity, concurrent criteria were considered. Finally, reliability was evaluated with the Cronbach alpha coefficient and the test-retest reliability, using a sub-sample of 10% (n = 37) of the patients, who responded to the questionnaire again.

Questionnaire finalization

The final questionnaire consisted of 92 items for each domain of satisfaction, based on four dimensions: communication, empathy, technical care, and care continuity. These dimensions were reinforced by the levels of care: promotion, prevention, healing, and rehabilitation, as indicators of the comprehensive care model. Each item was displayed in a Likert Scale either for expectations (Unimportant, Slightly important, Moderately Important, Important, and Very Important) or for experiences (Never, Rarely, Sometimes, Often and Always), both for general practitioner (GP) and family medicine nurse (FMN). Satisfaction was then categorized into three levels: Very satisfied, Satisfied, Unsatisfied.

A questionnaire was applied that included sociodemographic variables, as well as the developed survey to identify expectations and satisfaction [Table 1], which was filled up before and after medical consultation. Clinical activities recorded in the medical file were also registered, including a number of visits in the past year, and lab results for the past six months (i.e. blood levels of glucose, total cholesterol, triglycerides, serum creatinine, glomerular filtration rate, glycosylated hemoglobin). Blood pressure, weight, height, and body mass index were also registered.

Table 1

Sociodemographic and clinical characteristics of the patients (n=320)

Variable	n (n=320)	Percentage
Sociodemographic characteristics
Age Mean (ED*)	56.57 (10.70)
Sex
Masculine	116	36.3
Femenine	204	63.7
Disease characteristics
Time with diagnosis (years)
<10	288	90.0
>10	32	10.0
Comorbidities
Hypertension	203	54.3
Obesity	16	4.3
Dyslipidemia	24	6.4
Heart diseases	11	2.9
Cerebrovascular diseases	2	0.5
Others	25	6.7
Clinical actions
Visits
<10	59	18.5
>10	261	81.6
Glucose
<130 mg/dl	143	44.7
>130 mg/dl	177	55.3
Glucosylated hemoglobin
<7%	103	32.2
>7	161	50.3
Total cholesterol
<200 mg/dl	177	55.3
>200 mg/d	13	42.5
Triglycerides
<150 mg/dl	104	32.5
>150 mg/dl	208	65.0
Glomerular filtration rate
<60 ml/min	31	9.7
>60 ml/min	286	89.4
Body mass index
18.50-24.99 (Healthy)	49	13.0
25-29.99 (Overweight)	125	39.1
30-34.99 (Obese)	130	40.6
>40 (Extreme obesity)	23	7.2
Satisfaction
GP
Dissatisfied	126	39.4
Satisfied	95	29.7
Very satisfied	99	30.9
FMN
Dissatisfied	108	33.8
Satisfied	114	35.6
Very satisfied	98	30.6

Characteristics of the disease in the last 12 months. GP=general practitioner; FMN=family medicine nurse.

Ethics statement

The study was approved by the National Ethics Committee at IMSS (Comité Institucional de Ética) with registration number R-2015-101-40.

All the participants once they had agreed to participate signed a letter of informed consent.

Data analysis

Variables distribution of the population were described by means, standard deviation, frequencies, and percentages. The Kendall W concordance coefficient was calculated, considering a value of 1 for perfect agreement among evaluators, 0 representing the amount of agreement no greater than expected from random chance, and a negative value if the agreement was less than expected from random chance (a = < 0.05). Sampling adequacy was measured with the Kaiser–Meyer–Olkin (KMO) test, following the recommendation of Tabachnick and Fidell (< 0.3).[22] Bartlett sphericity test was also applied. Factorial analysis was performed using the varimax method of main components with rotation,[23] considering significant factorial loads ≥0.4. Spearman correlation was performed, considering moderate-elevated correlation (>0.4) and low or null (<0.3). Reliability was assessed between domains and dimensions estimating the Cronbach alpha, considering a value ≥0.70. For the test-retest reliability coefficient, the intraclass correlation coefficient (ICC) was estimated, with values between 0 and 1, and an ICC >0.5 was considered acceptable. An analysis of individual differences proposed by Bland and Altman was then conducted,[22] through a dispersion diagram. All analyses were performed with statistical package SPSS version 25[24] and the Atlas.ti software for qualitative analysis.

Results

A total of 320 patients with type 2 diabetes were included in the study. Most of them (60%) were younger than 60 years and included both married and female. Housewives were thus the most frequent occupation (46.3%, n = 148). Less than half had 5 years or less since diagnosis, and 36% had more than 11 years. The most prevalent comorbidities were metabolic syndrome (54%), hypertension, and dyslipidemia, as shown in [Table 1]. Half (50%) of the study population had a glycosylated hemoglobin >7%, and only 32.5% had adequate levels. Nearly two thirds had triglyceride levels above 150 mg/dl, and 42.5% had cholesterol over 200 mg/dl. While renal function was preserved in 89%, 10% had already some kidney impermeant. Factors like overweight and obesity were quite frequent.

Among general practitioners, satisfaction was equally distributed in the three tested categories, dissatisfied (39.4%), satisfied (29.7%), and very satisfied (30.9%), and were very similar for FMN.

While 128 items were evaluated for each of the expectations-experiences domains in the questionnaire, with a significant Kendall W concordance coefficient (0.93), as shown in [Table 2], following the expert’s consensus, 36 items were eliminated from each domain.

Table 2

Concordance between the observations of the experts

	EXPI*	EXPII*	EXPIII*	Sum of ranges (ΣR)
Sufficiency	1022	1005	982	3009
Coherency	1024	1024	1024	3072
Relevane	1008	1018	982	3008
Clarity	978	977	978	2933, Σ total=12022
	χ 2	df	P
Kendall W**	8,379	3	0,039

*EXP=Expert, **Kendall concordance coefficient, χ2=Chi-squared, df=degree of freedom

In the exploratory factorial analysis with varimax rotation of the 92 items for both domains, a KMO index >0.860 was obtained for GP and 0.894 for FMN for expectative, while for the experience these values were 0.864 for GP and 0.847 for FMN, with a Bartlett sphericity test < 0.05 (p = 0,001). Therefore, the factorial model was adequate to explain the data, as shown in [Table 3]. There were 11 items eliminated for GP in communication, 7 items in empathy, 11 items in technical care, and 7 items in care continuity, while for FMN there were 8 items in communication, 9 items in empathy, 8 items in technical care, and 7 items in care continuity. Spearman correlation values for expectations were 0.228-0.559 and for experience 0.400-0.629.

Table 3

Confirmatory factorial analysis

Rotated component array EXPECTATIONS
GP					FMN
ITEM	1	2	3	4	ITEM	1	2	3	4
COGP2			,864		COFMN17		,800
COGP3			,870		COFMN19		,811
COGP6			,734		COFMN22		,761
EGP42		,851			EFMN54			,740
EGP43		,896			EFMN57			,789
EGP47		,806			EFMNF59			,788
CTGP65	,900				CTMNF89	,858
CTGP69	,913				CTVFMN92	,787
CTGP73	,913				CTFMN95	,740
CAGP98				,868	CAFMN120				,759
CAGP99				,875	CEFMN122				,867
CAGPF108				,675	CAFMN123				,728
Variance (%)	21,262	20,744	19,247	18,431	Variance (%)	19,467	18,383	17,899	17,769
Total Variance (%) KMO=0.809	BST	χ2=2313,31	df=66	79,684, P=0.000	Total Variance (%) KMO=0.837	BST	χ2=1726,58	df=66	73,519, P=0.000
Rotated component array EXPERIENCES
GP					FMN
ITEM	1	2	3	4	ITEM	1	2	3	4
EXCOGP10				,744	EXCOFMN22		,748
EXCOGP14				,862	EXCOFMN27		,819
EXCOGP15				,681	EXCOFMN31		,725
EXEGPF34	,789				EXEFMN50		,791
EXEGP42	,847				EXEFMN58		,709
EXEGP43	,838				EXEFMN64		,762
EXCTGP75			,758		EXCTFMN83				,662
EXCTGP76			,774		EXCTFMN84				,842
EXCTGP79			,833		EXCTFMN89				,620
EXCAGP98		,853			EXCAFMN122	,806
EXCAGP99		,884			EXCAFMN124	,772
EXCAGP100		,745			EXCAFMN125	,766
Variance (%)	19,467	18,383	17,899	17,769	Variance (%)	19,608	19,393	18,987	15,056
Total Variance (%)				73,519	Total Variance (%)				73,045
KMO=0.863	BST	χ2=2213,25	df=66	P=0,000	KMO=0.877	BST	χ2=1814,95	df=66	P=0,000

KMO=Kaiser-Meyer-Olkin, BST=Bartlett sphericity test, df=degree of freedom, P=Significance, GP=General practitioner; FMN=Family medicine nurse; COM=Communication; E=Empathy; CT=Technical care; CA=Care continuity

Cronbach’ alpha values to assess internal consistency were above 0.7, with a general scale of 0.942 in 48 items. The dimension with the lowest value was technical care, as seen in [Table 4].

Table 4

Internal Consistency

	Expectations		Experiences
	GP	FMN	GP	FMN
Communication	0,834	0,789	0,821	0,821
Empathy	0,899	0,769	0,887	0,823
Technical care	0,905	0,816	0,794	0,692
Care continuity	0,801	0,791	0,845	0,828
Dimension	0,816	0,867	0,893	0,881
Domain	0,906	0,906	0,936	0,936
Total	0.942

*Cronbach alpha

The test re-test reliability coefficient was 0.849. The difference of means of the score in the test and the retest was statistically significant, 3 of the 4 factors for the domain of expectations presented reliability coefficients > 0.5 (boundaries from 0.459 to 0.653), indicating an acceptable concordance in the dimensions of communication, empathy, care continuity, and poor reliability in technical care. In the domain of experience, empathy, technical care, and care continuity had acceptable reliability, while there was poor reliability for the dimension of communication (0.430-0.893). Nevertheless, in the analysis of temporal stability using the Bland Altman method, most of the differences were between the mean of the variable difference and two standard deviations [See Annex 1].

Discussion

The developed questionnaire for evaluating type 2 diabetes patient satisfaction in this trial has shown adequate psychometric properties when assessing the health care provided by the general practitioner and the family medicine nurse. Contrary to other satisfaction evaluation tools, this questionnaire is unique since it is aimed to evaluate dual care and simultaneously the domains of satisfaction in expectations-experience.[12]

The use of this instrument in primary care will help experts to reinforcement of all satisfaction parameters in relation to levels of care, as well as the contribution of patients with type 2 diabetes, enriching the interaction physicians, patients and nurse.[252627]

Content validity based on expert’s judgement was performed as it has been previously suggested.[2829] Reliability estimation based on agreement allows eliminating subjective issues.[28] A high Kendall W concordance coefficient (0.931) indicates that there was a consensus in the process of classification and scoring, among the evaluators, supporting instruments interchangeability and reproducibility.[30]

One of the advantages of the herein presented questionnaire is that content validity was assessed as part of the psychometric properties of the instrument,[1531] in opposition to most evaluation tools that only include construct validity,[2] and do not perform a mixed analysis. Another plus of the instrument was the inclusion of type 2 diabetes patients, benefiting from using their words, expressions and accuracy of the items, to properly evaluate satisfaction with different interventions.[32]

The total variability of the instrument was >64% for expectations and 59% for experience considering four factors. Therefore, these factors could only explain a minimal amount of the instrument variability.

In spite of having similar factorial structures with other questionnaires, none of the previously published satisfaction assessment questionaries have considered all the domains included in this report, nor have they considered the two healthcare staff members included in this trial, general practitioners and family medicine nurses.

Since there is no gold standard to compare the scores from this questionnaire, only criteria items were used for validation.[18] A comparable Cronbach’s alpha coefficient to previously reported.[33] is encouraging, denoting an adequate correlation among included items.

The high observed value of the ICC supports an adequate concordance for the entire scale. The 0.5 cutoff value proposed for the ICC in this research,[20] led to an inadequate score in the mean difference of the test re-test for the technical care dimension in the domain of expectations (0.459), and the communication dimension in the domain of experience (0.430), although there was a good temporal stability of the instrument.[34]

In the present analysis women showed a higher satisfaction score, and so did older subjects, which has also been previously reported.[34]

The degree of satisfaction observed in the studied type 2 diabetes patients with this instrument, may be explained by the social desirability bias, present when participants in a study tend to provide answers that are socially acceptable due to the continuous use of health-care services. Other authors have suggested that fear of reprisal for negative answers may be the cause of the trend towards more favorable scores.

Patient dissatisfaction with care in diabetes is associated with poor self-care behaviors, low quality of life and inadequate blood glucose levels. In the primary care setting, where most type 2 diabetes should be controlled and treated, identifying patient satisfaction may be an important complementary tool to increase treatment adherence and compliance.[3536]

Nevertheless, the accurate and valid methodology employed in this research, may certainly counteract these limitations on satisfaction assessment. The lack of previous studies in Mexico, prevents comparison of the proposed instrument, and further research will properly validate it.

Conclusion

The items included in the development of this instrument provide theoretical support based on the theory and a model of the value of expectation in the setting of satisfaction. They also provide a one-on-one model (general practitioner and family medical nurse) of integral care, aimed to type 2 diabetes patients, considering four dimensions, communication, empathy, technical care, and care continuity, in the domains of expectations-experience. The main strength of this instrument, in addition to being considered a validated, adaptable instrument for other cultures, is that it can be used by the health care staff as well as in satisfaction evaluation research in the first level of diabetes health-care.

Key points

Primary care physicians play an important role in the control of diseases of the population, por lo que measure the satisfaction with integral care of the patient with type 2 diabetes will allow you to count on other tools to improve your services, optimizing their actions to maintain the health of patients.

Knowing the level of satisfaction of patients with the health personnel who is in charge of maintaining control of their disease is essential to understand the effect of treatment on disease control.

In addition, these results allow having elements for the design of strategies aimed at improving the relationship of health personnel with the patient.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

Satisfaction of the diabetes patient with the model (GP and/or FMN) with integral care in primary care. (with the items after the elimination)
EXPECTATIONS	5-Very important	4- Important	3-Moderately important	2-Of little importance	1-Unimportant
Instructions: Regarding your visit with your General Practitioner and/or the Family Medicine Nurse, I am going to ask you some questions, which you can answer with the options I will give you.
COMMUNICATION
GENERAL PRACTITIONER (GP)
2. How important is it that the General Practitioner explain your disease during the doctor’s visit?
3. How important is it that the General Practitioner explain the indications of the prescriptions?
6. How important is it that the General Practitioner explain the complications of diabetes? (The interviewer mentions examples, such as: low sugar, high sugar, eye disease, kidney disease, foot disease).
FAMILY MEDICINE NURSE (FMN)
17. How important is it that the nurse go to your home to orient you about educational programs?
19. How important is it that the nurse explain the activities that would help improve your health?
22. How important is it that the nurse explain hygienic habits you should have?
EMPATHY
GENERAL PRACTITIONER (GP)
42. How important is it that the General Practitioner nicely indicate the general care you should have?
43. How important is it that the General Practitioner nicely invite you to continue treatment with its medications?
47. How important is it that the General Practitioner show interest in your treatment in case of complications?
FAMILY MEDICINE NURSE (FMN)
54. How important is it that the Family Medicine Nurse explain the use of insulin, in case it is necessary?
57. How important is it that the Family Medicine Nurse show interest in going to your home to review your blood pressure?
59. How important is it that the Family Medicine Nurse explain in detail the benefits of being vaccinated against influenza?
TECHNICAL CARE
GENERAL PRACTITIONER (GP)
65. How important is it that the General Practitioner inform you in detail about your disease by telephone?
69. How important is it that the General Practitioner inform you by telephone about support groups to reduce the risk factors of your disease?
73. How important is it that the General Practitioner inform you by telephone about the effects of your medications?
FAMILY MEDICINE NURSE (FMN)
89. How important is it that the nurse teach you how to store, use and apply insulin?
92. How important is it that the nurse sends you to the General Practitioner in case of skin wounds that do not heal?
95. How important is it that the nurse suggest improvement goals in the care of your disease?
CARE CONTINUITY
GENERAL PRACTITIONER (GP)
98. How important is it that the General Practitioner give you an appointment for a visit every month?
GENERAL PRACTITIONER (GP)
99. How important is it that the General Practitioner give you monthly treatment?
108. How important is it that the treatments be coordinated between the General Practitioner and the Family Medicine Nurse?
FAMILY MEDICINE NURSE (FMN)
120. How important is it that the nurse send you to the General Practitioner when you are controlled?
122. How important is it that your disease is followed up by the Family Medicine Nurse?
123. How important is it that the nurse perform educational activities each time you come to a doctor’s visit?
EXPECTATIONS	5- Alwaysre	4- Almost always	2- Sometimesveces	3- Rarely	Never
Instructions: Regarding the doctor’s visit with the General Practitioner and/or the Family Medicine Nurse, I am going to ask you some questions, and you can answer with the options I will give you.
COMMUNICATION
GENERAL PRACTITIONER (GP)
10. Did the General Practitioner explain about how and when to measure your sugar (destrostix), before and after ingesting food?
14. Did the General Practitioner explain how you should take care of your skin to avoid complications?
15. Did the General Practitioner explain how medications could avoid complications?
FAMILY MEDICINE NURSE (FMN)
22. Did the nurse explain the hygienic habits you should have?
27. Did the nurse explain foot care?
31. Did the nurse explain the care you should have in case of presenting complications?
EMPATHY
GENERAL OPRACTITIONER (GP)
34. Did the General Practitioner show understanding of your health problems that caused you to go to the doctor’s visit?
42. Did the General Practitioner nicely indicate the general care you should have?
43. Did the General Practitioner nicely invite you to continue treatment with medications?
FAMILY MEDICINE NURSE (FMN)
50. Did the nurse treat you nicely during the doctor’s visit?
58. Did the nurse give you clear indications of the use of medications to maintain control?
64. Did the nurse send you in a timely fashion to the various health professionals to avoid complications?
TECHNICAL CARE
GENERAL PRACTITIONER (GP)
75. Did the General Practitioner give you a prescription for treatment based on the indications of the hospital specialist?
76. Did the General Practitioner send you for X-rays for possible complications?
79. Did the General Practitioner indicate applying insulin for uncontrolled sugar (glucose)?
FAMILY MEDICINE NURSE (FMN)
83. Did the nurse orient you on how to cut your nails?
84. Did the nurse send you to a dentist during these 6 months?
89. Did the nurse teach you how to store, use and apply insulin?
CARE CONTINUITY
GENERAL PRACTITIONER (GP)
98. Did you visit the General Practitioner every month?
99. Did the General Practitioner give you monthly treatment?
100. Did the General Practitioner send you to the health services you required?
FAMILY MEDICINE NURSE (FMN)
122. Did you receive follow up on your disease from the same Family Medicine Nurse?
124. Did you receive treatments coordinated between the Family Medicine Nurse and the General Practitioner?
125. Did you receive continuity in the follow-up from the nurse to reduce possible complications?