Development and validation of a questionnaire for a survey on perioperative fasting practices in India.

BACKGROUND AND AIMS: Perioperative fasting guidelines have been published and updated to standardise practices. Hence, Indian Society of Anaesthesiologists decided to conduct a survey to assess the fasting practices and the food habits across India, which would be subsequently used for developing preoperative fasting guidelines for the Indian population. We detail and discuss herewith the content validity of the questionnaire developed for the survey.
METHODS: Thirty-six questions related to perioperative fasting practices were framed based on the collected evidence and relevance to regional diet and concerns. Subsequently, an information sheet was prepared and sent to 10 experts to grade each question. The responses were tabulated, and item-wise content validity index (I-CVI), scale-wise content validity index (S-CVI) and modified kappa statistic were calculated in Microsoft Excel™ sheet.
RESULTS: Seven of the 10 experts completed the assessment and grading as per the instructions provided and submitted a completed proforma. S-CVI for relevance, simplicity, clarity and ambiguity was 0.72, 0.86, 0.72 and 0.72, respectively. S-CVI/average or average congruency percentagewas 0.95, 0.97, 0.95 and 0.95 for relevance, simplicity, clarity and ambiguity, respectively. Question 2 received an I-CVI of 0.71 in terms of clarity and question 23 received an I-CVI of 0.71. They were modified as persuggestions of the experts.
CONCLUSION: We conclude that our questionnaire designed to ascertain the preoperative fasting practices for a surveymet the content validity criteria both by qualitative and quantitative analyses.

INTRODUCTION

Perioperative fasting guidelines have been published and updated in continued attempts to standardise the practices in the surgical population. The guidelines provided by professional associations such as American Society of Anesthesiologists (ASA) and European Society of Anaesthesiologists (ESA) classify food items into solids, clear liquids, breast milk and non-human milk and advise duration of fasting for different subsets of patients based on evidence.[12] In India, in the absence of specific guidelines suited for local practice, the guidelines provided by ASA or ESA and others are followed. Hence, Indian Society of Anaesthesiologists decided to assess the fasting practices and the food habits across India, which would be subsequently used for developing preoperative fasting guidelines for the Indian population. We describe the content validity of the questionnaire developed.[3]

The process of development of a validated toolinvolves mainly two steps.[4] The first step involves extensive literature review and development of the questionnaire and the second step involves validation of the questionnaire based on expert opinion. Content validation is an essential step in instrument development as it signifies the extent to which a measurement reflects a specific intended domain of content.[5]

Quantification of the content validity is required and can be done using methods such as content validity index and multirater agreement tests.[5] In this article, we report the development and subsequent validation of a questionnaire to elicit the present perioperative fasting practices across institutions in various parts of India. We used the content validity index and modified kappa statistic (MKS) using the probability of chance agreement as described by Polit et al. to estimate the content validity of the questionnaire developed.[36]

METHODS

The content questions related to perioperative fasting practices were framed after an extensive literature search by six independent anaesthesiologists, from data bases that included PubMed, Google Scholar, EMBASE, Cochrane Library and guidelines framed by other societies all over the world; the collected information was collated. The search words and phrases used were 'NPO guidelines', 'NBM guidelines', 'fasting guidelines', 'preoperative fasting practices' and 'peri-operative fasting practices'. The bibliographic references were searched manually as well. Fifteen articles including recent review articles and latest guidelines by various societies were identified. Guidelines published after 2010 and recent review articles published after 2015 were taken into account. After collation, 31 questions were framed which underwent intensive scrutiny for clarity and relevance. Grammatical and spelling errors were ironed out. The questionnaire was expanded to 36 questions eliciting information on the perioperative fasting practices based on the collected evidence and with relevance to regional diet and concerns [Figure 1 and Appendix 1 is available online]. Subsequently, an information sheet was prepared for an expert group to grade each question based on relevance, simplicity, clarity and ambiguity of the framed question on a 4-point scale [Appendix 2 is available online]. This scale was adopted from a scale devised by Yaghmale[7] and subsequently used by Emmanueland Clow.[8] The questionnaire and the information sheet for grading the content were sent by e-mail to 10 experts with an experience of at least 10 years in the field of anaesthesiology and hailing from different regions.

Figure 1

The procedure followed for content validity of the questionnaire

A time frame of 1 month was provided to the experts to assess and grade the questionnaire. The target was receipt of response from at least 50% of the experts at the end of the month. The results were compiled and tabulated (grading of 3 or 4 was labelled as x and any grading of 1 or 2 as 0) in Microsoft Excel™ sheet. The item-wise content validity index (I-CVI) was calculated for each question from the questionnaire using the method described by Polit et al.[36] The scale-wise content validity index (S-CVI) and S-CVI/average or average congruency percentage (ACP) were also calculated based on the methods described by Polit et al.[36] Probability of chance agreement on relevance (Pc) was calculated using the formula Pc = [N!/{A!(N − A)!}] ×0.5N, where N = number of experts and A = number of experts in agreement on relevance. MKS was calculated using the following formula for each of the questions:

K = I-CVI − Pc/1 − Pc

If the I-CVI was less than 0.78, the group reached a consensus on whether to change the question as per the suggestion. Each item was also evaluated based on the evaluation parameters suggested by Cichetti and Fleiss.[910] The values obtained for I-CVI for relevance, simplicity, clarity and ambiguity were tabulated along with the calculated MKS and the evaluation of each item.

RESULTS

Seven of the 10 experts completed the assessment and grading as per the instructions provided and submitted a completed proforma. S-CVI for relevance, simplicity, clarity and ambiguity was 0.72 [Table 1], 0.86 [Table 2], 0.72 [Table 3] and 0.72 [Table 4] respectively. ACPs were calculated as 0.95 [Table 1], 0.97 [Table 2], 0.95 [Table 3] and 0.95 [Table 4] for relevance, simplicity, clarity and ambiguity respectively.

Table 1

Content validity of relevance

Question No.	I-CVI	Probability of chance agreement	Modified kappa statistic	Interpretation
1	1	0.008	1.0	Excellent
2	1	0.008	1.0	Excellent
3	1	0.008	1.0	Excellent
4	1	0.008	1.0	Excellent
5	0.85	0.055	0.8	Excellent
6	1	0.008	1.0	Excellent
7	1	0.008	1.0	Excellent
8	1	0.008	1.0	Excellent
9	1	0.008	1.0	Excellent
10	1	0.008	1.0	Excellent
11	0.85	0.055	0.8	Excellent
12	1	0.008	1.0	Excellent
13	0.85	0.055	0.8	Excellent
14	0.85	0.055	0.8	Excellent
15	1	0.008	1.0	Excellent
16	1	0.008	1.0	Excellent
17	0.85	0.055	0.8	Excellent
18	1	0.008	1.0	Excellent
19	1	0.008	1.0	Excellent
20	1	0.008	1.0	Excellent
21	1	0.008	1.0	Excellent
22	1	0.008	1.0	Excellent
23	1	0.008	1.0	Excellent
24	1	0.008	1.0	Excellent
25	0.85	0.055	0.8	Excellent
26	0.71	0.164	0.7	Good
27	0.85	0.055	0.8	Excellent
28	1	0.008	1.0	Excellent
29	0.85	0.055	0.8	Excellent
30	0.85	0.055	0.8	Excellent
31	1	0.008	1.0	Excellent
32	1	0.008	1.0	Excellent
33	1	0.008	1.0	Excellent
34	1	0.008	1.0	Excellent
35	1	0.008	1.0	Excellent
36	1	0.008	1.0	Excellent

Mean I-CVI - 0.95, S-CVI - 0.72, S-CVI/Average=0.95

Table 2

Content validity of simplicity

Question No.	I-CVI	Probability of chance agreement	Modified kappa statistic	Interpretation
1	1	0.008	1.0	Excellent
2	1	0.008	1.0	Excellent
3	1	0.008	1.0	Excellent
4	1	0.008	1.0	Excellent
5	1	0.008	1.0	Excellent
6	0.85	0.055	0.8	Excellent
7	1	0.008	1.0	Excellent
8	1	0.008	1.0	Excellent
9	1	0.008	1.0	Excellent
10	1	0.008	1.0	Excellent
11	1	0.008	1.0	Excellent
12	1	0.008	1.0	Excellent
13	1	0.008	1.0	Excellent
14	1	0.008	1.0	Excellent
15	1	0.008	1.0	Excellent
16	1	0.008	1.0	Excellent
17	1	0.008	1.0	Excellent
18	1	0.008	1.0	Excellent
19	1	0.008	1.0	Excellent
20	1	0.008	1.0	Excellent
21	1	0.008	1.0	Excellent
22	1	0.008	1.0	Excellent
23	0.85	0.055	0.8	Excellent
24	0.85	0.055	0.8	Excellent
25	1	0.008	1.0	Excellent
26	1	0.008	1.0	Excellent
27	0.85	0.055	0.8	Excellent
28	1	0.008	1.0	Excellent
29	1	0.008	1.0	Excellent
30	0.85	0.055	0.8	Excellent
31	1	0.008	1.0	Excellent
32	1	0.008	1.0	Excellent
33	1	0.008	1.0	Excellent
34	1	0.008	1.0	Excellent
35	1	0.008	1.0	Excellent
36	1	0.008	1.0	Excellent

I-CVI mean=0.9, S-CVI=0.86, S-CVI/Average=0.97

Table 3

Content validity of clarity

Question No.	I-CVI	Probability of chance agreement	Modified kappa statistic	Interpretation
1	0.85	0.055	0.8	Excellent
2	0.72	0.164	0.7	Good
3	1	0.008	1.0	Excellent
4	1	0.008	1.0	Excellent
5	1	0.008	1.0	Excellent
6	1	0.008	1.0	Excellent
7	1	0.008	1.0	Excellent
8	1	0.008	1.0	Excellent
9	0.85	0.055	0.8	Excellent
10	1	0.008	1.0	Excellent
11	1	0.008	1.0	Excellent
12	1	0.008	1.0	Excellent
13	0.85	0.055	0.8	Excellent
14	1	0.008	1.0	Excellent
15	0.85	0.055	0.8	Excellent
16	1	0.008	1.0	Excellent
17	1	0.008	1.0	Excellent
18	1	0.008	1.0	Excellent
19	1	0.008	1.0	Excellent
20	1	0.008	1.0	Excellent
21	1	0.008	1.0	Excellent
22	1	0.008	1.0	Excellent
23	0.85	0.055	0.8	Excellent
24	0.85	0.055	0.8	Excellent
25	1	0.008	1.0	Excellent
26	1	0.008	1.0	Excellent
27	0.85	0.055	0.8	Excellent
28	1	0.008	1.0	Excellent
29	1	0.008	1.0	Excellent
30	0.85	0.055	0.8	Excellent
31	1	0.008	1.0	Excellent
32	0.85	0.055	0.8	Excellent
33	1	0.008	1.0	Excellent
34	1	0.008	1.0	Excellent
35	1	0.008	1.0	Excellent
36	1	0.008	1.0	Excellent

Mean I-CVI=0.95, S-CVI=0.72, S-CVI/Average=0.95

Table 4

Content validity of ambiguity

Question No.	I-CVI	Probability of chance agreement	Modified kappa statistic	Interpretation
1	1.00	0.008	1.0	Excellent
2	1.00	0.008	1.0	Excellent
3	1.00	0.008	1.0	Excellent
4	1.00	0.008	1.0	Excellent
5	0.85	0.055	0.8	Excellent
6	1.00	0.008	1.0	Excellent
7	1.00	0.008	1.0	Excellent
8	1.00	0.008	1.0	Excellent
9	1.00	0.008	1.0	Excellent
10	1.00	0.008	1.0	Excellent
11	0.85	0.055	0.8	Excellent
12	1.00	0.008	1.0	Excellent
13	0.85	0.055	0.8	Excellent
14	0.85	0.055	0.8	Excellent
15	1.00	0.008	1.0	Excellent
16	1.00	0.008	1.0	Excellent
17	0.85	0.055	0.8	Excellent
18	1.00	0.008	1.0	Excellent
19	1.00	0.008	1.0	Excellent
20	1.00	0.008	1.0	Excellent
21	1.00	0.008	1.0	Excellent
22	1.00	0.008	1.0	Excellent
23	0.71	0.164	0.7	Good
24	1.00	0.008	1.0	Excellent
25	0.85	0.055	0.8	Excellent
26	1.00	0.008	1.0	Excellent
27	0.85	0.055	0.8	Excellent
28	1.00	0.008	1.0	Excellent
29	0.85	0.055	0.8	Excellent
30	0.85	0.055	0.8	Excellent
31	1.00	0.008	1.0	Excellent
32	1.00	0.008	1.0	Excellent
33	1.00	0.008	1.0	Excellent
34	1.00	0.008	1.0	Excellent
35	1.00	0.008	1.0	Excellent
36	1.00	0.008	1.0	Excellent

Mean I-CVI=0.95, S-CVI=0.72, S-CVI/Average=0.95

In terms of relevance [Table 1], 26 of the 36 questions were graded as 'relevant but need minor revision'or'very relevant' by all the experts (I-CVI-1, K-1). Nine questions were thought to be 'relevant but need minor revision' or 'very relevant' by six of the seven experts and one question was graded as 'relevant but need minor revision' or 'very relevant' by five experts out of seven (I-CVI 0.71, K 0.7).

Thirty-five questions received a grading of 'simple but need minor revision' or 'very simple' by all the seven experts (I-CVI = 1; K = 1), and five questions received a grading of 'simple but need minor revision' or 'very simple' by six of the seven experts (I-CVI = 0.85, K = 0.8) [Table 2].

Twenty-six of the 36 questions were graded as 'clear but need minor revision' or 'very clear' by the seven experts (I-CVI = 1, K = 1). Six of the seven experts graded nine questions as 'clear but need minor revision' or 'very clear' (1-CVI = 0.85, K = 0.8) and five of the seven experts graded one question as 'clear but need minor revision' or 'very clear' (I-CVI = 0.71, K = 0.7) [Table 3].

Twenty-six of the 36 questions were graded as 'no doubt but need minor revision' or 'meaning is clear' by all the seven experts (I-CVI = 1, K = 1). Six of the seven experts graded eight questions as 'no doubt but need minor revision' or 'meaning is clear' for nine questions (I-CVI = 0.85, K = 0.8), and six of the seven experts graded one question 'no doubt but need minor revision' or 'meaning is clear' (I-CVI = 0.71, K = 0.7) [Table 4]. Experts who graded questions as 1, 2 and 3 did give suggestions on how to revise the questions. Question 2 received an I-CVI of 0.71 in terms of clarity; the answers were modified as per the instructions of the experts to convey better clarity on the method of answer entry. The experts had advised to mention common kinds of food consumed specific to regions with examples. The examples on how to answer the question were included in the question in brackets. The experts had also suggested to add the term large/heavy meal instead of the term heavy meal. This was also complied with [Appendix 3 is available online].

Question 23 too received an I-CVI of 0.71 in terms of ambiguity and the experts had suggested that data needed to be collected for both elective and emergency surgeries. The questionnaire was initially modified to include both elective and emergency surgeries as shown in Appendix 3 is available online.

DISCUSSION

Content validity of an instrument is defined as the extent to which an instrument adequately samples the research domain of interest when attempting to measure aphenomena. It can be measured in two main steps: (a) identifying the domain through literature search and (b) developing the instrument items identified with domain of content which involves the quantification of content validity.[11] There are various methods of estimating the content validity of an instrument. They are mainly consistency estimates, consensus estimates and measurement estimates.[36] While estimating content validity researchers are usually guided by ease of computation, ease of understanding and communication, adjustment for chance agreement and focus on consensus agreement rather than consistency.

We chose content validity index and MKS as suggested by Polit et al.[36] in estimating the content validity of our scale. We used the criteria provided by Yaghmale[7] in measuring content validity. It included a 4-point scale in assessing four parameters for each question designed [Appendix 2 is available online].

According to Lynn,[12] an I-CVI of 0.78 and an S-CVI/Average of 0.90 are acceptable when more than six experts have graded the tool. Wynd et al.[11] quote theevaluation of magnitude of kappa coefficients by parameters given by Landis and Koch,[13] Cichetti,[10] and Fleiss.[11] We in our estimation used the parameters given by Cichetti and Fleiss as they had provided for four grades in comparison to those provided by Landis and Koch. They grade the kappastatistic as <0.40 = poor, 0.40–0.59 = fair, 0.60–0.74 = good and 0.75–1.00 = excellent.

In our estimation, of the 36 questions, 34 questions were quantitatively valid. One question had an I-CVI of 0.71 for ambiguity and a kappa statistic of 0.7 (good). One question had an I-CVI of 0.71 for clarity and a kappa statistic of 0.7. We considered that any I-CVI of >0.78 as acceptable and any item with an I-CVI <0.78 meant the item needed to be eliminated or rectified.

Considering these criteria, we accepted the suggestionsof the experts related to twoquestions (2 and 23) after assessing the suggestions and coming to a consensus among the experts. In question 2, the answers were modified to convey better clarity. The experts had advised to mention common kinds of food, which was duly attended to. Question 23was also modified as per the suggestions provided by the experts as mentioned above. A consultation was sought with the biostatistician regarding restructuring the questions without modifying the content to ease the analysis. As per those suggestions, the questionnaire was further modified. The final questionnaire constituted 40 questions. Question numbers 23 and 26 were split into two questions each to collect the data for emergency and elective cases separately, and questions which elicited the data on therapy and were related to questions 23 and 26were again duplicated to collect data specifically for emergency and elective cases. This resulted in the increase in the number of questions from 36 to 40.

CONCLUSION

We conclude that our questionnaire designed to ascertain the preoperative fasting practices met the content validity criteria both by qualitative and quantitative analyses. The two questions which did not meet the criteria were modified as per the suggestions given by the experts.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.