Determination of Visual Portfolio for Surgeons OverSeas Assessment of Surgical Needs Nigeria Study: Consensus Generation through an e-Delphi Process.

BACKGROUND: Surgery as a public health priority has received little attention until recently. There is a significant unmeasured and unmet burden of surgical illness in low- and middle-income countries (LMICs). Our aim was to generate a consensus among expert pediatric surgeons practicing in LMICs regarding the spectrum of pediatric surgical conditions that we should look out for in a community-based survey for Surgeons OverSeas Assessment of Surgical Needs Nigeria study.
MATERIALS AND METHODS: The Delphi methodology was utilized to identify sets of variables from among a panel of experts. Each variable was scored on a 5-point Likert scale. The experts were provided with an anonymous summary of the results after the first round. A consensus was achieved after two rounds, defined by an improvement in the standard deviation (SD) of scores for a particular variable over that of the previous round. We invited 76 pediatric surgeons through e-mail across Africa but predominantly from Nigeria.
RESULTS: Twenty-one pediatric surgeons gave consent to participate through return of mail. Thirteen (62%) answered the first round statements and 8 (38%) the second round. In general, the strength of agreement to all statements of the questionnaire improved between the first and second rounds. Overall consensus, as expressed by the decrease in the mean SD from 0.84 in the first round to 0.68 in the second round, also improved over time. The strength of consensus improved for 23 (74%) of the statements. The strength of consensus decreased for the remaining 8 (26%) of statements. Out of the 31 consensus-generating statements, 16 (51%) scored high agreement, 13 (42%) scored low agreement, and 2 (15%) scored perfect disagreement.
CONCLUSION: We have successfully identified the pediatric surgical conditions to be included in any community survey of pediatric surgical need in an LMIC setting.

INTRODUCTION

Surgical care is often erroneously perceived as a highly specialized and cost-ineffective area of medicine. On the contrary, it is essential at all levels of health-care delivery, including primary care, with provision of curative care for conditions and injuries.[1] This role of surgery as a public health priority has received little attention until recently, when the Lancet Commission on Global Surgery sheds light on the significant unmet burden of surgical illness in low- and middle-income countries (LMICs) and highlighted the significant health and economic benefits that would result from better access to safe surgical, anesthesia, and obstetric care.[2] Indeed, many illnesses such as cancers, trauma/road traffic injuries, and cardiovascular and metabolic diseases require surgery to restore health. These illnesses affect far more people than several other nonsurgical diseases combined.[3] It is estimated that about 5 billion people do not have access to safe, affordable surgery, and anesthesia when needed.[2] Among the approximately 300 million surgical procedures that are performed annually around the world, most occur in high-income countries (HICs), and it is largely unknown what the burden of surgical conditions are in LMICs.[3] Furthermore, the significant strides made in promoting the role of surgery in matters of global health have largely ignored the significant burden of childhood surgical conditions, which are often distinct and whose treatment mandates a particular set of skills, equipment, and infrastructure.

The Surgeons OverSeas Assessment of Surgical Needs (SOSAS) is a survey tool designed using the Demographic and Health Surveys, the WHO guidelines for conducting community surveys for injuries and violence, and a survey tool for road traffic injuries.[4] It has been used to estimate the unmet surgical needs in the communities in a few Asian and African countries.[4]

Those studies have, however, not included physical examination and were not tailored toward children. The SOSAS Nigeria study survey tool aims to include these modifications in addition to the use of a visual portfolio for congenital conditions in children.

The aim of this study was to determine the spectrum of conditions to be included in the visual portfolio using an e-Delphi consensus-generating technique among pediatric surgeons.

MATERIALS AND METHODS

Our study utilized the Delphi method, which seeks to attain a consensus among group members through a series of questionnaires. The chain of questionnaires is sent to a predetermined group of experts by mail or online.[5]

This approach ensures confidentiality for respondents, as questionnaires are answered anonymously and individually by each member of the group. Furthermore, the e-Delphi method prevents undue influence by any member of the group on the responses of the other participants. The answers are reviewed and collated, and then, an anonymous summary of responses is sent back to the group members along with the next iteration of the questionnaire.[6] This process is repeated until a group consensus is reached. This usually only takes two iterations but can sometimes take as many as six rounds before a consensus is reached.[7] A consensus is defined by an improvement in the standard deviation (SD) of the scores of a particular question.[8]

The study protocol was approved by the Lagos University Teaching Hospital Ethics and Research Board and conforms with the Declaration of Helsinki of ethical consent.

The experts answered the questionnaire in two rounds. After the first round, the participants were provided with an anonymous summary of the results with additional congenital anomalies nominated for inclusion. During the second round, experts were encouraged to review their earlier responses while considering the overall group's responses in the first iteration. This avoids direct influence from colleagues in the setting of a debate or live discussion.[9]

The findings were summarized in a series of statements. Participants were invited to rate their agreement with these statements using a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree).

Each round of the survey was administered online through a form designed on Microsoft Word® version 10 (Microsoft. Seattle, WA, USA).

In general, a sample size of about 10–15 is adequate for homogeneous panel of experts undergoing a Delphi survey.[10] Seventy-six pediatric surgeons from across Africa but predominantly from Nigeria were invited to answer the questionnaire. Individuals were solicited for participation in the survey through e-mail invitation. They were given the opportunity to ask questions regarding the study and were asked to sign an electronic consent form. The first set of questions was only sent to those that consented to participation through an e-mail response to the lead investigator. The survey remained open for 3 months, and reminder e-mails were sent every 2 weeks to the respondents.

The responses collected for each statement were used to measure the strength of agreement, based on the mean score, and the strength of consensus, based on the SD of the recorded scores.

Perfect agreement with a statement was defined as a second-round mean score of 5 and perfect disagreement as a second-round mean score of 0. Perfect consensus was defined as a second-round SD of 0.00. High and low scores were defined as second-round scores larger or smaller than the mean of all recorded second-round scores, respectively. Consensus agreement was defined by decreasing SD in subsequent rounds.

RESULTS

Participation

Of the 76 pediatric surgeons invited, 21 consented to participate through return of e-mail. Thirteen (62%) answered the first round statements and 8 (38%) responded in the second round [Table 1]. Among respondents, there were ten pediatric general surgeons, two pediatric urologists, and one pediatric orthopedic surgeon. The male-to-female ratio of the respondents was 3.3:1. About 85% of them work in the Federal Teaching Hospital, while 15% work with the state-owned teaching hospital as shown in Table 2.

Table 1

Respondent’s participation and mean standard distribution of responses at each round of the Delphi survey

Country of origin of respondents	n	Percentage of total invited	Percentage of total at given round	Percentage of previous round total	Mean SD
Invitation
Nigeria	53	70	-	-
Southern Africa	13	17	-	-
West Africa	10	13	-	-
Total	76	100	-	-
First round
Nigeria	8	10.5	61	-
Southern Africa	3	3.9	23	-
West Africa	2	2.6	16	-
Total	13	17	100	-	0.84*
Second round
Nigeria	6	7.9	75	46
Southern Africa	1	1.3	12.5	7.7
West Africa	1	1.3	12.5	7.7
Total	8	10.5	100	61.4	0.68*

*Overall an improvement in agreement observed (Mean standard deviation decreasing from 0.84 to 0.68) between the two rounds, indicating improved consensus

Table 2

Demographic characteristics of the respondents

Variables	n (%)
Gender-male:female	3.3:1
Specialty
Pediatric general surgery	10 (77)
Pediatric urology	2 (15)
Pediatric orthopedic surgery	1 (8)
Respondent hospital facility
Federal teaching hospital	11 (85)
State teaching hospital	2 (15)

Agreement and consensus

In general, the degree of consensus to all statements of the questionnaire improved between the first and second rounds, expressed by the decrease in the mean SD from 0.84 in the first round to 0.68 in the second round. This occurred despite the smaller number of respondents in the second round. Table 3 shows the summary of the mean scores and the SDs for each statement in both rounds. In general, consensus opinion was achieved for the following question items that are to be retained in the SOSAS Nigeria study: hydrocephalus, cleft lip, cleft palate, angular dermoid cyst, cystic hygroma, umbilical hernia, patent urachus, hypospadias, hydrocele, undescended testis, inguinoscrotal hernia, congenital clubfoot, and anorectal malformation with vestibular fistula.

Table 3

Mean score and standard deviation for each statement in the first and second rounds

Statement	First round			Second round
	Mean score	SEM	SD	Mean score	SEM	SD
I agree you should add hydrocephalus	4.46	0.237	0.855	5.00	0.000	0.000
Cleft lip	4.77	0.166	0.599	5.00	0.000	0.000
Cleft palate	4.62	0.241	0.870	5.00	0.000	0.000
Angular dermoid cyst	3.54	0.369	1.330	3.50	0.423	1.165
Thyroglossal cyst	4.23	0.323	1.166	4.25	0.412	1.165
Branchial cyst	3.77	0.426	1.536	3.85	0.479	1.356
Cystic hygroma	4.50	0.195	0.674	4.75	0.164	0.463
Umbilical hernia	4.31	0.365	1.316	4.75	0.250	0.707
Supraumbilical hernia	4.00	0.358	1.291	4.00	0.500	1.414
Patent urachus	3.92	0.329	1.188	4.50	0.267	0.756
Meatal stenosis	3.92	0.348	1.256	3.50	0.535	0.164
Hypospadias	4.77	0.166	0.599	4.75	0.164	0.463
Congenital chordee	4.00	0.424	1.528	4.75	0.164	0.463
Hydrocele	4.31	0.263	0.947	5.00	0.000	0.000
Undescended testes	5.00	0.000	0.000	5.00	0.000	0.000
Inguinoscrotal hernia	4.85	0.104	0.376	5.00	0.000	0.000
Congenital club foot	4.92	0.077	0.277	5.00	0.000	0.000
Congenital hip dislocation	4.31	0.201	0.751	4.00	0.327	0.926
Spinal bifida	4.85	0.104	0.376	5.00	0.000	0.000
Absent radius	4.08	0.383	1.382	4.00	0.423	1.195
Absent tibia	3.85	0.373	1.345	3.63	0.460	1.302
Absent fibula	3.85	0.373	1.345	3.63	0.460	1.302
Shortened femur	4.00	0.392	1.414	4.00	0.375	1.069
Tracheoesophageal fistula	5.00	-	-	4.38	0.324	0.916
ARM with vestibular fistula	5.00	-	-	4.38	0.324	0.916
Omphalocele ventral hernia	5.00	-	-	4.63	0.263	0.744
Syndactyl	5.00	0.000	0.000	4.50	0.327	0.926
Polydactyl	5.00	0.000	0.000	4.00	0.500	1.414
Osteogenesis imperfecta	5.00	-	-	-	-	-
Arthrogryposis multiplex	4.00	-	-	-	-	-
Absent digits	5.00	-	-	2.31	0.237	0.855

ARM: Anorectal malformation, SD: Standard deviation, SEM: Standard error of mean

The statements were also categorized based on the strength of the agreement by responders and the strength of the consensus. Some statements resulted in perfect agreement, but two of the statements resulted in perfect disagreement: the survey items for osteogenesis imperfecta and congenital arthrogryposis multiplex.

The strength of consensus improved for 23 (74%) of the statements and decreased for the remaining 8 (26%) of statements. Out of the 31 consensus-generating statements, 16 (51%) had a high score of agreement and 13 (42%) had a low score of agreement, while 2 (15%) had perfect disagreement as depicted in Table 4.

Table 4

Classification of statements by strength of agreement and quality of consensus

Statement	First round		Second round
	Mean score	SD	Mean score	SD
Strong score, improved consensus
I believe hydrocephalus is an important condition to survey	4.69	0.855	5.00	0.000
I believe cleft lip is an important condition to survey	4.77	0.599	5.00	0.000
I believe cleft palate is an important condition to survey	4.62	0.870	5.00	0.000
I believe cystic hygroma is an important condition to survey	4.58	0.674	4.75	0.463
I believe umbilical hernia is an important condition to survey	4.31	1.316	4.75	0.707
I believe congenital chordee is an important condition to survey	4.00	1.528	4.75	0.463
I believe hydrocele is an important condition to survey	4.31	0.947	5.00	0.000
I believe undescended testis is an important condition to survey	5.00	0.000	5.00	0.000
I believe inguinoscrotal hernia is an important condition to survey	4.85	0.376	5.00	0.000
I believe congenital clubfoot is an important condition to survey	4.92	0.277	5.00	0.000
I believe spinal bifida is an important condition to survey	4.85	0.376	5.00	0.000
Weak score, improved consensus
I believe patent urachus is an important condition to survey	3.92	1.188	4.50	0.756
I believe hypospadias is an important condition to survey	4.77	0.599	4.75	0.463
I believe syndactyl is an important condition to survey	5.00	0.000	4.50	0.926
I believe polydactyl is an important condition to survey	5.00	0.000	4.00	1.414
I believe thyroglossal cyst is an important condition to survey	4.23	1.166	4.25	1.165
Worsened consensus
I believe congenital hip dislocation is an important condition to survey	4.31	0.751	4.00	0.926
I believe absent radius is an important condition to survey	4.08	1.382	4.00	1.195
I believe absent tibia is an important condition to survey	3.85	1.345	3.63	1.302
I believe absent fibula is an important condition to survey	3.84	1.345	3.63	1.302
I believe angular dermoid is an important condition to survey	3.54	1.330	3.50	1.165

SD: Standard deviation

DISCUSSION

The study has allowed us to survey a panel of experts about the need to carry out a community survey, as well as the congenital surgical conditions that should be included in the survey tool. The Delphi method used was initially developed for forecasting purposes but is now gaining wide popularity in qualitative medical research for generating consensus by limiting potential biases resulting from the traditional individual or group interview methods.

The panelist are pediatric surgeons that practice in major tertiary hospitals and hence have a huge burden of referrals from the rural centers which may have influenced their responses on certain disease conditions included for the survey. The spectrum of conditions that were selected by Delphi is hydrocephalus, cleft lip, cleft palate, angular dermoid cyst, and cystic hygroma. The potential cosmetic challenges these conditions pose among other things may have contributed to their inclusion. Other conditions selected are umbilical hernia, patent urachus, hypospadias, hydrocele, undescended testis, inguinoscrotal hernia, congenital clubfoot, and anorectal malformation with vestibular fistula.

There are set criteria for selecting surgically treatable conditions. These criteria are that these conditions must be (I) identifiable by the patient, (II) objectively identifiable by examination, and (III) acceptable both ethically and culturally. These criteria were set by the original developer of the SOSAS instrument and were based on the fact that the surgical condition should be identifiable by the patient (hence, these exclude many internal conditions) and should be able to be confirmed by examination (objective assessment) in a culturally acceptable manner.[11]

Furthermore, it is worthy of note that not all surgically treatable conditions will be addressed in the survey, and therefore, the measured prevalence of surgically treatable conditions will be underestimating the real surgical burden of disease.[12] The developers of the instrument realize that ethical and cultural acceptability may be different for countries, and therefore, the SOSAS tool may need to be adapted to suit local customs wherever it is going to be used. It is meant to be open source, for anyone to use to determine the prevalence of surgically treatable conditions in LMICs or specific regions within them. It has been used in West Africa, and a pioneer study in Sierra Leone[12] has been replicated in Nepal, Rwanda,[13] and Uganda.[4]

This work is a modification of the SOSAS tool called the pediatric SOSAS to evaluate the need for surgery in the pediatric subpopulation. This modification includes three aspects which are to identify congenital anomalies in children and to do physical examination of children and use of picture portfolio which was not used in the previous study.[14]

Nonetheless, our study has its limitations: the low response rate of 17% of the total invited panelists in the first round which further decreased to about 10% in the second round has a tendency of biasing the SD away from the null. This implies that the consensus achieved after the second round might have been more significant if the response rate was higher. However, there was still consensus achieved for a majority of the statements which is evidenced by a decreasing mean SD in the second round.

The reasons for the low response rate of the e-Delphi methodology may be due to the unpopularity of the methodology for analysis among medical experts in our environment as evidenced by the paucity of literature utilizing it. The use of a wireless audience participation system at any gathering has made it possible to complete the various rounds in 1 day, thereby maximizing the response rate with advantages for both participants and the researcher.[10]

CONCLUSION

Based on the expert-generated consensus, we can conclude that the set of variables we achieved consensus on are necessary to be included in any community survey of pediatric surgical need in a LMIC. We believe that the findings of this Delphi survey could serve as a guideline to assist in setting out for a nation-wide community-based survey in Nigeria.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.