Development of tasks and evaluation of a prototype forceps for NOTES.

BACKGROUND AND OBJECTIVES: Few standardized testing procedures exist for instruments intended for Natural Orifice Translumenal Endoscopic Surgery. These testing procedures are critical for evaluating surgical skills and surgical instruments to ensure sufficient quality. This need is widely recognized by endoscopic surgeons as a major hurdle for the advancement of Natural Orifice Translumenal Endoscopic Surgery.
METHODS: Beginning with tasks currently used to evaluate laparoscopic surgeons and instruments, new tasks were designed to evaluate endoscopic surgical forceps instruments.
RESULTS: Six tasks have been developed from existing tasks, adapted and modified for use with endoscopic instruments, or newly designed to test additional features of endoscopic forceps. The new tasks include the Fuzzy Ball Task, Cup Drop Task, Ring Around Task, Material Pull Task, Simulated Biopsy Task, and the Force Gauge Task. These tasks were then used to evaluate the performance of a new forceps instrument designed at Pennsylvania State University.
CONCLUSIONS: The need for testing procedures for the advancement of Natural Orifice Translumenal Endoscopic Surgery has been addressed in this work. The developed tasks form a basis for not only testing new forceps instruments, but also for evaluating individual performance of surgical candidates with endoscopic forceps instruments.

INTRODUCTION

One recent and relatively disruptive medical advancement is Natural Orifice Translumenal Endoscopic Surgery (NOTES). NOTES procedures are conducted through a flexible endoscope, and at times with the assistance of a laparoscope, that is inserted into the body through the mouth, anus, vagina, or urethra. In addition to leaving no external scars, such procedures are anticipated to reduce the length of hospital stays, postoperative pain, and the risk of infection. NOTES at present is a quickly growing field in its early developmental stages. One of the significant obstacles for the advancement of NOTES is the current lack of sufficient instrumentation that is dedicated to the performance of complex procedures using a flexible surgical platform like an endoscope. The focus of this paper is 2-fold. The first objective is to develop a standardized set of tasks to assess both surgical skills and new surgical instruments for NOTES. The second is to utilize the assessment procedure to evaluate a new forceps prototype, via a side-by-side comparison with an existing endoscopic forceps instrument. Participant feedback is used to validate the prototype performance and, if necessary, to improve the design.

The common starting point for the evolution of NOTES is the January 2005 joint meeting of members of the Society of American Gastrointestinal Endoscopic Surgeons (SAGES) and a group of expert endoscopists representing the American Society for Gastrointestinal Endoscopy (ASGE). Important developments from this meeting were later published.[1] At this meeting, it was determined that endoscopes suitable for NOTES procedures already exist, although modifications were suggested, but the additional equipment needed to perform such procedures was limited to nonexistent.[1] The importance of overcoming these obstacles has been reaffirmed in the literature since then.[2-4]

To address the need for standardized tasks for NOTES, it is necessary to begin with testing used in other areas of medicine. The history of training and testing students in laparoscopic procedures is of particular importance, because it forms a basis for the tests developed here. Students who wish to become laparoscopic surgeons must undergo training that includes written standardized tests, subjective faculty evaluations, and technical testing via standardized tasks that measure a student's ability to perform different surgical tasks in a box trainer.[5-7] This work aims to modify and adapt several existing technical testing tasks to NOTES training. Because of the commonality of instrument requirements for NOTES, laparoscopic and endoscopic procedures, laparoscopic and endoscopic testing procedures were reviewed. Of the 6 tasks created here for NOTES, 3 were adapted from either the standardized laparoscopic technical training procedure (“Rosser station tests”) or an endoscopic testing procedure described in the literature.[5,8-10] The Rosser stations comprise a standard set of tests used for technical skills training of laparoscopic surgeons and instrument testing.[5] Several tasks, eg, the Cup Drop, are performed by students in a box trainer and timed for comparison amongst students.[5] The importance of Box Trainer testing was discussed by Rosser and has since been the topic of several other papers.[6,7,11,12] Other tasks that are described in the literature include the Sea-Spike Task and the Fuzzy Ball Task, which evaluate specific functions of the new prototype instrument.[8-10] Side-by-side comparison of surgical instruments is common with new designs and has numerous precedents.[8-10,13,14] The testing described in this article is based on similar testing procedures used to determine the adequacy of surgical instruments described in the literature.[10]

The second objective of this work focuses on the need for new instrumentation for use in NOTES. As progress has continued in the field of NOTES, many proposed procedures have yet to become a reality, because technological advancements have yet to yield the necessary instruments.[15] This is not to say that new instruments have not been developed, when in fact, many instruments, such as new endoscopes, have been designed and many more are currently under development.[4] Specifically, tools that can be used through 2-mm to 3-mm channels of the 100-cm to 200-cm flexible scopes are needed. Flexible tools in the millimeter to meso-scale, capable of accomplishing surgical tasks, are mostly nonexistent.

This general lack of adequate instrumentation has been acknowledged by the field, and in particular the lack of endoscopic suturing devices, scissors, and graspers has significantly slowed the advancement of NOTES.[15,16] Addressing the need for scissors and graspers is the main focus of the second objective of the work presented here. The specific prototype device that this research deals with is a 1.0-mm diameter forceps/scissors currently being developed at the Pennsylvania State University. While the instrument design provides for dual function as a forceps and a scissors, this article focuses only on the forceps functionality.[17-22]

METHODS

A set of 6 tasks was defined as NOTES tasks. These tasks include the Fuzzy Ball (A), Cup Drop (B), Ring Around (C), Material Pull (D), Simulated Biopsy (E), and Force Gauge (F) tasks, as presented in . The 3 tasks adopted from the literature were either directly utilized or modified to test instruments at this particular scale. The first of these adopted tasks is the Fuzzy Ball task ([A]). This task features a 1.5-cm diameter plush “fuzzy” ball that is repeatedly picked up and moved with each instrument for comparison. The Fuzzy Ball Task evaluates an instrument's dexterity and ability to grasp fine objects and was adopted from work by Aguirre et al,[19] but with one modification; in the present task only one instrument is used at a time through a single-channel endoscope. The version of the Fuzzy Ball Task presented in the literature requires that the ball be passed between 2 forceps instruments, which is not possible with a single-channel endoscope. The second adopted task is the Cup Drop Drill, which was adopted from laparoscopic testing, but with one modification. Instead of dried beans, which are too large for the endoscopic instruments under consideration here, 0.5-cm diameter fuzzy balls are used. These balls are similar to those used in the Fuzzy Ball task, except they are significantly smaller ([B]). Other Rosser station tasks could not be incorporated into the new task list for NOTES due to the inability to use multiple instruments simultaneously through a single-channel endoscope. The third task is a modified version of the Sea-Spike task that appears in the literature. In laparoscopic testing, this task involves multiple rings that are carefully positioned on asymmetric spikes and then removed with each instrument.[11,15] However, these rings are too thick for use with the endoscopic forceps. Therefore, a new apparatus was constructed that features a medium-gauge sewing needle, which acts as a spike, inserted into a block of wood at approximately a 10-degree angle to the horizontal ([C]). While completing this task, 0.75-cm diameter brass rings are picked up with each instrument and placed around the needle. This modified version of the sea spike drill is called the Ring Around Drill. Both the Cup Drop Drill and the Ring Around Drill are designed to test an instrument's ability to grasp and maneuver an object to a desired location.

Standardized NOTES Tasks in an Endoscopic Box Trainer: (A) Fuzzy Ball Task; (B) Cup Drop Task; (C) Ring Around Task; (D) Material Pull Task; (E) Simulated Biopsy; (F) Force Gauge.

In addition to adopting and modifying existing tasks, 3 new tasks were created to assess additional features of endoscopic forceps instruments. The first new task is the Material Pull, which is designed to simulate a forceps' ability to grasp and manipulate tissue by removing 2 pins from a piece of foam. Each pin is attached to a piece of 0.1-cm thick suede leather in a cone-like shape ([D]). The piece of suede roughly simulates tissue, while the motion involved in removing the pins simulates manipulating or tearing tissue. The second new task is the Simulated Biopsy task, which is designed to simulate tissue removal during a biopsy. This task involves a block of soft foam with multiple dots, each 0.2cm in diameter, drawn on it. The aim of the task is to completely remove only the material colored with the dot ([E]). The number of attempts required to completely remove the dot from the foam was recorded. The average number of attempts necessary to completely remove the dot was then used as a quantitative comparison between the instruments. The final task was the Force Gauge task. This task is designed to assess the grasping force of the forceps by measuring the maximum pull-off force of the forceps while it is used through the endoscope. During this task, the forceps is used to firmly grasp a piece of latex surgical tubing (outer diameter of 6.35mm and wall thickness of 1.59mm) that is connected to a force gauge. Once the tubing is grasped in the jaws, the forceps is then pulled away from the tubing until the tubing slips out of the jaws ([F]). The maximum measured force is defined as the maximum pull-off force.

The 6 tasks were used to evaluate 2 endoscopic forceps instruments. The first instrument is a prototype currently being designed at the Pennsylvania State University (. This 1.0-mm diameter forceps has been optimally designed to provide both a suitable opening at the jaws and grasping force while being able to be fabricated in large arrays using a new process called lost mold rapid infiltration forming.[19,20] The second instrument is a recently available comparable device, SpyBite φ1.0 mm, a single-use biopsy forceps instrument (.

Forceps Instruments in both Fully Open and Fully Closed Positions: (A) PSU 1.0 mm prototype forceps in the fully open position; (B) PSU 1.0 mm prototype forceps in the fully closed position; (C) SpyBite 1.0 mm single use forceps instrument in the fully open position; (D) SpyBite 1.0 mm single use forceps instrument in the fully closed position.

The forceps instruments were used and evaluated in an endoscopic box trainer by 12 test subjects who were recruited for the experiment. All subjects were practicing surgeons, gastroenterologists, or residents training with them. Before beginning the procedure, each participant viewed an instructional video demonstrating the tasks, read a written description of the research, and gave written consent to be recorded on video. During the testing, information was gathered by 3 video recording devices. The first device was a standard high-definition digital video recorder that was placed in the room and recorded the motion and activity outside of the box trainer as well as any comments made by the participants. The second video recorder was placed inside the endoscopic box trainer and recorded the actual use of the device from a fixed location. The third video device was the endoscope camera itself. The image recorded by the endoscopic camera was displayed on the endoscope tower and was recorded by a computer; this was the only video feed provided to the participants for completing the tasks. shows an example image of the video recorded through the endoscope, and shows an example image recorded by the stationary camera in the box. Both images were taken at the same time from the 2 different video sources.

Example Image Taken From Video Recorded Through the Endoscope.

Example Image Taken From Video Recorded by the Stationary Camera in the Box Trainer.

The video images were analyzed to help identify the approaches used by participants who found the tasks easier or got superior results. This information could then be used to infer whether any reported difficulty with a particular task stemmed from the instruments themselves or the approach used by the participants. The data pertaining to the number of attempts to remove the dot in the Simulated Biopsy task and the maximum pull off force as determined in the Force Gauge task were recorded manually by the researcher. In addition to recording data, the researcher served as the assistant to the participant, advancing and opening/closing the instrument in response to verbal commands from the participant.

The participants completed the testing one at a time. Each participant was asked to perform the tasks using an OLYMPUS Evis Exera GIF/CE/RCF Type 160 Series Endoscope ( [A]) in an endoscopic box trainer on an endoscopic tower ( [B]). The tasks were performed in the following order: Material Pull, Ring Around, Fuzzy Ball, Cup Drop, Simulated Biopsy, and Force Gauge task. All tasks were performed using the standard instrument first, followed by all 6 tasks using the prototype instrument. Immediately after the tasks were performed with both instruments, participants were asked to complete a written survey, which can be found in Appendix A. This survey was designed to assess the relative strengths and weaknesses of the instruments by asking participants to rate various aspects of each on a scale from 1 to 3, with 3 being superior and 1 being poor performance. The survey also solicited open-ended feedback and suggestions from the participants about both the prototype device design and the task list. This quantitative and qualitative data were then analyzed to determine what, if any, changes were necessary for the prototype design. In addition to gathering data about the performance of the prototype forceps, the surveys and videos also helped to assess the testing procedure and the tasks.

Endoscope, Endoscopic Box Trainer and Endoscopic Tower: (A) Endoscope and Box Trainer; (B) Endoscopic Tower and Video Monitor.

RESULTS

The number of attempts necessary to completely remove the dot during the simulated biopsy task and the maximum pull-off force measured during the force gauge task are reported in . There is little difference in the average number of attempts to remove the dot, with the standard instrument requiring an average of 4.73 attempts and the prototype instrument requiring an average of 4.36 attempts. It can be seen that there is a small difference in the average pull-off forces of the 2 instruments. The average pull-off force of the standard instrument was 0.926 N, and the average pull-off force of the prototype instrument was 0.761 N. The maximum measured pull-off forces using both instruments were identical (1.461 N).

The survey data on the adequacy of instrument features is summarized in . It can be seen that the prototype instrument was rated slightly better than the standard instrument in terms of pull-off force, with average ratings of 2.67 and 2.36 for the prototype and standard instruments, respectively. The prototype instrument was also rated somewhat higher, on average, in terms of the ability to control the open/closed positions of the jaws, with average ratings of 2.55 and 2.40 for the prototype and standard instruments, respectively, The standard instrument was rated slightly better in terms of amount of material removed in the simulated biopsy task, with average ratings of 2.17 and 2.64 for the prototype and standard instruments, respectively.

The relative ease of use of each instrument is reported in , where the average rating of the prototype instrument was 2.42, compared to an average rating of 2.27 for the standard instrument. The number of times the instruments' features were identified as making the tasks easy and difficult is reported in and , respectively. The distal jaw opening of the standard instrument was identified as making the tasks easier than the prototype instrument (28 compared to 15 times), whereas the jaw length and the ability of the prototype instrument to grasp firmly were identified as making tasks easier than that of the standard instrument (23 and 52, compared to 13 and 29, respectively). Similar trends are seen in , where the jaw opening of the prototype instrument and the standard instrument's jaw length and ability to grasp firmly were identified as making the tasks more difficult. However, there was not a large difference in the average relative importance of each feature, with the jaw length, jaw opening, and ability to grasp firmly being rated at 2.0, 2.1, and 2.4, respectively (.

A summary of the written comments made on the survey can be found in . This table lists the comments made in order of descending prevalence, as well as the number of times that participants made the comment or a similar comment. The verbal comments recorded during the testing were very similar to the written ones made on the surveys.

CONCLUSION

The quantitative survey results were consistent with the verbal and written comments. The comments indicate that the prototype instrument was preferred over the standard instrument in terms of the ability to control intermediate positions between the open and closed positions of the jaws. This comment is consistent with the data in , where the ability to control intermediate positions between the open and closed positions of the jaws is rated slightly higher for the prototype instrument. The comments also indicate that the prototype instrument's ability to grasp firmly is superior to that of the standard instrument. This comment is consistent with the data in , where the ability to grasp firmly was identified as making the tasks easy with the prototype instrument far more than with the standard instrument. However, the participants commented that the prototype is not well suited for biopsies, which is consistent with the standard instrument being rated higher than the prototype instrument in terms of amount of material removed (. Both the comments in and the data in indicate that the distal jaw opening of the prototype instrument is too small. Participants also commented that the prototype's jaw length is too short and should be increased; having longer jaws would further improve fine control and the ability to utilize intermediate positions. Another participant suggested that the prototype be made easier to rotate about its axis, which would make accurately positioning the jaw opening easier.

The feedback also included 2 comments regarding the visibility of the tip of the prototype. One comment states that the participant found it difficult to see if the jaws of the prototype instrument were open or closed. A prototype that could more easily rotate about its axis would make it easier to visually distinguish whether the forceps was fully open. The second comment indicated that the participant found it difficult to see around the actuation tube. This was largely due to epoxy present on the end of the tube nearest the camera. However, the epoxy would not be present on any future versions of the prototype.

Participant feedback regarding the testing procedure included comments that the testing procedure and the task list succeeded in evaluating the important features and abilities of the endoscopic forceps instruments, and that the testing was challenging and forced the participants to concentrate on the tasks at hand.

In summary, a new NOTES Task List, testing procedure, and NOTES instrument survey were developed. The protocol allows for objective testing of NOTES instruments and surgical skills. While extensive research has been conducted in theorizing and performing new NOTES procedures, little has been done previously to address the lack of adequate testing and evaluation methods. The work presented here helps to fill this void and establishes a new standard method for evaluating NOTES forceps. In addition, the new task list forms a basis that can be used to develop standard training methods for other instruments and eventually for interinstitutional comparisons. However, improvements should be made to the testing procedure. While most of the tasks on the task list effectively evaluated the intended features, the Force Gauge task should be improved. The data for this task exhibited significant scatter and little repeatability due to the wide range of approaches used by participants to grasp the rubber tubing. In the future, the force gauge should be mounted such that it is aligned with the entrance to the box trainer, rather than parallel to the bottom of the box trainer. The gauge should also be placed such that the length of the endoscope within the box trainer is minimized. These changes are expected to reduce the scatter in the force data by reducing the variability in the approach to the force gauge.

The prototype instrument was successfully tested by 12 test subjects, and their feedback on its relative strengths and weaknesses compared to those of a commercially available endoscopic forceps has been evaluated. The participants particularly appreciated the prototype's grasping force and its ability to open/close at intermediate positions. Though the prototype did not perform well at the biopsy task, it excelled at fine grasping. The prototype's distal jaw opening and jaw length should be increased to further improve its ability to operate at intermediate positions and to grasp larger objects. In addition, the instrument should be made easier to rotate around its longitudinal axis, facilitating improved visibility and grasping objects from various directions. When these changes are made, the prototype instrument design has the potential to be very useful in NOTES.

Table 1.

NOTES Tasks and Their Corresponding Figure Numbers

Task Name	Figure No.
Fuzzy Ball	1 (A)
Cup Drop	1 (B)
Ring Around	1 (C)
Material Pull	1 (D)
Simulated Biopsy	1 (E)
Force Gauge	1 (F)

Table 2.

Number	Participant ID#	#Attempts to Remove Dot		Pull-Off Force (N)
		Standard	Prototype	Standard	Prototype
1	2	5	7	0.646	0.728
2	22	5	5	1.080	1.461
3	14	5	5	0.801	1.278
4	17	4	6	0.857	0.715
5	13	5	5	1.045	0.400
6	23	5	3	1.461	1.129
7	5	4	3	0.890	0.690
8	3	Unable to Complete	Unable to Complete	0.712	0.316
9	8	5	3	1.037	0.316
10	15	5	4	1.422	1.009
11	14	4	3	0.691	0.701
12	12	5	4	0.473	0.386
		Mean (SD)
		4.73 (0.45)	4.36 (1.30)	0.926 (0.287)	0.761 (0.369)

Table 3.

Average Adequacy of Features of Both Instruments

	Mean (SD)
Standard Instrument
Maximum Pull-Off Force	2.36	0.643
Amount Of Material Removed	2.64	0.481
Ability to Control and Operate Device at Intermediate Jaw Openings Between Fully Open and Fully Closed	2.40	0.663
Prototype Instrument
Maximum Pull-Off Force	2.67	0.624
Amount Of Material Removed	2.17	0.687
Ability to Control and Operate Device at Intermediate Jaw Openings Between Fully Open and Fully Closed	2.55	0.498

Table 4.

Overall Mean (SD)

Standard Instrument	2.27 (0.731)
Prototype Instrument	2.42 (0.705)

Table 5.

Ability to Grasp Firmly was Identified As Making the Tasks Easy with the Prototype Instrument Far More Than with the Standard Instrument

Standard Instrument
Jaw Length	13
Distal Jaw Opening	28
Ability to Grasp Firmly	29
Prototype Instrument
Jaw Length	23
Distal Jaw Opening	15
Ability to Grasp Firmly	52

Table 6.

Distal Jaw Opening of the Prototype Instrument is Too Small

Standard Instrument
Jaw Length	20
Distal Jaw Opening	15
Ability to Grasp Firmly	11
Prototype Instrument
Jaw Length	11
Distal Jaw Opening	31
Ability to Grasp Firmly	4

Table 7.

Overall Average Relative Importance of Different Features

		Mean (SD)
Jaw Length	2.0	0.766
Distal Jaw Opening	2.1	0.771
Ability to Grasp Firmly	2.4	0.782

Table 8.

Survey Comments for Prototype Forceps

Paraphrased Comments	Number of Times Each Comment or a Similar Comment Was Made
Distal jaw opening is too small	5
Prototype jaw length is too short	4
Control of intermediate distances is essential, and this ability is better in the prototype instrument than the standard instrument	3
Prototype should be rotatable to allow for better grasping control	3
Prototype is not well suited for use in biopsies	2
The actuation tube is bulky and impairs vision	1
Prototype's ability to rotate is helpful	1
Prototype is better at grasping objects	1
Prototype instrument allows for more fine control	1
Smaller distal jaw opening allows for more precision when grabbing	1
The standard instrument features a visual cue as to whether the instrument is opened or closed and this feature should be incorporated into the prototype	1