Low-cost simulation models in Urology: a systematic review of the literature.

INTRODUCTION: Simulation models have been found to be effective and valid for training in Urology. Due to increasing costs of surgical training, there is a need for low-cost simulation models to enable Urology trainees to improve their skills.
MATERIAL AND METHODS: A literature review was performed using the PubMed and Embase databases until March 2020. A total of 157 abstracts were identified using the search criteria, of which 20 articles were identified describing simulation models for Urology training. Articles reviewed described simulation models created from materials costing less than $150. Data was extracted from the relevant articles in order to critically assess each paper for validity, ease of construct and educational impact.
RESULTS: Models were found pertaining to suprapubic catheterization (6), cystoscopy (3), percutaneous nephrolithotomy (5), scrotal examination (1), circumcision (1), ureteroscopy (1), transurethral resection of the prostate and bladder (2), and open prostatectomy (1). 18/20 (90%) assessed for either face, content, or construct validity. None of the papers evaluated assessed for transferability of skills to performance in real patients.
CONCLUSIONS: A plethora of low-cost simulation models for urological procedures are described in the literature, many of which can be easily constructed from cheap and accessible materials. However there is a need for further efforts to validate or assess for transferability of skills to clinical practice. Copyright by Polish Urological Association.

INTRODUCTION

Simulation has become a cornerstone in medical education, particularly in the fields of surgery and clinical skills education, and there is now an increasingly reliable evidence base to argue for further inclusion in surgical curricula [1]. Unfortunately, only 53.3% of UK trainees in Urology have access to facilities that provide such simulation outside of a course setting [1]. Across Europe, the number of trainees with access to simulators in Urology has in fact decreased over recent years [2]. In Urology, 'boot camp' style courses in which trainees are supervised performing simulated procedural skills have been popular and there is some evidence to suggest that these increase both self-rated procedural confidence, as well as competence on objective assessment [3]. However, courses teaching procedural skills in Urology are often expensive, with a UK based ‘boot camp’ course costing participants an estimated £500 per participant [3] Over the course of training, the estimate of individual costs to surgical trainees from courses alone is around £9105 [4]. Such financial barriers to optimal training are detrimental to the budding Urologist, emphasising the evidence that high surgical training is fraught with hidden costs.

Outside of courses, there has been a drive to develop high tech simulation models specific to Urological training, with evidence to suggest that they improve dexterity and time taken to improve tasks, particular for those in early stages of their training [5]. However, these are once again prohibitively expensive for the average surgical trainee. The ADAM™ model for cystoscopic urologic surgery produced by Karl Storz®, Germany retails at around $4474, with such complete virtual reality systems proving to be generally more costly than box trainer counterparts.

Therefore, there is a need for low cost simulation models that trainees in Urology may use to improve their skills. In the literature there are a number of authors identifying low-cost models for simulation that can often be constructed from cheap and easily obtainable materials; these may be synthetic or animal in nature. This article will systematically assess these low-cost simulation models in Urology for educational value, validity, ease of construction and cost. Educational value was defined as an attempt to demonstrate that the simulation model produced an improvement in performance in trainees.

MATERIAL AND METHODS

A literature review was carried out using PRISMA guidelines. Databases searched included PubMed and EMBASE. Search terms included: ‘low-cost’ OR (‘low’ AND ‘cost’) AND ‘urology’ AND (‘simulation’ OR ‘model’). The literature search was performed by three of the authors independently, TP, NS and RP. References from the reviewed studies were used to supplement the search where appropriate. There were no conflicts on the suitability of studies for inclusion between reviewers.

Studies were excluded if the components of the model could not be realistically expected to be accessible or available. Examples of this included items generated by 3D printers, the cheapest of which retail at greater than $250 on Amazon®. All Virtual Reality simulation tools were excluded due to their relative expense. Laparoscopic simulators not specific to Urology were also excluded as these have already been reviewed thoroughly elsewhere [6, 7]. The authors defined ‘low cost’ as being less than $150, with studies describing models with costs greater than this subsequently excluded. Conference abstracts were excluded as they uniformly lacked the requisite detail for critical analysis.

The included studies were reviewed by all authors with the use of a standardised proforma, and data was extracted regarding the following parameters: total cost, model type (animal/synthetic/other), ease of construction, number of study participants and their grade/experience, educational value, face, construct and content validity. Costs were converted to USD ($) for ease of comparison.

RESULTS

157 studies were identified through searching the PubMed and EMBASE databases. A further 5 articles were identified through review of the selected articles references, bringing the total to 162. Of these 41 abstracts were identified as relevant. A further 21 of these studies were subsequently excluded as per the aforementioned criteria. This can be viewed in the flowchart in Figure 1, adapted from the PRISMA flowchart [8].

Figure 1

PRISMA flowchart demonstrating article selection.

Suprapubic catheterisation

A total of 6 studies were identified relating to low-cost simulation models of suprapubic catheterisation. The costs and advantages of these models have been outlined in Table 1. Nonde et al. [9] described a box-type model for suprapubic aspiration involving commonly encountered Emergency Department (ED) equipment. This appeared easy to construct in under 10 minutes, for an estimated price of around $1–2. The model was felt to demonstrate face validity especially in terms of simulating the initial outflow of urine. Construct validity was not assessed. A similar but arguably more complicated model was developed by Singal et al. [10] from moulded urethrane and resin foam and a silicone moulded bladder. The model was assessed by 6 experienced urologists, who gave the model high scores for its visual and sensory representation of skills such as trocar insertion. However it appears complex to construct, and it seems likely that specialist equipment or knowledge would be required. In UCH Ibadan, Nigeria, a box-type model was constructed from a box with 9 compartments, each with a water filled balloon covered in a trilaminar abdominal wall replica made up of leather and foam [11]. Despite the model achieving positive feedback from participants, it was only deemed by 10% to be realistic and representative of anatomical landmarks. The UroEmerge Suprapubic catheter model [12] was constructed out of a Perspex® box (Perspex International, US), 3L irrigation fluid bag, and ‘Limbs and things’ abdominal closure model. There was no discussion of validation of this particular model. Another box-trainer constructed of Tupperware® (Tupperware Brands Corporation, US), using sponge, mefix and transpore to replicate the layers of the abdominal wall was constructed for less than 3$ [13]. 24 of 25 participants felt that the model was representative of a bladder. However, it should be noted that the participants prior experience of this procedure was not documented, and thus the reliability of their assessment is in question. Pavlyogli et al. [14] developed an ultrasound compatible model out of a replica bony pelvis and using a gelatin mould and non-rebreather (NRB) mask packaging to simulate the layers of the abdominal wall. This model scored favourably in terms of face and content validity amongst participants with prior experience of suprapubic cystostomy (SPC) insertion, and remained low cost, at approximately $60.

Table 1

Summary of suprapubic catheterisation models

SPC	Paper	Cost	Easy to construct	Educational impact	Validity
					Construct	Face	Content
SPC	Nonde et al.	<$2	√			√
	Singal et al.	$31		√		√
	Olapade-Olaopa et al.	NA
	Shergill et al.	NA	√	√
	Hossack et al.	<$10	√			√
	Palvolgyi et al.	$60			√	√

SPC – suprapubic cystostomy

Cystoscopy

A total of 3 studies relating to low cost simulation in rigid and flexible cystoscopy were identified (Table 2). These simulation models generally required access to flexible or rigid cystoscopes which were not included in the cost of the simulation, although it was felt that these were likely to be accessible to most urology trainees in a given department. Persoon et al. [15] created a low cost model for flexible cystoscopy constructed from a glass globe shaped food container, demarcated with bladder landmarks. Statistically significant improvements were seen in only one simple flexible cystoscopy task (p = 0.046), although they were perceived to have caused more urothelial ‘traumas’ (p = 0.001). The participants in the control group of medical students self-assessed as having found the task more difficult, which in the absence of an objective increase in proficiency may represent overconfidence on the behalf of the intervention group. Face validity was not formally assessed. A similar style of bench model was developed using a balloon with bladder markings drawn on it [16], costing less than $2 to assemble. Twenty-nine obstetrics and gynaecology trainees with varying degrees of experience were randomised to practice using the model vs a session of didactic teaching before performing their acquired skills on cadaveric specimens. Participants in the intervention group demonstrated a statistically significant decrease in time taken to assemble the scope (p <0.005), although there was no difference in time taken to perform the cystoscopy. After the use of the didactic simulator model, there was no difference in the assessed performance of more senior or more junior participants, suggesting construct validity. A further box model for cystoscopy was created from vegetable components such as hollowed out pumpkins [17] which would cost less than $10. The study reports positive feedback from students regarding these simulation models, but any detailed discussion regarding attempts to formally validate or assess these are lacking.

Table 2

Summary of endoscopic urological procedure models

Endourology	Paper	Cost	Easy to Construct	Educational Impact	Validity
					Construct	Face	Content
Cystoscopy	Persoon et al.	<$8	√	√
	Bowling et al.	<$10	√	√	√
	Hammond et al.	<$15	√
PCNL	Hammond et al.	<$10	√	√		√
	Hacker et al.	<$10	√
	Ewald et al.	$60*				√
	Sinha et al.	<$5	√		√
	Vijayakumar er al.	<$10	√		√	√	√
Ureteroscopy	Matsumoto et al.	$15	√	√
Transurethral	Bach et al.	$40	√	√	√		√

Requires 5 models to be made to distribute cost

PCNL – percutaneous nephrolithotrypsy

Percutaneous nephrolithotomy

Five studies were found that looked at PCNL access using low cost models, with varying techniques of construction, costing and validity assessment. These are listed in Table 2.

Of the 5 studies, 3 were constructed using animal materials, namely porcine kidneys implanted into chicken carcasses. Hammond et al. [18] described one such model, with pre-implanted pebbles to simulate stones, with university residents then tasked with using fluoroscopic percutaneous renal access to manipulate and fragment the stones as necessary. The simulation cost was not explicitly listed, but would likely have been <$10 based on the materials described. Anonymous feedback reported a ‘high degree of satisfaction with the model’ in terms of its effectiveness, suggesting face validity, but showed no evidence of attempting to obtain content validity. Hacker et al employed similar techniques [19]. Again, no costs were mentioned, but likely to be the same as that in Hammond et al’s model. No evidence of validity assessment was shown. A further animal model was described by Vijayakumar et al [20]. The model was well validated, with a panel of experts scoring it 5/5 on a Likert scale for resemblance to the real scenario. It was also able to differentiate effectively between junior surgeons and experts, with novices requiring twice as many attempts to achieve successful puncture of the calyceal system (P <0.001), as well as taking twice as long to complete the procedure (P<0.001). It scored favourability in terms of face validity with a commercially viable mannequin model, although it was noted that there was some differences between the bovine renal anatomy in comparison with the human counterpart.

Ewald et al used entirely synthetic materials from which to construct their model [21]. Using gelatin, surgical gloves, a ballistic gelatin block, acrylic base and chalk, a collecting system was constructed within a polyurethane foam housing. The authors state each model would cost $60, although required 5 models to be constructed simultaneously to divide the costs. The model was not formally tested to assess educational impact, although attempts were made to validate the simulation model. A visual analogue scale assessed various aspects of the model ranging from adequacy of calyceal visualization to comparability to human tissue. Twenty-two participants rated accuracy of the simulation at 8.3, with attendings rating it 8.4 (out of 10). It is worth noting that this simulation looked at percutaneous access alone and did not simulate stone retrieval. One study used a vegetable model; Sinha and Krishnamoorthy used a ‘bottle gourd’ or ‘calabash’ to mimic the abdominal wall with cotton pledges soaked in intravenous contrast spaced at intervals to simulate calyces. Eight expert participants trialled the simulator, 3 of whom were consultants and 5 of whom were residents [22]. Construct validity was shown using reverse validation with senior consultants obtaining a score of 99 and residents scoring a mean of 555. The simulation is likely to be the cheapest of all PCNL simulators (<$5). Again, this simulator only assessed percutaneous access.

Scrotal Examination

Just one study was identified looking at models to simulate testicular swelling (Table 3). Sarmah et al. used synthetic and animal materials largely revolving around balloons, eggs and other household items to simulate common scrotal pathologies [23]. These included epididymal cysts, tumours and other common swellings. The estimated cost was $11.55, with the models used to teach medical students between years 3 and 5. Sixty-six students trialled the models, however as >80% of these had never examined a scrotum before, the data obtained is likely ill-representative of any form of validity.

Table 3

Summary of open urological procedure models

Open surgery	Paper	Cost	Easy to Construct	Educational Impact	Validity
					Construct	Face	Content
Scrotal	Sarma et al.	$11.55	√
Circumcision	Kigozi et al.	$5-10	√
Open (multiple)	Rowley et al.	<$10	√	√		√	√

*Requires 5 models to be made to distribute cost

Circumcision

One study was identified relating to low cost male circumcision simulation (Table 3). Kigozi et al. [24] describe the manufacture and design of a synthetic penis made from wood and cloth in a low-resource setting estimated to cost $5–10 per unit. While the authors claim the model ‘greatly facilitated the transition from theory to practice’ they do not include any evidence of its impact on education, surgical practice or local outcomes and therefore it remains unvalidated as an educational tool.

Ureteroscopy

One study was identified relating to low cost simulation of URS. Matsumoto et al. [25] compared endourological skill performance (removal of a ureteric stone) in a novice cohort who had received a didactic teaching session on either low or high fidelity simulation training. Design and construction of the low fidelity synthetic bench model simulator is described with unit cost estimated at $20 with no specialist equipment required. The low fidelity simulator was found to be equally effective in training novice users as the high fidelity equivalent. Content validity was not formally assessed. The analysis of this study is outlined in Table 2.

Transurethral surgery

Only one study described truly low cost transurethral resection simulation models (Table 2) [26]. It utilised a combination of synthetic and animal materials including a garden hose, Tupperware® box and a variety of meat products to construct a box trainer costing less than $40. The artificial model demonstrated an average of 50% improvement in time for task completion in trainees of all levels. There is good construct validity as the consultants initially performed much better than their more junior counterparts. Even after training, the gap between consultant and junior performance was reduced but still maintained, confirming the simulator’s suitability for training. Schout and colleagues describe a more cost effective pig bladder model, however, there was no attempt at validation of its use in training surgeons [27].

Open prostatectomy

Rowley et al. described the use of common household objects to recreate urology specific simulation models [28]. The most successful of their models was the open prostatectomy which was created using an ‘orange in a milk jug’ and would cost less than $10 to recreate. Seven urology residents (PGY2-5) felt confident that they could recreate the simulation themselves, that it was useful for their training, and that it was realistic in its simulation whilst improving their confidence in suturing a prostate in a real patient (Face Validity). The analysis of this study is outlined in Table 3.

DISCUSSION

This review has demonstrated the recent expansion in the literature regarding low-cost procedural simulation in Urology. The many innovative approaches highlighted in this are testament to the need for lower cost approaches to simulation in order to combat the rising costs of surgical training as a whole. Although 18/20 (90%) of articles assessed validity to an extent, a significant proportion of the models described in this study failed to formally assess validity across multiple domains or adequately demonstrate educational value. None of the reviewed articles were able to formally assess for transfer of skills to performance in real patients. As has already been identified in the literature, methods and definitions used to validate simulator models vary significantly, and many of the validation attempts to places in distracting and uncontrolled settings such as conferences [29].

Despite this, several of the assessed simulators were well validated and Nonde et al’s model for suprapubic catheter insertion, appeared easy to construct, cost effective and scored highly for face validity amongst expert users [9]. The ‘Tupper’ model for transurethral resection was comprised of common household materials but displayed good construct validity [26]. Bowling’s balloon-based rigid cystoscopy model was found to be effective in improving cystoscopic skills on cadaveric specimens in comparison with a control arm[16]. Persoon’s glass globe model for flexible cystoscopy, although not assessed for validity, has demonstrated some impact as an educational tool to enable novice trainees familiarise themselves with flexible cystoscopy [15]. These models could be applied together to create a low-cost urological simulation course for urological trainees, for a total cost of approximately $60.

There is some evidence to suggest that low fidelity simulation is comparable to high fidelity simulation in terms of impact on training [30]. As such, low cost models are likely to be particularly valuable to novice trainees in terms of learning key steps of unfamiliar procedures, as well as general motor skills prior to learning for the first time on a real patient. It is also important to note that low cost models require little in terms of expense or labour to create and operate, and are thus able to supplement education without requiring any sacrifice in terms of study budget or departmental resources. Particularly in the current context of the COVID-19 pandemic, where courses and training opportunities are often cancelled due to social distancing measures, learning opportunities which trainees are able to create for themselves may become invaluable.

This review has several limitations. Although multiple reviewers were used and several databases accessed, it may be possible that some studies describing other low-cost simulation models were not reviewed. The $150 upper limit for material expense could be argued to represent an arbitrary exclusion criteria. However, this was felt to be a value which could be affordable to an individual trainee at an early stage in their career, without relying on departmental resources. Where material cost was not explicitly listed, costs were estimated by the reviewers based on local and internet market price, which may of course vary geographically.

Educational interventions such as those listed in this article are likely to add to the body of evidence that suggests low cost simulation increases participant confidence in performing a procedure [9, 15]. However, without adequate independent assessment of the participant’s ability to perform such procedures, it remains difficult to ascertain if this is false confidence, or whether it relates to a genuine increase in procedural skill. Nonetheless, the nature of low-cost simulation means that whilst there is little time or expense lost in the creation of these models, there is potential for a vast array of skills to be gained.

CONCLUSIONS

There are a number of low cost simulation models described in the literature pertaining to an array of urological procedures: suprapubic catheterisation, cystoscopy, PCNL, scrotal examination, circumcision, transurethral and open prostatectomy. Further efforts are required to formally validate these, as well as assess the transfer of acquired skills to performance in real patients.

CONFLICTS OF INTEREST

The authors declare no conflicts of interest.