First Italian validation of the "Satisfaction with simulation experience" scale (SSE) for the evaluation of the learning experience through simulation.

BACKGROUND AND AIM OF THE WORK: Training in simulation through "mannequins" is increasingly widespread among nursing students. In the Italian context, however, there are no tools that measure the degree of student satisfaction after clinical training through simulation. The aim of the study is to provide a first validation in Italian of the Satisfaction with simulation experience" (SSE) scale, a tool already validated in several languages.
METHODS: After obtaining the author's consent, the SSE was subjected to forward and backward translation. The content validity was assessed by 5 training experts by calculating the Content Validity Index by Item and by Scale (I-CVI and S-CVI); the face validity was tested on 4 nursing students who had participated in a simulation experience. Subsequently, the SSE was administered to 10 nursing students with test-retest after 7 days in order to evaluate the reliability by calculating the reliability coefficient (r) and Cronbach's α.
RESULTS: The author approved the final version of the SSE translated into Italian: I-CVI values>0.80 and S-CVI was 0.94. r is 0.88 and the α of the scale is 0.713.
CONCLUSIONS: The detected values of I-CVI and S-CVI are satisfactory, demonstrating the validity of the content of the SSE-ITA. The test-retest showed "optimal" reliability and the α was considered acceptable given the little deviation from the original (0.776). Although the results demonstrate satisfactory values, this is a first validation and other studies with larger samples are needed.

Background

The simulation consists in trying to recreate a scenario as realistic as possible (1) in order to teach the techniques and evaluate the performances that require a high degree of accuracy and sensitivity in the procedure (2). Its use dates back to the early 1900s in aviation, when the first flight simulator was used to reproduce real-world emergency situations, which otherwise would not have been possible to experience, in the absence of risk (3).

Simulation in the health sector has a long history, the first traces of which date back to 1920. In the last decade it has assumed a central role in the design of training courses in health care (4).

The simulation in healthcare training involves techniques ranging from low, to medium, up to high fidelity. The low-fidelity simulation uses static and non-reactive dummies, while the medium-fidelity simulation provides only basic human reactive functions such as heart rate or blood pressure detection (5, 6). In the high-fidelity simulation, mannequins connected to software are used that allow to recreate a complex clinical situation and to provide physiological, pathological and pharmacological parameters in real time. The learners participating in the experience are equipped with a microphone and are filmed, to allow a poster view of their work and to allow, even those who are not directly involved, to be involved (6, 7).

The primary purpose of simulation in healthcare is to integrate theoretical learning with practical experience, greatly increasing the learning potential. Some studies have shown that passive learning does not allow to learn more than 20-30% while active learning, or simulation, allows to reach a learning of up to 90% (7, 8). Simulation also allows experimenting without putting patients at real risk: the reproduction of complex clinical situations and their management in a team are an effective method to reduce errors attributable to the human factor (10).

There are several advantages that a training simulation can offer, such as improvement in the approach to the patient, learning without harming (11); improvement in team management, transmitting the skills of the decision-making process, favoring communication (6, 12–16) and collaboration (6, 11, 15, 16). By stimulating self-reflection (11), simulation also promotes self-confidence and self-efficacy (17). Finally, at the educational level, a strengthening of clinical knowledge was found (12–16). The simulation environment allows the learner to implement the technical skills until then learned only theoretically, in a safe environment, without worries about the progress of the task, in which the student feels free to make mistakes (18–20). Any errors made during the exercise are the focus of the debriefing, as it allows students to analyze the error, to understand it and to improve their future performance, in order to address their strengths and weaknesses before application of knowledge and skills in the workplace (17, 21, 22).

However, simulation also has some disadvantages, such as the fact that it takes a long time to develop and implement and can only be delivered to a small group of students at a time (6). Furthermore, some studies on the use of simulation report a scarcity of published articles on the subject carried out and carried out on adequate samples (23), as well as a lack of analysis of the costs that should be incurred and the need to produce reports on the effects in the short and long term (24).

In the cycle of studies of the three-year degrees of the health professions, simulation is often used in order to perform techniques and maneuvers learned in frontal lessons (6, 25), but in the Italian context there is still no tool for assessing student satisfaction.

Faced with this lack, we have decided to translate and adapt the “Satisfaction With Simulation Experience” (SSE) scale (26–29) to the Italian context, to carry out its first validation. The scale is of Australian origin (26), consists of 18 questions and has been validated in other languages (27–29). The tool does not measure the participant’s emotional sphere and non technical skills; in fact, it is not asked if the student was happy to collaborate in the group, or if and what were his difficulties. The scale instead refers purely to the satisfaction detected during the clinical component of the simulation, for example it is assessed if the student has received adequate feedback, if the simulation has allowed to test his skills and if the experience was positive from the point of view of clinical learning.

Methods

Participants and sampling

The study provided a sample made up of second year students of the Bachelor of Nursing Study (BNS) of the University of Parma based in Piacenza. Students were selected through convenience sampling. For this first phase of validation, the students of the second year of the course were involved because they were the only ones to have taken part in a simulation training at the Clinical Organizational Simulation Area (Area di Simulazione Clinico Organizzativa “ASCO”) (30). In Emilia-Romagna, at the hospital of Castel San Giovanni (Piacenza) there is in fact a simulation center (ASCO) which is aimed at private companies, healthcare companies, specialization schools and scientific societies that ask to participate in courses. ASCO provides the opportunity to participate in simulations using a high fidelity mannequin (30).

Tool

The SSE aims to assess the satisfaction of nursing students after a clinical learning experience through simulation with high and medium fidelity mannequins (26).

SSE is a 5-point Likert scale (Strongly disagree - disagree - not sure - agree - completely agree) consisting of 18 items that investigate 3 areas of the simulation experience: the first is “Debriefing and reflections “And is composed of 9 items that aim to question the validity and importance of the debriefing and moments of reflection (eg: “I had the opportunity to reflect on and discuss my performance during the debriefing”, “I received feedback during the debriefing that helped me to learn”); the second consists of 5 items and is intended to evaluate the effectiveness of the simulation on “Clinical Reasoning” (eg: “The simulation developed my clinical reasoning skills”); the third area concerns “Clinical learning”, consisting of 4 items and evaluates whether clinical skills have been acquired (eg: “The simulation helped me to apply what I learned from the case study”).

Traduction

The author of the SSE, Tracy Levett-Jones, was requested to authorize the adaptation of the instrument for the Italian context. Once the consent to use the SSE was obtained, a forward and backward translation was carried out (31, 32). Initially, two independent translations from English to Italian of the scale were made. The subjects who carried out the two translations had never seen the SSE before, had no contacts and had different cultural backgrounds. The two translators had TOEFL (Test of English as a Foreign Language) certification for the English language.

The two translations were compared and discrepancies were resolved, resulting in a third version.

The final Italian version has been retranslated into English by a language graduate. The English translation was submitted to the author’s attention for an opinion.

Face and content validity

The validation phase consists of the evaluation of the face validity (33, 34) and of content validity (32).

In order to test the face validity, 4 second year students of the SNB were involved who took part in the ASCO project (30) to which they were given the translated SSE. Subsequently, each student was offered a short semi-structured interview in which he asked if he had found difficulty in answering the items, if some items were confused or unclear, if the scale contained inadequate or offensive language or questions. Before proceeding to the interview, the purpose of the study was explained to the students and a consent was obtained for participation.

The validity of the content was assessed thanks to the contribution of a group of 5 experts in the field of training, who were given a 4-point Likert scale (from 1 “very significant” to 4 “not at all significant”), which ranged from to assess the relevance of each item of the translated SSE. In particular, the experts were: an head director of a BNS; a didactic tutor from a BNS with 25 years of experience; a physician expert in innovative teaching methodologies; a nurse from a local emergency service expert in training professionals and volunteers through simulation and a contract professor at a BNS; an expert in simulation and management of emotions related to the ASCO project. The results were then analyzed through the use of the Content Validity Index by item and by scale (I-CVI and S-CVI) (31). The experts were also able to give an opinion on the terms used and the position of the items on the scale.

From the changes made, the final SSE-ITA version was obtained, which was subsequently tested for reliability.

Reliability

Reliability was assessed by verifying the scale stability and its internal consistency.

The methodology used to verify stability was that of the test-retest which consists in administering the instrument to the same respondents at two different times. In this case, 10 SNB second-year students were involved who took part in the ASCO experience, who were asked to consent to participate. The scale was submitted to the same students one week apart, the results were then analyzed according to the reliability coefficient “r”.

The internal consistency was evaluated by calculating the Cronbach’s α index.

Data analysis

The scores of the Likert scale administered to the experts were divided into “relevant” for scores attributed to each item of 1 and 2 and “not relevant” for scores of 3 and 4. Subsequently the number of “relevant” for each item was divided by the total number of experts obtaining the value of I-CV. The mean of all I-CVIs determined the S-CVI value (35). I-CVI 0.78 and S-CVI 0.90 were considered acceptable (32).

The reliability of the tool was evaluated through the test-retest and by calculating the reliability coefficient “r”, if r > 0.70 the value was considered adequate, if r > 0.80 was considered optimal (36).

Internal consistency was evaluated by calculating Cronbach’s α and values were considered excellent if ≥ 0.90, good if ≥ 0.80, acceptable if ≥ 0.70, questionable if ≥ 0.60, poor if ≥ 0.50 and unacceptable if <0.50 (37).

Results

Table 1 shows the final version of the scale approved by the author (Table 1), which found no significant discrepancies between the original and the English version obtained from the Italian translation.

Table 1.

Original and italian version of SSE and α values

Original Version		Italian Version
Debrief and reflection		Debriefing e riflessioni
01	The facilitator provided constructive criticism during the debriefing	Il facilitatore ha provveduto a fornire critiche costruttive durante il debriefing
02	The facilitator summarised important issues during the debriefing	Il facilitatore ha riassunto le questioni più importanti durante il debriefing
03	I had the opportunity to reflect on and discuss my performance during the debriefing	Ho avuto l’opportunità di riflettere e discutere della mia performance durante il debriefing
04	The debriefing provided an opportunity to ask questions	Il debriefing ha previsto opportunità di porre domande
05	The facilitator provided feedback that helped me to develop my clinical reasoning skills	Il facilitatore ha provveduto a fornire feedback che mi hanno aiutato a sviluppare le mie capacità di ragionamento clinico
06	Reflecting on and discussing the simulation enhanced my learning	Le riflessioni e la discussione riguardo la simulazione hanno rafforzato il mio apprendimento
07	The facilitator’s questions helped me to learn	Le domande del facilitatore mi hanno aiutato ad apprendere
08	I received feedback during the debriefing that helped me to learn	Durante il debriefing ho ricevuto feedback che hanno favorito il mio apprendimento
09	The facilitator made me feel comfortable and at ease during the debriefing	Il facilitatore mi ha fatto sentire a mio agio e tranquillo/a durante il debriefing
	α 0.935	α 0.745
Clinical reasoning		Ragionamento clinico
10	The simulation developed my clinical reasoning skills	La simulazione ha sviluppato la mia capacità di ragionamento clinico
11	The simulation developed my clinical decision making ability	La simulazione ha sviluppato la mia capacità decisionale nella clinica
12	The simulation enabled me to demonstrate my clinical reasoning skills	La simulazione mi ha consentito di dimostrare le mie capacità di ragionamento clinico
13	The simulation helped me to recognise patient deterioration early	La simulazione mi ha aiutato a riconoscere precocemente il peggioramento del paziente
14	This was a valuable learning experience	Questa è stata una importante occasione di apprendimento clinico
	α 0.855	α 0.690
Clinical learning		Apprendimento clinico
15	The simulation caused me to reflect on my clinical ability	La simulazione mi ha spinto a riflettere sulle mie abilità cliniche
16	The simulation tested my clinical ability	La simulazione ha testato le mie abilità cliniche
17	The simulation helped me to apply what I learned from the case study	La simulazione mi ha aiutato ad applicare quello che ho imparato dal caso affrontato
18	The simulation helped me to recognise my clinical strengths and weaknesses	La simulazione mi ha aiutato a riconoscere i miei punti di forza e di debolezza riguardo le mie abilità cliniche
	α 0.850	α 0.635
	α tot. 0.776	α tot. 0.713

The Italian version therefore proved to be faithful to the original from a lexical point of view.

With regard to the analysis of the validity of the content, I-CVI values were obtained, all above 0.80, the S-CVI value obtained from the average of all the coefficients was equal to 0.94 (Table 2).

Table 2.

Item Score-Content Validity Index (I-CVI) e Scale-Content Validity Index (S-CVI)

ITEM	Expert 1	Expert 2	Expert 3	Expert 4	Expert 5	I-CVI
1	X	X	X	X	X	1,00
2	X	X	X	X	X	1,00
3	X	X	X	X	X	1,00
4	X	X	X	X	X	1,00
5	X	X	X	O	X	0,80
6	X	X	X	X	X	1,00
7	X	O	X	X	X	0,80
8	X	X	X	O	X	0,80
9	X	X	X	X	X	1,00
10	X	X	X	X	X	1,00
11	X	X	X	X	X	1,00
12	X	X	X	X	X	1,00
13	X	X	X	X	X	1,00
14	O	X	X	X	X	0,80
15	X	X	X	X	X	1,00
16	X	X	X	X	X	1,00
17	O	X	X	X	X	0,80
18	X	X	X	X	X	1,00
					S-CVI	0,94

X “Relevant” O “Not relevant”

Regarding face validity, come out observations about the choice of the lexicon and about the position of some items within the scale. Each observation was considered and evaluated by the authors of this study and the students interviewed:

The term “facilitatore” and item 17 “La simulazione mi ha aiutato ad applicare quello che ho imparato dal caso affrontato” were unclear, but the comparison with the students suggested not to change them.

Item 8 has been changed from “Durante il debriefing ho ricevuto feedback che mi hanno aiutato ad imparare” to “Durante il debriefing ho ricevuto feedback che hanno favorito l’apprendimento”.

Item 14 has been changed from “Questa è stata un’importante occasione di apprendimento” to “Questa è stata un’importante occasione di apprendimento clinico”.

Item 18 has been changed from “La simulazione mi ha aiutato a riconoscere i miei punti di forza e di debolezza riguardo la clinica” to “La simulazione mi ha aiutato a riconoscere i miei punti di forza e di debolezza riguardo le mie abilità cliniche”

The study of face and content validity shows that the scale is clear and immediate and that there were no offensive assertions.

With regard to stability, the Coefficient of Reliability was r = 0.88 (Table 3).

Table 3.

Reliability coefficients for item and total score

Item	r test - retest
1	0,42
2	0,82
3	0,82
4	0,80
5	0,80
6	1,00
7	0,80
8	1,00
9	0,60
10	0,82
11	0,82
12	0,51
13	0,51
14	0,56
15	0,52
16	0,51
17	0,65
18	0,61
r tot.	0,88

Regarding the Cronbach internal consistency α it was equal to 0.745; for the “Debriefing and reflections” block, equal to 0.69, for the “Clinical reasoning” block and equal to 0.635 for the “Clinical learning” block. The α of the scale considered as a whole was equal to 0.713 (Table 1).

Discussion

The educational path for students of the health professions aims at developing cognitive, gestural, relational and clinical judgment skills (38).

Simulation is an effective active teaching methodology that provides students with an authentic clinical experience in a controlled environment that is risk-free for both patients and themselves (17, 36). In the training of health professionals, simulation is increasingly used in order to increase the required skills, self-efficacy and self-confidence (40).

The experience of learning through simulation must also be assessed in terms of satisfaction as well as in terms of acquired skills. For this reason the SSE was developed (26) and for this reason the aim of the study was to provide a first contribution to the Italian validation of the scale.

The forward and backward translation method (30, 31) made it possible to obtain a final translation that can be superimposed on the original, avoiding distortions of the content of each item. The author of the scale found a good match between the original version and the English re-translation.

The measured values of I-CVI and S-CVI are satisfactory, therefore the content validity of the SSE-ITA can be ascertained (Table 4).

Table 4.

Final version of italian of Satisfaction With Simulation Experience scale (SSE-ITA)

DEBRIEFING E RIFLESSIONI
1	Il facilitatore ha provveduto a fornire critiche costruttive durante il debriefing	Fortemente in disaccordo -- in disaccordo -- non sono sicuro -- d’accordo --completamente d’accordo
2	Il facilitatore ha riassunto le questioni più importanti durante il debriefing	Fortemente in disaccordo -- in disaccordo -- non sono sicuro -- d’accordo -- completamente d’accordo
3	Ho avuto l’opportunità di riflettere e discutere della mia performance durante il debriefing	Fortemente in disaccordo -- in disaccordo -- non sono sicuro -- d’accordo -- completamente d’accordo
4	Il debriefing ha previsto opportunità di porre domande	Fortemente in disaccordo -- in disaccordo -- non sono sicuro -- d’accordo -- completamente d’accordo
5	Il facilitatore ha provveduto a fornire feedback che mi hanno aiutato a sviluppare le mie capacità di ragionamento clinico	Fortemente in disaccordo -- in disaccordo -- non sono sicuro -- d’accordo -- completamente d’accordo
6	Le riflessioni e la discussione riguardo la simulazione hanno rafforzato il mio apprendimento	Fortemente in disaccordo -- in disaccordo -- non sono sicuro -- d’accordo -- completamente d’accordo
7	Le domande del facilitatore mi hanno aiutato ad apprendere	Fortemente in disaccordo -- in disaccordo -- non sono sicuro -- d’accordo -- completamente d’accordo
8	Durante il debriefing ho ricevuto feedback che hanno favorito il mio apprendimento	Fortemente in disaccordo -- in disaccordo -- non sono sicuro -- d’accordo -- completamente d’accordo
9	Il facilitatore mi ha fatto sentire a mio agio e tranquillo/a durante il debriefing	Fortemente in disaccordo -- in disaccordo -- non sono sicuro -- d’accordo -- completamente d’accordo
RAGIONAMENTO CLINICO
10	La simulazione ha sviluppato la mia capacità di ragionamento clinico	Fortemente in disaccordo -- in disaccordo -- non sono sicuro -- d’accordo -- completamente d’accordo
11	La simulazione ha sviluppato la mia capacità decisionale nella clinica	Fortemente in disaccordo -- in disaccordo -- non sono sicuro -- d’accordo -- completamente d’accordo
12	La simulazione mi ha consentito di dimostrare le mie capacità di ragionamento clinico	Fortemente in disaccordo -- in disaccordo -- non sono sicuro -- d’accordo -- completamente d’accordo
13	La simulazione mi ha aiutato a riconoscere precocemente il peggioramento del paziente	Fortemente in disaccordo -- in disaccordo -- non sono sicuro -- d’accordo -- completamente d’accordo
14	Questa è stata una importante occasione di apprendimento clinico	Fortemente in disaccordo -- in disaccordo -- non sono sicuro -- d’accordo -- completamente d’accordo
APPRENDIMENTO CLINICO
15	La simulazione mi ha spinto a riflettere sulle mie abilità cliniche	Fortemente in disaccordo -- in disaccordo -- non sono sicuro -- d’accordo -- completamente d’accordo
16	La simulazione ha testato le mie abilità cliniche	Fortemente in disaccordo -- in disaccordo -- non sono sicuro -- d’accordo -- completamente d’accordo
17	La simulazione mi ha aiutato ad applicare quello che ho imparato dal caso affrontato	Fortemente in disaccordo -- in disaccordo -- non sono sicuro -- d’accordo -- completamente d’accordo
18	La simulazione mi ha aiutato a riconoscere i miei punti di forza e di debolezza riguardo le mie abilità cliniche	Fortemente in disaccordo -- in disaccordo -- non sono sicuro -- d’accordo -- completamente d’accordo

The reliability of the scale, assessed with test-retest, showed an “optimal” reliability (36) despite the rather low number of respondents, where even minimal variations greatly affect the final result.

The internal consistency measured per block produced an acceptable result for the “Debriefing and Reflections” section and questionable results for the remaining sections, unlike the original scale where the values were all > 0.80 (26). While the internal consistency of the whole scale was “acceptable”, with a value slightly lower than the α of 0.776 of the original scale, therefore we consider it valid. Again, the limited number of participants may have affected the result.

The Korean translation found significantly higher consistency values (0.85 - 0.79 - 0.91 for the blocks and 0.84 for the scale) (28), but the test was done on a different population than the original study so the data could hardly be comparable.

The study has some limitations: (i) although there are other studies in which the translated scale was tested on a limited number of subjects (41) and although some authors suggest testing the reliability on 10 subjects (36), the number of students who could have been enrolled may have induced numerous biases in the assessment of consistency and reliability; (ii) it was possible to enroll only students enrolled in the second year of the course as they are the only ones to have participated in a simulation experience, unlike the original study in which third-year students also participated; (iii) no comparison was made between the satisfaction assessed by the SSE and the skills acquired; (iiii) there is a lack of a convergent measure of validity and a discriminating measure of validity.

Finally, the SSE-ITA evaluates student satisfaction during clinical learning, so the technical skills, non technical skills and clinical skills acquired during the simulation must be evaluated through other tools.

Conclusion

In conclusion, it can be said that the SSE-ITA scale (table 4) seems to achieve the goal of assessing student satisfaction with clinical learning through simulation with high-fidelity mannequins.

In the future it would be interesting to evaluate the consistency of the SSE-ITA by involving a larger sample of students, involving students in nursing even in the 3 year course and validating the scale in other training areas, such as further education curricula of health professionals or training volunteers.

SCALA DI VALUTAZIONE PER LA SODDISFAZIONE DELL’ESPERIENZA DI SIMULAZIONE (SSE_ITA)

Leggi ogni asserzione e seleziona quella che meglio indica il tuo grado di accordo.

Fornisci una risposta ad ogni elemento, anche se può sembrare simile ad un altro.

Rispondi rapidamente, senza spendere troppo tempo su ognuna delle asserzioni.