Evaluation of the quality of the teaching-learning process in undergraduate courses in Nursing.

OBJECTIVE: to identify aspects of improvement of the quality of the teaching-learning process through the analysis of tools that evaluated the acquisition of skills by undergraduate students of Nursing.
METHOD: prospective longitudinal study conducted in a population of 60 secondyear Nursing students based on registration data, from which quality indicators that evaluate the acquisition of skills were obtained, with descriptive and inferential analysis.
RESULTS: nine items were identified and nine learning activities included in the assessment tools that did not reach the established quality indicators (p<0.05). There are statistically significant differences depending on the hospital and clinical practices unit (p<0.05).
CONCLUSION: the analysis of the evaluation tools used in the article "Nursing Care in Welfare Processes" of the analyzed university undergraduate course enabled the detection of the areas for improvement in the teachinglearning process. The challenge of education in nursing is to reach the best clinical research and educational results, in order to provide improvements to the quality of education and health care.

Introduction

The growing concern with the quality of higher education is a constant observed in every country in the world, among other things, due to the adequacy to social needs, to the alleged high cost and, consequently, the need to adapt to an increasingly demanding context, in which the universities must provide solutions to the needs posed by society 1.

This concern is expressed in the search for appropriate forms of organization of University teaching for change of paradigms, centered on the acquisition of competences according to the Lifelong Learning philosophy(2-3), or in the application of total quality management models 4, which involve external accreditation and internal processes, certificates or teachers. The process of change is universal and affects, since the end of last century, universities of the United States, Europe and Latin America 5.

In this context, in 2011 the undergraduate program in Nursing at the University Jaime I (Castelló, Spain) was started 6, accredited in the framework of the reform of the Spanish University system, after the implementation of the European Higher Education Area. Since the project began, the philosophy of the course is presented as an educational program inspired by quality management models: professors and students satisfied with the teaching-learning method and goals of quality care.

The Nursing course must ensure the training of competent professionals, who provide safe and quality care 7, as well as active and creative, able to respond to current and future demands of health 1, as well as adapt to the constantly evolving knowledge and technology. It is correct to think that the quality of nursing education can have an impact on the quality of medical care and professional development 6.

To achieve this, it is necessary to ensure the efficiency and quality of educational programs and nursing professors through the aforementioned accreditation programs. A cultural change is also required 2, providing for the participation of all stakeholders (students, professors, practice counselors and staff) for continuous quality improvement 8.

According to Denimg 9, the continuous improvement of quality is a process that involves four steps: planning, doing, assessing and acting. Improving the quality of teaching-learning processes is grounded on assessment 10, indispensable for obtaining relevant information, and educational innovation 1, for the implementation of improvement actions. A possible strategy is to consider the assessment and innovation as search fields within a cycle of continuous improvement, which allow to increase the quality of teaching-learning processes and to transform the professional practice of nursing.

In the course of Nursing of the University Jaime I 6, the teaching-learning process has as axis the competencies of each subject, formed by groups of disciplines. The acquisition of skills occurs through a sequence of learning that allows the acquisition of knowledge during classes of theoretical content, acquisition of related skills in laboratories and simulation classrooms and, from the second year, the demonstration and evaluation of learning outcomes in clinical spaces.

In this paper are presented the results that fall in the line of educational innovation of the post-doctoral program in the Department of Nursing of the University Jaime I, whose main objective is to make a formative assessment of the program of educational innovation for nursing students to acquire skills, and its impact on health care quality. Therefore, the goal is to detect aspects of improvement of the quality of the teachinglearning process through the analysis of tools to assess students' skills in this institution.

Method

This is a longitudinal, prospective study, based on tools that evaluate Nursing students acquisition of skills, students who were in the second year of the University Jaime I, throughout the course "Nursing Care in Welfare Processes", consisting of four disciplines: Basic Care; Nursing Care in Osteoarticular Processes; Nursing Care in Digestive, Renal and Endocrine Processes; and Nursing Care in Respiratory and Cardiovascular Processes.

The teaching method used integrates theory, simulated practice and clinical practice through learning outcomes and shared competencies in these four disciplines, guiding the content and avoiding disruption of the ongoing process of learning.

The rating system used considers the theoretical qualification (theory and practice simulated) as 50% of the final grade of each discipline. The other 50% correspond to the qualification obtained in clinical practice and are based on the evaluation carried out by clinical nurses accredited by the University with specific training to guide students (Reference Nurses) and professors through the following tools: (i) Guide of Evaluation of Clinical Practices (GEPC) - encompasses the verification and registration of learning activities defined for the achievement of objectives, with training assessments in the 3rd, 5th, 8th and 11th weeks, and a summative assessment in the 12th week, carried out by the Reference Nurses (RN) and by the professors of the disciplines; (ii) Defense of a clinical case - a case related to the contents of the subject and the results of learning that students develop and advocate in a public hearing; (iii) Portfolio - includes the registered clinical case, the PowerPoint presentation prepared for the defense of the case and a reflective diary, in which students contribute with their impressions, feelings and possible aspects of improvement; (iv) Electronic record system - a software designed for the monitoring of students from tools to help clinical decisions, standardized care plans according to the functional capacity, and methodology taught in classrooms,which are filled by students electronically in a tablet provided by the Nursing Department.

The studied population consists of 60 students enrolled in the four disciplines of the course "Nursing Care in Welfare Processes", taught by four professors, who perform clinical practices in five public hospitals and individuals, under the supervision of 41 RN. Through an intentional sampling, the records of students who are not enrolled in any of the four disciplines, of those who did not complete the period of practice and those that have not yet started it were excluded.

The studied variables, according to Figure 1, are the 30 learning activities incorporated in GEPC and the 10 items that include each of the checklists prepared by professors to assess the clinical case, portfolio and electronic records, so that each item is scored between 0 and 1, with a final score 10 for each assessment tool.

Figure 1

Studied variables. Assessment tools of the course "Nursing Care in Welfare Processes" of the undergraduate program in Nursing at the University Jaime I, Castelló de la Plana, Spain. Academic course 2012-2013

The main sources of information are the GEPC used by the RN to evaluate individually each student and the ad hoc records developed to evaluate the portfolio, the clinical case defense and the electronic records. Data collection is carried out simultaneously by the RN and by the professors of the disciplines in the formative evaluations in the clinical practice period (3rd, 5th, 8th and 11th weeks), from March 20 to June 26, 2013. The defense of the case, portfolio and electronic records are evaluated during the last week of clinical practice.

A descriptive analysis (mean, standard deviation, coefficient of variation and percentages) of items that contain elaborate ad hoc records to assess the defense of the case, the memory of practices and electronic records was carried out.

To detect aspects of improvement through tools for the assessment of the clinical case, portfolio and electronic records, the average score of each item is used. Therefore, considering that each item is scored on 1 point and trying to detect as many improvement aspects as possible, an average rating of less than 0.7 points was established as cut off point, being verified statistically with the Student t-test for one sample.

To detect the areas for improvement through the GEPC, the verification percentage of each learning activity was calculated, making the estimate with the Z test on proportions, if the percentage of each activity to check the set of GEPC included in the study was less than 80%. On the other hand, by the Chi-square independence test or Fisher's exact test, if the number of GEPC per group was n<5, the analysis of the verification of activities that did not reach the standard depends on the hospitals or units, using Chi-square test. Statistical analysis was performed with the Rcomander application of the software R 3.0.2. A level of bilateral statistical significance of p<0.05 was assumed when contrasting hypotheses.

Consensus among the faculty responsible for the disciplines included in the course "Nursing Care in Welfare Processes" is required, so that the results of the assessment in this study can be used. The anonymity of students and RN mentioned in the evaluation records was always kept, by means of a previous procedure of anonymity.

On the other hand, the educational innovation projects seem to have an impact on improving the training of future graduates, thus having an impact on the quality of nursing services, with social impact, since they address issues that interest and affect society as a whole. The projects related to the educational innovation and quality care should respect the fundamental principles of bioethics (beneficence, non-maleficence, autonomy and justice).

Results

Of the 60 students who formed the studied population, three were excluded because they were not enrolled in all subjects, and one for presenting a health problem that forced him to adapt his supervised clinics practices. Thus, the sample includes the assessment records of 56 students, of which 23.2% were male (n = 13) and 76.8% female (n = 43).

It is noted in the descriptive analysis of the evaluated items in the defense of the clinical case that the items "Assistance continuity after medical discharge" (x̄=0.68; s=0.24) and "Use of sources of evidence" (x̄=0.54; s=0.34) reached a score less than 0.7, although statistically it is only possible to say that the average rating is less than 0.7 in the second item (p-value<0.05).

When assessing the memory of clinical practice, the items "Use of bibliographical references" (x̄=0.69; s=0.4) and "Discussion on basic care and literature" (x̄=0.61; s=0.18) get an average rating of less than 0.7, with the results being statistically significant in the second item (p-value<0.05).

According to Table 1, in the evaluation of electronic records, the items that obtained a score less than 0.7 were: "Complete correctly the medical record" (x̄=0.68; s=0.23), "Record of evolution per shift" (x̄=0.68; s=0.19), "Performing the physical examination" (x̄=0.54; s=0.16), "Record of referred activities" (x̄=0.62; s=0.12) and "Identify consultations diagnostics" (x̄=0.48; s=0.32). These last three are statistically significant (p-value<0.05).

Table 1

Items of the assessment tools that do not reach the established quality standard. Course "Nursing Care in Welfare Processes" of the undergraduate program in Nursing at the University Jaime I, Castelló de la Plana, Spain.

On the other hand, the GEPC count with a total of 30 activities adapted to the learning outcomes, which are arranged in sequence, increasing their complexity during the entire period of the clinical practice. Five cases were excluded from that part of the analysis because the verification of registration activities was not collected.

It is possible to say with a confidence of 95% that 9 of the 30 learning activities included in the GEPC did not reach the standard of verification established in 80%, in accordance with Table 2(p<0.05). It is observed that most of these learning activities are part of the formative evaluation conducted during the 11th week. The results of the Chi-square test (X2) and Fisher's exact test (F) confirm that, in most of these activities, there are statistically significant differences depending on the hospital and clinical practices unit, respectively (p<0.05).

Table 2

Learning activities that do not reach the standard of verification and dependency of the practical units and hospitals. Course "Nursing Care in Welfare Processes" of the undergraduate program in Nursing at the University Jaime I, Castelló de la Plana, Spain.

Learning activities	%*	95% CI†	Z Test‡	X2§	F||
Assemble of practices	72.5	0-38%	0.9083	0.07	<0.01
Present assessment of deficits	74.5	0-36.5%	0.8365	<0.01	<0.01
Compare results with the bibliography	64.7	0-46.7%	0.9968	<0.01	<0.01
Assess results	58.8	0-52.6%	0.9999	<0.01	<0.01
Establish support relationship	88.2	0-21.1%	0.07074	<0.01	0.011
Identify the best evidence	84.3	0-25.7%	0.2206	0.068	0.343
Know information systems	80.4	0-30.1%	0.4721	0.134	0.094
Inform the division of labor	84.3	0-25.7%	0.2206	0.693	0.011
Deliver the final work to the supervisor	80.4	0-30.1%	0.4721	<0.01	0.013

*Percentage of verification activities

95% confidence interval

P-value of the Z test for one proportion

P-value of the Chi-square test

|P-value of Fisher's exact test

Discussion

The quality of education of nursing professionals has been in constant revision since the end of the last century, relating welfare quality problems with training problems 11that led to demands for change in the education of these professionals 12.

On literature review, although there is little evidence, it can be observed that it is possible to apply continuous quality improvement techniques in nursing training. These techniques are used in different ways in order to identify strengths and opportunities, as well as to develop improvements in educational programs, for example, through accreditation programs 13, establishing indicators related to the NCLEX testing in the United States 14, or using qualitative techniques to improve the quality of the clinical practice 15.

Our study provides another way to approach the continuous improvement of the quality of the teachinglearning process through the analysis of the assessment tools and their results, so that indicators are established to allow the identification of possible ways of improving the process and applying actions based on the best results of pedagogical research. This is what Figueroa describes as instructional design 1, although literature review has not found other similar studies in the field of Nursing.

The integrated methodology used in the

development of this study seems to offer satisfactory results, with success rate exceeding 90% in the four disciplines and with a correlation between the theoretical and practical skills that provide consistency and objectivity 16, although the obtained results reveal aspects of the teaching-learning process that should be reviewed.

On the one hand, analyzing all assessment tools, it is possible to observe shortcomings in the application of the methodology of nursing, as in the case of the initial assessment, the elaboration of diagnoses of consultations or planning and evaluation of results, and the establishment of a care relationship also did not reach the established quality threshold.

In our case, traditional lectures were used to explain the process of nursing. The cooperative learning or problem-based learning 17emerges as a possible alternative, since it places the student at the center of the teaching-learning process, enables significant

learning and provides good results 18, despite its difficult implementation.

It is important to note that the assessment tool of electronic records is the one that records the larger amount of items that do not reach the established threshold of quality. Students received prior training related to the methodology and the use of the system of records with the method of cases 19, although the innovative presence of nursing students using an electronic registration system with mobile devices during the clinical practice can cause resistance in the deployment process and hinder their access to electronic records 20, leading to possible impacts on the results of the evaluation. Despite these results, the use of software and similar tools can improve clinical skills of nursing students and encourage them in learning and implementing the nursing process during the clinical practice 21, therefore being necessary to promote the use of this tool among clinical nurses who guide students.

The learning model used integrates the practicebased evidence (PBE) with a transversal competence over the four academic courses 6. In this academic course, a strategy of progressive learning of PBE 22was developed and implemented, based on previous

experiences 23.

The obtained results show that the strategy should be reviewed, although it is also very likely that the difficulties related to the implementation process, such as hiring new professors or teaching methodologies used, have interfered in these results.

The results of the learning activities of the GEPC and significant differences in the centers and units where students perform clinical practices demonstrate the need to review the performance of the professors during clinical practice, since each professor assumes the supervision of one or two hospitals, serving as support for RN and participating in the learning and evaluation of students. In another work 24, it was possible to delve in the external or internal factors that may be interfering with participation of RN and in the quality of learning in the clinical scenario.

The limitations of the study were the size of the sample, since it was restricted to students who attended this course in its first year of implementation. On the other hand, the differences approached in the method of teaching, such as integrating theory and practice or evaluation by RN, compared to other colleges in the province with which they share practical spaces, may have interfered in the results obtained. In addition, the assessment tools were developed by consensus of a group of professors, but it still needs a study of validity and reliability.

Finally, in the continuous improvement of quality is

essential to involve all stakeholders(1,8,15). The analysis of the reflective journals included in the portfolio made by the students could help us identify other possible areas for improvement in the teaching-learning process 25. Despite these limitations, the results obtained are useful because they allow start developing strategies to improve the teaching-learning process.

Conclusion

The analysis of the assessment tools used for the course "Nursing Care in Welfare Processes" of the second-year undergraduate program in Nursing from the University Jaime I identified areas for improvement in the teaching-learning process.

Therefore, there is a need to evaluate the implementation of cooperative learning or problems based learning when conveying the nursing methodology. On the other hand, it is important to review the learning implementation strategy of PBE and encourage the participation of RN in the use of electronic registration system as a learning and assessment tool. Likewise, the collaboration between professors and RN during supervised clinical practice is essential but needs to be encouraged.

The challenge of education in nursing is to make use of the best clinical and educational research findings as the basis for education in order to ensure the quality of the teaching-learning process and the quality of the service that will be provided by future professionals. It is expected that the quality assurance of teaching in the undergraduate degree in nursing of the University Jaime I contributes to improve health care quality.