Mathematics learning and assessment using MathE platform: A case study.

Universities are encouraging the implementation of innovative methodologies and teaching strategies to develop an interactive and appealing educational environment where students are the focus of the learning process. In such a personalised learning environment, an increase of the students' engagement and the improvement of the outcomes arise. MathE has been developed to help achieve this goal. Based on collaborative procedures, internet resources - both pre-existing and freely available as well as resources specifically conceived by the project team - and communities of practices, MathE intends to be a tool to nurture and stimulate the learning of Mathematics in higher education. This study introduces and describes the MathE platform, which is divided into three sections: Student's Assessment, Library and Community of Practice. An in-depth description of the Student's Assessment section is presented and an analysis of the results obtained from students, when using this feature of the platform, is also provided. After this, and based on the answers to an online survey, the impact of the MathE platform among students and teachers of eight countries is shown. Although the number of collected results is still scarce, it allows the recognition of a trend regarding the use of the material of the Student's Assessment section for autonomous study. The results indicate the platform is well organized, with a satisfactory amount and diversity of questions and good interconnection between the various parts. Nevertheless, both teachers and students indicate that more questions should be introduced. The overall opinion about the MathE platform is very favourable.

Introduction

The results obtained by Portuguese students in their undergraduated studies in Mathematics are the target of regular scrutiny. Although there is evidence of an ongoing improvement path – the results obtained by Portugal in 2018 are close to the OECD average – the students’ performance in higher education courses that rely heavily on mathematical knowledge is still very low. The same pattern is observed all over Europe (OECD, 2019).

According to Prosser and Trigwell (1999) and Education and Commission (2011), the causes of this circumstance are related not only to individual aspects (low concentration, little motivation, feeling of inadequacy) but also to social prejudice regarding mathematics (often regarded as too difficult and ’not for everyone’), as well as to the nature of the subject itself (sequential and accumulative nature of the learning process, need for a big load of training and work). The use digital methodologies can improve the students’ interest in the study of mathematics and also offer them new opportunities of learning (Quigley et al., 2020). The combination of traditional classes and digital tools can engage the students with real world problems by providing a means of accommodating gaps in the required mathematical knowledge (Geiger et al., 2015).

University lecturers are encouraged to implement innovative methodologies and teaching tools in order to develop an interactive and appealing educational environment where students are the focus of learning and instructors are facilitators of the process, therefore increasing engagement and improving the student’s outcomes (Biggs, 2016). One way to achieve this goal is the creation of online collaborative teaching communities. Online instruction focuses on collaborative learning, acquisition of knowledge, and Internet applications to redesign education (Biggs, 2016). Investigation shows that a smart and versatile learning platform articulated with well-conceived team-based activities can increase students’ motivation, attitude and performance, therefore creating an environment for deeper learning and better consolidation (Biggs, 2016; Vygotsky, 1986).

The MathE platform was created to help to achieve this goal. Supported by an active learning methodology, the MathE platform is based on collaborative procedures, namely online discussion, group work, and individual procedures such as focused listening, formulating questions, note taking and annotating. All these activities are supported by ICT tools such as internet resources and communities of practices (Castro & Tumibay, 2021).

MathE can also provide significant support to students and teachers throughout the current pandemic situation with the consequent limitations to in-person classes and the incorporation of online strategies being promoted.

A set of anonymous surveys was applied to lecturers and students registered on the platform in order to evaluate the impact of the MathE platform. A standardized methodology was implemented in two phases: a qualitative phase for the development of the survey and a quantitative phase for the survey’s validation. In the qualitative phase, a literature review was performed and two discussion forums (students and teachers) were implemented. For the answering options, a five-point Likert scale was used. In the qualitative phase, the data was collected during the year 2020 (from November 15 until December 15), through an online survey. For convenience, the data was collected using the online platform Google Forms and the surveys were answered by 56 adult participants (students and teachers).

This paper is organized as follows. In Section 2, the contents of the MathE platform are described. Section 3 presents a SWOT analysis of relevant studies on online learning as well as a particular analysis of the MathE platform. The impact of MathE and the analysis of the usage of the platform are reported in Section 4. Section 5 presents a reflection about such impact and the last section, Section 6, concludes the paper and identifies future paths for further research on this subject.

MathE environment

MathE is an e-learning platform (mathe.pixel-online.org), with an associated Youtube Channel (youtube.com/channel/UCGKINlc7YgMrHzTIPp2rYcg), where students have free access to resources such as videos, exercises, training tests and pedagogical materials about several mathematical areas taught in higher education courses. It is an online collaborative environment where teachers and students can share knowledge, questions, reflections and learning strategies.

The purpose of MathE is to provide to students and teachers a new perspective on mathematical teaching and learning. MathE offers a dynamic and enthusiastic way to teach and learn mathematics, relying on digital interactive technologies that enable autonomous study (Pacheco et al., 2019; Pereira et al., 2020). In this way, MathE promotes creativity as well as an encouragement to search for unconventional practices in the process of learning mathematics, a subject that is often associated with monotony and boredom. Besides, MathE contributes to make the students’ more autonomous, allowing them to organize their studies at their own pace, a skill that can also be of utility for other school subjects. The design and building of environments that support learning and teaching processes are topics that have been object of abundant research, in the last years (Rodrigues et al., 2019; Borba et al., 2016). Moreover, considering that in the area of mathematics, particularly in higher education, teachers are not yet aware of this kind of research and of exisiting teaching environments, it is extremely important to spread the word about the existence of a freely available e-learning system such as MathE. This work presents an environment specially conceived for engineering students who are exposed to a variety of mathematical topics along their path in higher education.

MathE was developed and implemented by a consortium of seven institutional partners from five European countries: Polytechnic Institute of Bragança (Portugal), the Limerick Institute of Technology (Ireland), the University of Genova, Pixel (Italy), Kaunas University of Technology (Lithuania), Technical University of Iasi (Romania) and EuroED (Romania). Each of the partner institutions has built a solid community of teachers, in the corresponding countries, that have been actively collaborating and responding to the challenges of the project.

The MathE platform is organized into three main sections: Student’s Assessment, MathE Library and Community of Practice. The features of the MathE platform environment are shown in Fig. 1.

Fig. 1

MathE platform environment

At its current stage, the platform covers fourteen topics, among the ones that are at the classical core of graduate courses: Analytic Geometry, Complex Numbers, Differential Equations, Differentiation (including 3 subtopics: Derivatives, Partial Differentiation, Implicit Differentiation and Chain Rule), Fundamental Mathematics (2 subtopics: Elementary Geometry and Expressions and Equations), Graph Theory, Integration (2 subtopics: Integration Techniques and Double Integration), Linear Algebra (5 subtopics: Matrices and Determinants, Eigenvalues and Eigenvectors, Linear Systems, Vector Spaces and Linear Transformations), Optimization (2 subtopics: Linear Optimization and Nonlinear Optimization), Probability, Real Functions of a Single Variable (2 subtopics: Limits and Continuity and Domain, Image and Graphics), Real Functions of Several Variables (1 subtopic: Limits, Continuity, Domain and Image) and Statistics, as presented in Fig. 2.

Fig. 2

Mathematics topics and subtopics available on the MathE platform

The section Student’s Assessment is composed of multiple choice questions divided into topics, with two difficulty levels — basic and advanced — that can be selected by the students. The students can train and test their skills in the subsections Self Need Assessment and Final Assessment, respectively.

The goal of the Self Need Assessment section is to provide the student with training assessment to test if a certain topic that the he/she enrolled is already known and understood. If the student or the teachers believe that its understanding needs to be deepened, the student can choose answer another training assessment to measure its degree of confidence to perform a final assessment. Each training assessment will be randomly generated from the assessments database composed by questions/answers. In this way, the same student will be able to answer different training assessments on the same topic. When students answer a training assessment, they will have immediate access to the obtained mark: the test will randomly select questions from a given set and after the student submits the test, the mark will automatically appear, allowing self-evaluation. On the other hand, the goal of the Final Assessment section is to evaluate the student performance after practicing with training assessment questions (and all the related materials available on MathE portal). In the Final Assessment section, the questions are selected by the teacher and will be available for the students in a given moment, defined by the teacher. In this case, the student will submit the test and receive feed-back on the following day; the teacher will have access to the results at the end of the test, one day before the students.

In the section MathE Library the students can access a collection of videos and teaching materials about the topics/subtopics covered by the platform. There are two types of videos and teaching materials available on the platform: some are rigorously selected among the ones available on the Internet while others are produced by the elements of the MathE partnership. All resources make a point of being concise, appealing and having the ability to help students and teachers to focus on particular aspects of the covered subjects. It should be mentioned that there is a great number of video lessons and teaching materials available on the internet, but some of them have scientific errors and are not targeted at the higher education population. The MathE Library provides access to high quality, reviewed and certified academic support. Moreover, it is important to highlight the multi-language nature of the MathE Library: besides English, the mandatory language of the project in each item, it is possible to find available resources in different languages such as Portuguese, Italian, Lithuanian, Romanian and others.

The last section of the platform, Community of Practice, is a virtual meeting place where users can share teaching and learning experiences. In this way, a solid network of learning and teaching practices will be created, providing a place where teachers and students can share their experience, knowledge and information.

The MathE project offers an online tool for autonomous learning, available 24 hours per day, 7 days per week, where students can learn mathematics in an enjoyable way, less exhaustive and boring than the traditional methods. Mathematics is the pillar of several other courses, often perceived as a hard, dry subject. The use of collaborative methodologies and innovative tools that allow an alternative way to study and teach mathematics can demystify this paradigm. Furthermore, the skills developed while autonomously learning with the aid of MathE can be exported to other subjects as well as other areas of personal development.

SWOT analysis of online learning and the particular case of the MathE environment

The key of e-learning and blended learning consists of a combination of educational competence, from both students and teachers. The way that users deal with online learning determines the success or failure of a digital resource. Even if the students have great autonomy and responsibility in the pursue of their academic goals, teachers play an important role and must be consistent with the responsibility and the delivery of a curriculum to a diverse audience (Rovai & Downey, 2010); in such environments, it often requires more time and planning than in traditional learning methodologies, without the use of technological tools.

Although students have no difficulties with the technological aspects of dealing with e-learning methodologies (such as adaptation to new platforms and acquisition of knowledge), they may experience difficulties in the management of their daily lives (such as the adaptation of the schedules and the creation of new learning habits and rhythms), with clear consequences on their learning experiences and outcomes.

This section aims to investigate the strengths, weaknesses, opportunities and threats under relevant researches on online learning, presenting a SWOT analysis of methodologies that rely on online environments. The same analysis is conducted for the particular case of online learning with the MathE platform platform.

Strengths

Flexibility

with e-learning resources, people can study from anywhere, at anytime, being able to balance academic tasks with day-to-day work and family responsibilities (Arora & Mehta, 2018; Stone et al., 2016; Paul & Jefferson, 2019).

Accessibility

e-learning platforms offer numerous high quality education opportunities to students from diverse backgrounds and also supply interdisciplinary approaches to teaching and learning (Paul & Jefferson, 2019; Mahlangu, 2018).

Cultural plurality

a global environment for learning can be formed, in which culturally diverse students and teachers participate from their own countries. Besides, it provides a powerful academic environment for achieving global competence that may be applied to solve global problems (Rovai & Downey, 2010).

Learning communities

a digital ecosystem of discussions can be constructed, between people with similar interests, in order to share knowledge, thoughts, questions and experience (Persico et al., 2014; Tsiotakis & Jimoyiannis, 2016). Discussions are stimulated outside the classroom among students and teachers, making the class more interactive and productive through pre-work (Ali et al., 2019).

MathE strengths

to boost students interest in the science of mathematics, MathE promotes the introduction of unconventional practices for learning through the MathE library and Student’s Assessment. The platform is full time accessible to students and teachers worldwide. Besides, by means of the “Community of Practice” a solid network between MathE users is established, in which critical and reflexive discussions are offered to construct knowledge through group work.

Weaknesses

Less class control

some e-learning platforms give teachers low control since they do not allow them to upload materials or communicative tools to connect teachers and students (Limayem & Cheung, 2011). The teacher’s less control of the class may imply lack of students’ support and difficulties in pedagogical adaptation, lack of iteration and communication that compromises the e-learning performance (Rodrigues et al., 2019; Zaharias, 2009; Pape & Prosser, 2018).

User’s resistance

e-learning requires knowledge and skills to enhance study experiences; that can cause resistance if the person is not prepared to deal with technological educational tools (Zounek & Sudicky, 2013).

Students isolation

when there is no offline interaction, teachers and students are limited to electronic correspondence and may not pick-up on verbal and non-verbal cues (Paul & Jefferson, 2019). Therefore, students may feel isolated or abandoned in the virtual environment (Zounek & Sudicky, 2013). The absence of face-to-face interaction and continuous communication may make it difficult for teachers to identify their students personal education needs.

Time

developing material and online resources for virtual classes requests much more work and time compared to a face-to-face class preparation (Gewin, 2020; Bentley et al., 2003). Moreover, for students, it is necessary an effective time management to balance work, family, caring responsibilities and their studies (Farrel & Brunton, 2020).

No physical library access

being at a campus provides access to exclusive books and materials not available on internet. Besides, staff, such as as library technicians, can help learners edit their papers, locate valuable study material and improve study habits (Paul & Jefferson, 2019).

MathE weaknesses

Since MathE is a platform that aims to support traditional teaching, many of the mentioned weaknesses do not apply to the project. Although teachers can become MathE contributors by uploading and suggesting their materials, like any new tool, MathE is subject to user’s resistance, even though having been developed as a friendly interface.

Opportunities

Stimulate students’ interest

e-learning tools have the power to stimulate students’ interest and also increase their engagement and satisfaction with learning (Rodrigues et al., 2019; Kangas et al., 2017).

Know-how increment

the e-learning ecosystem enables the increment of teachers and students know-how through innovative tools and collaborative work (Rodrigues et al., 2019; Kangas et al., 2017).

Developing skills

by means of self-study, the students can develop responsibility to plan and organize time and tasks and solve problems by themselves. These skills are precious not only in education scenarios but also in professional careers (Mahlangu, 2018; Markova et al., 2017). Also, e-learning can be seen by an extra opportunity for learning, mainly for students who have difficulties with face-to-face methodologies (Paul & Jefferson, 2019; Tsiotakis & Jimoyiannis, 2016).

Reach expansion

e-learning allows universities to expand borders, increase collaborative learning and research activities. In this way, it becomes possible to identify new prospective students and to establish themselves as global educational providers (Arora & Mehta, 2018).

MathE opportunities

MathE can provide a dynamic, appealing and enthusiastic way to teach and learn mathematics. Students complement their learning through quality material developed or organized by professionals. Besides, MathE contributes to make the students more autonomous, leading to a different organization of their studies, at their own pace, also expanding these abilities to other school subjects.

Threats

Internet access

many people around the world do not have access to the internet, making the possibility of e-learning a utopian issue (Arora & Mehta, 2018; Valverde-Berrocoso et al., 2020). According to World Bank (Bank, 2020) data, in the year 2018, 84% of the citizens of member states of the European Union had access to the Internet, while in Latin America this value falls to 66%, and 18% in the least developed countries.

Financial investment

e-learning can contribute to increase educational inequality since it requires high financial investments not only by the government but also from users (Arora & Mehta, 2018; Ali et al., 2019; Mirata et al., 2020).

Adoption of new habits

due to lack of maturity and awareness, e-learning users are often not prepared to adopt new study habits, as self-time control, self-motivation, concentration, self-direction of their own study (Arora & Mehta, 2018; Paul & Jefferson, 2019; Boisselle, 2014). Thus, they are more likely to quit class if they do not like the instructor, the format, or the feedback (Paul & Jefferson, 2019).

Lack of technological abilities

frequently, the fail of e-learning is related to lack of investment in human resources training and also due to inconsistencies between teachers and student expectations when not getting immediate results (Persico et al., 2014; Rodrigues et al., 2019).

MathE threats

besides Internet connection, which is essential to all kinds of e-learning, it is verified within MathE the difficulty to motivate students to regularly access the platform to use extra materials, review a subject or try to answer questions without teacher supervision. The adoption of new habits by the students is a requirement to optimize the profit given by the MathE platform.

MathE impact analysis

As it was previously mentioned, the MathE platform is organized into 3 main structures: Student’s Assessment, MathE Library and Community of Practice. This section focuses on analyzing the students’ performance on the Student’s Assessment section, specifically on the Self Need Assessment, where students can train their competences.

Currently, there are 54 teachers and 659 students, from different nationalities, enrolled in the platform: Portuguese, Italian, Russian, Lithuanian, Irish, Spanish, Dutch and Romanian. As for the enrolled students, they come from several scientific areas. There is an expressive presence of engineering students, who have a strong mathematical basis in their course, but it is also noticed the presence of education, biology, financial and management students, who also have mathematical subjects in their academic path.

There are currently 981 questions available on the Self Need Assessment. These questions are divided into 14 topics and 17 subtopics, as detailed in Table 1. The MathE platform is dynamic since any teacher can submit questions for any of the subjects. All submitted questions are revised by a set of MathE reviewers and new topics or subtopics can be added.

Table 1

Questions available on MathE platform

Topic (T)	Subtopic	Basic	Advanced
		Questions	Questions
Differentiation (T1)	Derivatives (T1.1)	34	12
	Partial Differentiation (T1.2)	52	12
	Implicit Differentiation and Chain Rule (T1.3)	11	4
Integration (T2)	Integration Techniques (T2.1)	21	19
	Double Integration (T2.2)	27	13
Linear Algebra (T3)	Linear Systems (T3.1)	38	13
	Matrices and Determinants (T3.2)	30	10
	Vector Spaces (T3.3)	25	15
	Linear Transformations (T3.4))	23	17
	Eigenvalues and Eigenvectors (T3.5)	26	14
Real Functions of a Single Variable (T4)	Domain, Image and Graphics (T4.1)	8	9
	Limits and Continuity (T4.2)	15	10
Real Functions of Several Variables (T5)	Limits, Continuity, Domain and Image (T5.1)	28	26
Fundamental Mathematics (T6)	Expressions and Equations (T6.1)	27	15
	Elementary Geometry (T6.2)	38	9
Analytic Geometry (T7)	–	32	8
Complex Numbers (T8)	–	20	20
Differential Equations (T9)	–	31	10
Probability (T10)	–	28	18
Statistics (T11)	–	25	15
Graph Theory (T12)	–	32	16
Optimization (T13)	Nonlinear Optimization (T13.1)	30	13
	Linear Optimization (T13.2)	27	20
Numerical Methods (T14)	–	35	0
Total Questions	–	663	318

The available questions are divided into two difficulty levels: basic and advanced. There are 663 questions available in the basic level and 318 in the advanced level. Thereby, 67.58% of the questions available belong to the basic level, while 32.42% are advanced.

Figure 3 presents the total number of questions available on the platform, divided by topics. The topics composed by more subtopics have naturally more questions available than the topics with few or no subtopic. The topic Linear Algebra is the one with more subtopics (five in total) and is the topic with more questions. However, it is important to highlight that new questions are periodically inserted on all topics and subtopics, thus the number of questions available is constantly increasing.

Fig. 3

Total questions available on the MathE platform (per topics)

Collected results

As said before, there are 54 teachers and 659 students enrolled in the platform. Among the teachers, 37% are Portuguese and the others are divided into 7 nationalities. Whereas, among the universe of the students, 57% are from Portugal and the remainders belong to the other countries.

As already mentioned, this paper concentrates the efforts into the analysis of the Student’s Assessment section. Thus, the data collected and the performed analysis takes into consideration the information provided by the 122 students that are active and consistent users of the Student’s Assessment section, regularly answering and submitting self-assessment tests, in order to support their study and validate their progress. It is emphasized that other students use the platform to assess teaching materials and learning by the videos. Among these 122 students, 15 are from Italy (12%), 77 are from Portugal (63%), 25 from Lithuania (21%), 3 come from Ireland (2%), 1 is from Russia (1%) and 1 comes from Spain (1%).

During 2020, 2973 questions were answered. Such questions correspond to 24 topics/subtopics. Note that some of those questions can appear in more than one test, since each self need assessment test is randomly generated from the questions in the database. Table 2 presents the number of questions that were answered, divided into basic and advanced levels according to the correctness or incorrectness of the answers. Thus, among the 2973 questions answered, 2353 were basic and 620 were advanced. The sum of all answered questions, organized per topic is shown in Fig. 4.

Table 2

Questions answered on the MathE platform

Topic (T)	Subtopic	Basic Questions			Advanced Questions
		Correct	Incorrect	Total	Correct	Incorrect	Total
Differentiation (T1)	Derivatives (T1.1)	59	157	216	0	0	0
	Partial Diff. (T1.2)	11	9	20	0	0	0
	Implicit Diff and Chain Rule (T1.3)	0	0	0	0	0	0
Integration (T2)	Integration Tech. (T2.1)	24	47	71	4	3	7
	Double Int. (T2.2)	8	7	15	0	0	0
Linear Algebra (T3)	Linear Systems (T3.1)	94	68	162	0	3	3
	Mat. and Det. (T3.2)	102	94	196	14	15	29
	Vector Spaces (T3.3)	181	140	321	56	88	144
	Linear Transf. (T3.4)	132	128	260	25	60	85
	Eigenval/Eigenvec (T3.5)	30	36	66	13	5	18
Real Functions of a Single Variable (T4)	D, Im, Graph (T4.1)	20	22	42	0	5	5
	Lim. and Cont. (T4.2)	4	5	9	0	2	2
Real Functions of Several Variables (T5)	Limits, Continuity, D and Im (T5.1)	4	3	7	0	0	0
Fundamental Mathematics (T6)	Exp. and Eq. (T6.1)	95	177	272	11	16	27
	Elem. Geometry (T6.2)	66	78	144	5	5	10
Analytic Geometry (T7)	–	97	106	203	14	24	38
Complex Numbers (T8)	–	62	151	213	138	91	229
Differential Equations (T9)	–	36	24	60	0	0	0
Probability (T10)	–	10	26	36	11	5	16
Statistics (T11)	–	8	22	30	0	0	0
Graph Theory (T12)	–	2	8	10	7	0	7
Optimization (T13)	Nonlinear Opt. (T13.1)	0	0	0	0	0	0
	Linear Opt. (T13.2)	0	0	0	0	0	0
Numerical Methods (T14)	–	0	0	0	0	0	0
Total Questions	–	1045	1308	2353	298	322	620

Fig. 4

Total answered question (per topic)

Among the universe of the basic questions, 1045 were correctly answered and 1308 were incorrectly answered. On the other hand, among the advanced questions, 298 were correctly answered and 322 were incorrectly answered. As can be seen in Fig. 5, 45% of the total number of answered questions were correctly answered and 55% were incorrectly answered. Among the basic questions, the percentage of correct and incorrect answers was 44% and 56%, respectively. Considering the set of the advanced questions, the percentage of correctly answered was 48% and the percentage of incorrectly answered was 52%.

Fig. 5

Percentage of correct and incorrect answered

Figure 6 shows the percentage of correct and incorrect questions answered for each topic. In this case, each graphic contour indicates a percentage level and, thus, the further from the center, the greater the percentage associated with the variable under analysis. It is noteworthy that each graphic vertex represents a topic, as indicated by T1 until T12 and in accordance with Table 2. The topics Optimization (T13) and Numerical Methods (T14) which were recently added to the platform, have no questions answered and, for this reason, they were not considered in the graphics presented in Figs. 6 and 7.

Fig. 6

Percentage of correct and incorrect answers among the total of answered questions

Fig. 7

Percentage of basic and advanced answered questions

To identify the profile of the answered questions in each difficulty level, Fig. 7 compares the percentage of correct and incorrect answers per topic.

The topics with a higher percentage of correct answers are T9 (Differential Equations) and T5 (Real Functions of Several Variable) and the topic with the highest percentage of incorrect answers is T11 (Statistics). Considering the set of basic questions, the topics with a higher percentage of correct answers are T9 and T5 and the topic with the highest percentage of incorrect answers is T12 (Graph Theory). Considering the set of advanced questions, it is possible to state that Graph Theory, T12, corresponds to the highest percentage of correct answers.

MathE’s users feedback

To learn about the students’ and teachers’ impressions about the collaborative e-learning mode proposed by the MathE platform, two questionnaires were designed, one for students and the other one for teachers enrolled on the platform. Both surveys were anonymous and available online to be voluntarily filled out. The free tool Google Forms was used to conceive the surveys.

The classification levels used along the questions for the respondents to provide their input, follow the 5-level Likert scale (Norman, 2010) (from ”one” 1 to ”five” 5, in an ascendant order, for instance, 1 – “not appropriate at all” ,..., 5 – “extremely appropriate).

After the validation of the deployed questionnaire, it was disseminated among the users of the MathE portal. The respondents had the opportunity to give their opinions, indicating difficulties they had faced and possible suggestions of improvement. The results consider the opinion of the 32 students and 20 teachers who contributed, anonymously, through the voluntary supply of information by filling in the available online forms. A thorough description of the teachers and students’ feedback is presented below.

Students feedback

First, the students answered a question about “how much the MathE platform is a valuable help in their studies.” 40.6% said the platform was helpful, whereas 9.4% said the platform was not helpful in their study. On the other hand, another 9.4% said exactly the opposite, that is, that the platform was extremely helpful for them. To complete the percentage, 25% answered that the platform helped them a little and 15.6% that the platform helped a lot.

Considering the level of difficulty of the questions, all students considered that it was appropriate in all questions (more or less appropriated – 50%; very appropriated – 31.25%; extremely appropriated – 9.375%; little appropriated – 9.375%).

When questioned about the possibility of recommending the platform to a friend, 37.5% of the students said they would certainly recommend the platform to a friend, 21.875% said they would probably recommend it, 21.875% said it was highly likely they would recommend the platform, 18.75% answered it was unlikely for them to recommend the platform. No one affirmed they would certainly not recommend the platform.

Furthermore, the students were encouraged to suggest improvements, related to the amount and variety of teaching material, questions distribution and diversity of contents available on the platform. The obtained result is presented in Fig. 8.

Fig. 8

Students’ suggestions to improve the MathE platform

Increasing the number of questions for both difficulty levels as well as the diversity of teaching material were the main suggestions obtained from the students, in order to help improve the MathE platform.

In addition, the students were questioned about the main difficulties they experienced as MathE’s users; the collected answers are represented in Fig. 9.

Fig. 9

Students’ main difficulties

As it can be seen, 11 out of 32 students show no difficulties; the main difficulties found are related with the platform organization and the language. This kind of response is expected since the portal uses English as main language and most respondents are not native English speakers. It becomes obvious that the chosen working language is a decisive factor when it comes to reaching new users and broaden the range of the MathE community.

The final question in the survey consisted of an open question for additional comments, so that the students could express any other suggestions. In general, the students like the MathE platform and consider it a useful resource to support their study.

Teachers feedback

Considering the feed-back from the teachers, 85% of them mentioned MathE to their students and 95% of them did the same with their colleagues.

A relevant question that arises is related to the way to use the MathE platform to optimize the teaching/learning process. Considering the teachers survey, 35% use only the Self Need Assessment to deliver training tests, 10% only use the Final Assessment to do exams, 20% use both features, while 35% said they never used the platform during their classes. Thus, this last group of teachers only encourages their students to use the platform as an extra tool, outside classes.

Concerning the teachers’ contributions as question developers, 15% said they had already submitted questions for Self Need Assessment, 5% submitted questions for Final Assessment, 75% submitted for both sections and 5% never submitted any questions to MathE. Therefore, it is possible to conclude that the majority of the teachers have submitted questions to the platform.

Among the universe of the answers collected from teachers, it was observed that 90% of them consider that the available topics/subtopics are adequate for higher education students. They also consider that more elementary topics and more basic questions should be added, in order to help the students who struggle with difficulties with the fundamentals of mathematics and its basic concepts.

As the students, the teachers were also questioned about the adequacy of the level of difficulty of the available questions with 60% answering they were very appropriate, 10% extremely appropriated and the same amount of 15% was counted for the more or less appropriated and little appropriated categories. It is important to mention that no one said the difficulty level was not appropriated at all.

Regarding the main difficulties they had faced, the teachers’ majority, 85%, said they had not found any difficulty at all as MathE users. However, some teachers report difficulties with the platform and questions organization and also with the visualization of some questions, as can be seen in Fig. 10.

Fig. 10

Teachers’ main difficulties

Although few difficulties were mentioned, the teachers were encouraged to suggest improvements to be implemented on the platform. The obtained results are shown in Fig. 11.

Fig. 11

Teachers’ suggestions for improvement

Additionally, a blank space was also given to the teachers for them to include additional comments. Some of them said the platform is a great and useful resource. Another suggestion that was collected was the creation of an intermediate level of difficulty.

Discussion and reflection

The success of an e-leaning tool is linked to the level of satisfaction of the students and to the engagement of the teachers to explore such pedagogical models. Thus, finding ways to boost the students’ participation and the teachers’ engagement are crucial attitudes to improve the robustness, expansion and integrity of the MathE platform.

There is consensus that e-learning, either completely online or combined with in-class sessions, is a trend on general courses in higher education (Silverman & Hoyos, 2018). Moreover, this pattern could be observed during the COVID-19 pandemic, where the students had to remain at home, without attending traditional classes. The MathE platform, being a learning and teaching collaborative online tool, can perform a relevant supporting role in this context. Moreover, the platform aims to support the mathematics learning and teaching in a less conventional and more interactive manner than the traditional classroom provides, being a valuable asset when online teaching and autonomous study become mandatory.

It is known that the obstacles and successes when relying on e-learning are not simply technical matters and that they strongly depend on several human and social factors, such as the teachers’ digital competence and students’ access to fast broadband internet (McPherson & Nunes, 2004); also, the effectiveness of digital collaboration requires attention to individual’s (either teachers or students) activity (Silverman & Hoyos, 2018). Moreover, some research is devoted to the construction and evaluation of mathematics tutoring systems for supporting college students’ in a path of persistence and success. The discussion of new approaches to mathematics learning and collaboration among mathematics teachers through the use of Web platforms and communication tools is also addressed in Silverman and Hoyos (2018). In this context, this section includes a discussion about the preliminary results concerning the usage of the MathE platform and also the user’s opinions about its contents. Since various different points of view were considered, improvement will be possible, in order to provide teachers and students with the best possible tool to support the interdependent processes of teaching and learning mathematics.

Although there are students from different nationalities registered on the platform, the majority consists of Portuguese students (57% of the total of enrolled students). It has been noted that the participation of students is directly proportional to the number of teachers in each country. Since the largest number of teachers comes from Portugal, it is natural that Portuguese students are more encouraged to use the platform.

As expected, some topics/subtopics are more sought by students than others. That is obviously related with the existence/inexistence of each topic in the course curricula. Since the majority of enrolled students attend engineering courses, the topics/subtopics related to Calculus and Linear Algebra are among the more searched. Besides, Linear Algebra and Fundamental of Mathematics, which are the most answered topics, are elementary subjects, transversal to practically all curricula; these topics require special attention and a large number of available questions for the students to practice and test their basic skills.

Some topics, as T13 (Optimization) and T14 (Numerical Methods) as well as the subtopic T1.3 (Implicit Differentiation and Chain Rule), do not have any question answered, as can be seen in Tables 2 and Figure 4. This low level of participation is justified with the fact that these topic/subtopics were the last to be inserted in the platform; the interval of time since these topics were made available is too short for conclusions to be made. On the other hand, concerning the topics/subtopics that have been made available more time ago and are also very common on the curricula of engineering degrees, as T1 (Differentiation), T2 (Integration), and T9 (Differential Equations), it was noted the advanced questions are rarely used. This may be an indicator that the users struggle with very basic difficulties when studying these topics, so that they prefer to answer questions of basic level, not daring to try the advanced ones.

For all the topics, as can be perceived from Table 2, the number of basic questions answered is larger than the number of advanced ones. The largest audience of the platform consists of students who have difficulties apprehending basic mathematical concepts, so it is expectable that the basic questions are the more searched. Besides, advanced mathematics concepts are not required in most engineering courses, as they are for physics or mathematics courses.

In general, as Fig. 6 shows, the percentage of incorrect answers is higher than the number of correct answers, both when the total amount is considered as when they are analyzed by levels of difficulty (see Fig. 7). Also, analyzing Fig. 6, it is possible to verify that only in T5 (Real Functions of Several Variables), T9 (Differential Equations) and T12 (Graph Theory) the percentage of correct answered is slightly larger than the number of incorrect ones.

When the analysis takes into consideration the difficulty levels (see Fig. 7) the trend remains. For the basic level, only T3, T5 and T9 had a percentage of correct answers larger than the number of incorrect. Considering the advanced level questions, only T2, T8, T10 and T12 had a larger number of correctly answered than the number of incorrectly answered. However, when comparing with the absolute values presented in Table 2, the number of advanced T2, T10 and T12 answered questions is very small and not statistically relevant.

The expressive number of incorrect answers indicates that it is urgent and mandatory to review the level of difficulty of the questions and to redistribute them (although the surveys indicate that most teachers consider the level of difficulty adequate, most students indicate that it is merely more or less suitable). As was mentioned by a teacher, it would be useful to review the question’s level of difficulty, since a large number of incorrect answers for a topic indicates that some questions, that were initially considered basic, should migrate to the advanced level group. The fact that students make so many mistakes will discourage them from continuing to use the platform, so these results are useful alerts for the maintenance and expansion of the platform, which requires constant adjustments to fit the reality and the needs of the users.

It is also known that some teachers use the platform in the classroom as a way to ascertain the level of the students when starting the subject. This may be the cause to the high number of questions incorrectly answered, since many questions are answered before students have contact with the concepts in the classroom. On the other hand, the results collected for elementary topics reveal that, when reaching higher education, students do not have the necessary domain of elementary concepts, previously learned in high school. Given the cumulative nature of mathematical knowledge, this fact generates a cascade effect when students progress to higher education; for example, if the student has no knowledge of the basics about real-valued functions, it can be very painful to learn about limits and derivatives.

The fact that this study confronts students and teachers opinions, allows the implementation of punctual and effective improvements. Taking into account that most of the students pointed out that the platform helped them in some way, the authors take such observations as an indicator that the MathE platform is on the good path as reliable and useful tool to assist and support the process of learning mathematics.

Concerning the main difficulties experienced, shown in Fig. 9, the most voted option was “No difficulty”, an indicator that the platform is being adequately developed; however, adjusts are mandatory. Among the students who reported difficulties, the most voted option was “organization of the platform”; therefore strategies should be implemented to present all MathE tools in a more user-friendly manner. The difficulty with the language was also recurrently mentioned; although the platform offers resources in several languages, the ones in English are the majority. All the questions available in both sections of Student’s Assessment are also available only in English. Since MathE is intended to be used by students from all over the world, it is not possible to completely solve the language gap. The decision of the partners was to define English as main language, encouraging all collaborators to develop questions with simple and straight to the point statements, to make it easier for the non-English speaking students to apprehend the question (the fact that mathematical language is universal diminishes the language gap).

Regarding the teachers’ feedback about the features that require improvement, Fig. 11 shows that there is an expressive necessity to increase the diversity of the questions. Also, as was previously mentioned, there is also the urge for the creation of an intermediate level for better distribution and adjustment of the levels of difficulty of the questions.

Finally, the suggestion, given by the students, of releasing the full resolutions of the exercises as part of the provided feed-back, is a point that divides opinions among the platform’s collaborators. While this resource can be beneficial for some students, who will be able to check where their mistakes were made, compare answers and perhaps learn autonomously (especially those who do not have the aid of an available teacher), on the other hand, offering the full resolution at the end of an exam can stimulate students to quickly check where mistakes were made with no further stimulation to their curiosity and sense of investigation.

Conclusion and future work

The use of e-learning strategies became more prominent in recent years. However, with the world plunged into an unexpected pandemic that forced people to rethink all aspects of their lives, in 2020 it ceased to be an option, temporarily becoming the only safe way to attend classes for all levels of education. In this context, the implementation of e-learning methodologies became the highlight of many academic discussions and even teachers and institutions who had been reluctant to implement changes in their teaching practice, found no alternative.

This paper described different points of view about e-learning through a SWOT analysis. However, the performance of the MathE platform, as a collaborative e-learning tool, was the special focus of the paper. The MathE platform was presented, as well as the results collected within its database of assessment questions. The impressions collected from surveys answered by students and teachers about their experience as users of the platform are also herein presented.

It was possible to conclude that MathE is a useful and engaging tool to support the dynamics of teaching and learning mathematics, both from the point of view of teachers and students. Another very valuable conclusion achieved by the authors of this study, was that, as in every other field, there is much room for improvement, regarding the reorganization and the need for regular update of the questions that the platform provides its users. In the near future, the diversity of videos, teaching materials and questions available will be increased and a new intermediate level of difficulty will be introduced. Besides that, the pursue for innovation and the search for ways to engage new users from around the world, making MathE a truly global project, is also mandatory.