Analyzing Sequence Data with Markov Chain Models in Scientific Experiments.

Virtual reality-based instruction is becoming an important resource to improve learning outcomes and communicate hands-on skills in science laboratory courses. Our study attempts first to investigate whether a Markov chain model can predict the students' performance in conducting an experiment and whether simulations improve learner achievement in handling lab equipment and conducting science experiments in physical labs. In the present study, three cohorts of graduate students are trained on a microscopy experiment using different teaching methodologies. The effectiveness of the teaching strategies is evaluated by observing the sequences of students' actions, while engaging in the microscopy experiment in real-lab situations. The students' ability in performing the science experiment is estimated by sequential analysis using a Markov chain model. According to the Markov chain analysis, the students who are trained via a virtual reality software exhibit a higher probability to perform the steps of the experiment without difficulty and without assistance than their fellow students who attend more traditional training scenarios. Our study indicates that a Markov chain model is a powerful tool that can lead to a dynamic evaluation of the students' performance in science experiments by tracing the students' knowledge states and by predicting their innate abilities.

Introduction

In science courses, instructors do their best to communicate knowledge in terms of content and skills. A student who pursues to successfully complete a science course must not only have understood and assimilated the basic principles of specific science subjects, but he/she should have also acquired specific practical laboratory skills [21]. Besides, it is commonly acceptable that laboratory hands-on skills have always been a key pillar of science education. Μany researchers claim that the best way to obtain them is through practicing and not through passively watching face-to-face demonstrations in science labs or simply watching instructional videos [4].

Although practicing in a physical lab is an ideal way of being trained in experimental techniques, constantly equipping and maintaining these labs is becoming more and more expensive and prohibitive for educational institutions [34]. A robust and affordable solution to overcome: (a) the expensive update, (b) the safety issues that result from the misuse of sensitive and complex lab instruments from novice or unprepared students, and (c) the current needs for distance experimental training that arose from the new COVID-19 situations, is to immerse in a virtual lab and interact with the virtual lab equipment.

Virtual Reality (VR) is a cutting edge emerging technology that the last decades demonstrates a great potential to change and modernize the way learners are trained in hands-on skills in many fields such as medical, engineering, natural sciences, etc. [6, 29, 44, 49, 57, 60]. Makransky and Lilleholt [35] mention that many business analyses and reports (e.g., Belini et al. [3], Greenlight & RoadToVR [17]) predicted that VR would be the biggest future computing platform of all time as it could revolutionize the entertainment, gaming and education industries. Review papers mention the research methodologies used in the area of adaptive systems like 3D virtual learning environments [51]. Many supporters of VR technology claim that this alternative educational approach facilitates learning due to the ability of the human brain to perceive better and assimilate easier a three-dimensional (3D) computer-graphics representation than a simple text [11]. Many studies show that simulations can be a very promising and affordable tool for learning and instruction [30, 56, 64], especially for users who are not aware of information technologies [14]. Virtual laboratories have overall positive effects on students’ cognitive load, skills development and motivation [35]. Several VR educational applications have been designed for STEM (Science, Technology, Engineering, Mathematics) domains [9, 45, 59], as interaction with such environments has shown, among other things, gains in deep and certain conceptual understanding, experimental experience and problem-solving ability [23]. The exploratory character of the virtual worlds that offers free navigation using the first-person user viewpoint may be the reason why VR applications are educationally superior also to multimedia [66].

There are several research studies that explore whether in science courses, attending traditional educational scenarios in physical labs is more beneficial than following more novice teaching strategies that include interaction with virtual environments [5, 36, 37, 40, 43, 53]. All researchers agree on the fact that physical labs play a unique role as it is a sine-qua-non in science [20, 22, 69], but more and more the VR technology reveals its substantial contribution to the successful achievement of the learning outcomes in laboratory courses. Paxinou et al. [42] provided evidence in favor of the use of a VR educational software, as a supplementary tool to the traditional laboratory learning methods in Biology Makransky et al. [36, 37] claimed that simulations must be used as a tool for preparation for the lab experiments. Authors in Xu et al. [67] showed that in developmental biology the combination of a virtual oriented and a traditional methodology in teaching promotes effective student learning.

In a science laboratory course, a way to evaluate the effectiveness of a certain teaching procedure is to explore whether the learning outcomes have been fulfilled. Have the students understood the introduced concepts? Have the students managed to obtain the required laboratory skills? The level of the students’ understanding of the new topics can be easily assessed, for example, through scoring specially designed written tests, but assessing the gained practical skills is a quite multidimensional task [46].

Data, derived from the participation of students in educational research, are a powerful tool to researchers as they can utilize them to identify hidden patterns by using analytics techniques [15, 16, 27, 33, 41, 61, 63, 65]. This study is initially based on the assumption that the effective completion of all the steps comprised an experiment, is a robust indicator that the learner has a high perception of the lab environment and has also acquired all the necessary hands-on skills. Therefore, we trained three groups of students on the Microscopy Experiment, by applying three different teaching scenarios, to investigate the predominance of the best scenario. After the different educational interventions, the students’ ability to handle and operate properly a microscope, was evaluated through a specially designed worksheet. According to that worksheet, the microscopy experiment was divided into 13 steps. As is the case in every science experiment, the 13 steps had to be performed strictly in the given order and without skipping any of them. As a result, the data derived from the observed students’ actions when performing those 13 steps, were sequential. In this study, we decided to use a popular method for analyzing our sequential data, a Markov-based technique, and more specifically, a Markov chain model.

Markov model-based techniques are useful methods to analyze data, where order matters [32, 48]. Markov chain models and Hidden Markov models (HMMs) are both statistical models that belong in this category. Considerable research has been conducted on Markov chain models in many different settings, such as to predict enrolments for an education system or to model teachers’ behavior in the decision‐making process or finally to analyze genetic algorithms [26, 39, 47, 55]. On the other hand, HMMs have extensively been used to model the behavior of individual students regarding their engagement and their motivation towards the learning procedure [2, 10, 18, 50, 54, 62]. Arieli-Attali et al. [1] used a HMM to learn about test takers’ choice-making behavior in a self-adapted test, Shih et al. [52] proposed a HMM that could discover student learning tactics, Tadayon and Pottie [59] and He and Gao [19] used a HMM to analyze and make predictions of the students’ performance in educational games, and Jeong et al. [25] used HMMs to examine the effect of metacognitive prompting on students’ learning in the context of our computer-based learning-by-teaching environment.

Although the HMM is based on augmenting the Markov chain model, in this study we used the latter, as we aimed to model the sequences of observable events, like the observed student's actions when conducting an experiment in a science lab, and not any unobservable influences. To the best of our knowledge, no other studies have relied on a Markov chain model to provide evidence in favor of an educational intervention by modeling the students’ performance in a lab environment. Neither such a model has been used to further investigate whether the integration of Information and Communication Technologies into teaching, helps students acquiring those experimental skills that are necessary for performing successfully an experiment. Based on the fact that a science experiment, by nature, is a prototypical procedure to obtain sequential data, our main research question is whether a Markov chain model can offer additional information about the comparison of different teaching methods that are applied in laboratory science courses.

At this point, it is worth mentioning Deep Learning Techniques are also very popular techniques that could have been considered for the processing of the data in this study. Deep Learning comprises a state-of-the-art learning paradigm that sheds new light on neural network approaches. Long-Short Term Memory (LSTM) components, Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs) bring in the Machine Learning Community a great deal of innovation that provides accurate complex predictions on a diverse set of problems like speech processing, image processing, text processing and the like [24, 28, 68]. But, despite their groundbreaking impact on recent application problems like self-driving cars, in this study, we decided to choose simpler white box algorithms that fitted better for our problem where the understanding of the proposed solution was necessary.

The main aim of our study was twofold. First, we investigated whether a Markov chain model is a useful tool that can evaluate the students’ performance in science experiments by tracing their knowledge states and by predicting their innate abilities. Second, we interpreted the produced data to compare the effectiveness of three different teaching strategies applied in a laboratory science course. Such a comparison could lead to serious decisions regarding the improvement of teaching and learning practices. Furthermore, in our study, the Markov Chain methodology used for the sequence data analysis is presented as a step-by-step procedure in an attempt to make it totally understandable. Our results provided evidence in favor of using such a statistical model for modeling and predicting the students’ actions when conducting hands-on exercises. In addition, the Markov chain methodology helped us come to the conclusion that the students who were trained on the microscopy experiment by interacting with a VR educational application, exhibited a greater ability in performing the specific experiment in the physical lab.

Research Design and Methodology

Sixty-two, 4th-year, undergraduate students at the Department of Primary Education of University of Patras, in Greece, participated in this study. The students were enrolled in the Computers and Education course, where the learning outcomes are (a) to practice in computer use, (b) to be informed about the latest developments in educational software for primary and secondary education and (c) to be educated on technology-assisted teaching and learning.

This research focuses on the acquisition of hands-on skills after the training through a specific methodology. In particular, the participants are trained to operate a photonic microscope, the most basic and essential instrument in a biology lab. They are educated through three different teaching methods, one of which is the use of educational software. Our sample is a novice audience, as it brings a zero to minimum prior knowledge on the subject of microscopy. We chose the specific participants, as the students who attended the aforementioned graduate program, will face the challenge to train their young pupils, in simple experiments such as that of microscopy, when becoming teachers in primary schools.

To educate the 62 students on the use of the photonic microscope we separated them into three different groups; the T-Group who attended a traditional face-to-face demonstration of the microscopy experiment, the V-Group who watched an instructional video on the microscopy experiment, and the VR-Group who downloaded the Onlabs software (https://sites.google.com/site/onlabseap/) and interacted with the simulated microscope in a VR environment to perform the microscopy experiment. The specific experiment included the use of all of the 4 objective lenses of an optical (light) microscope to focus on human, animal and plant cells. Table 1 presents briefly the three phases of the educational experiment.

Table 1

A brief presentation of the designed educational experiment

1st Phase (Introduction to microscopy)—1 h
1. All students attended a general-oriented tutorial on the light microscopy 2. All students filled in a quiz on microscopy 3. The students were divided in three cognitive balanced groups based on their quiz scores
2nd Phase (Trained on the microscopy experiment)—1 h
T-Group	V-Group	VR-Group
The T-Group attended a live demonstration of the light microscopy experiment, performed by the lab tutor	The V-Group watched an instructional video on the light microscopy experiment	The VR-Group was trained on the light microscopy experiment by using the Instruction Mode of the Virtual Reality application Onlabs
3rd Phase (In the physical biology lab)—1/2 h
1. All students used a light microscopy experiment to focus on a specimen in the Biology lab 2. In parallel, all students filled in a worksheet designed to assess the acquired skills on operating an optical microscope

The current study is focusing on the 2nd Phase and especially on the 3rd Phase of the scenario. As presented in Table 1, in the 2nd Phase, the VR-Group was trained on the microscopy topic via the Onlabs, and more precisely by using the Instruction Mode of this VR application. The Onlabs is offered in three modes: The Instruction Mode, the Evaluation Mode and the Experimentation Mode. When using the Instruction Mode, specific instructions keep appearing on the screen guiding the students to operate properly the microscope. In case they cannot respond to the instruction, they have the option to click on the globe-button on the left up corner of the screen and a written hint is appearing to their help (Fig. 1).

Fig. 1

Screenshots from the instruction mode of onlabs; a a given instruction, b a given hint

If and only if, the students complete an instruction, they can move on to the next one. The students keep following the instructions and taking into consideration the hints until they manage to focus successfully on a specimen by using all the objective lenses of the microscope. Only once, the VR-Group interacted with Onlabs, the V-Group watched the instructional video and the T-Group attended the face-to-face tutorial.

According to the 3rd Phase of the project, after the training in the 2nd Phase, the three groups entered the biology lab. There, all students used their own optical microscope to set the instrument and focus on different cells. In parallel with conducting the specific microscopy experiment, the students had to fill in a worksheet (hard copy) following the method presented by Paxinou et al. [45].

According to this worksheet, the experiment was a procedure divided into 13 steps. For each step, an instruction was given. The students had to follow the given instruction to perform each step, in a specific order. Each time a student performed a step, he/she had to tick on one of the three given outcomes (states), A, B or C, (Table 2), before moving on to the next step.

Table 2

Three possible outcomes denote the students’ actions in the microscopy experiment

State A	I completed the step easily
State B	I finally completed the step but on difficulty
State C	I couldn’t complete the step by myself so I asked for help (from the supervisor or a fellow student)

Table 3 demonstrates the worksheet where instructions for each one of the 13 steps are given.

Table 3

The assessment worksheet

Step	Instruction	State			Step	Instruction	State
1	Turn the light on	A	B	C	8	Move your specimen by rotating the stage and the specimen knob so as your specimen gets in the light path (visual field)	A	B	C
2	Adjust the light intensity	A	B	C	9	Rotate the coarse adjustment knob to move the stage up or down until the image comes into focus. Ask for your supervisor to come and check your focus	A	B	C
3	Rotate the iris (diaphragm) lever to the leftmost position	A	B	C	10	Rotate the revolving nosepiece to set 10X objective lens into position	A	B	C
4	Lift the condenser knob up to its highest point	A	B	C	11	Focus on your specimen. This time use also the fine adjustment knob. Ask for your supervisor to check your focus	A	B	C
5	Rotate the revolving nosepiece so as to set the objective lens with the lowest magnification into position	A	B	C	12	Rotate the revolving nosepiece to set 40X objective lens into position	A	B	C
6	Place the specimen on the stage and stabilize it with the stage clips	A	B	C	13	Focus on your specimen by using only the fine adjustment knob. Ask for your supervisor to check your focus	A	B	C
7	Enter the microscoping mode by looking through the eyepieces. Slide the eyepiece housing to match the width of your eyes and then make the diopter adjustment	A	B	C

In this point it is important to highlight that in laboratory experiments the order of the given instructions must be followed in a strict way. As a result, the students had to perform each step in the order presented in Table 3, otherwise, the experiment would not have been successfully completed.

At the end of the assessment, each student’s data record included an ordered sequence of states, representing her or his performance on the microscopy experiment. The corresponding state sequence is, for example. Subsequently, the student performance is modeled using a Markov sequence model called the Markov chain model [38], which is presented in detail in the following section.

The Markov Chain Model

In this study, and in the context of an educational setting, we focus on the completion of an experiment executed by three groups of students (observed subjects) after been educated on microscopy, each by a different educational methodology. The experiment is presented as a sequence of steps, each with a number of possible outcomes. As an outcome, we consider the observable student’s action, among the possible ones, while performing a certain step in the experiment. If the probability of observing any of the possible outcomes in a single trial/step, depends on a predetermined time-invariant probability distribution, then we can claim that the experiment follows the so-called multinomial distribution model [8, 12]. Furthermore, we assume that the set of the possible outcomes is the same for all steps in the experiment.

The multinomial model is not a precise and accurate representation of a science experiment as the different steps of such an experiment might have varied difficulties, and because of that, varied probabilities of the observing outcomes. The coincidence of having different steps with the same difficulty may only happen in case different steps exhibit the same complexity, which is rarely the case in real-lab situations. Another assumption while using a multinomial model, which does not hold true for a science experiment, is the fact that the probability of observing a certain outcome at a particular step along the way, has a data distribution which is independent of the outcomes of the experiment in the nearby positions. Usually what happens in a series of actions is that the probability of observing a certain outcome at a particular position in the sequence is affected by the outcomes found at adjacent positions in the sequence.

Given the previous discussion, it appears that a Markov sequence model seems to be a more accurate representation of a science experiment. A Markov sequence model assumes that an outcome at a particular position in the sequence depends on the outcome in the previous position. That is, if an outcome Α is observed at the current position, then the probability of observing any one of the outcomes at the next position depends on a predetermined probability distribution. Markov models embody randomly changing systems [13] and it is a popular method to model sequential data. There are four common Markov models used in different situations: the Markov chain, the Hidden Markov Model, the Markov Decision Process and the Partially Observed Markov Decision Process. In our study, we use the Markov chain model which represents a class of stochastic processes of great interest for the wide spectrum of practical applications [7].

In a Markov chain model, there is a certain number of outcomes (states) which are observed while the subject (e.g. the student) is conducting a sequential process (e.g. a science experiment divided into numerous steps). The state space is represented as and is simply a set containing the possible states of the process, that is, [31]. Each state corresponds to a possible outcome that can be observed while performing a step in the aforementioned science experiment. The process starts in one of these states and moves on successively from one state to another, creating a sequence of states where and denotes the number of the steps and the length of the sequential process. The changes in the observed states across the sequence are called transitions. The probabilities associated with various state changes are called transition probabilities. For example, if the sequence at the step is at the state (current state) and when moving to the next step, is at state (next state), the transition probability is represented as where:

As it is already mentioned, this probability does not depend upon any states other than the current state. The process can also remain in the state it is in, and this occurs with probability . An initial probability distribution, defined on , specifies the starting state. Usually, this is done by specifying a particular state as the starting state. The overall process is then characterized by a state space, a probability transition matrix describing the probabilities of particular transitions and a probability distribution over the state space for the initial state.

By applying many times the Bayes’ Rule, for any probabilistic model of sequences, we can write the probability of the sequence as:

The key property of a Markov chain is that the probability of each symbol depends only on the value of the preceding symbol , not on the entire previous sequence, e.g. [10]. Therefore, Eq. 2 becomes

The main use of Eq. 3 is to find the values for a likelihood ratio test. To do that we use real data from our experiment. Our goal is to be able to discriminate between two Markov chain models and decide upon which one is the most probable to have generated a certain sequence. Please note that in our study we induce three such Markov chain models, each one corresponding to one of the three participating groups (T, V and VR-Group).

Given that in our study we observe the students’ performance from three different groups, it is safe to run three such discrimination tests (), each one corresponding to one pair from our three Markov chain models. For demonstration purposes, we present how to use the likelihood ratio test to discriminate between the Markov chain model induced by the VR-Group and the T-Group data. The transition probabilities for each one of these two models were set using the following equations:where is the number of times, state follows state in all of the sequences of students in VR-Group and T-Group (the so-called training sets), respectively. These are actually the maximum likelihood (ML) estimators for the transitions probabilities. To use these two models for discrimination we calculate the log-odds ratio (Score):where are the log-likelihood ratios of corresponding transition probabilities. Please observe that a positive value for the above function indicates that the sequence has been more probably generated from the Markov chain model of the VR-Group. On the other hand, a negative value indicates that the sequence has been more probably generated from the Markov chain model of the T-Group.

Discussion and Research Findings

In our study, the state space is a set of three states (Table 2). As a result, the state space is presented as , where . Each student has to perform individually the microscopy experiment which is divided into 13 steps. As a result, the length of the sequential process N appearing in Eq. 2, is now equal to 13. While following the given instructions and conducting the sequential steps, strictly in the given order, each student moves from state to state

At this point, we have to highlight that not knowing how to carry out an instruction for a specific step, leads to failure, as there is no option to move on to the next step without having completed the current one. Fortunately, lab supervisors, in physical lab environments, or avatars and help/hint buttons, in intelligent tutoring systems like VR environments, offer learners second chances and the capacity to move on. Based on that, in our study, state A corresponds to the “I completed the step easily” action, state B corresponds to the “I finally completed the step but on difficulty” action, and state C corresponds to the “I couldn’t complete the step by myself so I asked for help (from the supervisor or a fellow student)” action (Table 2).

After the completion of the experiment, sequential data that represent the specific actions each student executes while performing the steps, are produced. For example, in the VR-Group, the student with ID 16, exhibits the following pattern of actions in the 13 steps of the experiment: A, A, A, B, B, B, A, B, C, A, A, A, A. By observing this individual sequence, we count 12 transitions in total, when moving from one state to another. More specifically, we count five transitions from state A to itself, 2 transitions from state A to state B, zero transitions from state A to state C, one transition from state B to state A, 2 transitions from state B to itself, etc. Based on this counting, pAA = 5/7, pAB = 2/7, pAC = 0, pBA = 1/4, pBB = 2/4, etc. These transition probabilities can be organized in a transition matrix, such as the one presented in Table 4. A transition matrix is a common way to store the transition probabilities for a Markov chain model. The rows of the transition matrix represent the states observed at the current step in the sequence, while the columns represent the states that will be encountered at the next step in this sequence. It is easy to notice that in the transition matrix, the rows of the transition probabilities sum to 1.

Table 4

The transition probability matrix for the student with ID 16

State	A	B	C
A	5/7	2/7	0
B	1/4	2/4	1/4
C	1	0	0

The transition probabilities of students during the execution of the experiment comprise the basic parameters for a stochastic/probabilistic model such as the Markov chain model. These parameters are typically estimated from large sets of cases often called a training set. For instance, the probability pBC = 1/4 was estimated as the observed frequency of transitions from state B to state C, in the training set of the students’ performance data. In this way, we obtain the transition probabilities by counting the nine probabilities which represent the entire space of possible transitions among states in our experiment, as long as the training sequences are not systematically biased towards a peculiar state transition pattern.

We expect that these observed frequencies constitute reasonable estimates of the underlining transition probabilities of our Markov chain model. It can be shown that using the frequencies with which a student transitions from one state to another, as the aforementioned transition probabilities, maximizes the total probability of all the sequences given the specific Markov chain model (the likelihood). The specific way for estimating these models is known as Maximum Likelihood Estimation (MLE).

Table 5 presents the average transition probabilities matrices for T, V and VR-Group. According to these matrices, a T-Group student who is currently in state C and, therefore, ask for help to move on, has a 63.0% chance of moving into the higher ability state A at the next step. The diagonal entries represent the probabilities of remaining in the same state. For example, a VR-Group student has a 45.2% chance of remaining in state B at the next step. A VR-Group student who is currently in state C has a 0.0% chance of remaining in this state.

Table 5

Average transition probabilities matrices for T, V and VR-Group

All three transition probability matrices suggest that there is a degree of inertia regarding state A. The probabilities for persisting in state A are higher than those of shifting to state B or C (77.6%, 78.7% and 86.6% for T, V and VR-Group, respectively). On the other hand, regarding states B and C, the probabilities for shifting to state A are higher than any other probability, (except from the V-Group when the current state is C).

The powers of the transition matrices give us interesting information about the experiment as it evolves. Let be the transition matrix of the Markov chain for the T-Group:

The ijth entry of the matrix gives the probability that the Markov chain, starting in state s, will be in state s after steps. For example, let us compute which represents the probability that a Markov chain starting in state A will be in state C after 2 steps. We can easily observe that to move from state A to state C in 2 steps, we can either (a) remain to state A in the first step and move to state C in the second step, or (b) move from state A to state B in the first step and move from state B to state C in the second step or (c) move from state A to state C in the first step and remain to state C in the second step. This can be represented by the following equation:

Equation 7 indicates that is given by the dot product between the first row and the third column of the transition matrix . By substituting the corresponding values for the row and column vectors of the aforementioned matrix we get

The result above (0.097) is actually the value of the cell (1,3) in the matrix computed below:

Powers of transition matrices indicate the long-term behavior of a Markov chain model. In Table 6, successive powers of an adequate size of the transition probability matrices for T, V and VR-Group are presented. We note that after a certain number of steps for each group, the state predictions are independent of the current state.

Table 6

Powers of the transition probability matrices for T, V and VR-Group

Power of matrix	T-Group	V-Group	VR-Group
P1	0.7760.1280.0960.7380.1670.0950.6300.2590.111	0.7870.1280.0850.6000.3330.0670.4210.4740.105	0.8660.0860.0480.5160.4520.0320.9000.1000.000
P2	0.7570.1450.0970.7560.1470.0970.7500.1530.097	0.7310.1840.0850.7000.2620.0790.6600.2620.079	0.8380.1180.0440.7090.2510.0390.8310.1220.047
P3	0.7560.1460.0970.7560.1460.0970.7560.1470.097	0.7210.1950.0840.7160.2010.0830.7090.2090.082	0.8270.1290.0440.7800.1780.0420.8250.1310.044
P4	0.7560.1460.0970.7560.1460.0970.7560.1460.097	0.7200.1970.0830.7190.1980.0830.7180.1990.083	0.8230.1340.0440.8050.1510.0430.8220.1340.044
P5		0.7790.1970.0830.7190.1970.0830.7790.1980.083	0.8210.1350.0440.8150.1420.0440.8210.1350.044
P6		0.7190.1970.0830.7190.1970.0830.7190.1970.083	0.8200.1360.0440.8180.1380.0440.8200.1360.044
P7			0.8200.1360.0440.8190.1370.0440.8200.1360.044
P8			0.8200.1360.0440.8200.1360.0440.8200.1360.044

For instance, looking at the fourth power of the transition matrix for T-Group, , the transition probabilities for the three states A, B and C are 0.756, 0.146 and 0.097 respectively, no matter where the chain started. This is an example of a type of Markov chain called a regular Markov chain. For this type of chain, it is true that the long-range predictions of states are independent of the starting state.

Subsequently, we illustrate the long-term behavior of a Markov chain, when it starts in a state chosen by a probability distribution on the set of states. In our study, a probability distribution is a row vector that comprises three components whose entries are non-negative and sum to one. If is such a probability distribution which represents an initial state of a Markov chain, then we view the component of as representing the probability in Eq. 3, that the chain starts in state s. To denote this, we choose to be the probability vector with its entry equal to 1 and all other entries equal to 0. For example, the probability vector is the initial probability distribution when our Markov chain starts in state B. We note that if we want to estimate the distribution of the states after a number of steps, we simply need to multiply the initial probability distribution with the corresponding power of the transition matrix of the Markov chain model. So, if our initial probability distribution vector is , then the probability distribution vector after three steps is going to be . If in the formula before, we consider the transition matrix , as the transition matrix for the VR-Group, the probability distribution vector after three steps is going to be .

A probability distribution satisfying the equation , is stationary because the transition matrix does not change the probability of the states of the process. Such a distribution is unique in a regular Markov chain and in our study is (0.756 0.146 0.097) for T-Group, (0.719 0.197 0.083) for V-Group and (0.820 0.136 0.444) for VR-Group (Table 7). In other words, although increases theoretically without a bound, after a certain number of steps its value attains a limit (for example, becomes 0.756 for the T-Group, 0.719 for the V-Group and 0.820 for the VR-Group).

Table 7

The stationary transition probabilities after steps, for

	piAn	piBn	piCn
T-Group	0.756	0.146	0.097
V-Group	0.719	0.197	0.083
VR-Group	0.820	0.136	0.444

Since state A is related to the response I completed the step easily, it is obviously a state that corresponds to higher student’s ability than state B and C, in terms of how to treat and operate successfully a microscope. As a result, the observation where  >  >  is an indicator that the interaction with the VR lab environment helped the students in VR-Group become more capable of handling the microscope than their fellow students in other groups. These findings agree with the students’ scores in a written assessment test based on the subject of microscopy. In detail, in a zero to ten scale, the students in the T-Group received a score of 6.52 ± 1.31, in the T-Group 6.64 ± 1.19 and in the VR-Group 7.39 ± 1.18 (T-Group: t =  − 2.538; df = 29; p < 0.05, V-Group: t =  − 4.353; df = 28; p < 0.001, VR-Group: t =  − 8.823; df = 23; p < 0.001). Based on the results presented in Table 7, it is also noteworthy that watching passively an instruction video is a less effective teaching method than attending a face-to-face live demonstration of the experiment, as it is less probable that the V-Group students will be in state A than the students in the T-Group.

While in the previous discussion we argued upon the collective performance of the three groups of students, we now proceed to characterize the performance of each individual student, based on the observed sequence of actions in the experiment that he/she conducted. To accomplish this, we make use of Eq. 6 that assigns a score to a student, based on the sequence that he/she exhibits. The score in Eq. 6 can be used to discriminate between a pair of Markov chain models by summing over the logarithm of ratios of corresponding transition probabilities, in these two models.

Let us consider the transition matrices for T and VR-Group presented in Table 5. To apply Eq. 6, we need to take the ratio of these two matrices, element by element, as shown in the following equation:followed by the logarithm base 2 of the ratio of the two matrices above, which is displayed below:

If we now consider any possible sequence observed by a student and by applying Eq. 6 along with the information in matrix , we can easily tell whether the sequence has been generated by the Markov chain model of the VR-Group or of the T-Group. Let us consider the sequence that the student with ID 11 exhibits: . It is obvious that this sequence contains eight AA transitions, one AB transition, two BB transitions and one BA transition. For each one of these transitions, Eq. 6 tells us to consider the corresponding value of the cell from the matrix. In this way, based on Eq. 6 for the sequence of the student with ID 11,  = , we get

The positive value (3.045) of the score for the given sequence suggests that the sequence conforms to the Markov chain model of the VR-Group. In the following table (Table 8), we present the data derived from the observed sequences of the 19 students in the VR-Group. For students with IDs 7, 9, 13 and 15, the is negative (all the elements that correspond to these students are marked in bold in Table 8). This finding indicates that although these three students originally belonged to the VR-Group, the group where the students have the highest probability to perform the experiment without difficulty and without any help, the log-likelihood discrimination test assigned them to the T-Group.

Table 8

The VR-Group student ID followed by his/her sequence along with the computed by Eq. 6 and the assigned group

ID	Sequence	Score(x)	Group
1	AACAAAAAAAAAA	1.104438	VR
2	AAAAAAAAAAAAA	1.900220	VR
3	AABAAAAABBBAA	1.635277	VR
4	AAAAAAAAAAAAA	1.900220	VR
5	AAAAAAAAAABAA	0.487979	VR
6	AAAAAAAAAAAAA	1.900220	VR
7	AAABAAAACBBAA	– 1.592277	T
8	AAAAAAAABAAAA	0.487979	VR
9	AAAABABABAAAA	– 2.336503	T
10	AAAAAAAAAAAAA	1.900220	VR
11	AAAAAABBBAAAA	3.047518	VR
12	AAAAAAAAACAAA	1.104438	VR
13	AAABAAAACAAAA	– 0.307802	T
14	AAAAAAAAAAAAA	1.900220	VR
15	AAACACAAAACAA	– 0.487125	T
16	AAABBBABCAAAA	0.946543	VR
17	AAAAABAAABBAA	0.355508	VR
18	ABBBBBAABBBAA	6.754355	VR
19	AACAAAAAACAAA	0.308657	VR

The analogue table for the T-Group is presented below (Table 9). For students with IDs 8, 12–18 and 23, the is positive. As a result, although these students originally belonged to the T-Group, the log-likelihood discrimination test assigned them to the VR-Group.

Table 9

The T-Group student ID followed by his/her sequence along with the computed by Eq. 6 and the assigned group

ID	Sequence	Score(x)	Group
1	AAAAABAAABAAA	– 0.924262	T
2	AAAAAACBAABAA	– 2.872046	T
3	AACAABAABAAAA	– 1.720043	T
4	AABACBACBAAAA	– 6.232072	T
5	AACAABBABAAAB	– 1.178088	T
6	AABAABACAAAAA	– 1.720043	T
7	AACCAAAAABAAA	– 9.788082	T
8	AAAAAABABBBAA	1.635277	VR
9	AAABCABAABAAA	– 3.025236	T
10	AAAAACAABAAAA	– 0.307802	T
11	BAACAACAAACAA	– 1.161552	T
12	AAAAAAAAAAAAA	1.900220	VR
13	AABAAAAAAAAAA	0.487979	VR
14	AABAAAAAAAAAA	0.487979	VR
15	AABCABBBBAAAA	2.226313	VR
16	BAAAAAAAAAAAA	1.225793	VR
17	AAAACAAAAAAAA	1.104438	VR
18	AACAAAAAAAAAA	1.104438	VR
19	ABCBCAAAAACAA	– 4.249514	T
20	AAAACAAACCBAC	– 2.888718	T
21	ABABAACBBAAAB	– 3.742332	T
22	AAACAACBABAAA	– 3.667828	T
23	AAAAAAAAAAAAA	1.900220	VR
24	AACCAABAAAAAA	– 9.788082	T

Figure 2 presents the histogram of the scores for all the sequences of the T and VR-Groups. The T-Group sequences are shown with light blue, the VR-Group sequences with salmon, whereas the purple color results from the overlap of the light blue and the salmon color. This overlap of the values is around zero.

Fig. 2

Distributions of scores of students for the T and VR-Groups

Given our ability to discriminate sequences coming from different groups of students, we can easily segment a given (possibly) unknown sequence, to the most probable group path. For example, if we were segmenting the sequence of the student from the T-Group with ID 15 (Table 10), we would see that in the 1st and from 7th to 12th step his/her actions are closer to the VR-Group while during the 2nd–6th step are closer to the T-Group.

Table 10

The probable group path for the student with ID 15 from the T-Group

Step	Score(x)	Group
1	0.15835	VR
2	– 0.42111	T
3	– 1.98299	T
4	– 1.46756	T
5	– 2.04703	T
6	– 0.60890	T
7	0.82921	VR
8	2.26733	VR
9	1.75125	VR
10	1.90961	VR
11	2.06796	VR
12	2.22631	VR
13	2.22631	VR

Conclusions

In this paper we investigated the capability of a Markov chain model to evaluate the students’ performance in science experiments by tracing the students’ knowledge states and by predicting their innate abilities. Consequently, by analyzing the data derived from the Markov model-based techniques, we tried to compare the effectiveness of three different teaching methodologies in communicating experimental skills to students. To accomplish that, three groups of students were trained on microscopy through three different teaching scenarios: the T-group attended a classical face-to-face lab tutorial, the V-Group watched, in class, an instructional video, and the VR-Group entered the Computers lab, downloads a VR educational software and interacts with a simulated microscope. After the educational interventions, the students assessed their gained hands-on skills by performing a microscopy experiment of 13 steps, in real lab situations. There, we recorded through a worksheet, the students’ actions and subsequently we analyzed their states. Each time a student performed one of the 13 steps, he/she had to tick on one of the three given states A, B or C, before moving on to the next step. State A corresponded to “I completed the step easily”, State B to “I finally completed the step but on difficulty” and State C to “I couldn’t complete the step by myself so I asked for help”.

Using training data from three groups of students, we came up with three different Markov chain models each one represented by a different transition matrix. We made use of the stationary distribution property of the transition matrices to assess the performance of each one of the three groups. We exploited the underlying Markov property of a sequence to build a scoring function that discriminates between a pair of input models. It was also shown that the Markov chain models are a promising solution in educational lab environments to model students’ actions and make the prediction throughout the experiment. The results indicate that examining sequential data from the experiment could lead to dynamic evaluation of student experimental skills before he/she even finishes the specific experiment. The three Markov chain models presented in this project could assign every student to the T, V or VR-Group, (or in other words, to the low, medium, or high ability group), giving the instructor the potentiality to make a beneficial intervention and provide the students' efficient feedback.

Our results indicated that a VR-oriented learning procedure is more beneficial and more effective in helping the students to acquire the necessary experimental skills for a specific lab experiment. Students from the VR-Group were better prepared and exhibited a higher probability to conduct the steps of the experiment easily and without any help than the students in V and T-Group. Although nowadays video is the most important digital media on the Internet, the role of the watchers is still the same; they are a passive audience. Therefore, according to the Markov chain analysis, the students in the V-Groups had a lower probability to conduct the steps easily and by themselves than the students in T-Group who also attended passively a demonstration of the experiment but this demonstration was live and they had the opportunity to stop the tutor and pose questions. The average transition probability of the V-Group students to move on to the higher ability State A at their next step, when they were in State C, was 42.1% whereas this probability was 63.0% and 90.0% for students in T and VR-Group, respectively. Furthermore, the probabilities for persisting in State A were higher for VR-Group (86.6%) compared to the percentages of 77.6% and 78.7%, for T and V-Groups, respectively.

Recommendations

Our findings reinforce the point of view that educational institutions should take advantage of technological innovations such as VR, and enrich the conventionally applied learning methods and curriculums in an attempt to successfully engage their students and satisfy the initially defined learning outcomes. A VR application such Onlabs could be offered to students that attend laboratory courses to engage them in a creative way into the educational procedure and prepare them for the real lab environment. The higher performance of the VR-Group was probably a consequence of the active participation, the high interactivity and the individualization, features that are highlighted when interacting with a VR environment. Furthermore, models like the one suggested in this study, could be a useful tool for educators to predict their students’ performances in science experimentation through tracking their knowledge state. Such information is an interesting insight for academic management and supervision.

Future Research Focus

Experimental data indicated the good fitness of our models to the training data. To validate the accuracy of our findings, further experimentation with larger training data sets is necessary. A focus of the future research will be on designing an empirical study where a HMM can be applied to model students’ emotional states like frustration, anxiety or interest, when performing a science experiment in a physical lab.