Reliability and validity of free software for the analysis of locomotor activity in mice.

Background: Kinovea software that tracking semi-automatically the motion in video screen has been used to study motion-related tasks in several studies. However, the validation of this software in open field test to assess locomotor activity have not been studied yet. Therefore, this study aimed to examine the reliability and validity of this software in analyzing locomotor activities.
Methods: Thirty male Institute Cancer Research mice were subjected in this study. The results examined by this software and the classical method were compared. Test-retest reliability and inter-rater reliability were analyzed with Pearson's correlation coefficient and intraclass correlation coefficient (ICC). The validity of this software was analyzed with Pearson's correlation coefficient.
Results: This software showed good test-retest reliability (ICC=0.997, 95% confidence interval [CI]=0.975-0.994, p<0.001). This software also showed good inter-rater reliability (ICC=0.987, 95% CI=0.973-0.994, p<0.001). Furthermore, in three analyses for the validity of this software, there were significant correlations between two methods (Pearson's correlation coefficient=0.928-0.972, p<0.001). In addition, this software showed good reliability and validity in the analysis locomotor activity according to time interval.
Conclusion: This study showed that this software in analyzing drug-induced locomotor activity has good reliability and validity. This software can be effectively used in animal study using the analysis of locomotor activity.

INTRODUCTION

Locomotor activity in animals refers to the movement of an animal from one place to another [1]. Animal studies on brain function usually use rats or mice to measure locomotor activity. In addition, changes in locomotor activity are used to evaluate the effectiveness of drugs [2]. Locomotor activity is also an indicator that shows manic-like behavior and depressive-like behavior [3,4]. Moreover, it is used to assess anxiety or addiction tendencies [5,6]. Thus, locomotor activity is a useful behavioral indicator that can reflect various mental functions.

Locomotor activity has been mostly assessed in the open field test [7]. The open field test measures locomotor activity by allowing the experimental animal to move freely in a square transparent box [7]. Classical method analyzing locomotor activities in the open field test have been measured indirectedly by the number of times the mouse passes crossed lines [8,9]. Although this analyzing method has been widely used, it takes a long time to measure the locomotor activity because it analyzes this activity mannually [8,9].

In recent years, laboratory instruments and equipment have been developed to automatically measure locomotor activity. One of them measures locomotor activity in an open field where an infrared beam is installed [10]. To use this method, it is necessary to have equipment that automatically detects the number of times the experimental animal passes a certain section of the infrared beam. However, this method still indirectly measures the locomotor activity, and information on the positions to which the experimental animal has moved is limited. Another method is to record the movement of the experimental animal and analyze the video to measure the locomotor activity [11,12]. This method can measure the natural movement of the experimental animal, and it can automatically analyze the distance that the animal has moved and its location information. Thus, it is a useful method for studying locomotor activity. However, to use this method, video recording equipment and a computer analysis program must be included.

In recent years, the Kinovea software (version 0.8.24, http://www.kinovea.org) that tracks the movement of objects in videos has been released at no cost to the user. This software have already been used in other research fields [13-20], so it could be used in an effective analysis of locomotor acitivity. If this software has sufficient reliability and validity for analyzing locomotor activity, it will lead to reducing the budget of an animal experiment or using the same budget for other tests, which may promote research involving locomotor activity. However, this software does not yet have the reliability or validity for analyzing the locomotor activity in mice. Therefore, this study investigated whether the Kinovea software have an adaptable reliability and validity in analyzing the locomotor activity of mice, and wheather the accuracy of Kinovea analysis is maintained according to time-interval.

MATERIALS AND METHODS

1. Study subjects and procedure

Thirty male white Institute of Cancer Research (ICR) mice aged 16-17 weeks were used in this study. The experimental animals were subjected to a 1-week acclimatization process and were allowed to freely drink water and consume feed. The temperature of the cage was maintained at 20℃±1℃, and the humidity was 40-60%.

The locomotor activity of experimental animals was measured by an open field test. The following two interpretation methods were used to analyze the results of the open field test: a classical method and a method using the Kinovea software. The results were used to evaluate the reliability and validity of the Kinovea software. This study was conducted after approval of the Animal Experiment Ethics Committee of Pusan National University Hospital.

2. Measurement of locomotor activity

The open field test was used to measure locomotor activity. It was conducted in an open field of 50×50×30 cm for 60 min. The bottom of the space was black. The test was recorded with a digital camcorder (HDR-CX560, Sony, Tokyo, Japan). The recorded videos were analyzed using two methods: a classical method and method using the Kinovea software. In the classical method, a space of 50×50 cm was divided into 25 squares by drawing horizontal and vertical lines at intervals of 10 cm, and the number of times that the forelegs of the mouse crossed a section was counted by a reader (Fig. 1A). This value was used as a standard indicator of the locomotor activity.

Fig. 1.

Classical method measures locomotor activity in the open field test by counting the number of times that the forelegs of the mouse pass crossed lines (A), and Kinovea method measures locomotor activity by tracking movements of the mouse (B).

The Kinovea software was designed as semi-automated video tracking software that allows a computer program to automatically track movement (Fig. 1B). In the analysis using the Kinovea software, the recorded videos were loaded into the Kinovea software. A tracking detector was attached to the center of the mouse’s body to track the movement of the mouse in this software. However, if the software cannot track the mouse properly, the reader can reset the tracking in order to improve accuracy. When the video is analyzed with the Kinovea software, the time and spatial information of the mouse’s movements are obtained as x and y coordinate values. Based on this data, the total travel distance was calculated using MATLAB R2013b (MathWorks, Inc., Natick, MA, USA), and the value was taken as an indicator of the amount of locomotor activity.

3. Reliability assessment of Kinovea software

Test-retest reliability and inter-rater reliability were investigated to measure the reliability of the analysis method for locomotor activity using the Kinovea software. With regard to test-retest reliability, the locomotor activity was analyzed in 30 mice by using the Kinovea software. After 1 month, the same researcher analyzed the same video by using the Kinovea software. For inter-rater reliability, the data on locomotor activity in 30 mice, which was analyzed by two different investigator using the Kinovea software, were compared. All investigators were blinded to each other’s results.

4. Validity assessment of Kinovea software

To assess the validity of the analysis method for locomotor activity using the Kinovea software, the method using the Kinovea software was compared with the conventional classical method. The validity of the data on locomotor activity using two methods in 30 mice was compared.

5. Reliability and validity assessment of Kinovea software over time

To evaluate the reliability and validity of the Kinovea software over time, the results of the analysis of locomotor activity over time were divided into intervals of 10 min. The test-retest reliability, inter-rater reliability, and validity of the Kinovea software were analyzed using data collected every 10min.

6. Statistical analysis

G*power was used to determine the number of experimental animals required for the reliability and validity assessment. In the correlation analysis, the correlation coefficient was 0.50, the power was 0.80, and the significance level was 0.05. As a result, the appropriate number of samples was 26, and the reliability and validity were evaluated in 30 mice.

The intraclass correlation coefficient (ICC) was obtained to assess the test-retest reliability and inter-rater reliability when measuring locomotor activity using the Kinovea software. The Pearson correlation coefficient was calculated to evaluate the validity of the Kinovea software. A statistical analysis was performed using the IBM SPSS version 22.0 (IBM Co., Armonk, NY, USA).

RESULTS

1. Reliability of Kinovea software in assessing total locomotor activity

To confirm the reliability of the Kinovea software, the test-retest reliability and inter-rater reliability were determined. The ICC was 0.988 (95% confidence interval [CI]=0.975-0.994, p<0.001), which was at the excellent level when testretest reliability was analyzed at 1-month intervals (Fig. 2). In addition, when two readers analyzed the same videos to determine the inter-rater reliability, the ICC was 0.987 (95% CI=0.973-0.994, p<0.001) (Fig. 3).

Fig. 2.

This graph showed good test-retest reliability of Kinovea tracking software program in two times analyses at a month-interval (ICC=0.988 [95% CI=0.975-0.994], p<0.001). ICC, intraclass correlation coefficients; CI, confidence interval.

Fig. 3.

This graph showed good inter-rater reliability of Kinovea tracking software program through the same video clip by two persons in two times analyses (ICC=0.987 [95% CI=0.973- 0.994], p<0.001). ICC, intraclass correlation coefficients; CI, confidence interval.

2. Validity of Kinovea software in assessing total locomotor activity

To verify the validity of the Kinovea software, the results of the classical analysis and the results of the analysis using the Kinovea software were compared. For the analysis results using the Kinovea software, the results that were used in the inter-rater and test-retest reliabilities evaluation were applied. Pearson’s correlation coefficients were statistically significant with values of 0.928 (p<0.001) to 0.972 (p<0.001) in the three-time analysis results (Fig. 4).

Fig. 4.

This graph showed good validity of Kinovea tracking software program compared to classical method in order to assess locomotor activity in open field test in two times analyses (first analysis: Pearson's correlation coefficient=0.928, p<0.001; second analysis: Pearson's correlation coefficient=0.942, p<0.001; third analysis: Pearson's correlation coefficient=0.972, p<0.001).

3. Reliability and validity of Kinovea software in assessing local activity over time

The reliability and validity of changes in locomotor activity over time were analyzed at intervals of 10min using the Kinovea software. When comparing the two tests using Kinovea in the assessment of test-retest reliability, the ICC was within the range of 0.979 to 0.990, and the results showed a significant correlation at all intervals (Table 1). The ICC was found to be in the range of 0.975 to 0.990 when two different raters assessed the locomotor activity over time by using Kinovea. The results showed a significant correlation at all intervals (Table 1). The correlation coefficients were found to be in the range of 0.911 to 0.986 when using the classical method and the Kinovea method, respectively, to assess locomotor activity. The correlations were significant in most of the intervals (Table 2).

Table 1.

Intraclass correlation coefficient of inter-rater and testretest reliabilities for Kinovea analysis according to time interval

Time interval (min)	Test-retest reliability	Inter-rater reliability
0-10	0.984 (0.967-0.993)[a)]	0.980 (0.957-0.990)[a)]
10-20	0.979 (0.957-0.990)[a)]	0.975 (0.948-0.988)[a)]
20-30	0.987 (0.974-0.994)[a)]	0.990 (0.979-0.995)[a)]
30-40	0.987 (0.973-0.994)[a)]	0.987 (0.972-0.994)[a)]
40-50	0.995 (0.989-0.998)[a)]	0.980 (0.959-0.991)[a)]
50-60	0.990 (0.979-0.995)[a)]	0.988 (0.976-0.994)[a)]
Total 60	0.988 (0.975-0.994)[a)]	0.987 (0.973-0.994)[a)]

Values mean intraclass correlation coefficient (95% confidence interval).

p<0.01.

Table 2.

Pearson’s correlation coefficient according to time interval between classical method and three Kinovea analyses

Time interval (min)	Kinovea analysis 1	Kinovea analysis 2	Kinovea analysis 3
0-10	0.939[a)]	0.953[a)]	0.961[a)]
10-20	0.911[a)]	0.939[a)]	0.968[a)]
20-30	0.963[a)]	0.962[a)]	0.976[a)]
30-40	0.943[a)]	0.951[a)]	0.986[a)]
40-50	0.933[a)]	0.947[a)]	0.933[a)]
50-60	0.931[a)]	0.947[a)]	0.965[a)]
Total 60	0.928[a)]	0.942[a)]	0.972[a)]

Kinovea analysis 1 means the results of correlation analysis between classical method and first Kinovea analysis.

Kinovea analysis 2 means the results of correlation analysis between classical method and Kinovea analysis for test-retest reliability.

Kinovea analysis 3 means the results of correlation analysis between classical method and Kinovea analysis for inter-rater reliability.

Values mean Pearson’s correlation coefficient.

p<0.01.

DISCUSSION

This study confirmed that the Kinovea program, a freely available software package, has adequate reliability and validity when analyzing locomotor activity in mice. Pearson’s correlation coefficient was found to be 0.90 or higher in all analyses, and the ICC was also 0.90 or higher. According to Fleiss et al., the reliability can be considered as excellent if the ICC is 0.75 or higher [21,22]. Based on this criterion, the test-retest reliability and inter-rater reliability in this study can be considered as very high. The validity of the Kinovea software was also high with a Pearson’s correlation coefficient of 0.90 or higher. The validity can be considered to be superior if the Pearson correlation coefficient is 0.90 or higher [23].

The results of this study showed that Kinovea is useful in the analysis of locomotor activity in mice, as shown in previous studies [13-20]. A computer software program for animal experiments must be purchased with video equipment, and used only by researchers with the equipment. Researchers who do not have such equipment must spend a lot of time recording locomotor activity and analyzing video. However, similar analysis results can be obtained in less time by using the publicly available Kinovea program instead of the classical analysis method.

This study is significant because it confirmed that using Kinovea in animal studies to measure locomotor activity in mice is advantageous in terms of time and economics. In addition, this study showed high reliability and validity in the analysis of locomotor activity over time. When using the Kinovea program, the results of the analysis include not only the total locomotor activity but also the variation pattern over time and location information in the open field. Therefore, this software can be useful for studying changes over time rather than simply analyzing total locomotor activity. Kinovea also includes spatial location information, so the results analyzed by Kinovea can be used in various ways depending on the study purpose.

When analyzing the information obtained through Kinovea using a mathematical operation program such as MATLAB, various behavior analyses are possible. Recent studies on locomotor activity used animal location inforamtion to analyze animal behavior in various aspects [5,11,12]. One study investigated emotional disposition by analyzing the tendency to stay for a long time in a location close to familiar objects or a location that is distant from a place of which the animal fears [5]. Thus, the locomotor activitiy analyzed by Kinovea contains various information that can be useful for studying animal behavior in mice [24,25].

The limitations of this study are as follows. First, in terms of validity, the usefulness was not compared with that of other methods developed recently. Follow-up studies may need to compare the usefulness of the Kinovea analysis method with that of other analysis methods. Second, in this study, the reliability and validity results over time are limited because only time intervals of 10min were compared. Additional studies are needed to determine the reliability and validity for smaller time units. Third, the reliability and validity of spatial information were not analyzed in this study. The reliability and validity of assessing location information over time using Kinovea should be determined in future studies. Fourth, since the experimental animals used in this study were only ICR mice, this study does not indicate that mice of different colors or species would show similar results. As ICR mice are white, it is likely that tracking of the videos was easy without any other corrections. For example, black mice may show different results when their videos are analyzed; thus, a separate video processing method may be required.

However, the results of this study suggest that the Kinovea analysis method has a usefulness that is similar to that of the classical method. Therefore, the Kinovea analysis method can be used as a basis for analyzing the locomotor activity in mice. Considering that Kinovea is open-source software, it is of high value to the researchers as they can now utilize their research budget more efficiently. In follow-up studies, it is necessary to confirm the usefulness of various analysis methods of Kinovea and whether the accuracy is reproduced in other speices of experimental animals.