Use of information modelling techniques to understand research trends in eye gaze estimation methods: An automated review.

Eye gaze tracking has been used to study the influence of visual stimuli on consumer behavior and attentional processes. Eye gaze tracking techniques have made substantial contributions in advertisement design, human computer interaction, virtual reality and disease diagnosis. Eye gaze estimation is considered critical for prediction of human attention, and hence indispensable for better understanding human activities. In this paper, Latent Semantic Analysis is used to develop an information model for identifying emerging research trends within eye gaze estimation techniques. An exhaustive collection of 423 titles and abstracts of research papers published during 2005-2018 were used. Five major research areas and ten research trends were classified based upon this study.

Introduction

Eye gaze trackers (EGTs) are smart devices, which are used to estimate the direction of eye gaze. Eye gaze is defined as the line of sight of an individual and it represents focus of attention. Initially, EGTs were used in neurology, ophthalmology and psychology to study oculomotor patterns with respect to different cognition states of an individual [1, 2]. Recently, EGTs have been used to decide appropriate marketing mix, advertisement design, human computer interaction, virtual reality, disease diagnosis and to study human behavior [3, 4, 5, 6, 7]. Tracking the gaze of an eye is used to study the influence of visual stimuli on consumer behavior and attentional processes [3, 8, 9]. EGTs have also been extensively used in web page design to predict the salient regions of web pages [10]. EGTs can be broadly classified into two categories namely intrusive and non-intrusive techniques. Intrusive techniques make use of electrodes, contact lenses and head mounted EGTs to record eye gaze, whereas non-intrusive techniques rely upon high precision cameras to capture eye images and gaze direction [11]. Commercially available EGTs are pretty expensive, making it economically unavailable for most of the user and researchers [12, 13]. Manuscripts pertaining to eye gaze tracking and computational models for measuring gaze of an eye has been published by many researchers. Studies providing an insight into contemporary status of eye gaze research and outcomes is available at renowned research databases. Literature considering algorithms, system configuration, user conditions and performance issues for existing gaze tracking systems has too been reviewed by many researchers. Most of the available reviews are done manually and may suffer from opinion bias resulting from experience, expertise and analytical skills of the reviewer [11, 14]. Semi-automated topic modelling algorithms imbued with established methods to conduct a systematic review, could be an alternative to restrict this opinion bias to a great extent. Further, it may also help in identifying research trends within eye gaze tracking research [15, 16, 17]. Furthermore, semi-automated review methods to find the core research trends have been adopted by many researchers in many domains [18, 19, 20, 21]. To the best of our knowledge there is no empirical study suggesting EGT research trends available so far [11, 14]. In this paper a quantitative method called Latent Semantic Analysis (LSA) is used for open ended text analysis of literature associated with EGTs and their applications [22]. LSA is a fully automatic mathematical technique for extracting and inferring meaningful relations from the contextual usage of words [22]. This method provides textual meaning to identified topic solutions using an automated approach thereby eliminating human bias [15, 20]. Primary aim of this work is to gain a realistic understanding of prominent research trends and EGT applications as promulgated by EGT researchers [22]. Subsequently, the relationships amongst them were investigated to achieve following objectives:

To find the leading researchers and prominent publications in Eye Gaze Tracking.

To find out most prominent research topics associated with Eye Gaze Tracking.

To anticipate the future research directions in applications of Eye Gaze Tracking.

Rest of the paper is structured in five sections. Section two elaborates methodology used in finding research trends and prominent researchers. Section three details about the results obtained after implementation. Section four provides a discussion about research objectives and probable future directions. Section five provides limitations of the study and last section includes conclusion drawn from the study.

Methodology

Bibliographic datasets like IEEE Xplore, ScienceDirect, DBLP computer science bibliography, ArXiv, Google Scholar, Mendeley, Directory of Open Access Journals (DOAJ), Association for Computing Machinery Digital Library (ACM DL), SPIE, Journal of Eye Movement Research, Hindawi and CiteSeerX were referred to collect literature dataset. Further, Taylor and Francis, Wiley, the MDPI journal bibliographic database were also searched to locate any suitable literature. EGT manuscripts published from 2005 till 2018 were considered based upon the keywords and stipulations as discussed in section 2.1.

Data acquisition

Aforesaid databases were manually searched to find suitable literature. The articles were selected using “Eye Gaze” OR “Gaze Points” OR “Eye gaze trackers (EGTs)” OR “Eye Gaze Technology” OR “Eye and gaze” OR “Limbus tracking” OR “Video-oculography” OR “Pupil tracking” OR “Purkinje image” OR “Eye gaze applications” as search keywords. Mendeley was used for the purpose of collection, screening, selection and corpus preparation. Literature accumulated was then manually reviewed in Mendeley to filter out articles based upon inclusion and exclusion criteria as mentioned in Table 1. Few of the articles, which were out of scope or were duplicated were eliminated as detailed in Table 2. The dataset was later converted into comma-separated values (CSV) using an export filter. The exported file included titles, abstracts and year of publication. Year wise distribution of publications are presented in Figure 1. The previously reported research in EGTs being subjective and qualitative, might not offer an insight towards top researchers and majorly contributing journals. Based on the number of occurrences within dataset, the top ten researchers with most of the publications on EGTs during the period 2005–2018 and top twelve journals publishing articles related to EGTs are presented in Tables 3 and 4, respectively.

Table 1

Inclusion and Exclusion criteria.

S.No	Inclusion criteria	Exclusion criteria
1	The articles must either be published in the proceedings of reputed conferences or journals during the period 2005–2018.	Articles not directly relevant to eye gaze tracking techniques.
2	The articles must have focus on eye gaze tracking and applications	Articles which are published before 2005 were not reporting on the study and development of EGT.

Table 2

Paper count during data pre-processing.

S. No	Steps	Paper Count
1	Online Database Search	1041
2	After Filtered for particular keywords	763
3	After elimination of duplicate articles	649
4	After elimination of non-relevant articles	423

Figure 1

Year wise distribution of publications during 2005–2018.

Table 3

Top 10 researchers in Eye gaze research.

S.No	Steps	Paper Count
1	Yusuke Sugano	31
2	Roberto Valenti	24
3	Qiang Ji	19
4	Andrew Duchowski	16
5	Soussan Djamasbi	14
6	Dan Witzner Hansen	14
7	Carlos Hitoshi Morimoto	12
8	Takashi Nagamatsu	9
9	Zhiwei Zhu	7
10	Xucong Zhang	7

Table 4

Top Journals publishing research on Eye gaze.

S.No	Journal Name	Number of Publications
1	IEEE Transactions on Pattern Analysis and Machine Intelligence	29
2	IEEE Transactions on Human Machine Systems	18
3	Pattern Recognition	15
4	IEEE Transactions on Image Processing	14
5	ACM Transactions on Graphics	12
6	Multimedia tools and applications	11
7	IEEE Transactions on Biomedical Engineering	11
8	Computer Vision and Image Understanding	11
9	International Journal of Computer Vision	10
10	Developmental Cognitive Neuroscience	9
11	Cognitive, Affective and Behavioural Neuroscience	9
12	Journal of Vision	7
13	Journal of Eye Movement Research	6
14	Expert systems with applications	3
15	Frontiers in Human Neuroscience	3

Application of Latent Semantic Analysis

An information model can be defined as representation of concepts, operations, rules and relationships between data and semantics for a chosen domain. An information model offers an organized structure of domain information requirements, which is not only stable but can also be shared. Researchers have proposed and experimented with numerous information modelling techniques like Latent Semantic Analysis (LSA), Latent Dirichlet Allocation (LDA), Probabilistic Latent Semantic Analysis (PLSA) and Correlated Topic Modelling (CTM). Choosing an appropriate information model was a real challenge to execute this study. A thorough comparison of modelling techniques revealed the fact that, LSA seem most appropriate to carry this study forward. Comparison of various information modelling techniques has been summarized in Table 5. Latent semantic analysis (LSA) is a natural language processing approach which can identify research trends within large literature dataset [22, 23]. LSA can not only summarize any text dataset but can also search, organize and understand it in an automated fashion. It is capable to examine prevalent relationships within documents and terms within dataset to reveal associated concepts and trends. LSA is an unsupervised learning approach based upon Singular Vector Decomposition (SVD). SVD creates a low dimensional space to reveal topics and relationships by comparing documents [18, 22]. Moreover, it is an established approach for identifying research trends prevalent within large literature dataset [17, 23]. Recommendations for application of the methodology were taken from Evangelopoulos et al. [17]. Since the aim of the study was to explore the latent structure of the dataset, factor analysis extension was applied to LSA in association with fast truncated incremental stochastic single pass SVD algorithm. Document loading and term loading matrices are the two primary outcomes of LSA. Term loading matrix includes trending topics with highly loaded terms associated with them. The document loading matrix includes trending topics with highly loaded documents associated with them. High loading values is an indication to higher familiarity with a topic [18]. Corpus prepared as given in section 2.1 was fed into LSA model to subsequently uncover latent semantic structure of the dataset. Five topic solution exemplifying five latent classes, associated keywords and their labels are exhibited in Table 6. It also includes highly loaded terms obtained from LSA driven empirical analysis of literature dataset. The detailed procedure followed is discussed in following sections.

Table 5

Comparison of information modelling techniques.

Technique and Reference	LSA [22, 24]	LDA [25, 26]	PLSA [27, 28]	CTM [29, 30]
Characteristics	Quick and efficient	Suitable for short length documents	Generative model, different words generated from different topics.	Allows word occurrences in more than one topic
	Peculiar and distinct words within topic
	Captures unique words
Limitations	Difficult to decide on the number of topics. In this study, dimensionality reduction has been performed using SVD that factorizes any matrix into product of three matrices i.e. U*S*V which provides topic coherence useful in determining the number of topics	Difficult to identify relations among topics	Not suitable for lengthy documents	Requires complex computations
		Results are difficult to interpret and analyze	No probabilistic model at the level of documents	Results are difficult to interpret and analyze
			Overfitting
Polysemy (same word with different meaning)	Handles polysemy partially	Does not handle	Handles partially	Does not handle
Synonymy (different words with same meaning)	Handles synonymy	Does not handle	Handles partially	Does not handle
Applications	Automatic essay grading	Anti- phishing	Image retrieval	Image retrieval
	Spam filtering	Word sense disambiguation	Classification	Query classification

Table 6

Five topic solution along with high loaded terms.

Topic Id	Topic Label	High loading terms
T5.1	Real time head pose estimation	based pose estimation real time head robust intrusive illumination localization
T5.2	Corneal fixation for pupil monitoring	accurate pupil recognition active feature analyse predict corneal fixation monitoring
T5.3	Movement tracking and detection	computer movement video analyses system detection attention interface reliable measure
T5.4	Commercial use of eye gaze tracking	technique images web advertise intelligence commercial computing motion interface heatmap
T5.5	Interactive human computer applications	system active human calibration pupil computer interactive applications disable cognitive

Pre-processing and term filtering

The first step towards LSA corpus preparation was pre-processing and term filtering. Characters, words and sentences discovered during pre-processing act as tokens for further processing by LSA. It would not only reduce the dictionary size but would improve efficacy of LSA. This step will further improve the efficiency of the text mining approach [31]. As per the recommendations of Evangelopoulos et al. [17], names, numbers, acronyms, abbreviations and punctuations were removed from the corpus. The below mentioned steps were followed or corpus preparation in Python using NLTK (Natural Language processing toolkit):

Tokenize titles and abstracts for each document within corpus.

Tokens were converted into lowercase letters.

Punctuations like periods, commas, question marks and apostrophes were eliminated.

Numbers were filtered to textual terms only.

Refine the corpus by removing English stopwords, and common keywords (“eye gaze”, “gaze points”, “eye gaze trackers (EGTs)”, “Eye Gaze Technology” or “Eye and gaze”) from all the publications.

N-character filtering was done to filter out words with less than three characters.

Corpus is further refined by removing words that appear only once in the whole corpus.

Create transformations by using TF-IDF model.

Now corpora are prepared for LSI modelling.

Select highest appearance of tokens based on topic solution.

Initially the dataset had 2640 tokens. After pre-processing, the count was reduced to 267 tokens. 423 sparse vectors were created with 267 tokens. The dataset of 423 documents was thus converted to a vector space where rows represent 267 terms or dimensions of 423 columns, each corresponding to an article. Each document was subsequently converted into bag of words. This mapping would allocate an integer identity (ID) to the terms within bag and would also count their occurrences within each document producing a dictionary. This dictionary is further used to create a weighted matrix as shown in Figure 2.

Figure 2

Two dimensional representation and documents (columns) for a particular topic solution.

Term frequency and inverse document frequency

A TF-IDF weighting scheme was deployed to identify the significance of a given entity in comparison to other entities (term or document) within corpus. It is used as a weighting factor to proportionately increase the weight of any term depending upon the number of its occurrences within document and an offset by the frequency of term in corpus. TF-IDF is helpful in adjusting the weights of words [18]. The approach followed in this paper is presented in (1) below. Wherein the terminologies used in the equation are defined as:

W (i, j) = TF-IDF weight obtained.

tf = term frequency.

d = number of documents in the corpus.

n = TF-IDF weight obtained

The term frequency (tf) in Eq. (2) measures the number of occurrences of a term in a document.

Using the same weighting scheme discussed in Eq. (1), a 267*423 term-document weighted matrix is created for i term in j document of n documents in corpus and is used in all the identified topic solutions.

Rank lowering using Singular Vector Decomposition

The weighted matrix TF-IDF was provided to the SVD to further perform rank lowering. The SVD model is used to perform matrix X factorization into variables [22, 32]. The following terminology is used in Eqs. (3) and (4).

U: Initial rotation.

∑: Scaling.

V: Final rotation

The mathematical expression XX and XX provides term-loading and document-loading respectively. represents the weights of the topics in descending order. The maximum number of topics generated was equal to the number of documents in the corpus. For extracting a few topics (k), the topmost k singular values were taken from the matrix [17, 33]. The procedure to be adopted for rank lowering and SVD is explained with in following example:

Text is represented as a matrix of form X = U∑Vt such that each row stands for unique word and each column represents unique document. Each cell represents the frequency of the word with which it appears in a document as shown in Table 7.

Table 7

Terms frequency per document matrix (X).

Terms/Docs	Doc1	Doc2	Doc3	:	Doc423
movement	0	1	1	:	0
images	1	0	1	:	0
head	1	0	1	:	1

Apply preliminary transformation wherein weights have been assigned describing word importance in particular document w. r.t all other documents. The dimension reduction step had structured the matrices in such a way that words that did not appear originally in some contexts now do appear, at least fractionally as shown in Table 8.

Table 8

Initial rotation (U).

Terms/topics	Topic1	Topic2	Topic3	:	Topic10
movement	0.031	0.432	-0.154	:	0.021
images	-0.346	-0.119	0.02	:	-0.102
head	0.181	0.135	0.399	:	-0.034

Apply SVD which decomposed the original matrix into the product of three other matrices. One component of one matrix describes the original row entities as shown in Table 8, another describes the original column entities as shown in Table 10 and third matrix contains scaling values diagonally in Table 9. The three components are matrix multiplied in such a way to reconstruct the original matrix as shown in Table 11.

Table 10

Final rotation (V).

Documents/topics	Topic1	Topic2	Topic3	:	Topic10
Doc1	0.2	0.21	0.06	:	0.14
Doc2	0.06	0.17	-0.13	:	-0.23
:	:	:	:	:	:
Doc423	-0.11	-0.3	0.21	:	0.07

Table 9

Scaling matrix (∑).

Topics/topics	Topic1	Topic2	Topic3	:	Topic10
Topic1	2.432			:
Topic2		2.167		:
Topic3			2.043	:
:	:	:	:	:	:
Topic10				:	0.721

Table 11

Terms-documents matrix after SVD transformation (Xt).

Terms/documents	Doc1	Doc2	Doc3	:	Doc423
movement	0.16	-0.4	0.38	:	0.47
images	-0.14	0.37	0.33	:	0.4
head	0.15	0.51	-0.36	:	0.41

Complete SVD transformation of matrix is shown in Table 11. The table below shows the two dimensional reconstruction of the original matrix after applying LSA transformations which induces the similarity relations between the documents.

Term loading XXt matrix represents terms loaded for a particular topic solution. Each cell contains term weight for a particular topic giving more weightage to that topic solution as per the specified threshold value as described in Table 12.

Table 12

Term loading (XXt).

Topics/terms	Movement	Images	Head	Pose
Topic1	0.124	0.091	0.327	0.385
Topic2	0.025	0.112	0.125	0.014
:	:	:	:	:
Topic10	0.002	0.182	0.109	0.051

Document loading matrix X tX represents documents loaded for a particular topic solution. Each cell contains document weight for a particular topic giving more weightage to that topic in terms of number of documents loaded for that topic as per the specified threshold value as shown in Table 13.

Table 13

Document Loading (X tX).

Documents/topics	Topic1	Topic2	Topic3	:	Topic10
Doc1	0.182	0.092	0.182	:	0.011
Doc2	0.019	0.002	0.129	:	0.118
:	:	:	:	:	:
Doc423	0.117	0.009	0.114	:	0.091

Selecting optimal topic solutions

Optimal topic solutions are attained through dimensionality reduction. Dimensionality reduction is a process of selecting k largest singular values from the singular matrix generated by SVD. Selection of an optimal dimension has been a key challenge associated with this process as it requires extensive understanding and numerous iterations to reach optimal value [17]. As recommended by Deerwester et al. [22], the optimal number of topic solutions for 423 corpus of documents is approximately 10. It may suffice to predict trends within Eye gaze research. In addition, three and five topic solutions were considered optimal to express core research areas.

Selecting threshold values

The term loading and document loading matrices indicate corresponding weights for uncovered topics. It means every cell of the within term and document loading matrix has a loading value corresponding to the term/document (row) and topic (column). Values of loading matrices could be both negative and positive. Varimax rotation was done to interpret results obtained from loading matrices. This resulted in increased loading for one topic in comparison to other topics [34]. The number of loaded documents for a particular topic defines importance of that topic. A heuristic empirical tail distribution approach was applied to differentiate between significant and insignificant loading [23]. For example, for ten topic solution, the loading values of (423) documents in each ten topics were transformed to a one dimensional matrix (vector) having 4230 elements. To obtain the threshold value, the vector is sorted in descending order thereby retaining the 1/423th term of high loading values. After performing certain calculations using tail distribution the threshold values obtained for three, five and ten topic solutions are 0.196, 0.213, and 0.227 respectively. Therefore, the documents having the loading values and more than the specified threshold were considered significant for the topic.

Labelling of topic

After sorting the loading values in descending order from both term loading and document loading matrix, an iterative approach was followed for the labelling of topics based on highly loaded terms in term loading matrix for each topic. The highly loaded values are grouped together based on their occurrences and weightage in term loading for creating a label for each topic as shown in Tables 14, 15, and 16. The topic labelling was done manually and is subject to human bias as topical coherence varied significantly. Owing to limited computing resources, topic solutions were obtained from only titles and abstract of the articles primarily focusing on eye gaze instead of taking the complete article.

Table 14

Core Eye gaze research areas for three and five topic solution.

Topic no	Topic label	2005–2018	2005–2011	2012–2018
T3.1	Real time head pose estimation	127	31	96
T3.2	Movement tracking and detection	93	28	65
T3.3	Appearance based estimation	12	3	9
T5.1	Real time head pose estimation	103	41	62
T5.2	Corneal fixation for pupil monitoring	17	5	12
T5.3	Movement tracking and detection	69	23	46
T5.4	Commercial use of pattern analysis	19	5	14
T5.5	Interactive human computer applications	17	5	12

Table 15

Five topic solution with high-loading research papers.

Topic No.	Topic Labels	High-loading Papers	Loading Values
T5.1	Real time head pose estimation	[40]	0.586
		[41]	0.545
		[42]	0.524
		[43]	0.514
T5.2	Corneal fixation for pupil monitoring	[44]	0.631
		[45]	0.524
		[46]	0.501
		[47]	0.463
T5.3	Movement tracking and detection	[48]	0.431
		[49]	0.327
		[50]	0.325
		[51]	0.287
T5.4	Commercial use of eye gaze tracking	[52]	0.497
		[53]	0.369
		[54]	0.360
		[55]	0.325
T5.5	Interactive human computer applications	[56]	0.533
		[57]	0.408
		[58]	0.369
		[59]	0.339

Table 16

Research trends in Eye Gaze Tracking.

Topic No	Topic Label	2005–2018	2005–2011	2011–2018
T10.1	Real time head pose estimation	127	39	88
T10.2	Appearance based gaze estimation	24	9	15
T10.3	Calibration methods	8	2	6
T10.4	Neural networks for gaze recognition	51	19	32
T10.5	Human computer interaction for disabled	29	6	23
T10.6	Interdisciplinary use of eye gaze tracking	30	6	24
T10.7	Cognitive applications	19	5	14
T10.8	Gaze points using oculography	13	4	9
T10.9	Pupil tracking	18	10	8
T10.10	Iris calibration	5	2	3

Results

Summary of topic solutions

LSA resulted into three, five and ten topic solutions representing core research areas in eye gaze tracking. Topic labels and number of publications associated with core research areas for three different time periods in between 2005-2018 are shown in Table 14. Topic solutions are represented as Ti,j, which represents the jth factor of the ith topic solution. For instance, T3.2 represents second factor of the third topic solution. The number of articles associated with each topic solution indicates the value of respective research area within that particular topic solution. The mapping displayed in Table 17, presents the connections between core research areas and the research trends identified using cross-loading analysis.

Table 17

Mapping of core Eye gaze research areas and research trends.

Topic No	Five Topic Labels	Ten Topic no	Ten Topic Labels
T5.1	Real time head pose estimation	T10.1	Real time head pose estimation
		T10.2	Appearance based gaze estimation
T5.2	Corneal fixation for pupil monitoring	T10.3	Calibration methods
		T10.9	Pupil tracking
		T10.10	Iris calibration
T5.3	Movement tracking and detection	T10.8	Gaze points using oculography
		T10.4	Neural networks for gaze recognition
T5.4	Commercial use of pattern analysis	T10.6	Interdisciplinary use of eye gaze tracking
T5.5	Interactive human computer applications	T10.5	Human computer interaction for disabled
		T10.7	Cognitive applications

Core research areas associated with eye gaze

The core research areas presented in Table 14 for three topic solution were focused on “real time head pose estimation”, “movement tracking and detection” and “appearance-based estimation”.

The core research areas presented in five topic solution were “real time head pose estimation”, “Corneal fixation for pupil monitoring”, “Movement tracking and detection”, “Commercial use of pattern analysis”, “Interactive human computer applications”. The documents with high loading values for each topic are presented in Table 15. Most of the papers had been loaded to one research area “Real time head pose estimation” in three and five topic solution. Loading numbers in five and three topic solution may vary, as in five topic solution more research areas have been emerged out. Another research area that emerges from the results is “movement tracking and detection” with the loading of 69 papers [14, 35, 36, 37, 38, 39]. The estimation of eye gaze between a group of people communicating with each other or with robot is estimated through correlation between gaze points and head pose movement as stated by Masse et al [36]. Duchowski et al. gives a review on eye gaze tracking techniques and its applications [39]. Some other literature review on detection and tracking of eye movements is discussed by many researchers [11, 14]. Moreira et al. discussed eyes and eyebrows detection using simple webcam for animation type applications [37]. Corcoran et al. discussed real time detection of gaze in combination with human emotions useful in gaming applications [38].

Eye gaze research trends

The ten topic solutions revealing research trends are displayed in Table 16, along with the count of highly loaded papers for a particular topic solution. Papers having loading values of 0.227 or greater than that were only considered relevant for ten topic solution. The ten topic solution emphasized on emerging research trends “real time head pose estimation” (T10.1), “neural networks for gaze recognition” (T10.4) and “interdisciplinary use of eye gaze tracking” (T10.6).

Real time head pose estimation

The major research trend that has maximum loading of papers that has been emerged from ten topic solution is “real time head pose estimation” with the maximum loading of 127 papers. Real time head pose estimation plays a major role in EGT applications. Variation in head pose and slight illumination change may affect the results in estimating the gaze of an eye using EGTs. Researchers have worked upon numerous head pose based gaze estimation algorithms [40, 41, 60, 61]. Initially, head pose estimate is not known, some initialization is required for accurate pose tracking and gaze estimation [42, 60, 62, 63].

Neural networks for gaze recognition

Eyes plays a major role in understanding human social interactions. In the field of cognitive and behavioral neuroscience, eye gaze processing involving the use of neural networks is used in understanding the abnormal activities in a pathological condition [64, 65]. The author proposed a camera based eye tracker using artificial neural network to estimate eye gaze. As neural networks works directly on eye image and based upon that gaze points are estimated [63, 66]. Some other researchers applied convolutional neural networks for regression and prediction problems i.e. human eye fixations and pose estimation [67, 68].

Appearance based gaze estimation

It is analysis of a person's eye appearance. It utilizes natural gaze of eyes as seen from a commodity camera. One of the biggest advantage of appearance based estimation is that, it does not require any special equipment. Appearance based gaze estimation can be done with the help of ordinary cameras as stated by Sugano et al., proposed a novel method using visual saliency computed from video clips for eye gaze estimation [69]. Adaptive linear regression, neural networks or other learning measures are some of the prominent appearance based eye gaze estimation methods [70, 71]. As learning based methods requires labelled training data for appearance based gaze estimation, an alternative method proposed by Wood et al. solve this problem using synthesized eye images [72, 73]. The method proposed by Zhang et al. makes use of multimodal convolutional neural networks tested on their own dataset, worked well under varied head movements and different lighting conditions [74].

Pupil tracking

Robust, accurate and real time tracking of pupil is a key component in online eye gaze estimation [75]. Model based gaze estimation techniques predict eye gaze using 3D geometric model and camera calibration basis of stereo-vision system [76, 77]. Nagamatsu et al. proposed a stable and fast calibration free method for gaze points estimation using 3D eye model [77]. The other model based methods proposed by many makes use of RGB camera for real time gaze estimation with improved accuracy [78, 79]. K-nearest neighbor, random forest regression method and support vector regressor machine are the few listed model based methods used by researchers for detection of pupil images [80, 81, 82, 83]. Another model based approach proposed by Xiong et al. considered optical and visual axis deviation using spherical eye model for eye gaze estimation [84].

Human computer interaction for disabled

Eye gaze has its applications in human computer interaction wherein input is taken from gaze of an eye and is used to further control the computer system basis of certain features and then the computer executes the commands based on gaze location on the screen [64, 85, 86, 87]. Another application of EGT is in simulation, where eye tracker helps in analyzing the attention of pilots in realistic situations [88]. With the use of eye tracking technique in e-learning has made possible to estimate the focus of learner in real time [7, 89]. Human robotic interaction is the another area where researchers have focused upon, navigational portable robot worked well with user's eye gaze [90, 91]. Smart home and smart television are some other prominent researched application using eye gaze for disabled people in effectively controlling smart devices [92].

Gaze points using oculography

The eye movement recording, the author proposed an algorithm reducing the effect of eye blinks while measuring gaze points. Various techniques using oculography have been proposed by researchers to find gaze points [93, 94, 95].

Calibration methods

Eye gaze tracking is the process of finding a location on screen where exactly the user is looking at. In order to find angle between horizontal and vertical eye movements, system calibration needs to be done for every user to get accurate gaze points [96]. The method proposed by Zhu et al. estimates eye gaze without any head movement restriction. The 3D gaze estimation method so proposed minimize calibration procedure thereby providing a more accurate gaze tracking solution [97].

Cognitive applications

Eye gaze tracking and detection helps in getting deep insights in decision making and problem solving. The author proposed multimodal eye gaze interface for people with physical disability and other locked in diseases [7, 98, 99, 100].

Iris calibration

Most of the EGTs estimates eye gaze with few restrictions such as head movement restriction and calibration need for every user. A calibration procedure is required to compute eye orientations. The author presents a calibration procedure for which user has to look at a particular set of targets based upon which the corresponding calibration function and gaze points have been estimated [101, 102].

Interdisciplinary use of eye gaze tracking

EGTs have been widely used in interactive and diagnostic applications such as in marketing field, e-commerce, psychology, in augmented and virtual reality [103, 104, 105, 106]. Eye gaze tracking has been used to study the influence of human visual behavior on consumer behavior and other attentional processes [9, 53, 104, 107].

Mapping between core research areas and research trends

Mapping between the core research areas and research trends is shown in Table 17. The connection between core research areas and research trends was made manually on the basis of loading of research papers loaded for topic solutions as stated in numerous study [18, 23]. A manual connection has been made between core research areas and research trends based upon the loading of papers. LSA gives weight factors based upon which one document can be loaded into more than one topic solution as mentioned in number of studies [18, 23]. The mapping provides the connection between low aggregated topic loading with the high aggregated topic loading. In this study, most of the documents are clustered around one topic solution i.e. “real time head pose estimation”.

Discussions about research objectives and potential future applications

This section tenders a discussion about how LSA has contributed towards answering research objectives mentioned in the introduction.

Objective 1: To find the leading researchers and prominent publications in Eye Gaze Tracking

Top journals and authors worked in eye gaze tracking are shown in Table 4 and Table 3. Journals include IEEE transactions on pattern analysis and machine intelligence, IEEE transactions on human machine systems and Pattern Recognition. Research on eye gaze has been done by researchers all over the world especially from Eastern Asia and Europe. Yusuke Sugano [69], Roberto Valenti [42], Andrew Duchowski [39] are some of the prominent researchers published their articles on eye gaze. Sugano et al. worked on appearance based gaze estimation that take input images from the video clip and based on that gaze points are estimated using visual saliency [69]. Hansen et al. gives a comparative analysis on methods of gaze estimation based on their geometric features and discussed about various Eye gaze estimation techniques along with their accuracies [14].

Objective 2: To find out most prominent research topics associated with Eye Gaze Tracking

The outcome of study indicates that “real time head pose estimation”, “movement tracking and detection” are the most widely researched topics in Eye gaze. Appearance based estimation though simple and easy to setup, utilizes natural gaze of eyes as seen from a commodity camera. The advantage of appearance based gaze estimation is, it does not require any special equipment. Appearance based gaze estimation can be done with the help of ordinary cameras as stated by Sugano et al [69]. Eye tracking and detection can be done with many tools and equipment's [108]. From this research area many methods and techniques have been identified focusing on Eye Gaze along with its applications in various fields [109].

Objective 3: To anticipate the future research directions in applications of Eye Gaze Tracking

Contemporary EGTs are expensive and are beyond the reach of masses as they require dedicated laboratories and expert handling. Although the said EGTs are highly calibrated and produce accurate results, yet they are not the only option to track eye gaze. Advancement in technology has resulted into the integration of high resolution cameras within smartphones and portable devices like laptops. It would not be long when these integrated cameras assisted solutions would eventually offer a substitute for these expensive EGTs. It would also offer a cost effective yet reliable method of eye gaze tracking. Eye gaze is a nascent and emerging area in computer vision and neuroscience, hence there is an ample scope for expansion in its applications [110]. Few of the prominent prospective applications of EGT are enlisted as under:

Smart Phones: Convenience was the main factor which brought revolution in the mobile industry. Firstly, bulky mobiles transformed into sleeker ones, key pad gave way to touch screens, then came gesture control. Now eye gaze can further revolutionize mobile industry. Through eye gaze one can unlock the phone, scroll through all the applications, open or close any app, click pictures by just blinking eye, making a phone call by just gazing at the name on the screen, can write a message or search a particular app by just gazing at the required alphabets in the correct order. There are endless possibilities wherein eye gaze can be associated with smartphones making them even more smarter [56, 111, 112, 113].

Driver assistance system: Eye gaze finds numerous applications not only in guiding the drivers through all types of terrains but also in preventing them from over speeding, breaking traffic rules thereby decreasing on road accidents. Combined with GPS eye gaze tracking can also warn driver of any unforeseen obstruction coming in his path like traffic jams, road blocks etc. which comes in handy especially when navigating in dense fog conditions, thereby reducing accidents considerably. In fact if coupled with proximity sensors eye gaze tracking can play a major role in averting accidents in the form of preinstalled accident preventing system in vehicles, thereby saving thousands of lives which are lost every year due to such mishaps [109, 114, 115, 116, 117, 118].

Security and authentication: Conventional password based security systems are still widely used, but they pose a serious threat to the entire system as they are susceptible for being misused by unauthorized persons. Eye gaze tracking provides an impeccable solution in the form of retina scan, retina being unique in every individual makes it virtually impossible for unauthorized persons to gain access in critical areas. It can be one of the best possible solution for Home security, wherein only individual whose retina scan is stored in the system would be allowed to unlock the door. Any unauthenticated person who tries to enter would be thwarted by sending out loud alarms alerting nearby residents and sending distress call to the house owner, thereby avoiding any loss to the owner [119, 120].

Robotics: With the advancements in Robotic, it is just a matter of time that robots will become an indispensable part of human kind. Through eye gaze now robots can anticipate what the person might be requiring, it may be just a glass of water or even a walk in the park [91, 121]. They can act as a security guards of our homes in our absence, recording each and every movement in and around the house. Through eye gaze they can even assist doctors in complex surgeries and with the help of sophisticated sensors it can also guide doctors in performing such surgeries with utmost precision. Robots being indispensable part of industries, various technological upgradations are carried out as well as researched to make robots more efficient, time is money. The lesser the time taken to complete a task the more is the output and hence the profit. Robots can be assigned various task, the sequence of which can be decided upon the eye gaze of the user [122, 123].

Virtual and augmented reality: Virtual reality is a relatively new technology which gives user an immersive experience of a particular place without being actually present there. This technology is made possible only through eye gaze tracking, as virtual reality is based upon the gaze of the user wherein user wears some device around eyes, creating such an environment that the user start relating himself to the virtual world. In whichever direction user looks, the gadget creates the environment as per his gaze thereby giving him an immersive experience [124, 125, 126].

Entertainment: It can be very useful in various fields for example, entertainment wherein it can be used as an excellent promotional tool by giving the user an experience of the movie sets without being actually there. It creates an excitement among the audience, in medical it can assist young doctors by showing them various medical aspects like Human Anatomy, Surgeries in a virtual environment thereby giving them in depth knowledge of these aspects which might had been difficult in the real environment [127, 128].

Gaming: Gaming is an ever evolving field. Every now and then new innovative ideas are being introduced to make gaming experience all the more interesting. Now gaming consoles can not only be controlled by hands but also through the EGTs. Eye movements are tracked through these sophisticated trackers which performs corresponding actions on the gaming screen. For example, in a racing game car can be controlled through eye gaze, or in a puzzle game the puzzle pieces can be controlled onscreen through eye movements and can brought together to complete the puzzle. Other applications include strategy game like PUBG player can look around the scene by just tilting his head and make the character move in any direction based on the player's eye gaze direction making the gameplay experience more thrilling and engaging [38, 128].

Smart homes and TV control: Smart homes and TVs have been built to provide convenience to disabled person in their daily routine. Various methods have been proposed by researchers in designing these devices by analyzing gaze on the basis of face recognition and gaze estimation. Some examples include, in channels changing by just looking on the extreme left or extreme right of the television screen, increasing and decreasing volume by looking at the extreme top and bottom of the screen respectively [92, 129, 130].

Medicare: Eye gaze tracking has its applications in medicine, treatment of eye cancer and in surgical operations [131, 132, 133]. Non-intrusive technique of eye gaze tracking is utilized here in the treatment and detection of eye diseases.

Cervical treatment: It can help the patients suffering from cervical spondylitis and other backache problems by correcting their postures and warning them when their posture becomes wrong. The main reason for such problems is sitting or standing in incorrect posture for long period of time, it can also help others avoid such ailments by helping them take adequate precautions by correcting and maintaining their posture [134].

A means of communication for the disabled: Communicating becomes very difficult for a disabled person specifically those whose limbs are paralyzed. Unable to speak, write and move makes it difficult even to ask for water and food etc. Eye gaze tracking can come as a blessing for such patients. By entering keywords in the computer through eye gaze, one can convey his thoughts, requirements to the concerned person thereby making their life easier [57, 135, 136, 137, 138].

Banking System: In digital era of banking industry eye gaze can be a game changer. For instance, in fully automated branches customer's eye gaze can be detected to know exactly what is he looking for. Whether he wants to update his passbook, deposit or withdraw cash or even open an account [139].

Simulation: Aircrafts are among the most expensive vehicles available on planet and so is there training. A small mistake can lead to huge losses of life and money. An ideal pilot must be able to deal with all kinds of situations be it bad weather or heavy air traffic. For him to reach that level of expertise he would initially need extensive training. Training him directly on a trainer jet might prove very costly, hence giving him an immersive virtual experience would be a cost effective option. For instance, flying through rough weather is a very tedious, requires lot of patience and complete control over the plane. This can be practiced in the simulative environment where the user can experience such situations and can learn how to tackle it. Here eye gaze can help in studying and analyzing the behavior of the person in emergency situations [140, 141, 142].

User Interface design and evaluation: Eye gaze tracking has its application in user interface design in finding which areas of a webpage are getting attention from users [143, 144]. Eye tracking has been used in evaluating graphical user interface within web environments in order to improve the usability in designing web pages [145, 146]. Using scan paths information, analysts can get an estimate which areas of the interface are getting attention or which have been ignored [147].

Appraising product packaging: In today's market, product packaging plays an important role to influence customer's decision in purchasing a product. Using eye movements can help in identifying users focus of attention which in turn helps in increasing product market by noticing user gaze points [148, 149, 150].

Social Analysis: Eye gaze helps in understanding the ability of an individual in paying attention to any useful information [151]. Gaze direction along with magnetic resonance imaging technique helps in developing potential interpersonal interaction [152, 153].

Sports: Eye movements helps athletes in anticipating where exactly the ball will bounce [154]. Eye gaze tracking can be used in enhancing skills and performance by analyzing eye movements of the players in cricket or squash type sports [155, 156, 157].

The comparison of the above discussed eye gaze tracking applications is listed in Table 18 on the basis of usage of intrusive and non-intrusive technique along with varying head pose estimation.

Table 18

Applications of eye gaze tracking.

Sr. No.	Applications	Reference Number	Intrusive	Non -intrusive	Head pose estimation
1	Virtual reality	[125, 126, 127, 128]	✓
2	IPTV Controlling	[92]	✓
3	Medicine	[131, 132, 133, 158, 159]		✓
4	Sports	[154]	✓
5	Simulator	[140, 141]	✓
6	Augmented Reality	[160, 161]	✓		✓
7	Marketing	[110, 149, 162]	✓
8	Driver assistance system	[109, 115, 116, 163]		✓	✓
9	E-learning	[89]	✓	✓
10	Gaming	[128]	✓
11	Robotics	[91, 121, 164]		✓	✓
12	Smartphone based object detection	[56, 112, 113]		✓	✓
13	Security and authentication	[119, 120]		✓
14	Smart homes and TV control	[129, 130]		✓	✓

Limitations

Some issues might have arisen while compiling literature dataset on Eye gaze tracking techniques. It relied upon factors like the literature source, the query used and the identification and selection of final literature used to prepare corpus. The keywords “Eye Gaze”, “Gaze Points”, “Eye gaze trackers (EGTs)”, “Eye Gaze Technology”, “Eye and gaze”, “Limbus tracking”, “Video-oculography”, “Pupil tracking”, “Purkinje image”, “Eye gaze applications” were incorporated to find out suitable research papers. To generate a good reliable dataset, the other research databases, which did not look up in the automated search, were checked manually. The research papers collected for the present study were extensively checked for refining the data by applying inclusion and exclusion criteria as listed in Table 1. However, there might be a possibility that few relevant papers may have been omitted. There is a possibility of bias even when using LSA and it may depend upon the choice of keywords, inclusion and exclusion criteria etc. To reduce this bias to the extent possible, a heuristic approach was followed to identify suitable threshold to be used by algorithm. Although LSA can significantly improve the vector space model with the inclusion of synonyms yet it cannot automatically decide the appropriate number of required topic solutions. To mitigate impact of this limitation, thorough discussions with domain experts were made before choosing optimal topic solutions. Further, topic labelling was carried out on the basis of human judgement, which might have induced some subjective bias. There might be some limitations in relation to generalization of results. Owing to limited computing resources, only topic solutions were obtained from titles and abstract of the articles primarily focusing on eye gaze instead of taking the complete article. The identification of research trends and core research areas were based on experimental design that involved pre-processing of literature, selection of literature, utilization of SVD, term document matrix creation and topic labelling. Each of these subsequent choices may influence results. However, data verification was conducted through manual review so the results must be reliable enough to achieve generalization.

Conclusion

This study primarily investigates prominent research trends by deploying information modelling techniques on 423 research articles published from 2005 till 2018. Using LSA, K- topic solutions were identified based on document loadings and corresponding terms. LSA revealed five core research areas and ten research trends followed by EGT research community. Yusuke Sugano and Roberto Valenti were found to major contributors towards EGT articles and IEEE Transactions on Pattern Analysis and Machine Intelligence and IEEE transactions on human machine systems are pioneer in promulgating EGT research literature. Real time head pose estimation, interdisciplinary use of eye gaze tracking along with neural networks for eye gaze recognition came out as prominent contemporary research trends within EGT research community. The study also concludes the fact that, inducting EGT with state of art augmented and virtual reality, medicare, gaming, artificial intelligence, social analytics, sports, entertainment and Internet of Things has opened numerous new horizons for EGT research.

Declarations

Author contribution statement

Nandini Modi: Conceived and designed the experiments.

Jaiteg Singh Khaira: Analyzed and interpreted the data; Wrote the paper.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Competing interest statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.