Gaze Based Mind Wandering Detection Using Deep Learning

Document Type

Thesis

Degree Name

Master of Science (MS)

Department

Computer Science and Info Sys

Date of Award

Summer 2021

Abstract

Mind wandering (MW) is a phenomenon where a person shifts their attention from task-related to task-unrelated information. Mind Wandering is an omnipresent phenomenon for human beings. The consequence of mind wandering can impact a person’s performance negatively. Reorienting the mind’s attention using technology shows great promise to improve the performance and productivity of people in learning or other performative tasks. In this research, we investigate 62 eye gaze features by dividing them into four sets of global features: eye movement descriptive features, pupil diameter descriptive features, blink features, and miscellaneous features to detect mind wandering during reading from a computer interface. Our dataset, which was collected from a previous study, contains a mind wandering report where 135 participants were recorded “mind wandering” or “not mind wandering” using self-reporting during a computerized reading task. During this process, a remotely placed eye tracker tool recorded eye gaze data. Models were created using six supervised conventional machine learning (ML) algorithms: logistic regression, k-nearest neighbors (k-NN), support vector machine (SVM), decision tree, random forest and naive Bayes. Machine learning models were trained on eye gaze dataset and evaluated using 5-fold cross validation. We measured the performance using area under the receiver operating characteristics (AUC-ROC) score, AUC-ROC curve, and confusion-matrix. To further improve the AUC-ROC score and other evaluation metrics, we trained standard neural networks and deep learning models using the data. Four sets of deep learning architectures were trained and evaluated. We found that dense neural network with one dimensional convolutional layer (DNN+Conv1D) outperformed the performance of conventional machine learning models. Naïve Bayes achieved mean test AUC-ROC score of 0.6595 and mean test accuracy of 0.6416. DNN+Conv1D beat the AUC-ROC score and achieved a score of 0.8024 and mean test accuracy of 0.7278. Our implementation used missing data values rather than discarding them which in fact improved our results. Our findings also showed that an automated mind wandering detection using deep learning models generalize well for new participants. This finding may help laboratory studies of mind wandering and for building systems to detect attention of inattentive drivers, students, or people in other work contexts that need focus to improve performance on a task.

Advisor

Derek Harter

Subject Categories

Computer Sciences | Physical Sciences and Mathematics

Link to Full Text

Share

COinS