The relationship between neural network assessments of a pilot's emotional states and the quality of functional task performance

Abstract

The article presents the results of an analysis of diagnostic instrumental data from pilots, specifically focusing on recorded emotional states during the following experiments: performing multiple landing approaches using flight director bars on a flight navigation display; performing object search tasks. Deep convolutional neural networks were employed to identify emotional states. Mathematical statistics methods were used to analyze the experimental data, including principal component analysis to reduce the dimension of the data matrix across the 7 studied emotional categories while minimizing information loss. In the landing approach experiments, the relationship between the average number of emotions per minute for a specific category and the RMS (root mean square) deviation from the glideslope was evaluated. Data analysis was conducted throughout the entire approach. Furthermore, the relationship between the average number of emotions of a given category at the beginning of the approach and the average total deviation from the glideslope when nearing the runway was assessed, aiming to predict landing approach accuracy. In the object search experiments, the probabilities of registering emotions over the object search time were evaluated. Data analysis revealed that the emotional category "Sad" serves as an indicator of focus on task perfomance and a indicator of the pilot's concentration. The emotional categories "Fear" and "Surprise" were identified as indicators of confusion and concern. Generalized correlation matrices were also calculated, demonstrating that emotions are correlated with each other, with piloting precision metrics (deviations from the glideslope), and with task completion time (object search time).