Assessment of In-game Scenes and Forecasting of Pre dominating Player Emotions
In a groundbreaking development for the gaming industry, researchers have unveiled EMOGRAPH, a system designed to predict players' emotions during horror games. The system was tested on the popular horror game, Outlast.
The EMOGRAPH system introduces an innovative emotion prediction method that utilizes the OCC variables from the model of emotions' cognitive evaluation by Ortony, Clore, and Collins [1]. This approach combines eye movements and facial expressions to annotate game objects with dominant emotions, using the game scene's design goals to predict player emotions.
EMOGRAPH provides a solution for computer-assisted emotional analysis of game sessions. The system is named EMOGRAPH (Emotional Graph).
How EMOGRAPH Works
EMOGRAPH integrates multiple data streams—eye movements, facial expressions, questionnaire responses, and brain wave activity (EEG)—to predict player emotions during horror game sessions.
- Eye Movements: EMOGRAPH tracks players’ eye gaze patterns, fixations, saccades, and pupil dilation. These ocular features provide insight into the player’s attention focus, arousal level, and emotional responses to in-game stimuli.
- Facial Expressions: The system analyzes facial muscle movements using computer vision techniques to detect expressions linked to basic emotions like fear, surprise, or disgust.
- Questionnaires: Self-reported emotional states and subjective experiences are collected from players through standardized emotion questionnaires administered before, during, or after gameplay.
- EEG Brain Wave Activity: EEG measures neural oscillations that correlate with emotional states, such as changes in alpha, beta, theta, and gamma bands.
Combining the Data
EMOGRAPH employs machine learning algorithms that fuse these heterogeneous data streams at either the feature level or decision level. By integrating physiological (EEG), behavioral (eye and facial data), and subjective data (questionnaires), the system obtains a richer, more robust representation of emotional states.
Multimodal feature extraction synchronizes features from each modality along the timeline of the gameplay session, so that emotions predicted correspond precisely to particular game events or gameplay moments.
Using labeled emotions from questionnaires, supervised learning models are trained on the combined dataset to recognize patterns across modalities that correspond with specific emotions induced by the horror game scenario. In some implementations, the system can provide near real-time predictions of players’ emotional states, adapting to the dynamic nature of the game and the evolving player experience.
The "Outlast" Experiment
"Outlast" is a psychologically intense horror game with unpredictable scares, atmospheric tension, and high-stress moments. EMOGRAPH leverages the heightened physiological signals (e.g., increased pupil dilation, micro-expressions of fear), EEG indicators of stress/arousal, and subjective fear ratings from questionnaires to accurately pinpoint when and how players experience fear, anxiety, or relief throughout gameplay.
The experiment involving EMOGRAPH was conducted with 21 participants playing Outlast. The results show the effectiveness of the method in identifying emotions and their triggers. The prediction results from the EMOGRAPH system show promise in widening possibilities in game design.
[1] Ortony, A. C., Clore, G. L., & Collins, A. (1988). The Cognitive Structure of Emotions. Cambridge University Press.
In summary, EMOGRAPH combines eye tracking, facial expression analysis, self-report questionnaires, and EEG data through machine learning-based multimodal fusion to predict players’ emotional states during horror gaming, delivering a comprehensive and dynamic understanding of their affective responses in Outlast. This innovative system could pave the way for a new era of personalized and emotionally engaging gaming experiences.
The EMOGRAPH system could potentially expand beyond horror games, applying its technology and artificial-intelligence capabilities to sports, such as football, for analyzing players' emotional responses during high-stress moments. This analysis could help coaches better understand the emotional state of their players and make informed decisions.
By combining eye tracking, facial expression analysis, questionnaires, and EEG data, EMOGRAPH could provide valuable insights into the emotional experiences of football players during crucial moments in a match, offering a new perspective on sports psychology and improving player performance.
