Human-centred Emotion Detection

Interpreting emotions in text-based social media communication is a challenging task due to the absence of non-verbal cues such as tone, body language, and facial expressions. While advances in Natural Language Processing (NLP) and Machine Learning (ML) have improved sentiment analysis, existing emotion detection models struggle with inconsistencies in emotion labeling, lack validation against human interpretations, and fail to capture the nuances of real-world emotional expression. Misinterpretations can lead to miscommunication, social isolation, and psychological stress in online interactions.

‍

This research aims to introduce a novel methodology for training and validating ML-based emotion detection models, incorporating human evaluation to bridge the gap between machine-generated emotion classifications and human perception. Our goal is to:

✔️ Train emotion detection models using multiple psychological emotion theories (Categorical and Dimensional emotion models).

✔️ Conduct human-centered validation by comparing ML outputs with human annotations of social media text.

✔️ Investigate the impact of emotional intelligence on human emotion interpretation and how it affects agreement with ML predictions.

‍

Stage 1: Human-Centered Model Training

• Create custom datasets based on social media posts labeled using different psychological emotion models.

• Train ML models (RoBERTa-based NLP models) on these labeled datasets.

Stage 2: Human-Centered Model Validation

• Conduct a user study with human evaluators to compare their emotional interpretations with ML model predictions.

• Use crowdsourcing platforms (e.g., Amazon Mechanical Turk) to analyze human agreement with machine-labeled emotions.

• Measure participants’ emotional intelligence (EI) scores to examine their effect on labeling behavior.

‍