Emotional content is a crucial ingredient in user-generated videos. However, the sparsity of emotional expressions in the videos poses an obstacle to visual emotion analysis. In this paper, we propose a new neural approach, Bi-stream Emotion Attribution-Classification Network (BEAC-Net), to solve three related emotion …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,932 papers · 148 categories
Trend · papers per month
48 results for “Emotional Video Classification”
End-to-end neural network extracts graph structure from EEG signals for improved emotional video classification.
problem Challenges in achieving accurate EEG classification for emotional video analysis.
method Proposes an end-to-end neural network model that learns an appropriate multi-layer graph structure from raw EEG signals.
result Improves performance in emotional video classification compared to manually defined connectivity structures.
Generates music with video emotion using deep neural networks.
problem Generating music with video emotion.
method Hybrid deep neural network combining ANFIS and LSTM.
result Low mean absolute errors and similar global features in spectrograms.
Enhances speech emotion recognition by integrating visual data with attention mechanisms.
problem Improving emotion detection accuracy by combining multiple modalities.
method Introducing an attention mechanism to combine audio, text, and video modalities.
result Significant improvement of 3.65% in weighted accuracy.
A fusion approach combines audio and video features for emotion recognition.
problem Continuous emotion recognition using both visual and auditory modalities.
method Pre-trained CNN features from video frames and minimalistic auditory descriptors. Fusion at feature or prediction level. SVR for prediction.
result Improves CCCs of 0.749 and 0.565 for arousal and valence respectively.
Deep Fusion: An Attention Guided Factorized Bilinear Pooling for Audio-video Emotion Recognitioncs.LG
Automatic emotion recognition (AER) is a challenging task due to the abstract concept and multiple expressions of emotion. Although there is no consensus on a definition, human emotional states usually can be apperceived by auditory and visual systems. Inspired by this cognitive process in human beings, it's natural to…
Simpler CNN model with spatial attention and temporal pooling outperforms complex models.
problem Emotion recognition from videos with small face deformations and identity variations.
method Spatial attention mechanism and temporal softmax pooling applied to a pre-trained CNN.
result The approach achieves higher accuracy than state-of-the-art methods on the EmotiW dataset.
This paper describes audEERING's submissions as well as additional evaluations for the One-Minute-Gradual (OMG) emotion recognition challenge. We provide the results for audio and video processing on subject (in)dependent evaluations. On the provided Development set, we achieved 0.343 Concordance Correlation Coefficien…
This paper improves sentiment classification by combining text, audio, and video data using DCCA.
problem Improving sentiment classification accuracy using multi-modal data.
method Deep Canonical Correlation Analysis (DCCA) for combining text, audio, and video embeddings.
result One-Step DCCA outperforms current state-of-the-art in multi-modal embedding learning.
An adversarial learning framework for preserving users' anonymity in face-based emotion recognitioncs.LG
Paper proposes a framework to protect user anonymity in emotion recognition.
problem Preserving user anonymity in face-based emotion recognition systems.
method Adversarial learning framework using CNN architecture.
result The proposed approach minimizes identity-specific information and maximizes emotion-dependent information.
In this work, we train fully convolutional networks to detect anger in speech. Since training these deep architectures requires large amounts of data and the size of emotion datasets is relatively small, we use transfer learning. However, unlike previous approaches that use speech or emotion-based tasks for the source …
Paper proposes ICCN to learn correlations between text, audio, and video for multimodal sentiment analysis.
problem Improving multimodal sentiment analysis by learning hidden correlations between text and audio/video features.
method Interaction Canonical Correlation Network (ICCN) using deep canonical correlation analysis (DCCA).
result Empirical results confirm the effectiveness of ICCN in capturing useful information from all three views.
This paper presents our approach to the One-Minute Gradual-Emotion Recognition (OMG-Emotion) Challenge, focusing on dimensional emotion recognition through visual analysis of the provided emotion videos. The approach is based on a Convolutional and Recurrent (CNN-RNN) deep neural architecture we have developed for the …
StarGAN model generates and recognizes emotions from facial expressions.
problem Emotion recognition and generation from facial expressions.
method Used StarGAN model to train on a new emotion dataset of 4K videos.
result Trained StarGAN model can generate and recognize emotions based on valence arousal scores.
This short paper describes our solution to the 2018 IEEE World Congress on Computational Intelligence One-Minute Gradual-Emotional Behavior Challenge, whose goal was to estimate continuous arousal and valence values from short videos. We designed four base regression models using visual and audio features, and then use…
AI in Pursuit of Happiness, Finding Only Sadness: Multi-Modal Facial Emotion Recognition Challengecs.CV
AI misidentifies facial expressions in videos, often misinterpreting happiness as sadness.
problem Automated facial emotion recognition in videos, especially with mixed modalities (visual and audio).
method Applied state-of-the-art visual and temporal networks, explored feature fusion methods.
result Machine learning models misclassify emotions, particularly happiness as sadness.
Unimodal approach for group-level emotion recognition without individual features.
problem Privacy issues in individual-based emotion recognition models.
method Frugal approach using global features, state-of-the-art and synthetic corpora.
result 59.13% accuracy on VGAF test set, 11th place in EmotiW Challenge 2020.
Multi-task learning is a method for improving the generalizability of multiple tasks. In order to perform multiple classification tasks with one neural network model, the losses of each task should be combined. Previous studies have mostly focused on multiple prediction tasks using joint loss with static weights for tr…
Project creates new dataset for 'in the wild' emotions recognition.
problem Recognition of emotions in real-life unpredictable situations.
method Design and implement a new dataset and deep learning model.
result Demonstrates effectiveness of the new deep learning model for real-life emotions recognition.
New system for automatic music emotion recognition considers multiple emotions simultaneously.
problem Automatic recognition of simultaneous and multiplicity of emotions in music.
method Comparison of multilabel and multiclass machine learning algorithms on the Emotify dataset.
result The Geneva Emotional Music Scale 9 is adopted for multilabel and multiclass classification of music emotions.
Beyond Sentiment Classification: A Generative Framework for Emotion Intensity Evaluation in Textcs.CL
Generative model evaluates text emotion intensity, outperforming classification.
problem Limitations of discrete emotion classification in applied domains.
method Fine-tuning generative language models to output continuous emotion intensity scores.
result Generative model outperforms classification baselines and reveals generalization capabilities.
In recent years, the use of bio-sensing signals such as electroencephalogram (EEG), electrocardiogram (ECG), etc. have garnered interest towards applications in affective computing. The parallel trend of deep-learning has led to a huge leap in performance towards solving various vision-based research problems such as o…
Automatic understanding of human affect using visual signals is a problem that has attracted significant interest over the past 20 years. However, human emotional states are quite complex. To appraise such states displayed in real-world settings, we need expressive emotional descriptors that are capable of capturing an…
Unified tensor factorization for efficient 3D convolutions in spatio-temporal emotion analysis.
problem Training deep 3D convolutions is computationally expensive and requires large datasets.
method Tensor factorization framework for separable higher-order convolutions.
result Improved spatio-temporal emotion estimation on large datasets.
Model improves emotion recognition using multiple physiological signals.
problem Single physiological signal is insufficient for accurate emotion recognition.
method Fused multiple modal physiological signals (EEG, EMG, EOG) for emotion classification.
result Best classification accuracy of 94.42% on arousal and 94.02% on valence in two-class tasks.
Learning Representations of Emotional Speech with Deep Convolutional Generative Adversarial Networkscs.CL
Automatically assessing emotional valence in human speech has historically been a difficult task for machine learning algorithms. The subtle changes in the voice of the speaker that are indicative of positive or negative emotional states are often "overshadowed" by voice characteristics relating to emotional intensity …
Variational Autoencoders for Learning Latent Representations of Speech Emotion: A Preliminary Studycs.SD
Learning the latent representation of data in unsupervised fashion is a very interesting process that provides relevant features for enhancing the performance of a classifier. For speech emotion recognition tasks, generating effective features is crucial. Currently, handcrafted features are mostly used for speech emoti…
New method transfers emotions in facial images.
problem Transforming facial images to different emotions.
method Infinite task learning and vector-valued reproducing kernel Hilbert spaces.
result Achieves low reconstruction cost and high emotion classification accuracy.
Investigating Audio, Visual, and Text Fusion Methods for End-to-End Automatic Personality Predictioncs.AI
We propose a tri-modal architecture to predict Big Five personality trait scores from video clips with different channels for audio, text, and video data. For each channel, stacked Convolutional Neural Networks are employed. The channels are fused both on decision-level and by concatenating their respective fully conne…
Gait patterns reveal emotions, offering a non-invasive method for automated recognition.
problem Automated emotion recognition from gait patterns.
method Data collection, preprocessing, and classification techniques.
result Gait patterns can indicate different emotion states, making them a promising source for emotion detection.
A Multi-Task Learning & Generation Framework: Valence-Arousal, Action Units & Primary Expressionscs.CV
Over the past few years many research efforts have been devoted to the field of affect analysis. Various approaches have been proposed for: i) discrete emotion recognition in terms of the primary facial expressions; ii) emotion analysis in terms of facial Action Units (AUs), assuming a fixed expression intensity; iii) …
StockEmotions dataset for financial sentiment and emotion analysis.
problem Limited resources for financial sentiment analysis.
method Collects 10,000 English comments from StockTwits, categorizes emotions into 12 classes.
result DistilBERT outperforms other models in sentiment classification, and Temporal Attention LSTM model achieves best performance in multivariate time series forecasting.
Study shows stress affects emotion recognition models, improving generalizability.
problem Stress affects emotion recognition models, reducing their generalizability.
method Used adversarial networks to control for stress effects on emotion recognition.
result Emotion recognition models that control for stress during training have better generalizability.
Deep learning framework detects emotions from EEG data.
problem Detecting emotions from EEG signals.
method Temporal and spatial convolutional layers learn discriminative representations.
result TSception achieves 86.03% classification accuracy, significantly outperforming other methods.
Study shows emotion affects speaker recognition and vice versa.
problem Dependencies between emotion and speaker recognition.
method Transfer learning and fine-tuning for emotion classification.
result Fine-tuning improves emotion recognition performance by 30.40% on IEMOCAP, 7.99% on MSP-Podcast, and 8.61% on Crema-D.
Deep architecture learns transferable features for robust speech emotion recognition.
problem Robust and discriminative features for diverse speech emotion domains.
method Jointly uses CNN for domain-shared features and LSTM for domain-specific emotion classification.
result Transferable features provide gains up to 18.4% in speech emotion recognition.
Bayesian model predicts emotion from fitness tracker heartbeat data.
problem Predicting emotional valence from consumer fitness tracker heartbeat data.
method End-to-end Bayesian deep learning model using PPG data.
result Peak F1 score of 0.7 for emotional valence classification.
Improved multi-modal emotion recognition using deep learning.
problem Combining acoustic and text modalities for emotion recognition.
method Proposes a deep learning-based approach to fuse text and acoustic data using SincNet for acoustic features and parallel DCNN and Bi-RNN branches for text processing with cross attention.
result Achieves 3.5% improvement in weighted accuracy compared to existing methods.
End-to-end model detects emotions and predicts Facebook reactions.
problem Detecting emotions and predicting reactions on Facebook posts.
method Jointly learns emotions and reactions using neural model with logic formulas as constraints.
result Model improves both emotion classification and reaction prediction.
In emotion recognition, it is difficult to recognize human's emotional states using just a single modality. Besides, the annotation of physiological emotional data is particularly expensive. These two aspects make the building of effective emotion recognition model challenging. In this paper, we first build a multi-vie…
Study enhances cryptocurrency sentiment analysis using TikTok and Twitter data.
problem Lack of comprehensive sentiment analysis in cryptocurrency markets.
method Multimodal analysis of TikTok and Twitter data using large language models.
result TikTok's video sentiment influences speculative assets and short-term trends.
This paper addresses the problem of emotion recognition from physiological signals. Features are extracted and ranked based on their effect on classification accuracy. Different classifiers are compared. The inter-subject variability and the personalization effect are thoroughly investigated, through trial-based and su…
Bardo Composer generates tabletop RPG music based on player speech.
problem Creating immersive background music for tabletop RPGs.
method Speech recognition, emotion classification, and music generation using a novel beam search algorithm.
result Generated music pieces can be accurately identified by human subjects as conveying the intended emotion.
Tactile enhanced multimedia is generated by synchronizing traditional multimedia clips, to generate hot and cold air effect, with an electric heater and a fan. This objective is to give viewers a more realistic and immersing feel of the multimedia content. The response to this enhanced multimedia content (mulsemedia) i…
Recently, generative adversarial networks and adversarial autoencoders have gained a lot of attention in machine learning community due to their exceptional performance in tasks such as digit classification and face recognition. They map the autoencoder's bottleneck layer output (termed as code vectors) to different no…
Two-stream model recognizes affect from audio and video.
problem Human affect recognition in real-world settings.
method Two-stream aural-visual analysis model with separate audio and visual processing.
result Model achieves promising results on Aff-Wild2 database.
Self-supervised ECG learning improves emotion recognition.
problem Improving emotion recognition from ECG signals.
method Multi-task deep learning framework with signal transformations as pretext tasks.
result Significant performance improvement in emotion classification.
Joint Binary Neural Network for Multi-label Learning with Applications to Emotion Classificationcs.LG
Recently the deep learning techniques have achieved success in multi-label classification due to its automatic representation learning ability and the end-to-end learning framework. Existing deep neural networks in multi-label classification can be divided into two kinds: binary relevance neural network (BRNN) and thre…