Sensor fusion has wide applications in many domains including health care and autonomous systems. While the advent of deep learning has enabled promising multi-modal fusion of high-level features and end-to-end sensor fusion solutions, existing deep learning based sensor fusion techniques including deep gating architec…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,932 papers · 148 categories
Trend · papers per month
48 results for “deep fusion”
Study improves robustness of deep fusion models against single source noise.
problem Ensuring robustness of deep fusion models against noise added to a single input source.
method Proposed two approaches: a carefully designed loss function and a convolutional fusion layer.
result Deep fusion models become robust against noise applied to a single source, preserving performance on clean data.
This paper reviews deep learning for multi-modality medical image segmentation.
problem Improving segmentation accuracy in medical images using multiple modalities.
method Overview of deep learning and multi-modal medical image segmentation, analysis of different network architectures and fusion strategies.
result Later fusion of modalities can lead to more accurate segmentation results.
A new memory-based fusion layer improves multi-modal deep learning performance.
problem Improving performance of multi-modal deep learning by addressing long-term dependencies.
method Introducing a Memory based Attentive Fusion (MBAF) layer that incorporates both current and long-term dependencies.
result The MBAF layer enhances fusion and improves performance across different modalities and networks.
EmbraceNet fusion model for multi-sensor activity recognition.
problem Human activity recognition using multiple sensors is challenging.
method Independent processing of each sensor, feature fusion with EmbraceNet, post-processing, and additional processes.
result Improved performance in SHL recognition challenge.
Deep convolutional neural networks for multi-scale time-series classification and application to disruption prediction in fusion devicesphysics.plasm-ph
Deep CNN predicts disruptions in fusion plasmas with high accuracy.
problem Predicting plasma events in fusion devices with multi-scale, multi-physics characteristics.
method Deep convolutional neural networks (CNN) with dilated convolutions trained on ECEi diagnostic data.
result Deep CNN achieves an F1-score of ~91% on disruption prediction.
Meta Fusion integrates various multimodal data fusion strategies into a unified framework.
problem Improving predictive power of machine learning methods across diverse applications.
method Meta Fusion constructs a cohort of models based on latent representations across modalities, sharing soft information to boost performance.
result Meta Fusion consistently outperforms conventional fusion strategies in simulation and real-world applications.
Multi-Modal Deep Learning for Credit Rating Prediction Using Text and Numerical Data Streamsq-fin.GN
Paper analyzes deep learning models for credit rating prediction using text and numerical data.
problem Improving credit rating prediction using multi-modal deep learning.
method Testing different deep learning models and fusion strategies for structured and unstructured datasets.
result CNN-based multi-modal model with two fusion strategies outperformed other models.
Sensor Fusion using Backward Shortcut Connections for Sleep Apnea Detection in Multi-Modal Datacs.LG
Improved sleep apnea detection using sensor fusion and backward shortcut connections.
problem Untreated sleep apnea leads to severe health consequences; automated detection is needed.
method Late sensor fusion using backward shortcut connections to improve deep learning models.
result Significant improvement in predictive performance over single sensor methods.
Layer fusion reduces deep neural network layers with minimal loss in accuracy.
problem Model compression to reduce neural network size and computation.
method Fusion of similar layers to reduce model size with minimal performance loss.
result Deep networks can be compressed up to 3.33x with minimal accuracy loss.
Human Action Recognition Using Deep Multilevel Multimodal (M2) Fusion of Depth and Inertial Sensorscs.LG
New deep fusion methods improve human action recognition using depth and inertial sensor data.
problem Existing multimodal HAR frameworks lack mid-level feature fusion.
method Proposes three deep multilevel multimodal fusion frameworks, transforming depth and inertial sensor data into images and using convolution with Prewitt filter to create modality within modality.
result Supremacy of proposed fusion frameworks over existing methods on three publicly available datasets.
MNIST-NET10 fusion improves MNIST classification to 0.1% error rate.
problem Improving MNIST classification accuracy.
method Complex heterogeneous fusion architecture using degree of certainty aggregation.
result MNIST-NET10 achieves 0.1% error rate with 10 misclassifications.
Sensor fusion is a key technology that integrates various sensory inputs to allow for robust decision making in many applications such as autonomous driving and robot control. Deep neural networks have been adopted for sensor fusion in a body of recent studies. Among these, the so-called netgated architecture was propo…
Proposes Fusion Recurrent Neural Network for sequence data.
problem Improving sequence learning for practical applications.
method Fusion module and Transport module for sequence data.
result Fusion RNN performs comparably to state-of-the-art RNNs.
In this paper, we propose a deep multimodal fusion network to fuse multiple modalities (face, iris, and fingerprint) for person identification. The proposed deep multimodal fusion algorithm consists of multiple streams of modality-specific Convolutional Neural Networks (CNNs), which are jointly optimized at multiple fe…
Late fusion of clinical notes and physiological data improves ICU mortality prediction.
problem Improving ICU mortality prediction using multimodal data.
method Late fusion of clinical notes and physiological time series data with a deep learning architecture.
result Late fusion approach provides statistically significant improvement in mortality prediction performance.
New method learns fusion rules from few images using granular ball priors.
problem Challenges in supervised learning for image fusion with limited data.
method Introduces incomplete priors and Granular Ball Pixel Computation (GBPC) algorithm.
result Lightweight neural network learns effective fusion rules from few images.
Two solutions for multi-modal record linkage using Deep Learning inspired by Visual Question Answering.
problem Matching records from multiple sources representing the same entity.
method Two fusion modules: Recurrent Neural Network + Convolutional Neural Network and Stacked Attention Network. A Siamese Neural Network computes similarity.
result Recurrent Neural Network + Convolutional Neural Network fusion module outperforms a simple model.
Generalized Bilinear Deep Convolutional Neural Networks for Multimodal Biometric Identificationcs.LG
In this paper, we propose to employ a bank of modality-dedicated Convolutional Neural Networks (CNNs), fuse, train, and optimize them together for person classification tasks. A modality-dedicated CNN is used for each modality to extract modality-specific features. We demonstrate that, rather than spatial fusion at the…
Comparing Deep Learning Models for the Task of Volatility Prediction Using Multivariate Dataq-fin.ST
Study compares deep learning models for volatility prediction using multivariate data.
problem Predicting volatility using multivariate data.
method Evaluated multiple deep learning models including MLP, RNN, TCN, and Temporal Fusion Transformer.
result Temporal Fusion Transformer and TCN variants outperform classical models and shallow networks.
Proposes a deep Auto-Encoder-like framework for visual-tactile fusion object clustering.
problem Combining visual and tactile information for better object clustering.
method Deep Auto-Encoder-like Non-negative Matrix Factorization framework, graph regularizer, modality-level consensus regularizer, alternating minimization strategy.
result Improves object clustering performance by leveraging both visual and tactile modalities.
Human Activity Recognition (HAR) based on motion sensors has drawn a lot of attention over the last few years, since perceiving the human status enables context-aware applications to adapt their services on users' needs. However, motion sensor fusion and feature extraction have not reached their full potentials, remain…
This paper improves model fusion by training-time neuron alignment, reducing barriers in multi-model fusion.
problem Diverse neuron permutations across different settings hinder model fusion performances.
method Training-time neuron alignment using fixed neuron anchors to reduce training-time permutations.
result Training-time neuron alignment improves fusion of pretrained models and federated learning performances.
AI model predicts stock prices using social media data and hybrid neural networks.
problem Predicting stock price movements during the COVID-19 pandemic.
method Integrates social media trends and historical stock data using a hybrid CNN-BLSTM framework.
result The proposed framework outperforms traditional models in predicting stock price movements.
Study predicts turbulent electric fields in fusion plasmas using deep learning.
problem Predicting turbulent electric fields in fusion plasmas.
method Physics-informed deep learning, drift-reduced Braginskii theory, experimental data.
result Neutrals broaden turbulent field amplitudes and increase shearing rates.
Domain Fusion uses GANs to augment data for low-volume target datasets.
problem High costs in data development for deep learning applications.
method Multi-domain learning GANs to generate new samples.
result Domain Fusion achieves better classification accuracy with less data.
CMTF improves financial market forecasting by fusing multiple data types.
problem Lack of effective integration of diverse financial data sources.
method Transformer-based deep learning framework with tensor interpretation and auto-training.
result CMTF outperforms classical and deep learning models in price direction classification.
In the past, Acoustic Scene Classification systems have been based on hand crafting audio features that are input to a classifier. Nowadays, the common trend is to adopt data driven techniques, e.g., deep learning, where audio representations are learned from data. In this paper, we propose a system that consists of a …
Polynomial fusion layer improves speech-driven facial animation.
problem Recent facial synthesis relies on low-dimensional representations and concatenation, ignoring higher-order interactions.
method Proposes a polynomial fusion layer to model higher-order interactions of facial encodings.
result Demonstrates improved video quality, audiovisual synchronisation, and blink generation.
Early recognition of abnormal rhythms in ECG signals is crucial for monitoring and diagnosing patients' cardiac conditions, increasing the success rate of the treatment. Classifying abnormal rhythms into exact categories is very challenging due to the broad taxonomy of rhythms, noises and lack of large-scale real-world…
We present convolutional neural network (CNN) based approaches for unsupervised multimodal subspace clustering. The proposed framework consists of three main stages - multimodal encoder, self-expressive layer, and multimodal decoder. The encoder takes multimodal data as input and fuses them to a latent space representa…
Physics-informed machine learning techniques for edge plasma turbulence modelling in computational theory and experimentphysics.plasm-ph
Physics-informed ML models improve turbulence understanding in fusion plasmas.
problem Improving turbulence modeling in fusion plasma devices.
method Physics-informed deep learning framework constrained by PDEs.
result Direct quantitative comparisons of turbulent fields between theory and gyrokinetic models.
Simultaneous Long-tailed Recognition and Multi-modal Fusion for Highly Imbalanced Multi-modal Datacs.CV
Framework for handling long-tailed multi-modal data.
problem Class imbalance and long-tailed distributions in multi-modal data.
method Multi-expert architecture with modality-specific networks and dynamic fusion weights.
result Framework outperforms existing methods in long-tailed, class-imbalanced scenarios.
FuseInit optimizes neural network initialization by fusing layers from deeper networks.
problem Initialization sensitivity of neural networks.
method Fusing neighboring layers from deeper networks trained with random initialization.
result Shallower networks initialized with FuseInit can perform as well as deeper networks.
HighRes-net enhances satellite imagery by fusing multiple low-res views.
problem Super-resolving satellite imagery for reliable monitoring of human impact.
method End-to-end deep learning approach for multi-frame super-resolution.
result HighRes-net outperformed in European Space Agency's MFSR competition.
Kernel fusion is a popular and effective approach for combining multiple features that characterize different aspects of data. Traditional approaches for Multiple Kernel Learning (MKL) attempt to learn the parameters for combining the kernels through sophisticated optimization procedures. In this paper, we propose an a…
We consider the task of multimodal music mood prediction based on the audio signal and the lyrics of a track. We reproduce the implementation of traditional feature engineering based approaches and propose a new model based on deep learning. We compare the performance of both approaches on a database containing 18,000 …
Classification using multimodal data arises in many machine learning applications. It is crucial not only to model cross-modal relationship effectively but also to ensure robustness against loss of part of data or modalities. In this paper, we propose a novel deep learning-based multimodal fusion architecture for class…
Incomplete Multi-view Clustering via Hierarchical Semantic Alignment and Cooperative Completioneess.IV
HSACC improves multi-view clustering of incomplete data.
problem Challenges in clustering incomplete multi-view data.
method Hierarchical Semantic Alignment and Cooperative Completion framework.
result HSACC outperforms state-of-the-art methods on benchmark datasets.
Learning Pose Estimation for UAV Autonomous Navigation andLanding Using Visual-Inertial Sensor Datacs.LG
Improved UAV navigation and landing using deep learning.
problem Autonomous navigation and landing of UAVs with high accuracy.
method Multimodal fusion of visual and inertial sensor data using deep neural networks.
result 25% improvement in pose estimation accuracy compared to traditional methods.
With the increasing popularity of video sharing websites such as YouTube and Facebook, multimodal sentiment analysis has received increasing attention from the scientific community. Contrary to previous works in multimodal sentiment analysis which focus on holistic information in speech segments such as bag of words re…
CoCoAFusE fuses expert predictions to model complex patterns with interpretability and uncertainty.
problem Modeling complex patterns with interpretability and uncertainty quantification.
method Competitive/Collaborative Fusion of Experts (CoCoAFusE) that fuses expert distributions in addition to mixing.
result CoCoAFusE avoids multimodality artifacts and provides tighter credible bounds on the response variable.
A machine learning approach to record fusion with high accuracy.
problem Aggregating multiple records corresponding to the same entity.
method Constructing feature vectors from attribute-level, record-level, and database-level signals; using a stagewise additive model to learn a classifier.
result Average precision of ~98% with source information and ~94% without source information across diverse datasets.
Automated methods for Alzheimer's disease (AD) classification have the potential for great clinical benefits and may provide insight for combating the disease. Machine learning, and more specifically deep neural networks, have been shown to have great efficacy in this domain. These algorithms often use neurological ima…
In order to maximize detection precision rate as well as the recall rate, this paper proposes an in-vehicle multi-source fusion scheme in Keyword Spotting (KWS) System for vehicle applications. Vehicle information, as a new source for the original system, is collected by an in-vehicle data acquisition platform while th…
MCFNet recovers spatial detail and fuses it with semantic information for real-time segmentation.
problem Recovering spatial detail information and fusing it with semantic information in real-time.
method Proposes a new architecture (MCFNet) with feature refinement and fusion modules, and a gating unit.
result Achieves competitive performance with high speed (75.5% mIOU, 151.3 FPS on Cityscapes).
Turbulent field fluctuations in gyrokinetic and fluid plasmasphysics.plasm-ph
Deep learning compares turbulence models in plasma physics.
problem Predicting edge plasma turbulence in magnetic fusion reactors.
method Physics-informed deep learning framework for comparing two-fluid and gyrokinetic models.
result Good overall agreement between two-fluid theory and gyrokinetic models in turbulent field fluctuations.
DPC uses physics and neural nets to solve SDEs.
problem Solving stochastic differential equations with missing physics.
method Physics-data fusion with conditional maximum mean discrepancy (CMMD) loss.
result DPC achieves highly accurate solutions on benchmark examples.