Novel fusion network combines polarization and radiomics features for liver cancer classification.
problem Challenges in histopathological diagnosis of HCC and ICC.
method Two-tier fusion approach: feature-level and classification-level.
result Significantly enhances classification accuracy, even at reduced resolutions.
This paper reviews deep learning for multi-modality medical image segmentation.
problem Improving segmentation accuracy in medical images using multiple modalities.
method Overview of deep learning and multi-modal medical image segmentation, analysis of different network architectures and fusion strategies.
result Later fusion of modalities can lead to more accurate segmentation results.
The study evaluates cross-modal knowledge fusion methods.
problem Combining knowledge from text, KGs, and images.
method Evaluation of different fusion methods using embeddings.
result Potential of cross-modal knowledge fusion.
New method learns fusion rules from few images using granular ball priors.
problem Challenges in supervised learning for image fusion with limited data.
method Introduces incomplete priors and Granular Ball Pixel Computation (GBPC) algorithm.
result Lightweight neural network learns effective fusion rules from few images.
3-D-CNN fusion improves high-resolution HS images from noisy MS and HS data.
problem Improving high-resolution hyperspectral images from noisy multispectral and hyperspectral data.
method 3-D-Convolutional Neural Network (3-D-CNN) for image fusion, followed by dimensionality reduction.
result The proposed method significantly reduces computational time and noise robustness compared to conventional methods.
Study fusion methods for financial image views to improve robustness against attacks.
problem Improving robustness of financial image views for next-day direction prediction.
method Same-source multi-view learning with early fusion and late fusion, using OHLCV and technical-indicator views, and evaluating pixel-space L-infinity attacks.
result Early fusion can suffer negative transfer under noisy settings, while late fusion is more reliable once labels stabilize.
New deep fusion methods improve human action recognition using depth and inertial sensor data.
problem Existing multimodal HAR frameworks lack mid-level feature fusion.
method Proposes three deep multilevel multimodal fusion frameworks, transforming depth and inertial sensor data into images and using convolution with Prewitt filter to create modality within modality.
result Supremacy of proposed fusion frameworks over existing methods on three publicly available datasets.
Framework fuses satellite and OSM data for local climate zone classification.
problem Heterogeneity of multimodal satellite and OSM data.
method Separate processing of satellite and OSM data, followed by fusion at the model level.
result Framework outperforms baseline by 6-2% in classification accuracy.
Study improves product categorization on Amazon using multi-modal fusion.
problem Multi-label product categorization in e-commerce.
method Late fusion of image, description, and title modalities using modified CNN and ResNet-50 models.
result Tri-modal late fusion model achieved an F1 score of 88.2%, significantly better than single modal models. MKPN predicts varying-sized kernels for burst image denoising.
problem Denoising burst images corrupted by noise.
method Deep neural network (MKPN) predicts and fuses kernels of varying sizes.
result MKPN outperforms state-of-the-art on synthetic datasets.
A new method for feature fusion in U-Net decoders using difference-based gating.
problem Precise fusion of high-level semantics and low-level details in U-Net decoder reconstruction.
method Proposes two difference-based gating approaches: Feature-difference gating (FDG) and Entropy-difference gating (EDG).
result Both FDG and EDG methods outperform existing attention-based fusion methods, with EDG showing superior performance.
This paper proposes a deep learning method to improve thermal image resolution.
problem Improving the resolution of thermal infrared images.
method Integrates high-frequency information from visual images to enhance thermal image resolution.
result The proposed multimodal fusion model outperforms state-of-the-art methods in super-resolution.
This work proposes a novel autoencoder for fusing visible and infrared images.
problem Challenging task to combine spatial and spectral information from visible and infrared images.
method Spatially constrained adversarial autoencoder with residual architecture and adversarial regularizer.
result Generates a more realistic fused image with enhanced spatial and spectral information.
Domain Fusion uses GANs to augment data for low-volume target datasets.
problem High costs in data development for deep learning applications.
method Multi-domain learning GANs to generate new samples.
result Domain Fusion achieves better classification accuracy with less data.
A lot of attention has been devoted to multimedia indexing over the past few years. In the literature, we often consider two kinds of fusion schemes: The early fusion and the late fusion. In this paper we focus on late classifier fusion, where one combines the scores of each modality at the decision level. To tackle th…
PointPainting fuses lidar and image data for better 3D object detection.
problem Lidar-only methods outperform fusion methods on 3D object detection benchmarks.
method Sequential fusion by projecting lidar points into image segmentation output and appending class scores.
result Significant improvements on state-of-the-art 3D object detection methods on KITTI and nuScenes datasets.
Flexible band grouping and kernel fusion for hyperspectral image processing.
problem Large dimensionality in hyperspectral imaging.
method Non-contiguous and contiguous band grouping for dimensionality reduction; improved visual clustering; unsupervised clustering algorithms; diverse features via different proximity metrics and kernel functions; l∞-norm multiple kernel learning. result Heterogeneous features and kernels lead to performance gain.
MNIST-NET10 fusion improves MNIST classification to 0.1% error rate.
problem Improving MNIST classification accuracy.
method Complex heterogeneous fusion architecture using degree of certainty aggregation.
result MNIST-NET10 achieves 0.1% error rate with 10 misclassifications.
New method handles missing data in multimodal brain imaging.
problem Missing data in multimodal brain imaging.
method Full Information Linked ICA (FI-LICA) algorithm.
result FI-LICA outperforms current practices in classification and prediction.
Framework fuses RGB images and depth maps for self-driving car control.
problem Fault tolerance in self-driving cars with sensor failures.
method Deep neural network architecture for sensor fusion.
result Framework can learn to use relevant sensor information even when one fails.
Fusion of transformer networks using optimal transport for improved performance.
problem Improving performance of transformer-based models through fusion.
method Exploiting optimal transport for soft alignment of transformer components.
result Consistently outperforms vanilla fusion and individual parent models.
Deep CNN predicts disruptions in fusion plasmas with high accuracy.
problem Predicting plasma events in fusion devices with multi-scale, multi-physics characteristics.
method Deep convolutional neural networks (CNN) with dilated convolutions trained on ECEi diagnostic data.
result Deep CNN achieves an F1-score of ~91% on disruption prediction.
A CNN-based model improves stock price prediction accuracy.
problem Overfitting in image-based stock prediction models.
method SMSFR-CNN combining CNN and image features.
result SMSFR-CNN achieves high predictive accuracy on A-share stocks.
A method extracts binary features directly from CS measurements for compressive image classification.
problem Efficiently classify images using compressive sensing without reconstruction.
method DCT-based approach for binary feature extraction from CS measurements, feature fusion with CNN features.
result Fused features outperform state-of-the-art methods in image classification.
Study predicts turbulent electric fields in fusion plasmas using deep learning.
problem Predicting turbulent electric fields in fusion plasmas.
method Physics-informed deep learning, drift-reduced Braginskii theory, experimental data.
result Neutrals broaden turbulent field amplitudes and increase shearing rates.
Current high-throughput data acquisition technologies probe dynamical systems with different imaging modalities, generating massive data sets at different spatial and temporal resolutions posing challenging problems in multimodal data fusion. A case in point is the attempt to parse out the brain structures and networks…
Finite image of mapping class group representations proved using graph embeddings.
problem Finiteness of images of mapping class group representations in twisted Dijkgraaf-Witten theory.
method Translation of problem into graph manipulation, using TVBW representations and spherical fusion categories.
result Finiteness of images of mapping class group representations in twisted Dijkgraaf-Witten theory is proven.
Paper tackles label noise in large datasets, purifying noisy data with a nonparametric framework.
problem Label noise in large-scale datasets with coarse labels.
method Develops a model-agnostic nonparametric framework for classification.
result Framework purifies noisy data using a small clean dataset and manages ambiguous samples.
Deep feature fusion improves mitosis counting accuracy.
problem Manual mitosis counting by pathologists is time-consuming and inconsistent.
method Combines Faster R-CNN for object detection with UNet segmentation features and RGB image features.
result Achieved an F-score of 0.508 on mitosis counting challenge dataset, outperforming state-of-the-art methods.
Study compares MRI and PET for Alzheimer's classification using deep learning.
problem Balanced comparison of MRI and PET for Alzheimer's disease classification.
method Deep learning with ADNI dataset, fusion of MRI and PET.
result Deep learning shows benefits of using both MRI and PET for Alzheimer's classification.
Framework for handling long-tailed multi-modal data.
problem Class imbalance and long-tailed distributions in multi-modal data.
method Multi-expert architecture with modality-specific networks and dynamic fusion weights.
result Framework outperforms existing methods in long-tailed, class-imbalanced scenarios.
Radiomics models improved by combining features from multiple modalities and classifiers.
problem Reduced predictive performance from combining features from a single modality and challenges in selecting optimal classifiers.
method Developed a reliable classifier fusion strategy using modality-specific classifiers and an analytic evidential reasoning (ER) rule.
result ER rule-based radiomics models outperformed traditional models.
Improved robustness in multi-modal sensor fusion with deep learning.
problem Inconsistency in fusion weights leading to poor performance under sensor failures.
method Proposes deep multi-modal sensor fusion architectures with fusion weight regularization and target learning.
result Proposed architectures outperform existing deep learning methods under sensor failures.
Improved VQA accuracy with generalized fusion operators.
problem Enhancing multimodal fusion for better VQA performance.
method Generalized Hadamard-Product fusion operators with Nonlinearity Ensembling, Feature Gating, and post-fusion layers.
result 1.1% absolute improvement on VQA 2.0 test-dev set.
Proposes Fusion Recurrent Neural Network for sequence data.
problem Improving sequence learning for practical applications.
method Fusion module and Transport module for sequence data.
result Fusion RNN performs comparably to state-of-the-art RNNs.
Tensor models improve joint EEG and fMRI analysis.
problem Jointly analyzing EEG and fMRI for brain function studies.
method Soft and flexible coupling of tensor decompositions for EEG and fMRI.
result Tensorial methods outperform ICA in multi-modal analysis.
Meta Fusion integrates various multimodal data fusion strategies into a unified framework.
problem Improving predictive power of machine learning methods across diverse applications.
method Meta Fusion constructs a cohort of models based on latent representations across modalities, sharing soft information to boost performance.
result Meta Fusion consistently outperforms conventional fusion strategies in simulation and real-world applications.
Survey examines challenges and tools for integrating multiple types of omics data.
problem Integrating multiple types of omics data for healthcare applications.
method Categorizes fusion approaches, collects open-source tools, explores datasets.
result Identifies challenges and gaps in multimodal learning for multi-omics.
New insights into knot fusion numbers via cabling.
problem Understanding fusion numbers of ribbon knots and their behavior under cabling.
method Utilizing knot Floer homology and cabling formulas to analyze fusion numbers.
result The fusion number and strong homotopy fusion number of (p,1)-cable knots are preserved.
Study improves robustness of deep fusion models against single source noise.
problem Ensuring robustness of deep fusion models against noise added to a single input source.
method Proposed two approaches: a carefully designed loss function and a convolutional fusion layer.
result Deep fusion models become robust against noise applied to a single source, preserving performance on clean data.
A new memory-based fusion layer improves multi-modal deep learning performance.
problem Improving performance of multi-modal deep learning by addressing long-term dependencies.
method Introducing a Memory based Attentive Fusion (MBAF) layer that incorporates both current and long-term dependencies.
result The MBAF layer enhances fusion and improves performance across different modalities and networks.
Optimized deep learning architectures improve sensor fusion performance.
problem Sensor fusion in autonomous systems.
method Proposed two optimized architectures: coarser-grained and two-stage gated.
result Significant performance improvements and robustness in noisy conditions.
Innovative 2-categories create 4-manifold invariants.
problem Constructing invariants for 4-manifolds.
method Semisimple 2-categories, fusion 2-categories, and state-sum construction.
result Construct a state-sum invariant for 4-manifolds.
MISA combines multiple datasets for better feature extraction.
problem Combining diverse datasets for better feature extraction.
method MISA combines multiple heterogeneous datasets using Kotz distribution and combinatorial optimization.
result MISA produces robust generalization of ICA, IVA, and ISA.
Framework improves marine mammal monitoring in noisy underwater environments.
problem Underwater bioacoustic monitoring challenges due to overlapping calls and variable noise.
method Multi-step attention-guided framework with segmentation and mid-level fusion.
result Improved signal discrimination, reduced false positives, reliable representations.
This paper improves model fusion by training-time neuron alignment, reducing barriers in multi-model fusion.
problem Diverse neuron permutations across different settings hinder model fusion performances.
method Training-time neuron alignment using fixed neuron anchors to reduce training-time permutations.
result Training-time neuron alignment improves fusion of pretrained models and federated learning performances.
We classify all fusion categories for a given set of fusion rules with three simple object types. If a conjecture of Ostrik is true, our classification completes the classification of fusion categories with three simple object types. To facilitate the discussion we describe a convenient, concrete and useful variation o…
Paper develops Riemannian geometry for SPSD matrices with DA applications.
problem Riemannian geometry of SPSD matrices for DA.
method Closed-form expressions, approximations of geodesic path, PT, canonical representation.
result Proposes an algorithm for DA with improved performance.