A lot of attention has been devoted to multimedia indexing over the past few years. In the literature, we often consider two kinds of fusion schemes: The early fusion and the late fusion. In this paper we focus on late classifier fusion, where one combines the scores of each modality at the decision level. To tackle th…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “late fusion”
In the past few years, a lot of attention has been devoted to multimedia indexing by fusing multimodal informations. Two kinds of fusion schemes are generally considered: The early fusion and the late fusion. We focus on late classifier fusion, where one combines the scores of each modality at the decision level. To ta…
Late fusion of clinical notes and physiological data improves ICU mortality prediction.
problem Improving ICU mortality prediction using multimodal data.
method Late fusion of clinical notes and physiological time series data with a deep learning architecture.
result Late fusion approach provides statistically significant improvement in mortality prediction performance.
Study fusion methods for financial image views to improve robustness against attacks.
problem Improving robustness of financial image views for next-day direction prediction.
method Same-source multi-view learning with early fusion and late fusion, using OHLCV and technical-indicator views, and evaluating pixel-space L-infinity attacks.
result Early fusion can suffer negative transfer under noisy settings, while late fusion is more reliable once labels stabilize.
Sensor Fusion using Backward Shortcut Connections for Sleep Apnea Detection in Multi-Modal Datacs.LG
Sleep apnea is a common respiratory disorder characterized by breathing pauses during the night. Consequences of untreated sleep apnea can be severe. Still, many people remain undiagnosed due to shortages of hospital beds and trained sleep technicians. To assist in the diagnosis process, automated detection methods are…
One of the core tasks in multi-view learning is to capture relations among views. For sequential data, the relations not only span across views, but also extend throughout the view length to form long-term intra-view and inter-view interactions. In this paper, we present a new memory augmented neural network model that…
In this study, we investigated multi-modal approaches using images, descriptions, and titles to categorize e-commerce products on Amazon. Specifically, we examined late fusion models, where the modalities are fused at the decision level. Products were each assigned multiple labels, and the hierarchy in the labels were …
Human Activity Recognition (HAR) based on motion sensors has drawn a lot of attention over the last few years, since perceiving the human status enables context-aware applications to adapt their services on users' needs. However, motion sensor fusion and feature extraction have not reached their full potentials, remain…
Framework for multi-task learning with semiparametric models and nuisance parameters.
problem Improving parameter estimation from diverse, heterogeneous datasets.
method Late fusion multi-task learning framework with two-step process: individual task learning followed by adaptive aggregation.
result The method achieves faster convergence rates compared to individual task learning when tasks share similar parametric components.
A new multi-view clustering method that is fast, scalable, and easy to use.
problem High computational complexity, one-stage fusion, and dataset-specific hyperparameter tuning in multi-view clustering.
method Random view groups, hybrid early-late fusion, diversified base clusterings, and unified bipartite graph.
result Almost linear time and space complexity, no dataset-specific tuning required.
Meta Fusion integrates various multimodal data fusion strategies into a unified framework.
problem Improving predictive power of machine learning methods across diverse applications.
method Meta Fusion constructs a cohort of models based on latent representations across modalities, sharing soft information to boost performance.
result Meta Fusion consistently outperforms conventional fusion strategies in simulation and real-world applications.
Cooperative Perception for 3D Object Detection in Driving Scenarios using Infrastructure Sensorscs.CV
3D object detection is a common function within the perception system of an autonomous vehicle and outputs a list of 3D bounding boxes around objects of interest. Various 3D object detection methods have relied on fusion of different sensor modalities to overcome limitations of individual sensors. However, occlusion, l…
In this paper we seek methods to effectively detect urban micro-events. Urban micro-events are events which occur in cities, have limited geographical coverage and typically affect only a small group of citizens. Because of their scale these are difficult to identify in most data sources. However, by using citizen sens…
We present convolutional neural network (CNN) based approaches for unsupervised multimodal subspace clustering. The proposed framework consists of three main stages - multimodal encoder, self-expressive layer, and multimodal decoder. The encoder takes multimodal data as input and fuses them to a latent space representa…
The Acoustic Camouflage Phenomenon: Re-evaluating Speech Features for Financial Risk Predictioncs.SD
Study shows integrating acoustic features in financial forecasting models can degrade performance.
problem Predicting stock market volatility from corporate earnings calls using speech features.
method Empirical investigation of acoustic feature extraction in teleconference environments using a two-stream late-fusion architecture.
result Integrating acoustic features via late fusion significantly degraded performance, reducing recall to 47.08%.
DMFAW improves multi-view clustering with adaptive weights and feature selection.
problem Lack of effective feature selection and empirical hyperparameter selection in existing deep matrix factorization methods.
method Introduces Deep Matrix Factorization with Adaptive Weights (DMFAW) for multi-view clustering, incorporating feature selection and dynamically updating weights using Control Theory.
result DMFAW outperforms state-of-the-art methods in clustering performance.
Novel fusion network combines polarization and radiomics features for liver cancer classification.
problem Challenges in histopathological diagnosis of HCC and ICC.
method Two-tier fusion approach: feature-level and classification-level.
result Significantly enhances classification accuracy, even at reduced resolutions.
DRAGON improves learning for rare classes in unbalanced datasets using class descriptions.
problem Learning rare classes in unbalanced datasets with deep models.
method DRAGON is a late-fusion architecture that corrects bias towards frequent classes and fuses class-descriptions to improve tail-class accuracy.
result DRAGON outperforms state-of-the-art models on new benchmarks for long-tail learning with class descriptors.
In the past, Acoustic Scene Classification systems have been based on hand crafting audio features that are input to a classifier. Nowadays, the common trend is to adopt data driven techniques, e.g., deep learning, where audio representations are learned from data. In this paper, we propose a system that consists of a …
Paper classifies movie genres using multimodal data.
problem Challenging task of multi-label movie genre classification.
method Created dataset from video clips, subtitles, synopses, and posters. Extracted features using various descriptors. Evaluated using different classifiers and late fusion strategy.
result Best F-Score result of 0.628 achieved by combining LSTM on synopses and CNN on movie trailer frames.
Spectral and matrix factorization methods for consistent community detection in multi-layer networksstat.ML
We consider the problem of estimating a consensus community structure by combining information from multiple layers of a multi-layer network using methods based on the spectral clustering or a low-rank matrix factorization. As a general theme, these "intermediate fusion" methods involve obtaining a low column rank matr…
Multimodal Deep Learning for Finance: Integrating and Forecasting International Stock Marketsq-fin.CP
In today's increasingly international economy, return and volatility spillover effects across international equity markets are major macroeconomic drivers of stock dynamics. Thus, information regarding foreign markets is one of the most important factors in forecasting domestic stock prices. However, the cross-correlat…
MSB framework improves survival prediction in immunotherapy patients with missing data.
problem High dimensionality and blockwise missingness in multimodal clinical data.
method MSB is a late-fusion framework that independently models modality-specific features before aggregating predictions via cross-validated stacking.
result MSB outperformed baseline algorithms in predicting progression-free survival in lung cancer patients.
We study a two-level multiview learning with more than two views under the PAC-Bayesian framework. This approach, sometimes referred as late fusion, consists in learning sequentially multiple view-specific classifiers at the first level, and then combining these view-specific classifiers at the second level. Our main t…
A new method for combining multiple data views in supervised learning.
problem Combining multiple data views in supervised learning, especially in biology and medicine.
method Cooperative learning combines squared error loss with an agreement penalty to encourage predictions from different data views to agree.
result Cooperative learning achieves higher predictive accuracy on simulated and real multiomics data.
Deep neural networks solve Raven's Progressive Matrices with high accuracy.
problem Testing relational reasoning in machine learning systems.
method Combining Wild Relation Networks with Multi-Layer Relation Networks and introducing Magnitude Encoding.
result Deep neural networks achieve 98.0 percent accuracy, significantly improving over previous methods.
New model recognizes emotions with missing modalities, improving accuracy.
problem Handling missing modalities in emotion recognition.
method Transformer-based architecture with cross-attention and self-attention mechanisms.
result Improvement of 37% in predicting arousal values and 30% in valence values compared to baseline.
Improved neural networks by averaging late-stage weights.
problem Improving the performance of neural networks.
method Ensemble late-stage weights and average them.
result Augmenting standard models with late-phase weights improves generalization.
New method finds precise late-time behavior of wave equations.
problem Analyzing late-time behavior of wave equations with inverse-square potentials.
method Physical-space-based method for deriving late-time asymptotics.
result Sharp, uniform decay estimates in time for asymptotic late-time tails.
Study assesses drought and late-frost risks in Bavaria using vine copulas.
problem Assessing risks of late-frost and drought in Bavaria due to climate change.
method Used vine copula models for non-Gaussian and asymmetric dependencies, with univariate and bivariate regression analyses.
result Identified 'at-risk' regions for forest adaptation.
This paper presents a light-weight and accurate deep neural model for audiovisual emotion recognition. To design this model, the authors followed a philosophy of simplicity, drastically limiting the number of parameters to learn from the target datasets, always choosing the simplest earning methods: i) transfer learnin…
Second part of series studying charged scalar fields on Reissner--Nordström spacetimes.
problem Analyzing late-time behavior and stability of charged scalar fields on black hole backgrounds.
method Purely physical-space based methods, energy estimates, inverse-power laws.
result First pointwise decay estimates for charged scalar fields on black hole backgrounds.
New framework optimizes deep learning training by deferring large batch sizes to late stages.
problem Optimizing batch size scheduling for deep learning training efficiency.
method Introduced the functional scaling law (FSL) framework to analyze and optimize batch size scheduling.
result Large batch sizes can be deferred to late training stages without sacrificing performance.
DAGR improves navigation by refining goal representations conditioned on the current state.
problem Goal-conditioned reinforcement learning lacks state awareness, leading to inefficient policy recovery.
method DAGR refines static goal embeddings into state-conditioned ones using gated cross-attention with a state-goal discrepancy map.
result DAGR improves navigation tasks on OGBench, matching or outperforming base methods.
LITE models improve query-document relevance with learnable late interactions.
problem Improving query-document relevance with lower latency and storage.
method Proposes learnable late-interaction models (LITE) that use factorized query and document embeddings followed by a learnable scorer.
result Empirically, LITE outperforms previous late-interaction models in re-ranking tasks.
Sensor fusion has wide applications in many domains including health care and autonomous systems. While the advent of deep learning has enabled promising multi-modal fusion of high-level features and end-to-end sensor fusion solutions, existing deep learning based sensor fusion techniques including deep gating architec…
Proposes Fusion Recurrent Neural Network for sequence data.
problem Improving sequence learning for practical applications.
method Fusion module and Transport module for sequence data.
result Fusion RNN performs comparably to state-of-the-art RNNs.
We propose a generalized class of multimodal fusion operators for the task of visual question answering (VQA). We identify generalizations of existing multimodal fusion operators based on the Hadamard product, and show that specific non-trivial instantiations of this generalized fusion operator exhibit superior perform…
SAM selects flatter minima late in training, improving generalization.
problem Improving neural network generalization under various settings.
method Sharpness-Aware Minimization (SAM) applied late in training.
result SAM efficiently selects flatter minima late in training, improving generalization.
New insights into knot fusion numbers via cabling.
problem Understanding fusion numbers of ribbon knots and their behavior under cabling.
method Utilizing knot Floer homology and cabling formulas to analyze fusion numbers.
result The fusion number and strong homotopy fusion number of (p,1)-cable knots are preserved.
JD.com uses a new CNN model to improve ad click prediction.
problem Improving CTR prediction for ads with visual content.
method Proposes Category-specific CNN (CSCNN) to incorporate category knowledge early in the feature extraction process.
result CSCNN outperforms existing methods in CTR prediction.
We introduce semisimple 2-categories, fusion 2-categories, and spherical fusion 2-categories. For each spherical fusion 2-category, we construct a state-sum invariant of oriented singular piecewise-linear 4-manifolds.
Estimates LATE using combined datasets, overcoming data limitations.
problem Estimating LATE when compliance is incomplete and data is split.
method Combines separately observed datasets to estimate LATE using direct and weighted least squares methods.
result Proposes a stable and practical estimator for LATE.
A new memory-based fusion layer improves multi-modal deep learning performance.
problem Improving performance of multi-modal deep learning by addressing long-term dependencies.
method Introducing a Memory based Attentive Fusion (MBAF) layer that incorporates both current and long-term dependencies.
result The MBAF layer enhances fusion and improves performance across different modalities and networks.
Algorithms that fuse multiple input sources benefit from both complementary and shared information. Shared information may provide robustness against faulty or noisy inputs, which is indispensable for safety-critical applications like self-driving cars. We investigate learning fusion algorithms that are robust against …
This paper improves model fusion by training-time neuron alignment, reducing barriers in multi-model fusion.
problem Diverse neuron permutations across different settings hinder model fusion performances.
method Training-time neuron alignment using fixed neuron anchors to reduce training-time permutations.
result Training-time neuron alignment improves fusion of pretrained models and federated learning performances.
We classify all fusion categories for a given set of fusion rules with three simple object types. If a conjecture of Ostrik is true, our classification completes the classification of fusion categories with three simple object types. To facilitate the discussion we describe a convenient, concrete and useful variation o…
New proof and formula linking fusion trees to quantum knot invariants.
problem Quantum knot invariants encoding in non-semisimple TQC.
method Connection between fusion trees and Lawrence representations, using graphical calculus.
result Explicit encoding of quantum knot invariants via fusion trees.