A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New method for multi-modal depth prediction challenges.
Study reduces redundant information in multi-modal datasets.
Data balancing reduces variance in machine learning models.
Framework corrects model form errors in structural dynamics predictions.
We propose a novel method, Modality-based Redundancy Reduction Fusion (MRRF), for understanding and modulating the relative contribution of each modality in multimodal inference tasks. This is achieved by obtaining an -way tensor to consider the high-order relationships between modalities and the output laye…
SDR outperforms IDR in multimodal data analysis, especially with fewer samples.
In this paper, we propose a deep multimodal fusion network to fuse multiple modalities (face, iris, and fingerprint) for person identification. The proposed deep multimodal fusion algorithm consists of multiple streams of modality-specific Convolutional Neural Networks (CNNs), which are jointly optimized at multiple fe…
In this paper, we propose to employ a bank of modality-dedicated Convolutional Neural Networks (CNNs), fuse, train, and optimize them together for person classification tasks. A modality-dedicated CNN is used for each modality to extract modality-specific features. We demonstrate that, rather than spatial fusion at the…
GPCCA integrates multi-modal data with missing values, improving clustering accuracy.
Assistive robots can potentially improve the quality of life and personal independence of elderly people by supporting everyday life activities. To guarantee a safe and intuitive interaction between human and robot, human intentions need to be recognized automatically. As humans communicate their intentions multimodall…
SiMLR reduces complex biomedical data into simpler, interpretable forms.
In many classification systems, sensing modalities have different acquisition costs. It is often {\it unnecessary} to use every modality to classify a majority of examples. We study a multi-stage system in a prediction time cost reduction setting, where the full data is available for training, but for a test example, m…
A new modal autoencoder improves feature extraction by constraining the decoder.
This work improves manifold learning for multi-modal data.
In linear inverse problems, the goal is to recover a target signal from undersampled, incomplete or noisy linear measurements. Typically, the recovery relies on complex numerical optimization methods; recent approaches perform an unfolding of a numerical algorithm into a neural network form, resulting in a substantial …
Structural damage due to excessive loading or environmental degradation typically occurs in localized areas in the absence of collapse. This prior information about the spatial sparseness of structural damage is exploited here by a hierarchical sparse Bayesian learning framework with the goal of reducing the source of …
Bayesian approach reduces FL communication cost by one-shot.
Adaptive anchor methods improve multi-modal learning by balancing intra-modal and inter-modal information.
Many machine learning problems, especially multi-modal learning problems, have two sets of distinct features (e.g., image and text features in news story classification, or neuroimaging data and neurocognitive data in cognitive science research). This paper addresses the joint dimensionality reduction of two feature ve…
Proposes MPCA for robust PCA using mode estimation.
KD-Net transfers knowledge from multi-modal to mono-modal segmentation networks.
MHVAE learns cross-modality inference inspired by human cognition.
The paper analyzes and proposes an algorithm for multi-modal nonlinear embeddings with theoretical performance bounds.
COBRA reduces modality gap in cross-modal tasks.
Proposes a novel network for CTR prediction by learning modality-specific and modality-invariant representations.
Combining complementary information from multiple modalities is intuitively appealing for improving the performance of learning-based approaches. However, it is challenging to fully leverage different modalities due to practical challenges such as varying levels of noise and conflicts between modalities. Existing metho…
We investigate deep generative models that can exchange multiple modalities bi-directionally, e.g., generating images from corresponding texts and vice versa. Recently, some studies handle multiple modalities on deep generative models, such as variational autoencoders (VAEs). However, these models typically assume that…
Improves video search by balancing text and visual modalities.
Multimodal sentiment analysis is a core research area that studies speaker sentiment expressed from the language, visual, and acoustic modalities. The central challenge in multimodal learning involves inferring joint representations that can process and relate information from these modalities. However, existing work l…
MoCA uses a novel autoencoder to analyze multi-modal health data.
We explore training an automatic modality tagger. Modality is the attitude that a speaker might have toward an event or state. One of the main hurdles for training a linguistic tagger is gathering training data. This is particularly problematic for training a tagger for modality because modality triggers are sparse for…
Obtaining common representations from different modalities is important in that they are interchangeable with each other in a classification problem. For example, we can train a classifier on image features in the common representations and apply it to the testing of the text features in the representations. Existing m…
Framework for handling long-tailed multi-modal data.
TAP transfers knowledge from unlabeled data to improve cross-modal learning.
Learning generative models that span multiple data modalities, such as vision and language, is often motivated by the desire to learn more useful, generalisable representations that faithfully capture common underlying factors between the modalities. In this work, we characterise successful learning of such models as t…
DNN-based cross-modal retrieval has become a research hotspot, by which users can search results across various modalities like image and text. However, existing methods mainly focus on the pairwise correlation and reconstruction error of labeled data. They ignore the semantically similar and dissimilar constraints bet…
This work improves multi-modal generative models by using permutation-invariant neural networks.
A framework for uncertainty-aware multimodal learning using conformal Shapley intervals.
MAESTRO improves multimodal learning for dynamic time series with adaptive attention and robustness.
Develops multi-modal neural network models for improved prediction and uncertainty quantification.
This paper studies the nonparametric modal regression problem systematically from a statistical learning view. Originally motivated by pursuing a theoretical understanding of the maximum correntropy criterion based regression (MCCR), our study reveals that MCCR with a tending-to-zero scale parameter is essentially moda…
In this study, we investigated multi-modal approaches using images, descriptions, and titles to categorize e-commerce products on Amazon. Specifically, we examined late fusion models, where the modalities are fused at the decision level. Products were each assigned multiple labels, and the hierarchy in the labels were …
Centroid-Encoder reduces high-dimensional data for better visualization.
A new method simplifies sampling from complex distributions without using diffusions.
Symile learns joint representations across multiple modalities, outperforming pairwise CLIP.
With the emergence of diverse data collection techniques, objects in real applications can be represented as multi-modal features. What's more, objects may have multiple semantic meanings. Multi-modal and Multi-label (MMML) problem becomes a universal phenomenon. The quality of data collected from different channels ar…
ROME improves density estimation for multi-modal, non-normal data.
Anomaly detection is a fundamental problem in data mining field with many real-world applications. A vast majority of existing anomaly detection methods predominately focused on data collected from a single source. In real-world applications, instances often have multiple types of features, such as images (ID photos, f…