Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

24487195 · May 202619922001200920172026
48 results for Mixed Modality

The paper improves SMC algorithm for multi-modal distributions by proving variance bounds.

problem Problems with SMC on multi-modal distributions, especially in terms of mixing time.
method Proves variance bounds for SMC on multi-modal distributions using soft decomposition.
result Bounds on SMC variance depend on local rather than global mixing times.

Integrates neural encoders into GLMMs for multimodal data analysis.

problem Scalable Bayesian inference for GLMMs assumes low-dimensional tabular predictors and does not handle high-dimensional modalities.
method Jointly learns modality-specific neural encoders with GLMM objective, performs variance-corrected stochastic-gradient MCMC.
result Preserves interpretable fixed and random effects while scaling to large longitudinal datasets.

Graphical causal models are an important tool for knowledge discovery because they can represent both the causal relations between variables and the multivariate probability distributions over the data. Once learned, causal graphs can be used for classification, feature selection and hypothesis generation, while reveal…

2017-04-09abs ↗pdf ↗

Sampling from posterior distributions using Markov chain Monte Carlo (MCMC) methods can require an exhaustive number of iterations, particularly when the posterior is multi-modal as the MCMC sampler can become trapped in a local mode for a large number of iterations. In this paper, we introduce the pseudo-extended MCMC…

2017-08-17abs ↗pdf ↗

We study the problem of sampling from the power posterior distribution in Bayesian Gaussian mixture models, a robust version of the classical posterior. This power posterior is known to be non-log-concave and multi-modal, which leads to exponential mixing times for some standard MCMC algorithms. We introduce and study …

2019-12-11abs ↗pdf ↗

Paper proposes a method to improve MCMC sampling for energy-based models.

problem MCMC sampling of energy-based models is often not mixing in high-dimensional data.
method Proposes using a flow-based model as a backbone to correct the energy-based model, enabling mixing in latent space.
result MCMC sampling of the corrected EBM in the latent space mixes well and traverses modes in the data space.

The paper introduces new estimators for multivariate functions using Fourier methods.

problem Estimating multivariate functions like densities and regression functions.
method Monte Carlo estimators based on the Fourier integral theorem.
result Established rates of convergence for new estimators, often superior to existing methods.

UncertaintyPlayground simplifies uncertainty estimation in Python.

problem Uncertainty estimation in supervised learning tasks.
method Sparse and Variational Gaussian Process Regressions for normally distributed outcomes, Mixed Density Networks for mixed distributions.
result Fast and simplified uncertainty estimation through Python library.

Adaptive anchor methods improve multi-modal learning by balancing intra-modal and inter-modal information.

problem Fixed anchor methods limit multi-modal learning by over-reliance on a single modality and inadequate cross-modal correlation.
method Adaptive anchor methods using centroid-based anchors from all modalities.
result Adaptive anchor methods like CentroBind consistently outperform fixed anchor methods across various datasets.

KD-Net transfers knowledge from multi-modal to mono-modal segmentation networks.

problem Limited acquisition of multiple imaging modalities in clinical settings.
method Generalized distillation framework adapted for mono-modal networks.
result The student network outperforms baseline mono-modal networks in brain tumor segmentation.

MHVAE learns cross-modality inference inspired by human cognition.

problem Cross-modality inference in multimodal data.
method Hierarchical multimodal generative model with modality-specific and joint-modality distributions.
result MHVAE performs on par with state-of-the-art models on multimodal datasets.

The paper analyzes and proposes an algorithm for multi-modal nonlinear embeddings with theoretical performance bounds.

problem Generalizability of multi-modal nonlinear embeddings to unseen data.
method Theoretical analysis and a multi-modal nonlinear representation learning algorithm motivated by performance bounds.
result The proposed algorithm yields promising performance in multi-modal image classification and cross-modal image-text retrieval applications.

New adaptive temperature selection improves parallel tempering efficiency.

problem Enhancing mixing in multi-modal distributions using parallel tempering.
method Adaptive temperature selection using policy gradient approach.
result Lower integrated autocorrelation times achieved compared to traditional methods.

COBRA reduces modality gap in cross-modal tasks.

problem Joint embedding spaces fail to sufficiently reduce modality gap in multi-modal tasks.
method COBRA trains image and text modalities in a joint fashion using Contrastive Predictive Coding and Noise Contrastive Estimation.
result COBRA significantly reduces the modality gap and generates robust joint-embedding space.

Modern datasets are becoming heterogeneous. To this end, we present in this paper Mixed-Variate Restricted Boltzmann Machines for simultaneously modelling variables of multiple types and modalities, including binary and continuous responses, categorical options, multicategorical choices, ordinal assessment and category…

2014-08-06abs ↗pdf ↗

Proposes a novel network for CTR prediction by learning modality-specific and modality-invariant representations.

problem Learning good representation of items from multimodal features in E-commerce is challenging due to redundant information across modalities.
method Introduces a Multimodal Adversarial Representation Network (MARN) that calculates modality-specific weights and learns modality-invariant representations.
result Consistently achieves remarkable improvements over state-of-the-art methods in CTR prediction.

Improves likelihood-free inference by using a new sampling approach to avoid biased data collection.

problem Efficient Bayesian inference without likelihood evaluation for real-world datasets.
method Introduces Neural Proposal (NP) to sample simulation inputs i.i.d. for unbiased posterior inference.
result Demonstrates improved performance, especially for multi-modal posteriors, through experiments.

Study analyzes bias interactions in multimodal models using simulation-based methods.

problem Analyzing dynamic bias interactions in multimodal models to ensure fairness and equity.
method Simulation-based heuristic approach to compute bias scores for text-only, image-only, and multimodal embeddings.
result Multimodal bias interactions can be amplification, mitigation, or neutral, with text bias often dominant.

Combining complementary information from multiple modalities is intuitively appealing for improving the performance of learning-based approaches. However, it is challenging to fully leverage different modalities due to practical challenges such as varying levels of noise and conflicts between modalities. Existing metho…

2018-05-29abs ↗pdf ↗

We investigate deep generative models that can exchange multiple modalities bi-directionally, e.g., generating images from corresponding texts and vice versa. Recently, some studies handle multiple modalities on deep generative models, such as variational autoencoders (VAEs). However, these models typically assume that…

2016-11-07abs ↗pdf ↗

MoCA uses a novel autoencoder to analyze multi-modal health data.

problem Challenges in analyzing continuous multi-modal health data from wearable devices.
method Proposes MoCA, a self-supervised learning framework combining transformer and masked autoencoder methods.
result Demonstrates strong performance boosts across reconstruction and classification tasks.

Obtaining common representations from different modalities is important in that they are interchangeable with each other in a classification problem. For example, we can train a classifier on image features in the common representations and apply it to the testing of the text features in the representations. Existing m…

2016-12-23abs ↗pdf ↗

Framework for handling long-tailed multi-modal data.

problem Class imbalance and long-tailed distributions in multi-modal data.
method Multi-expert architecture with modality-specific networks and dynamic fusion weights.
result Framework outperforms existing methods in long-tailed, class-imbalanced scenarios.

TAP transfers knowledge from unlabeled data to improve cross-modal learning.

problem Improving supervised learning performance using unlabeled data from a different modality.
method Probabilistic approach for missing information estimation, kernel regression, cross-attention module, TAP neural network.
result TAP significantly improves generalization across different domains and neural network architectures.

We propose a novel method, Modality-based Redundancy Reduction Fusion (MRRF), for understanding and modulating the relative contribution of each modality in multimodal inference tasks. This is achieved by obtaining an (M+1)(M+1)-way tensor to consider the high-order relationships between MM modalities and the output laye…

2018-11-30abs ↗pdf ↗

DNN-based cross-modal retrieval has become a research hotspot, by which users can search results across various modalities like image and text. However, existing methods mainly focus on the pairwise correlation and reconstruction error of labeled data. They ignore the semantically similar and dissimilar constraints bet…

2017-03-21abs ↗pdf ↗

This work improves multi-modal generative models by using permutation-invariant neural networks.

problem Improving multi-modal generative models with tighter variational objectives.
method Developed more flexible aggregation schemes based on permutation-invariant neural networks.
result Our variational objective and flexible aggregation models can better approximate the true joint distribution.

FBMS R package simplifies Bayesian model selection and averaging.

problem Complex regression settings with multi-modal posterior landscapes.
method Efficient MJMCMC and GMJMCMC algorithms for Bayesian model exploration.
result FBMS effectively handles Bayesian generalized linear and nonlinear models.

A framework for uncertainty-aware multimodal learning using conformal Shapley intervals.

problem Uncertainty and modality level importance in multimodal learning.
method Introduces conformal Shapley intervals to quantify modality level importance and uncertainty.
result Demonstrates meaningful uncertainty quantification and strong predictive performance.

MAESTRO improves multimodal learning for dynamic time series with adaptive attention and robustness.

problem Challenges in multimodal learning, especially in healthcare and daily living.
method Dynamic intra- and cross-modal interactions, symbolic tokenization, adaptive attention budgeting, sparse cross-modal attention, MoE mechanism.
result Average relative improvements of 4% and 8% over existing multimodal and multivariate approaches, respectively, under complete observations.

Develops multi-modal neural network models for improved prediction and uncertainty quantification.

problem Improving prediction accuracy and uncertainty quantification for multi-modal data.
method Multi-modal Bayesian neural network models with conjugate last-layer estimation using SVI.
result Improved prediction accuracy and uncertainty quantification compared to uni-modal models.

This paper studies the nonparametric modal regression problem systematically from a statistical learning view. Originally motivated by pursuing a theoretical understanding of the maximum correntropy criterion based regression (MCCR), our study reveals that MCCR with a tending-to-zero scale parameter is essentially moda…

2017-02-20abs ↗pdf ↗