New method learns robust joint representations by translating between modalities.
problem Learning robust joint representations from noisy or missing modalities.
method Cyclic translations between modalities with cycle consistency loss.
result Achieves state-of-the-art results on multimodal sentiment analysis datasets.
Joint diffusion models improve data representation for both generation and prediction.
problem Inconsistent performance between generation and classification tasks in joint models.
method Extended vanilla diffusion model with a classifier for joint end-to-end training.
result Joint diffusion model outperforms state-of-the-art hybrid methods in classification and generation.
We investigate deep generative models that can exchange multiple modalities bi-directionally, e.g., generating images from corresponding texts and vice versa. A major approach to achieve this objective is to train a model that integrates all the information of different modalities into a joint representation and then t…
Adding attributes for nodes to network embedding helps to improve the ability of the learned joint representation to depict features from topology and attributes simultaneously. Recent research on the joint embedding has exhibited a promising performance on a variety of tasks by jointly embedding the two spaces. Howeve…
Unpaired multi-domain causal representation learning is possible with sufficient conditions.
problem Learning shared causal representation from unpaired data across domains.
method Identify sufficient conditions for joint distribution and shared causal graph recovery.
result Practical method to recover shared latent causal graph from marginal distributions.
Improved multimodal variational models capture more complex joint distributions.
problem Limited expressiveness of multimodal variational models.
method Used normalizing flows to approximate and transform a simple parametric joint posterior into a more complex one.
result The model improves on state-of-the-art multimodal variational methods on various tasks.
We investigate deep generative models that can exchange multiple modalities bi-directionally, e.g., generating images from corresponding texts and vice versa. Recently, some studies handle multiple modalities on deep generative models, such as variational autoencoders (VAEs). However, these models typically assume that…
Paper aims to find joint representation between vocal tract geometry and speech sound acoustics.
problem Finding a joint latent representation between articulatory and acoustic domains for vowel sounds.
method Invertible neural network models, convolutional autoencoder, normalizing flows, semi-supervised learning.
result Satisfactory performance in articulatory-to-acoustic and acoustic-to-articulatory mapping.
DeepBDC improves few-shot classification by measuring joint distributions of image features.
problem Few-shot classification with limited training data.
method DeepBDC method using deep learning and Brownian Distance Covariance.
result DeepBDC significantly outperforms existing methods on various benchmarks.
MHVAE learns cross-modality inference inspired by human cognition.
problem Cross-modality inference in multimodal data.
method Hierarchical multimodal generative model with modality-specific and joint-modality distributions.
result MHVAE performs on par with state-of-the-art models on multimodal datasets.
Unified theorem for deep and shallow joint-equivariant machines.
problem Universal approximation of joint-equivariant machines.
method Constructive universal approximation theorem based on ridgelet transform.
result Unified approximation of deep and shallow networks.
Proposes a copula-driven framework for multimodal learning.
problem Aligning and fusing representations from multiple modalities with complex interactions.
method Copula model for joint distribution of modalities, Gaussian mixture for marginal distributions.
result Superior performance on public MIMIC datasets.
Instantaneous volatility of logarithmic return in the lognormal fractional SABR model is driven by the exponentiation of a correlated fractional Brownian motion. Due to the mixed nature of driving Brownian and fractional Brownian motions, probability density for such a model is less studied in the literature. We show i…
Many natural signals exhibit a sparse representation, whenever a suitable describing model is given. Here, a linear generative model is considered, where many sparsity-based signal processing techniques rely on such a simplified model. As this model is often unknown for many classes of the signals, we need to select su…
Dyn-VGAE learns evolving network structures.
problem Learning dynamic network representations.
method Dynamic joint Variational Graph Autoencoders (Dyn-VGAE).
result Dyn-VGAE captures temporal evolution in dynamic networks.
Symile learns joint representations across multiple modalities, outperforming pairwise CLIP.
problem Pairwise contrastive learning fails to capture joint information between multiple modalities.
method Symile uses a flexible, architecture-agnostic objective to learn modality-specific representations by deriving a lower bound on total correlation.
result Symile outperforms pairwise CLIP on cross-modal classification and retrieval across various datasets.
FairNN learns fair representations and decisions by optimizing a multi-objective loss function.
problem Fairness in machine learning models for decision-making.
method Joint feature representation and classification with multi-objective loss function.
result Joint approach outperforms separate treatment of fairness in representation learning or supervised learning.
Improved Q-learning for multi-agent reinforcement learning by weighting joint action values.
problem QMIX restricts Q-values to monotonic mixtures, limiting complex value functions. method Introduced weighted projection to recover optimal policies, improving performance.
result CW QMIX and OW QMIX outperform baseline QMIX on multi-agent tasks.
New model learns from missing modalities and class labels.
problem Conflict between learning joint representations and modalities in multi-modal data.
method Introduces a novel conditional multi-modal discriminative model using an informative prior distribution and a likelihood-free objective function.
result Our model achieves state-of-the-art results in downstream classification, acoustic inversion, and image and annotation generation.
FJS method improves multinomial classification accuracy.
problem Improving multinomial classification accuracy under dataset shift.
method Derive FJS representation and propose alternative methods.
result Factorizable joint shift is not fully identifiable without additional assumptions.
Proposes joint LCA for multiview data to identify shared and view-specific components.
problem Extracting shared components sequentially from multiview data.
method Formulates a matrix decomposition model with joint and individual structures, proposes a penalty term objective function, and employs a refitting procedure.
result Achieves simultaneous estimation and rank selection for cross covariance.
A fair PCA method using JEVD ensures balanced data representation.
problem PCA's bias in data with demographic characteristics.
method Joint Eigenvalue Decomposition (JEVD) for fair PCA.
result JEVD optimally balances fairness and PCA's data structure.
MIM learns joint distributions with mutual information and low divergence.
problem Learning joint distributions over observations and latent variables.
method Probabilistic auto-encoder with three design principles: low divergence, high mutual information, and low marginal entropy.
result MIM learns representations with high mutual information, consistent encoding and decoding distributions, effective latent clustering, and comparable data log likelihood to VAE.
CLIP learns joint image-text representations for zero-shot learning.
problem Understanding and improving zero-shot transfer performance in CLIP.
method Formal study of transferrable representation learning and analysis of zero-shot transfer performance.
result Proposes a new CLIP-type approach that outperforms existing methods.
MIM learns useful representations with high mutual information.
problem Learning useful representations for downstream tasks.
method Symmetric Jensen-Shannon divergence and mutual information regularizer in an encoder/decoder framework.
result MIM learns high mutual information representations without posterior collapse.
Unsupervised methods have proven effective for discriminative tasks in a single-modality scenario. In this paper, we present a multimodal framework for learning sparse representations that can capture semantic correlation between modalities. The framework can model relationships at a higher level by forcing the shared …
COBRA reduces modality gap in cross-modal tasks.
problem Joint embedding spaces fail to sufficiently reduce modality gap in multi-modal tasks.
method COBRA trains image and text modalities in a joint fashion using Contrastive Predictive Coding and Noise Contrastive Estimation.
result COBRA significantly reduces the modality gap and generates robust joint-embedding space.
Paper proposes IABF to improve NECST robustness.
problem Limited robustness of NECST learned coding networks.
method Infomax Adversarial-Bit-Flip (IABF) to improve stability and robustness.
result IABF achieves state-of-the-art performances on compression and error correction benchmarks.
Paper presents J-RFDL for robust DL in compressed space, improving data representation robustness and accuracy.
problem Improving data representation robustness and accuracy in the presence of noise and outliers.
method Joint Robust Factorization and Projective Dictionary Learning (J-RFDL) in a factorized compressed space.
result Delivers superior performance in data representation and classification over state-of-the-art methods.
Study on pairwise counter-monotonicity, a type of negative dependence.
problem Understanding and quantifying extremal negative dependence structures.
method Established stochastic representation and invariance property; showed implications and connections.
result Pairwise counter-monotonicity implies negative association and joint mix dependence.
JECL clusters images and captions by jointly learning representations and assignments.
problem Clustering image-caption pairs with limited structured training data.
method Parallel encoders trained with clustering and alignment objectives, minimizing KL divergence and maximizing Jensen-Shannon divergence, with regularizers.
result JECL outperforms single-view and multi-view methods on large image-caption datasets.
TensorFlow Probability introduces JointDistributions for probabilistic programming.
problem Specifying models in probabilistic programming languages.
method Declarative representations of directed graphical models.
result JointDistributions for TensorFlow Probability.
The paper proposes a method to create domain-invariant representations using Wasserstein distance.
problem Domain shifts in training data affect machine learning model performance across different domains.
method The method combines classification/regression losses with a GAN-type discriminator to minimize the Wasserstein distance between domains.
result The approach produces the highest minimum classification accuracy and most invariant representation across domains.
ACVAEs improve on CVAEs by learning more flexible latent correlations.
problem Learning latent representations with correlated structure.
method Adaptive prior distribution and belief propagation.
result ACVAEs outperform CVAEs in link prediction and hierarchical clustering.
A new model generates samples with a succinct common representation using Wyner's common information.
problem Generating samples with a succinct common representation.
method Proposes a variational Wyner model trained to minimize symmetric Kullback-Leibler divergence with regularization terms.
result Demonstrates utility through joint and conditional generation experiments.
Scaling algorithms improve joint training of deep energy-based models.
problem Joint training of deep energy-based models often fails and delivers worse results.
method Proposed online and offline scaling algorithms to fix joint training.
result Scaling algorithms improve joint training and deliver better results.
PCI combines perception and control using Bayesian inference with object-based representations.
problem Separate perception and control in reinforcement learning.
method Joint Perception and Control as Inference (PCI) framework with Object-based Perception Control (OPC).
result OPC achieves good perceptual grouping quality and outperforms baselines in accumulated rewards.
New approach tackles domain adaptation without assuming domain invariant representations.
problem Learning models on source domains for target domains with labeled and unlabeled data.
method Hidden Covariate Shift hypothesis; learning representation to match joint distributions.
result State-of-the-art performance on Amazon Reviews dataset.
VJE learns latent representations without contrastive learning, providing probabilistic semantics.
problem Learning latent representations without contrastive signals.
method VJE maximizes a symmetric conditional evidence lower bound (ELBO) on paired encoder embeddings, using a Student-t distribution on a polar representation.
result VJE outperforms standard non-contrastive baselines in ImageNet-1K, CIFAR-10/100, and STL-10.
Novel bounds for deep MDA algorithms improve performance and efficiency.
problem Improving performance of MDA algorithms with few target labels and pseudo labels.
method Information-theoretic tools and novel deep MDA algorithm.
result Algorithm-dependent generalization bounds for MDA.
We formalize the problem of learning interdomain correspondences in the absence of paired data as Bayesian inference in a latent variable model (LVM), where one seeks the underlying hidden representations of entities from one domain as entities from the other domain. First, we introduce implicit latent variable models,…
Learning multimodal representations is a fundamentally complex research problem due to the presence of multiple heterogeneous sources of information. Although the presence of multiple modalities provides additional valuable information, there are two key challenges to address when learning from multimodal data: 1) mode…
Proposes a new model to better handle correlation risk in credit risk calculations.
problem Empirical evidence shows correlation risk is significant in credit risk models.
method Introduces a stochastic correlation extension of the Vasicek model using circular diffusion.
result Demonstrates how correlation volatility and persistence affect joint default and survival probabilities.
Derives integral representations for a Lévy process and its extremum, hitting time, with fast evaluation.
problem Efficiently evaluating the joint probability density function of a Lévy process, its supremum, and hitting time.
method Integral representations, Laplace-Fourier transforms, summation by parts, conformal deformation, trapezoid rules, Gaver-Wynn-Rho algorithm.
result Explicit calculations and fast evaluation of the joint cpdf for Lévy processes.
Novel approach for estimating joint probability densities using tensor decompositions and dictionaries.
problem Estimating joint probability densities of mixed discrete and continuous variables.
method Low-rank tensor decomposition combined with dictionary learning.
result Better classification and lower error rates compared to existing methods.
The paper analyzes multivariate Hawkes processes and their induced population processes.
problem Analyzing the time-dependent joint probability distribution of multivariate Hawkes processes.
method Exact and asymptotic analysis of general multivariate Hawkes processes and their induced population processes.
result Full characterization of the time-dependent joint transform of the multivariate population process and its intensity process.
Paper shows deep nets can't guarantee domain adaptation without invariant features.
problem Learning domain-invariant features for successful domain adaptation.
method Proposes a counterexample and a new generalization upper bound.
result Shows conditional shift and characterizes a fundamental tradeoff.
Proposes methods to identify and estimate counterfactual distributions with confounding.
problem Estimating counterfactual distributions in the presence of confounding.
method Nonparametric identification and semiparametric estimation using conditional copulas and machine learning.
result Valid inference for individual-level effects and nonparametric identifiability of latent confounding subspace.