New method tightens variational representations of divergences for faster learning.
problem Improving tightness of variational representations of divergences for faster statistical estimation.
method Improved objective functionals constructed via an auxiliary optimization problem, leveraging neural network approximation.
result Tighter variational representations can result in significantly faster learning and more accurate estimation of divergences.
Minimal variations guide unsupervised learning for better downstream tasks.
problem Efficiently describing raw data for various future tasks.
method Minimal variations as a guiding principle for unsupervised representation learning.
result Unveiling minimal variations as a principle behind unsupervised learning.
New method learns disentangled discrete representations using categorical variational autoencoders.
problem Learning disentangled representations from discrete latent spaces.
method Replaced standard Gaussian VAE with a categorical VAE to mitigate rotational invariance.
result Categorical distributions improve learning of disentangled representations.
Improves NF for complex data distributions with multiple modes.
problem Difficulty in handling data distributions with multiple isolated modes.
method Proposes a new framework using variational latent representation to improve NF.
result Significantly more powerful for generating data distributions with multiple modes.
The recently developed variational autoencoders (VAEs) have proved to be an effective confluence of the rich representational power of neural networks with Bayesian methods. However, most work on VAEs use a rather simple prior over the latent variables such as standard normal distribution, thereby restricting its appli…
Representation learning is a central challenge across a range of machine learning areas. In reinforcement learning, effective and functional representations have the potential to tremendously accelerate learning progress and solve more challenging problems. Most prior work on representation learning has focused on gene…
We extend Stochastic Gradient Variational Bayes to perform posterior inference for the weights of Stick-Breaking processes. This development allows us to define a Stick-Breaking Variational Autoencoder (SB-VAE), a Bayesian nonparametric version of the variational autoencoder that has a latent representation with stocha…
New definition of disentanglement for non-independent factors of variation.
problem Current disentanglement definitions assume independent factors of variation, limiting their applicability.
method Definition based on information theory, related to Information Bottleneck Method, proposed measurement method.
result Proposed method correctly measures disentanglement with non-independent factors of variation.
We investigate the problem of learning representations that are invariant to certain nuisance or sensitive factors of variation in the data while retaining as much of the remaining information as possible. Our model is based on a variational autoencoding architecture with priors that encourage independence between sens…
New variational approach for privacy and fairness in data representations.
problem Learning private and fair representations while preserving relevant information.
method Variational formulation of privacy and fairness optimization problems using Lagrangians.
result Control over the trade-off between utility and privacy/fairness through a Lagrange multiplier parameter.
PRI-VAE learns disentangled representations by optimizing principle-of-relevant-information.
problem Learning disentangled representations under VAE framework remains unknown.
method Proposes PRI-VAE, a novel learning objective to optimize disentanglement.
result Demonstrates effectiveness of PRI-VAE on four benchmark datasets.
VJE learns latent representations without contrastive learning, providing probabilistic semantics.
problem Learning latent representations without contrastive signals.
method VJE maximizes a symmetric conditional evidence lower bound (ELBO) on paired encoder embeddings, using a Student-t distribution on a polar representation.
result VJE outperforms standard non-contrastive baselines in ImageNet-1K, CIFAR-10/100, and STL-10.
Bayesian algorithm improves word representations using semantic taxonomy.
problem Improving word representations in semantic taxonomy.
method Bayesian Hierarchical Words Representation (BHWR) learning algorithm combining Variational Bayes and semantic taxonomy modeling.
result BHWR produces better representations for rare words.
DGA and DVGA learn disentangled graph representations to improve graph analysis.
problem Holistic graph auto-encoders fail to capture latent factors effectively.
method Design disentangled graph convolutional network and component-wise flow, impose independence constraints.
result Improved disentangled graph representations enhance graph analysis tasks.
We develop a framework for incorporating structured graphical models in the \emph{encoders} of variational autoencoders (VAEs) that allows us to induce interpretable representations through approximate variational inference. This allows us to both perform reasoning (e.g. classification) under the structural constraints…
VDA improves disentanglement of latent representations in complex signals.
problem Learning disentangled and interpretable representations in nonstationary, high-dimensional time-evolving signals.
method Variational decomposition autoencoding (VDA) framework, incorporating signal decomposition, contrastive self-supervised task, and variational prior approximation.
result DecVAEs surpass state-of-the-art VAE-based methods in disentanglement quality and generalization.
DISCoVeR learns disentangled representations by separating shared and condition-specific factors.
problem Learning disentangled representations for multi-condition data.
method Dual-latent architecture, parallel reconstructions, max-min objective.
result DISCoVeR achieves improved disentanglement on various datasets.
We would like to learn a representation of the data which decomposes an observation into factors of variation which we can independently control. Specifically, we want to use minimal supervision to learn a latent representation that reflects the semantics behind a specific grouping of the data, where within a group the…
Study variation spaces for neural networks, linking them to approximation theory.
problem Understanding the variation spaces of shallow neural networks.
method Examined variation spaces defined by convex hulls and integral representations for a dictionary of functions.
result Found that Barron space, spectral Barron space, and Radon BV space are variation spaces for certain neural networks.
Variational Auto-Encoders (VAEs) have been widely applied for learning compact, low-dimensional latent representations of high-dimensional data. When the correlation structure among data points is available, previous work proposed Correlated Variational Auto-Encoders (CVAEs), which employ a structured mixture model as …
Optimal hedging framework with variational preferences under convex risk measures.
problem Optimal hedging with variational preferences under convex risk measures.
method Theoretical hedging optimization framework with dual representation of risk measures and utilities.
result Derivation of optimality and indifference pricing conditions.
Develops VAEs for learning complex physical systems from data.
problem Learning low-dimensional representations of nonlinear physical systems.
method Variational Autoencoders with manifold latent spaces.
result Effective in learning nonlinear Burgers equation and constrained mechanical systems.
We show that the variational representations for f-divergences currently used in the literature can be tightened. This has implications to a number of methods recently proposed based on this representation. As an example application we use our tighter representation to derive a general f-divergence estimator based on t…
New method learns robust representations by modeling environment variation.
problem Learning invariant representations across varying environments.
method Explicitly modeling variation across environments and marginalizing it out.
result Proposed method outperforms invariant-learning methods in various settings.
Improved multimodal variational models capture more complex joint distributions.
problem Limited expressiveness of multimodal variational models.
method Used normalizing flows to approximate and transform a simple parametric joint posterior into a more complex one.
result The model improves on state-of-the-art multimodal variational methods on various tasks.
EVGAE improves VGAE's latent representation learning by mitigating over-pruning.
problem Over-pruning in VGAE limits latent variable capacity and diversity.
method EVGAE uses epitomic approach with multiple sparse VGAE models (epitomes) to increase active latent units and improve generative ability.
result EVGAE outperforms VGAE in generative ability and link prediction on citation networks.
The paper proposes a method to learn structured representations from unlabeled data using mutual information maximization.
problem Learning structured representations from unlabeled data.
method Adversarial maximization of mutual information between a structured latent variable and a target variable.
result The proposed method outperforms current baselines in document hashing and yields highly compressed interpretable representations.
AutoBayes automates Bayesian graph exploration for robust machine learning.
problem Learning representations invariant to nuisance variations in machine learning.
method Automated Bayesian inference framework exploring different graphical models.
result Significant performance improvement with nuisance-invariant machine learning pipelines.
Adversarial CCA improves representation learning by allowing more flexible priors.
problem Improving representation learning through more flexible priors.
method Adversarial techniques applied to Deep Variational CCA (VCCA and VCCA-Private).
result Adversarial CCA offers stronger and more flexible priors for representation learning.
SAMI learns disentangled representations from data.
problem Learning disentangled representations from data.
method Combines diffusion models and VAEs to learn disentangled representations.
result SAMI learns disentangled representations that are interpretable and useful.
The paper studies the dimension of limit sets using variational principles and stationary measures.
problem Calculating the Hausdorff dimension of limit sets of Anosov representations and the Rauzy gasket.
method Established variational principles for affinity exponents and Rauzy gaskets, combined with dimension formulas of stationary measures.
result Yields the equality between the Hausdorff dimensions and affinity exponents in both settings.
Paper summarizes unsupervised learning challenges for disentangled representations.
problem Unsupervised learning of disentangled representations without inductive biases.
method Theoretical and practical analysis of existing approaches.
result Unsupervised disentanglement is fundamentally impossible without inductive biases.
Variational Auto-Encoders (VAEs) are capable of learning latent representations for high dimensional data. However, due to the i.i.d. assumption, VAEs only optimize the singleton variational distributions and fail to account for the correlations between data points, which might be crucial for learning latent representa…
VICE embeds concepts in a vector space using human data.
problem Developing numerical models for mental representations of object concepts.
method Variational Interpretable Concept Embeddings (VICE) using variational inference and triplet odd-one-out task data.
result VICE outperforms SPoSE at predicting human behavior and provides more reproducible object representations.
VIR model improves regression accuracy and uncertainty estimation for imbalanced data.
problem Imbalanced regression datasets lead to poor model accuracy and uncertainty estimation.
method VIR model uses probabilistic smoothing and reweighting to estimate latent representations and uncertainty.
result VIR model outperforms state-of-the-art models in accuracy and uncertainty estimation.
Mean representations of VAEs are correlated but still useful for tasks.
problem Correlation between mean and sampled representations of VAEs.
method Selective posterior collapse to identify active and passive variables.
result Passive variables in mean representations are correlated but uncorrelated in sampled ones.
We introduce a new approximation of f-divergences for machine learning.
problem Variational representations of f-divergences for machine learning. method Definition and analysis of Moreau-Yosida approximation of f-divergences with the Wasserstein-1 metric. result Generalization and relaxation of hard Lipschitz constraints in f-divergences. New Variational InfoMax objective improves neural network performance.
problem Optimizing neural networks using Bayesian Inference and Information Bottleneck.
method Derive Variational InfoMax (VIM) objective that maximizes InfoMax directly.
result VIM improves model performance in accuracy, robustness, and representation quality.
The increasing availability of electrocardiogram (ECG) data has motivated the use of data-driven models for automating various clinical tasks based on ECG data. The development of subject-specific models are limited by the cost and difficulty of obtaining sufficient training data for each individual. The alternative of…
Let M be a compact oriented three-manifold whose interior is hyperbolic of finite volume. We prove a variation formula for the volume on the variety of representations of M in SLn(C). Our proof follows the strategy of Reznikov's rigidity when M is closed, in particular we use Fuks' appro…
We examine the influence of input data representations on learning complexity. For learning, we posit that each model implicitly uses a candidate model distribution for unexplained variations in the data, its noise model. If the model distribution is not well aligned to the true distribution, then even relevant variati…
The paper develops a new approach to conditional risk measures using modular convex analysis.
problem Developing a new method for conditional risk measures.
method Random modular approach to conditional certainty equivalents and niveloids in the conditional L∞-space. result Retrieves a conditional variational formula for optimized certainty equivalents and applies it to the conditional entropic risk measure.
Multimodal fusion is considered a key step in multimodal tasks such as sentiment analysis, emotion detection, question answering, and others. Most of the recent work on multimodal fusion does not guarantee the fidelity of the multimodal representation with respect to the unimodal representations. In this paper, we prop…
Recent advances in neural variational inference have spawned a renaissance in deep latent variable models. In this paper we introduce a generic variational inference framework for generative and conditional models of text. While traditional variational methods derive an analytic approximation for the intractable distri…
Earth observation embeddings can convert discrete biome maps into continuous representations that better capture ecological variation.
problem Biome maps impose categorical boundaries that compress continuous variation in biotic communities.
method Fit a linear classifier on Earth observation embeddings to predict biome labels.
result Continuous biome representation outperforms discrete biome labels for predicting species occurrence.
We investigate deep generative models that can exchange multiple modalities bi-directionally, e.g., generating images from corresponding texts and vice versa. A major approach to achieve this objective is to train a model that integrates all the information of different modalities into a joint representation and then t…
Modality-agnostic compression improves across diverse data types.
problem Efficiently compressing data across multiple modalities.
method Functional view of data, Implicit Neural Representation (INR), modality-agnostic latent representations, variational compression.
result Improved performance compared to existing methods, especially for diverse modalities.
We combine conditional variational autoencoders (VAE) with adversarial censoring in order to learn invariant representations that are disentangled from nuisance/sensitive variations. In this method, an adversarial network attempts to recover the nuisance variable from the representation, which the VAE is trained to pre…