DisARM improves gradient estimation for binary latent variables.
problem Challenges in training models with discrete latent variables.
method Uses antithetic sampling over continuous augmentation.
result DisARM consistently outperforms ARM and baseline methods in log-likelihood and variance.
The study analyzes how data augmentation helps isolate content from style in self-supervised learning.
problem Understanding how data augmentation affects the separation of content and style in self-supervised learning.
method Formulated a latent variable model with content and style components, studied identifiability of latent representation, and introduced a dataset to test the theory.
result Sufficient conditions for identifying the invariant content partition in self-supervised learning.
Autoencoders improve anomaly detection by deriving features and augmenting data.
problem Poor performance of one-class classifiers in high-dimensionality and sparsity.
method Uses autoencoders to derive meaningful latent variables and augment data for OCC training.
result Enhances OCC algorithms' performance and outperforms other methods.
We develop a general variational inference method that preserves dependency among the latent variables. Our method uses copulas to augment the families of distributions used in mean-field and structured approximations. Copulas model the dependency that is not captured by the original variational distribution, and thus …
LADD models improve discrete diffusion for faster language generation.
problem Practical discrete diffusion models ignore cross-token dependencies, degrading performance.
method Introduces a learnable auxiliary latent channel, diffusing over the joint (token, latent) space.
result LADD models yield improvements on unconditional generation metrics.
New method disentangles style features from data augmentations.
problem Difficulty in deducing which data attributes are 'style' and should be discarded.
method Structured data augmentation with multiple style embedding spaces, maximizing joint entropy.
result Empirically demonstrates benefits on synthetic and real-world data.
In this work we address the problem of approximating high-dimensional data with a low-dimensional representation. We make the following contributions. We propose an inverse regression method which exchanges the roles of input and response, such that the low-dimensional variable becomes the regressor, and which is tract…
Efficient inference for nonparametric Hawkes processes using Pólya-Gamma augmentation.
problem Efficient inference for nonparametric Hawkes processes.
method Pólya-Gamma augmentation, EM algorithm, mean-field variational inference.
result The proposed algorithms can recover well the underlying prompting characteristics efficiently.
A&R method reduces computational cost for large categorical distributions.
problem High computational cost for large categorical distributions.
method Latent variable augmentation and stochastic variational inference.
result A&R provides a tighter bound on the marginal likelihood and better predictive performance.
LOL method simplifies forming linear combinations of latent variables.
problem Lack of general-purpose methods for manipulating latent variables.
method Latent Optimal Linear combinations (LOL) method.
result LOL simplifies creation of expressive low-dimensional representations.
GNMT uses latent variables to improve machine translation, especially with missing words.
problem Improving machine translation, especially with missing words.
method GNMT adds a latent variable to an encoder-decoder model to capture sentence semantics.
result GNMT achieves competitive BLEU scores and is superior when source sentences are missing words.
Bayesian optimization tackles mixed discrete-continuous problems with Gaussian processes.
problem Optimizing problems with both discrete and continuous variables using costly simulations.
method Relaxing discrete variables into continuous latent variables, using Bayesian optimization, and incorporating compatibility constraints with Lagrangians.
result Comparative analysis of different mixed Bayesian optimization approaches.
Study improves Gaussian Process Latent Variable Model for noisy longitudinal data.
problem Noisy and incomplete longitudinal data makes learning representations difficult.
method Augment variational approximation with systematic samples of unseen observations.
result Demonstrates improved learning of Gaussian Process Dynamical Systems in noisy data.
Unbiased gradient estimation improves VAE performance.
problem Training VAEs via maximum likelihood is difficult due to intractable integrals.
method Introduced unbiased estimators of the log-likelihood gradient using coupled Markov chains.
result Unbiased estimators lead to better predictive performance in VAEs.
Improved generalization with semantic perturbations using normalizing flows.
problem Overfitting in deep neural networks training.
method Use normalizing flows for generating semantically meaningful perturbations in latent space.
result Achieved 96.6% test accuracy on CIFAR-10 with ResNet-18, outperforming existing methods.
Max-margin learning is a powerful approach to building classifiers and structured output predictors. Recent work on max-margin supervised topic models has successfully integrated it with Bayesian topic models to discover discriminative latent semantic structures and make accurate predictions for unseen testing data. Ho…
ARM estimator improves gradient backpropagation in binary networks.
problem Improving gradient backpropagation through stochastic binary layers.
method ARM estimator using augment-REINFORCE-merge approach.
result ARM estimator achieves state-of-the-art performance in binary models.
Bayesian neural networks simplified with input augmentation.
problem Uncertainty in deep learning models.
method Layer-wise input augmentation to induce uncertainty distributions.
result State-of-the-art performance in uncertainty representation.
Bayesian inference for factorial hidden Markov models is challenging due to the exponentially sized latent variable space. Standard Monte Carlo samplers can have difficulties effectively exploring the posterior landscape and are often restricted to exploration around localised regions that depend on initialisation. We …
Unified approach to training stochastic RNNs with latent variables.
problem Training generative latent variable models with autoregressive decoders.
method Amortized variational inference with backward RNN conditioning and auxiliary reconstruction cost.
result Improved performance on speech and sequential MNIST benchmarks.
Analysis of pretrained models' effectiveness in downstream tasks.
problem Understanding why pretrained models perform well in NLP tasks.
method Analyzed head and prompt tuning approaches using latent variable models.
result Prompt tuning provides stronger guarantees than head tuning.
DPGDS models sequential count data with deep hierarchical structure and temporal dependencies.
problem Modeling sequentially observed multivariate count data with hierarchical and temporal dependencies.
method Developed deep Poisson-gamma dynamical systems with data augmentation and MCMC inference.
result Demonstrated excellent predictive performance and interpretable latent structure.
New method AnInfoNCE uncovers latent factors in contrastive learning with practical variability.
problem Theoretical assumptions of contrastive learning loss overlook practical variability in positive pairs.
method AnInfoNCE, a generalization of InfoNCE, models anisotropic variability to uncover latent factors.
result AnInfoNCE increases recovery of latent factors in CIFAR10 and ImageNet, albeit at the cost of accuracy.
SDREM models complex network data with deep learning, improving link prediction.
problem Modeling latent structures in relational data with high-order node dependence.
method Scalable deep generative relational model (SDREM) incorporating high-order neighbourhood structure and novel data augmentation.
result Improved link prediction performance on real-world datasets.
Enhances reinforcement learning with hierarchical policies using latent variables.
problem Improving performance in reinforcement learning tasks with hierarchical policies.
method Training each layer of a hierarchical neural network to solve tasks directly, with latent variables controlling lower layers.
result Improves performance on standard benchmark tasks and complex sparse-reward tasks.
New method prevents posterior collapse in generative models.
problem Posterior collapse weakens generative model capacity or requires complex objectives.
method Proposes δ-VAEs that constrain the posterior variational family to a minimum distance from the prior. result Achieves state-of-the-art log-likelihood on CIFAR-10 and ImageNet 32x32.
Enhances generative models by improving expressivity without high computational cost.
problem Improving expressivity in generative models without increasing computational complexity.
method Proposes a new family of generative flows on an augmented data space, proving they can approximate a Hamiltonian ODE as a universal transport map.
result Demonstrates state-of-the-art performance on flow-based generative modeling benchmarks.
Framework explains how dual deep networks learn features from unlabeled data.
problem Understanding self-supervised learning with dual deep networks.
method Theoretical framework and hierarchical latent tree model.
result Deep ReLU networks learn latent variables through contrastive SSL.
Develops a Bayesian non-parametric approach for signal separation with varying components.
problem Signal separation with varying components across different input locations.
method Augments Gaussian Process Latent Variable Models with weighted sums of pure component signals and incorporates priors for linear weights.
result Framework allows for non-linear variations in signals and incorporates useful priors for linear weights.
Improves low-shot learning with novel GAN for diverse example generation.
problem Overfitting and forgetting in small data settings.
method Covariance-Preserving Adversarial Augmentation Networks (CPGANs).
result Significant improvement on ImageNet benchmark.
This paper improves probabilistic latent models on hyperbolic spaces.
problem Uncertainty in predictions due to geodesics crossing low-data regions.
method Augmenting hyperbolic manifold with a pullback metric for probabilistic pullback metrics.
result Geodesics on pullback metric respect both geometry and data distribution, reducing uncertainty.
New method infers network couplings from spin trajectories in continuous time.
problem Inferring network couplings from observed spin trajectories in continuous time.
method Introducing latent variables to linearize and make likelihood quadratic, deriving EM and variational algorithms.
result Demonstrated performance on simulated data and biologically plausible network.
Generative models learn latent process to match target distributions.
problem Training flow-matching models with auxiliary stochastic dynamics.
method Introduces latent process generator matching, treating generative state as a deterministic image of a Markov process.
result Learn generator of a stochastic process with same marginal distributions.
Efficiently infers Gaussian process density models with Gibbs sampling and variational methods.
problem Density estimation for complex, nonparametric models.
method Augmented likelihood with latent variables, Gibbs sampling, and variational mean field approximations.
result Efficient inference for Gaussian process density models with up to thousands of data points.
Particle MCMC involves using a particle filter within an MCMC algorithm. For inference of a model which involves an unobserved stochastic process, the standard implementation uses the particle filter to propose new values for the stochastic process, and MCMC moves to propose new values for the parameters. We show how p…
Proposes FarmHazard model for hazard regression with correlated covariates.
problem Model selection challenges in high-dimensional data with correlated covariates.
method Factor-Augmented Regularized Model for Hazard Regression (FarmHazard) that learns latent factors and idiosyncratic components.
result Proves model selection and estimation consistency under mild conditions.
The paper proposes a proactive method to improve model reliability by removing unstable relationships in the training data.
problem Improper generalization of predictive models due to dataset shift.
method Proactively removing unstable relationships using causal mechanisms and latent counterfactual variables.
result Models that remove vulnerable variables and use estimates of latent variables transfer better, often outperforming in the target domain.
We propose a new data-augmentation strategy for fully Bayesian inference in models with binomial likelihoods. The approach appeals to a new class of Polya-Gamma distributions, which are constructed in detail. A variety of examples are presented to show the versatility of the method, including logistic regression, negat…
Generative models enhance weak supervision for better image classification.
problem Lack of labeled data in supervised learning.
method Fusion of generative adversarial networks and weak supervision.
result Model improves multiclass image classification performance.
Framework learns disentangled continuous and categorical representations.
problem Learning disentangled representations of continuous and categorical data.
method Variational autoencoder with relaxed discrete distribution, controlling latent units.
result Framework disentangles continuous and categorical factors on various datasets.
Improved cross-entropy estimator for likelihood-free inference.
problem Efficient inference for complex models with intractable likelihoods.
method Use neural networks as surrogate models and augment training data with joint likelihood ratio and score.
result New cross-entropy estimator provides improved sample efficiency.
Combines deep learning with constraints for better image generation.
problem Generating natural images with deep learning.
method Expectation-maximization framework with handcrafted constraints.
result Improved image generation using neural networks and constraints.
Seq-CVAE learns a latent space for each word position to capture sentence intention.
problem Capturing diversity in image captioning models.
method Seq-CVAE learns a sequential latent space for each word position, mimicking future sentence summaries.
result Significantly improves diversity metrics on MSCOCO dataset compared to baselines.
Black box variational inference allows researchers to easily prototype and evaluate an array of models. Recent advances allow such algorithms to scale to high dimensions. However, a central question remains: How to specify an expressive variational distribution that maintains efficient computation? To address this, we …
Efficient multi-class classification with well-calibrated uncertainty.
problem Trade-off between uncertainty calibration and speed in multi-class Gaussian process classification.
method Proposes a new likelihood function leading to a conditionally conjugate model with efficient variational inference.
result Up to two orders faster than state-of-the-art methods with well-calibrated uncertainty estimates.
New method identifies latent causal factors from observational data alone.
problem Identifying latent causal factors without interventions or graphical restrictions.
method Characterization of latent factors in nonlinear causal models with additive Gaussian noise and linear mixing, using a practical algorithm based on solving a quadratic program over observed data.
result Latent causal variables can be identified up to a layer-wise transformation, and further disentanglement is not possible.
iGCL preserves graph semantics in latent space augmentations.
problem Manual tuning of augmentation ratios and unexpected graph changes.
method iGCL uses a Variational Graph Auto-Encoder to learn augmentations in the latent space, optimizing an upper bound for contrastive loss.
result iGCL achieves state-of-the-art performance on graph-level and node-level tasks.
A new framework for nonlinear ICA using auxiliary variables and contrastive learning.
problem Recovering underlying latent variables from data.
method Augmenting data with auxiliary variables and using contrastive learning for identification.
result General theoretical framework and practical algorithm for nonlinear ICA.