New path-gradient estimator for continuous normalizing flows.
problem Limitation of simple Gaussian variational distributions in complex applications.
method Proposed a path-gradient estimator for continuous normalizing flows.
result Empirical evidence of superior performance of the new estimator.
This paper develops variational continual learning (VCL), a simple but general framework for continual learning that fuses online variational inference (VI) and recent advances in Monte Carlo VI for neural networks. The framework can successfully train both deep discriminative models and deep generative models in compl…
EVCL combines VCL and EWC to prevent forgetting new tasks.
problem Preventing catastrophic forgetting in continual learning.
method Hybrid model integrating VCL and EWC.
result Consistently outperforms baselines in learning new tasks.
Improves continual learning with Variational Continual Learning framework.
problem Avoiding catastrophic forgetting and efficient model capacity use in sequential tasks.
method Mean-field variational Bayesian neural networks approach.
result Significantly improved results on continual learning benchmarks.
CLPF models continuous time-series data with improved representational power and variational approximations.
problem Fitting continuous time-series data with existing models faces challenges in representational power and variational quality.
method CLPF uses a time-dependent normalizing flow driven by a stochastic differential equation to decode continuous latent processes into continuous observables. Maximum likelihood optimization is achieved through a novel variational posterior process.
result CLPF outperforms state-of-the-art baselines on synthetic and real-world time-series data.
Develops a new method for learning discrete distributions without embedding them in a continuous space.
problem Challenges in learning discrete distributions using current methodologies.
method Introduces a MAD invertible map and a mixed variational flow (MAD Mix) for discrete distributions.
result MAD Mix produces more reliable approximations than continuous-embedding flows.
Improved continual learning method using variational inference and FiLM layers.
problem Training models on new tasks and datasets in an online fashion.
method Generalized Variational Continual Learning (GVCL) with likelihood-tempering and FiLM layers.
result GVCL outperforms existing baselines in both small and large datasets, providing better calibration.
Proposes a new method for continual learning in neural networks.
problem Challenges in applying sequential Bayesian inference to neural networks.
method Sequential function-space variational inference.
result Neural networks trained with the proposed method achieve better predictive accuracy.
SFSVI uses Gaussian mixtures to approximate neural network outputs for continual learning.
problem Learning new tasks without forgetting old ones in neural networks.
method Sequential function-space variational inference with Gaussian mixture approximation.
result Gaussian mixture SFSVI outperforms other methods in continual learning.
TVO tightens variational inference bounds for deep models.
problem Improving variational inference bounds for deep models.
method Introduces thermodynamic variational objective (TVO) connecting variational inference and thermodynamic integration.
result TVO provides tighter lower bound to log marginal likelihood than ELBO.
Optimizes decision-making with variational Bayesian methods for continuous utilities.
problem Inference approximations for continuous utilities without full posterior knowledge.
method Automatic pipeline that co-opts continuous utilities into variational inference algorithms.
result Consistent improvement in decision-making when calibrating approximations for specific utilities.
Study examines stability of image-reconstruction algorithms using variational regularization.
problem Stability and robustness of image-reconstruction algorithms in medical imaging.
method Review and novel stability results for ℓp-regularized linear inverse problems, focusing on p∈(1,∞). result Guarantees Lipschitz continuity for small p and Hölder continuity for larger p in Lp(Ω) function spaces. VCoTTA uses variational Bayesian methods to adapt models under continuous domain shifts.
problem Error accumulation in continual test-time adaptation.
method VCoTTA employs variational Bayesian techniques to update a Bayesian Neural Network (BNN) during testing, combining priors from source and teacher models.
result VCoTTA effectively mitigates error accumulation in CTTA, as shown by experimental results on three datasets.
New algorithm for continuous-time switching systems using variational inference.
problem Inference in time-series data with continuous-time switching systems.
method Developed a variational inference algorithm combining Gaussian process approximation and posterior inference for Markov jump processes.
result Bayesian latent state estimates and point estimates of unknown parameters for arbitrary points on the real axis.
VAR-GPs solve continual learning by updating posteriors sequentially.
problem Catastrophic forgetting in sequential learning tasks.
method Sparse inducing point approximations and auto-regressive variational distribution.
result VAR-GPs prevent catastrophic forgetting and outperform baselines.
This paper tackles continuous domain generalization, improving model performance across unseen domains.
problem Existing domain generalization approaches fail to capture the complex, multidimensional nature of real-world variation.
method Introduces Continuous Domain Generalization (CDG), a principled framework grounded in geometric and algebraic theories. Proposes a Neural Lie Transport Operator (NeuralLio) for structure-preserving parameter transitions and a gating mechanism for robust generalization.
result Demonstrates significant improvement in generalization accuracy and robustness across various datasets.
We present a framework for learning disentangled and interpretable jointly continuous and discrete representations in an unsupervised manner. By augmenting the continuous latent distribution of variational autoencoders with a relaxed discrete distribution and controlling the amount of information encoded in each latent…
Earth observation embeddings can convert discrete biome maps into continuous representations that better capture ecological variation.
problem Biome maps impose categorical boundaries that compress continuous variation in biotic communities.
method Fit a linear classifier on Earth observation embeddings to predict biome labels.
result Continuous biome representation outperforms discrete biome labels for predicting species occurrence.
Unified framework for variational inference tackles federated and continual learning.
problem Fragmented literature in variational inference.
method Partitioned Variational Inference (PVI) framework.
result New methods outperform state-of-the-art in federated and continual learning.
This paper examines challenges and solutions for solving variational inequalities.
problem Stability issues in solving variational inequalities, especially in multi-objective scenarios.
method Continuous-time analysis to understand and improve stability of algorithms.
result Understanding continuous-time dynamics can help in designing more stable algorithms for variational inequalities.
Two approximate lifted variational methods for hybrid domains improve inference scalability and accuracy.
problem Efficient inference in hybrid probabilistic relational models with multi-modality and continuous evidence.
method Two approximate lifted variational approaches applicable to hybrid domains, exploiting model symmetries.
result The proposed variational methods are scalable and can leverage approximate model symmetries, outperforming existing message-passing approaches.
A method for disentangling discrete and continuous factors of data without using a discriminator network.
problem Unsupervised disentanglement of discrete and continuous factors of data.
method A procedure that minimizes total correlation of continuous latent variables and a separate discrete inference procedure.
result The method significantly outperforms current disentanglement methods based on disentanglement score and inference network classification score.
Variational Bayes simplifies Bayesian neural networks for uncertainty quantification.
problem Quantifying uncertainty in neural networks' outputs.
method Approximates intractable Bayesian integrals using variational methods.
result Comparison of various approximation methods in literature.
Max-Entropy approach improves variational inference for complex posterior distributions.
problem Efficient inference with simple families vs. accuracy in variational inference.
method Greedy approximation of the posterior distribution with Max-Entropy approach.
result Demonstrated ability to capture complex multimodal posterior distributions.
Extends GP models for sequential data, scalable and robust.
problem Handling sequential input-output observations in multi-task settings.
method Variational inference with sparse approximations and recursive GP priors.
result Tractable continual learning with KL divergences and recursive reconstruction.
VSDN models sporadic time series with neural SDEs.
problem Modeling irregular and sparse time series data.
method Variational Bayesian method and neural SDEs.
result VSDNs outperform state-of-the-art models in prediction and interpolation.
Develops a method to efficiently compute Wasserstein barycenters with variational distributions.
problem High computational burden in computing Wasserstein barycenters for high-dimensional and continuous settings.
method Introduces a variational distribution to approximate the continuous Wasserstein barycenter, reformulating the problem as an optimization with c-cyclical monotonicity.
result The method provides a tractable dual formulation for efficient computation of Wasserstein barycenters, demonstrated on real applications.
Continuous-time Bayesian networks (CTBNs) constitute a general and powerful framework for modeling continuous-time stochastic processes on networks. This makes them particularly attractive for learning the directed structures among interacting entities. However, if the available data is incomplete, one needs to simulat…
We consider a square-integrable semimartingale and investigate the convex order relations between its discrete, continuous and predictable quadratic variation. As the main results, we show that if the semimartingale has conditionally independent increments and symmetric jump measure, then its discrete realized variance…
Proves solution uniqueness for biomembrane shape prediction.
problem Proving solution uniqueness for the genus one Canham variational problem.
method Combining numeric analytic continuation and singularity analysis to prove non-negativity of a sequence.
result Proves positivity of the sequence, leading to solution uniqueness.
This work proposes a model to prevent forgetting in continual classification learning.
problem Preventing forgetting in continual classification learning.
method The approach builds on lifelong generative capabilities and derives a new variational bound.
result The model prevents catastrophic forgetting in continual classification learning.
This note continues investigation of randomness-type properties emerging in idealized financial markets with continuous price processes. It is shown, without making any probabilistic assumptions, that the strong variation exponent of non-constant price processes has to be 2, as in the case of continuous martingales.
Improved inference for models with continuous latent variables.
problem Inference accuracy with traditional variational methods is limited.
method Reparameterized Variational Rejection Sampling (RVRS) using a proposal distribution with a reparameterized gradient estimator.
result RVRS offers a better trade-off between computational cost and inference fidelity.
Study on reducing forgetting in neural networks using compression theory.
problem Catastrophic forgetting in neural networks.
method Defined forgetting as increased description lengths, compared variational posterior approaches to prequential coding methods.
result Proposed a new continual learning method combining ML plug-in and Bayesian mixture codes.
Discond-VAE separates continuous and discrete factors in data.
problem Separating shared and class-specific variations in real-world data.
method Introduces private and public latent variables to represent continuous and discrete factors, respectively.
result Discond-VAE successfully disentangles class-dependent continuous factors from discrete factors.
New method prevents forgetting in learning new tasks.
problem Poor ability of models to solve new problems without forgetting.
method Task-agnostic hierarchical information-theoretic optimality principle with Mixture-of-Variational-Experts layer.
result Demonstrated competitive performance in continual supervised and reinforcement learning.
A new method uncovers discrete and continuous factors in gene expression data.
problem Jointly identifying discrete and continuous factors of variability without supervision.
method cpl-mixVAE framework using multiple interacting networks.
result The method successfully uncovers discrete and continuous factors in gene expression data.
Proves continuity and singular set dimension for 2D maps with Q values.
problem Interior regularity of 2D Q-valued maps. method Strong concentration-compactness theorem for equicontinuous maps.
result 2D Q-valued maps are Hölder continuous with singular set dimension ≤1. We develop variational integrators from discrete Hamiltonian systems with external forces.
problem Creating accurate discrete models of continuous Hamiltonian systems.
method Constructing discrete Hamiltonian systems with external forces, analyzing symplectic structure, and combining methods to build variational integrators.
result We derive variational integrators that approximate continuous Hamiltonian systems with high accuracy.
VASE uses Bayesian neural networks to improve exploration in sparse reward environments.
problem Exploration in environments with continuous control and sparse rewards.
method VASE uses a Bayesian neural network model of the environment dynamics and variational inference to alternately update the model's accuracy and policy.
result VASE outperforms other surprise-based exploration techniques in continuous control sparse reward environments.
This paper tackles continuous domain adaptation with a new approach.
problem Learning in non-stationary environments, especially domain drift.
method Variational domain-agnostic feature replay, composed of inference, generative, and solver modules.
result Demonstrates the effectiveness of the proposed approach for practical usage.
New method handles unknown task boundaries in continual learning.
problem Catastrophic forgetting in neural networks.
method Fixed-point equations for online variational Bayes optimization.
result Approximates online Bayes update for non-stationary data.
A scalable framework for inference in continuous Cox processes using Gaussian processes.
problem Inference in inhomogeneous Poisson processes with continuous intensity functions.
method Structured variational approximation of likelihood through augmentation with superposition of Poisson processes.
result Structured variational approximation captures dependencies across variables and outperforms mean-field methods and sampling schemes.
We show how to use a variational approximation to the logistic function to perform approximate inference in Bayesian networks containing discrete nodes with continuous parents. Essentially, we convert the logistic function to a Gaussian, which facilitates exact inference, and then iteratively adjust the variational par…
Continuous semi-implicit models enable faster training and better performance in generative modeling.
problem Slow convergence in hierarchical semi-implicit models during training.
method CoSIM, a continuous semi-implicit model that incorporates a continuous transition kernel for efficient training.
result CoSIM achieves superior performance on image generation tasks compared to existing methods.
Theory broadens GFlowNets to handle continuous spaces.
problem Limitation of GFlowNets to discrete spaces.
method Developed a theory for generalized GFlowNets.
result Empirical results show strong performance in continuous cases.
The variational autoencoder (VAE) is a generative model with continuous latent variables where a pair of probabilistic encoder (bottom-up) and decoder (top-down) is jointly learned by stochastic gradient variational Bayes. We first elaborate Gaussian VAE, approximating the local covariance matrix of the decoder as an o…
Improved algorithms for convex-concave min-max optimization and monotone variational inequalities.
problem Efficiently solving constrained convex-concave min-max problems and monotone variational inequalities.
method Higher-order methods achieving iteration complexities of O(1/T^{ rac{p+1}{2}}) for p-th order derivatives.
result Achieved improved convergence rates for min-max and monotone variational inequalities.