This paper introduces Wasserstein variational inference, a new form of approximate Bayesian inference based on optimal transport theory. Wasserstein variational inference uses a new family of divergences that includes both f-divergences and the Wasserstein distance as special cases. The gradients of the Wasserstein var…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “variational techniques”
Minimal variations guide unsupervised learning for better downstream tasks.
problem Efficiently describing raw data for various future tasks.
method Minimal variations as a guiding principle for unsupervised representation learning.
result Unveiling minimal variations as a principle behind unsupervised learning.
Improves gradient estimation for discrete distributions with variance reduction techniques.
problem Excessive variance in gradient estimation for discrete distributions.
method Stein operators for discrete distributions and control variates.
result Substantially lower variance in gradient estimation.
Paper analyzes dynamics of nonholonomic systems with collisions using variational techniques.
problem Analyzing the dynamics of nonholonomic mechanical systems with impacts.
method Variational techniques extended to nonsmooth context for collisions.
result Variational formulation for implicit nonholonomic mechanical systems with energy-momentum preserving collisions.
We use neural networks as control variates with geometric integration techniques.
problem Analytic integration of neural network approximations for variance reduction.
method Integration domain subdivision using computational geometry for MLPs with continuous piecewise linear activation functions.
result Neural networks can be used as control variates with geometric integration methods.
In recent years, data have become increasingly higher dimensional and, therefore, an increased need has arisen for dimension reduction techniques for clustering. Although such techniques are firmly established in the literature for multivariate data, there is a relative paucity in the area of matrix variate, or three-w…
Quantum variational circuits improve reinforcement learning efficiency.
problem Improving reinforcement learning algorithms using quantum computing.
method Investigation of quantum variational circuits for DQN and Double DQN, encoding classical data for quantum circuits.
result Quantum variational circuits can solve reinforcement learning tasks with a smaller parameter space.
New method accelerates energetic variational inference using particle dynamics.
problem Efficiently solving variational inference problems with reduced computational cost.
method Particle-based variational inference with implicit scheme, inspired by energy quadratization and operator splitting.
result Significantly reduces computational cost compared to existing methods.
Stochastic optimization techniques are standard in variational inference algorithms. These methods estimate gradients by approximating expectations with independent Monte Carlo samples. In this paper, we explore a technique that uses correlated, but more representative , samples to reduce estimator variance. Specifical…
We give a way of constructing real variations of mixed Hodge structures over compact Kähler manifolds by using mixed Hodge structures on Sullivan's -minimal models of certain differential graded algebras associated with real variations of Hodge structures.
New proof of Gaffney's inequality for differential forms on manifolds with boundary.
problem Proving Gaffney's inequality for differential forms on manifolds with boundary.
method Variational approach combined with Bochner's technique.
result New proof of Gaffney's inequality for differential forms.
Variational Prediction simplifies Bayesian inference without test time costs.
problem Bayesian inference's computational costs and posterior predictive distribution marginalization.
method Variational Prediction learns a variational approximation to the posterior predictive distribution using a variational bound.
result Directly learns a variational approximation to the posterior predictive distribution without test time marginalization costs.
New algorithms accelerate SVGD convergence using deep unfolding.
problem Improving the speed of SVGD convergence.
method Integrating deep unfolding into SVGD for parameter learning.
result Proposed algorithms achieve faster convergence in various tasks.
Inspired by the seminal work on Stein Variational Inference and Stein Variational Policy Gradient, we derived a method to generate samples from the posterior variational parameter distribution by \textit{explicitly} minimizing the KL divergence to match the target distribution in an amortize fashion. Consequently, we a…
This paper presents a multi-dimensional computational method to predict the spatial variation data inside and across multiple dies of a wafer. This technique is based on tensor computation. A tensor is a high-dimensional generalization of a matrix or a vector. By exploiting the hidden low-rank property of a high-dimens…
Hidden regular variation is a sub-model of multivariate regular variation and facilitates accurate estimation of joint tail probabilities. We generalize the model of hidden regular variation to what we call hidden domain of attraction. We exhibit examples that illustrate the need for a more general model and discuss de…
Our research extends the Bilingual Evaluation Understudy (BLEU) evaluation technique for statistical machine translation to make it more adjustable and robust. We intend to adapt it to resemble human evaluation more. We perform experiments to evaluate the performance of our technique against the primary existing evalua…
New method reduces training cost by using approximate gradients.
problem Training neural networks is computationally expensive.
method Uses control variates to approximate gradients without full backward pass.
result Efficacy demonstrated on a vision transformer classification task.
We study linear models under heavy-tailed priors from a probabilistic viewpoint. Instead of computing a single sparse most probable (MAP) solution as in standard deterministic approaches, the focus in the Bayesian compressed sensing framework shifts towards capturing the full posterior distribution on the latent variab…
Recent work used importance sampling ideas for better variational bounds on likelihoods. We clarify the applicability of these ideas to pure probabilistic inference, by showing the resulting Importance Weighted Variational Inference (IWVI) technique is an instance of augmented variational inference, thus identifying th…
This paper combines three techniques to reduce communications in distributed variational inequalities.
problem Efficiently communicating solutions in large-scale distributed variational inequalities.
method Combining similarity, compression, and local steps to reduce communication rounds and cost.
result Best theoretical guarantees of communication complexity and superior performance in adversarial learning experiments.
Monte Carlo (MC) sampling algorithms are an extremely widely-used technique to estimate expectations of functions f(x), especially in high dimensions. Control variates are a very powerful technique to reduce the error of such estimates, but in their conventional form rely on having an accurate approximation of f, a pri…
We propose a flexible framework that deals with both singer conversion and singers vocal technique conversion. The proposed model is trained on non-parallel corpora, accommodates many-to-many conversion, and leverages recent advances of variational autoencoders. It employs separate encoders to learn disentangled latent…
VSI model predicts survival distributions efficiently.
problem Predicting time-to-event distributions in clinical applications.
method Variational framework for non-parametric distribution estimation.
result Improved performance over competing solutions.
Batch normalisation doesn't affect variational inference but fails for larger batch sizes.
problem Failure of Monte Carlo Batch Normalisation (MCBN) for capturing epistemic uncertainty in larger batch sizes.
method Investigated MCBN as an approximate inference technique for Bayesian neural networks, showing its limitations and providing insights for improvement.
result For larger batch sizes, MCBN fails to capture epistemic uncertainty, requiring the batch size to be a variational parameter.
Variational inference is a scalable technique for approximate Bayesian inference. Deriving variational inference algorithms requires tedious model-specific calculations; this makes it difficult to automate. We propose an automatic variational inference algorithm, automatic differentiation variational inference (ADVI). …
Unified view of GP approximations improves efficiency.
problem Disparate variational features limit GP efficiency.
method View GP as a Banach space to unify feature selection.
result Unified understanding of existing and new features.
Adversarial CCA improves representation learning by allowing more flexible priors.
problem Improving representation learning through more flexible priors.
method Adversarial techniques applied to Deep Variational CCA (VCCA and VCCA-Private).
result Adversarial CCA offers stronger and more flexible priors for representation learning.
Screening is an effective technique for speeding up the training process of a sparse learning model by removing the features that are guaranteed to be inactive the process. In this paper, we present a efficient screening technique for sparse support vector machine based on variational inequality. The technique is both …
We address the problem of graph classification based only on structural information. Inspired by natural language processing techniques (NLP), our model sequentially embeds information to estimate class membership probabilities. Besides, we experiment with NLP-like variational regularization techniques, making the mode…
Unnormalised latent variable models are a broad and flexible class of statistical models. However, learning their parameters from data is intractable, and few estimation techniques are currently available for such models. To increase the number of techniques in our arsenal, we propose variational noise-contrastive esti…
In this short paper, we re-derive the Bochner formula for the Laplacian by considering local variations of volume. The derivation is rooted in the fact that the Laplacian of a function measures the volume variation along the flow of the gradient vector of the function. Possible extensions of this approach/technique are…
We discuss intrinsic aspects of Krupka's approach to finite-order variational sequences. We give intrinsic isomorphisms of the quotient subsheaves of the short finite-order variational sequence with sheaves of forms on jet spaces of suitable order, obtaining a new finite-order (short exact) variational sequence which i…
Clarifies EM algorithm and variational Bayesian inference concepts.
problem Gaps in AI literature understanding of EM and variational concepts.
method Tutorial presentation of EM algorithm, variational Bayesian inference, and autoencoded variational Bayes.
result Establishes clear links between EM and variational methods.
The holy grail in deep neural network research is porting the memory- and computation-intensive network models on embedded platforms with a minimal compromise in model accuracy. To this end, we propose a novel approach, termed as Variational Student, where we reap the benefits of compressibility of the knowledge distil…
The recognition network in deep latent variable models such as variational autoencoders (VAEs) relies on amortized inference for efficient posterior approximation that can scale up to large datasets. However, this technique has also been demonstrated to select suboptimal variational parameters, often resulting in consi…
K-Means is one of the most used algorithms for data clustering and the usual clustering method for benchmarking. Despite its wide application it is well-known that it suffers from a series of disadvantages; it is only able to find local minima and the positions of the initial clustering centres (centroids) can greatly …
Many real-valued stochastic time-series are locally linear (Gassian), but globally non-linear. For example, the trajectory of a human hand gesture can be viewed as a linear dynamic system driven by a nonlinear dynamic system that represents muscle actions. We present a mixed-state dynamic graphical model in which a hid…
We develop stochastic variational inference, a scalable algorithm for approximating posterior distributions. We develop this technique for a large class of probabilistic models and we demonstrate it with two probabilistic topic models, latent Dirichlet allocation and the hierarchical Dirichlet process topic model. Usin…
This work proposes using zero-variance control variates to reduce variance in pathwise gradient estimators for variational inference.
problem Pathwise gradient estimators in variational inference have high variance, leading to inefficient optimization.
method Apply zero-variance control variates to pathwise gradient estimators.
result Zero-variance control variates can significantly reduce the variance of pathwise gradient estimators without requiring complex assumptions.
CDVI improves variational inference for survival analysis by considering censoring mechanisms.
problem Challenges in applying variational methods to survival data, especially the dependence on censoring.
method Censor-dependent variational inference (CDVI) tailored for latent variable models in survival analysis.
result Significant improvements in estimating individual survival distributions.
Deep neural networks with their large number of parameters are highly flexible learning systems. The high flexibility in such networks brings with some serious problems such as overfitting, and regularization is used to address this problem. A currently popular and effective regularization technique for controlling the…
Quantum annealing improves VB inference, avoiding local minima.
problem Variational Bayes inference stuck in local minima.
method Quantum annealing approach to VB inference.
result Quantum annealing variational Bayes (QAVB) outperforms classical VB.
Study on 4D Riemannian manifolds solves curvature problem.
problem Resonant prescribed T-curvature problem on compact manifolds.
method Variational theory, energy and gradient estimates, Morse lemma, Liouville technique.
result New existence results for critical points at infinity.
Learning a generative model from partial data (data with missingness) is a challenging area of machine learning research. We study a specific implementation of the Auto-Encoding Variational Bayes (AEVB) algorithm, named in this paper as a Variational Auto-Decoder (VAD). VAD is a generic framework which uses Variational…
Solves wealth maximization problem using variational analysis.
problem Maximizing expected utility of terminal wealth.
method Variational analysis, forward-backward stochastic differential equation (FBSDE).
result Characterization and solutions for various utility functions.
Optimal hedging strategies for exotic options using vanilla options.
problem Hedging exotic options with illiquid vanilla options.
method Simple approximations and variational techniques in a market model and stochastic volatility model framework.
result Optimal Delta and Vega hedging strategies can be computed easily.
Reinforcement Learning for Non-Stationary Markov Decision Processes: The Blessing of (More) Optimismcs.LG
New RL method tackles dynamic MDPs with evolving rewards and states.
problem Dynamic MDPs with evolving rewards and states.
method Sliding Window Upper-Confidence bound for Reinforcement Learning (SWUCRL2-CW) and Bandit-over-Reinforcement Learning (BORL).
result Achieves dynamic regret bound for non-stationary MDPs.