arXiv research

Study compares L1 and VG sparsity priors in inverse problems.

Variable selection for high-dimensional linear models has received a lot of attention lately, mostly in the context of l1-regularization. Part of the attraction is the variable selection effect: parsimonious models are obtained, which are very suitable for interpretation. In terms of predictive power, however, these re…

In this paper, we propose a variable selection method for general nonparametric kernel-based estimation. The proposed method consists of two-stage estimation: (1) construct a consistent estimator of the target function, (2) approximate the estimator using a few variables by l1-type penalized estimation. We see that the…

Paper develops a method to predict cancer patient survival using molecular profiles.

Developed a new thresholding method that connects soft and hard thresholding.

New variational principle found for non-variational differential equations.

Derives Lagrangian for minimal surfaces, proving tangential variations vanish.

Improved Bayesian uncertainty quantification using variational bagging.

This work improves variational inference by reducing gradient variance.

Adaptive variational Bayes framework improves inference adaptively.

A new EVI framework improves ParVI methods by maintaining variational structure and reducing KL-divergence.

We propose a family of variational approximations to Bayesian posterior distributions, called $α$-VB, with provable statistical guarantees. The standard variational approximation is a special case of $α$-VB with $α=1$. When $α\in(0,1]$, a novel class of variational inequalities are developed for linking the Bayes risk …

New examples of variational bivectors found that are not Poissonian.

Variational Prediction simplifies Bayesian inference without test time costs.

Improved VAE estimation from incomplete data using variational mixtures.

In the paper "On Truncated Variation of Brownian Motion with Drift" (Bull. Pol. Acad. Sci. Math. 56 (2008), no.4, 267 - 281) we defined truncated variation of Brownian motion with drift, $W_t = B_t + μt, t\geq 0,$ where $(B_t)$ is a standard Brownian motion. Truncated variation differs from regular variation by neglect…

Semi-Implicit Variational Inference (SIVI) is improved with SIVI-SM using score matching.

This work proposes using zero-variance control variates to reduce variance in pathwise gradient estimators for variational inference.

New classification of hypersurfaces with conformal variations.

Variational inference is increasingly being addressed with stochastic optimization. In this setting, the gradient's variance plays a crucial role in the optimization procedure, since high variance gradients lead to poor convergence. A popular approach used to reduce gradient's variance involves the use of control varia…

Variational approach to basic manifold structures.

The paper introduces structured variational families to improve scalability in black-box variational inference.

New method reduces inference variance for faster optimization.

We derive a formula for the first variation of horizontal perimeter measure for $C^2$ hypersurfaces of completely general sub-Riemannian manifolds, allowing for the existence of characteristic points. For $C^2$ hypersurfaces in vertically rigid sub-Riemannian manifolds we also produce a second variation formula for var…

Paper variates Navier-Stokes-Fourier system for thermodynamic consistency.

We develop unbiased implicit variational inference (UIVI), a method that expands the applicability of variational inference by defining an expressive variational family. UIVI considers an implicit variational distribution obtained in a hierarchical manner using a simple reparameterizable distribution whose variational …

Variational inference is an umbrella term for algorithms which cast Bayesian inference as optimization. Classically, variational inference uses the Kullback-Leibler divergence to define the optimization. Though this divergence has been widely used, the resultant posterior approximation can suffer from undesirable stati…

Variational inference transforms posterior inference into parametric optimization thereby enabling the use of latent variable models where otherwise impractical. However, variational inference can be finicky when different variational parameters control variables that are strongly correlated under the model. Traditiona…

Variational Inference is a powerful tool in the Bayesian modeling toolkit, however, its effectiveness is determined by the expressivity of the utilized variational distributions in terms of their ability to match the true posterior distribution. In turn, the expressivity of the variational family is largely limited by …

Variational inference methods for latent variable statistical models have gained popularity because they are relatively fast, can handle large data sets, and have deterministic convergence guarantees. However, in practice it is unclear whether the fixed point identified by the variational inference algorithm is a local…

A new optimization algorithm for Gaussian Variational Inference on precision matrices.

Variational inference for latent variable models is prevalent in various machine learning problems, typically solved by maximizing the Evidence Lower Bound (ELBO) of the true data likelihood with respect to a variational distribution. However, freely enriching the family of variational distribution is challenging since…

Variational inference is a scalable technique for approximate Bayesian inference. Deriving variational inference algorithms requires tedious model-specific calculations; this makes it difficult to automate. We propose an automatic variational inference algorithm, automatic differentiation variational inference (ADVI). …

We investigate the use of alternative divergences to Kullback-Leibler (KL) in variational inference(VI), based on the Variational Dropout \cite{kingma2015}. Stochastic gradient variational Bayes (SGVB) \cite{aevb} is a general framework for estimating the evidence lower bound (ELBO) in Variational Bayes. In this work, …

The recognition network in deep latent variable models such as variational autoencoders (VAEs) relies on amortized inference for efficient posterior approximation that can scale up to large datasets. However, this technique has also been demonstrated to select suboptimal variational parameters, often resulting in consi…

We show a very simple and general total second variation formula for Perelman's $\mathcal{W}$-functional at arbitrary points in the space of Riemannian metrics. Moreover we perform a study of the properties of the variations of Kähler structures. We deduce a quite simple and general total second variation formula for P…

The calculus of variations for lagrangians which are not functions on the tangent bundle, but sections certain affine bundles is developed. We follow a general approach to variational principles which admits boundary terms of variations.

A setting for global variational geometry on Grassmann fibrations is presented. The integral variational functionals for finite dimensional immersed submanifolds are studied by means of the fundamental Lepage equivalent of a homogeneous Lagrangian, which can be regarded as a generalization of the well-known Hilbert for…

New formulas for coassociative submanifolds' volume variation.

The paper studies curves in Riemannian manifolds using total variation flow.

We introduce incremental variational inference and apply it to latent Dirichlet allocation (LDA). Incremental variational inference is inspired by incremental EM and provides an alternative to stochastic variational inference. Incremental LDA can process massive document collections, does not require to set a learning …

CDVI improves variational inference for survival analysis by considering censoring mechanisms.

We characterize how to vary the Abel-Jacobi map in terms of Schiffer variation. From this characterization, we will interpret the relation of hyperellipticity of curves with Schiffer variation and describe the deformation of elliptic solitons under Schiffer variation.

Improves bandit convex optimization with gradient variations.

Hidden regular variation defines a subfamily of distributions satisfying multivariate regular variation on $\mathbb{E} = [0, \infty]^d \backslash \{(0,0, ..., 0) \}$ and models another regular variation on the sub-cone $\mathbb{E}^{(2)} = \mathbb{E} \backslash \cup_{i=1}^d \mathbb{L}_i$, where $\mathbb{L}_i$ is the $i…

Survey of methods for solving smooth stochastic variational inequalities.

In Bayesian machine learning, the posterior distribution is typically computationally intractable, hence variational inference is often required. In this approach, an evidence lower bound on the log likelihood of data is maximized during training. Variational Autoencoders (VAE) are one important example where variation…

Improved sampling method using regularized Stein Variational Gradient Flow.