Log-density gradient estimation is a fundamental statistical problem and possesses various practical applications such as clustering and measuring non-Gaussianity. A naive two-step approach of first estimating the density and then taking its log-gradient is unreliable because an accurate density estimate does not neces…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “log density gradient”
Proposes log density gradient to improve reinforcement learning sample complexity.
problem Residual error in gradient estimation in policy gradient methods.
method Log density gradient method to correct residual error, using state-action discounted distributional formulation.
result Min-max optimization method to approximate log density gradient with on-policy samples, achieving sample complexity of .
Improved VI with Price's gradient estimator for target log-density.
problem Approximating target distributions from unnormalized log-densities.
method Stochastic gradient-based variational inference with Price's gradient estimator.
result Identifies Price's gradient as the key to WVI's superior performance.
We consider the problem of sampling from a strongly log-concave density in , and prove an information theoretic lower bound on the number of stochastic gradient queries of the log density needed. Several popular sampling algorithms (including many Markov chain Monte Carlo methods) operate by using stochas…
Mean shift clustering finds the modes of the data probability density by identifying the zero points of the density gradient. Since it does not require to fix the number of clusters in advance, the mean shift has been a popular clustering algorithm in various application fields. A typical implementation of the mean shi…
Modal regression is aimed at estimating the global mode (i.e., global maximum) of the conditional density function of the output variable given input variables, and has led to regression methods robust against heavy-tailed or skewed noises. The conditional mode is often estimated through maximization of the modal regre…
A new sampling method, RC-LMC, reduces computational cost for high-dimensional log-concave distributions.
problem High computational cost of LMC in high dimensions.
method RC-LMC updates only one coordinate at a time, adding noise.
result RC-LMC is more efficient than LMC in high dimensions, especially for skewed distributions.
GBHT uses gradient boosting for density estimation with theoretical guarantees.
problem Density estimation for unsupervised learning.
method Gradient Boosting Histogram Transform (GBHT) with Negative Log Likelihood loss.
result GBHT achieves faster convergence rates and better performance than base learners in density estimation.
Algorithm samples from composite log-concave distributions using gradient evaluations and restricted Gaussian oracles.
problem Sampling from composite log-concave distributions with limited gradient evaluations.
method Proximal gradient algorithm with RGO for and strong/strongly convex conditions for .
result Achieves error in total variation distance in iterations.
BBVI converges nearly dimensionally independent for log-concave targets.
problem Efficiently optimizing variational parameters in high-dimensional spaces.
method Proved convergence rate of BBVI with reparametrization gradient for log-concave targets.
result BBVI converges with nearly independent dimension dependence for log-concave targets.
Non-Gaussian component analysis (NGCA) is aimed at identifying a linear subspace such that the projected data follows a non-Gaussian distribution. In this paper, we propose a novel NGCA algorithm based on log-density gradient estimation. Unlike existing methods, the proposed NGCA algorithm identifies the linear subspac…
Paper formulates particle flow using variational inference and Fisher-Rao gradient flow.
problem Estimating posterior densities in probabilistic models.
method Variational formulation of particle flow, Fisher-Rao gradient flow, Gaussian and Gaussian mixture approximations.
result Gaussian and Gaussian mixture approximations of Fisher-Rao particle flow reduce to Exact Daum and Huang particle flow under linear Gaussian assumptions.
A new metric tensor improves Riemann manifold Monte Carlo for Bayesian models.
problem Improving sampling efficiency in Bayesian hierarchical models.
method Metric tensor derived from log-density gradient covariance matrices.
result Metric tensors enhance sampling for complex Bayesian models.
The Riemannian Langevin Algorithm samples from manifolds efficiently.
problem Sampling from distributions on manifolds with log-Sobolev inequality.
method Riemannian Langevin Algorithm, log-Sobolev inequality, self-concordance extension, stochastic smoothness bounding.
result The Riemannian Langevin Algorithm converges rapidly to the target density.
Study on sample complexity of policy gradient for stabilizing linear systems under multiplicative noise.
problem Learning optimal feedback gain for stabilizing linear systems with multiplicative noise.
method Analyzes the sample complexity of policy gradient methods, addressing the cusp obstruction and using symmetry to control divergent parts of the gradient.
result Proves that projected mini-batch policy gradient attains total sample complexity of O(1/η) when noise density is known and O(η^(-(2s+1)/(2s))) when estimated, for C^s noise densities with s ≥ 2.
In this paper, we study the problem of sampling from a given probability density function that is known to be smooth and strongly log-concave. We analyze several methods of approximate sampling based on discretizations of the (highly overdamped) Langevin diffusion and establish guarantees on its error measured in the W…
Improved convergence for non-log-concave sampling.
problem Sampling from non-log-concave distributions.
method Novel conductance analysis of SGLD with auxiliary Markov Chain.
result SGLD achieves ε-sampling error with fewer evaluations.
New method estimates densities using Sobolev regularization, outperforming existing algorithms.
problem Non-parametric density estimation with clear inductive bias.
method Regularizes Sobolev norm of density, approximates kernel via sampling, uses natural gradients for optimization.
result Method ranks second best on ADBench anomaly detection benchmark.
Langevin Monte Carlo (LMC) is an iterative algorithm used to generate samples from a distribution that is known only up to a normalizing constant. The nonasymptotic dependence of its mixing time on the dimension and target accuracy is understood mainly in the setting of smooth (gradient-Lipschitz) log-densities, a seri…
Flow-based generative models parameterize probability distributions through an invertible transformation and can be trained by maximum likelihood. Invertible residual networks provide a flexible family of transformations where only Lipschitz conditions rather than strict architectural constraints are needed for enforci…
Recently, some works have suggested methods to combine variational probabilistic inference with Monte Carlo sampling. One promising approach is via local optimal transport. In this approach, a gradient steepest descent method based on local optimal transport principles is formulated to transform deterministically point…
AR-DAE approximates entropy gradient for machine learning models.
problem Intractable computation of entropy gradient for continuous distributions.
method Amortized residual denoising autoencoder (AR-DAE) to approximate entropy gradient.
result AR-DAE provides an unbiased gradient approximation for entropy.
A new method normalizes EBM training by introducing a learnable parameter.
problem Training energy-based models with maximum likelihood is challenging due to intractable normalisation constants.
method Proposes a self-normalised log-likelihood (SNL) objective that introduces a learnable parameter representing the normalisation constant.
result The SNL objective is a lower bound of the log-likelihood and can be directly optimised using stochastic gradient techniques.
A Jacobian inequality for gradient maps on the sphere and its application to directional statisticsmath.DG
In the field of optimal transport theory, an optimal map is known to be a gradient map of a potential function satisfying cost-convexity. In this paper, the Jacobian determinant of a gradient map is shown to be log-concave with respect to a convex combination of the potential functions when the underlying manifold is t…
The reparameterization trick is widely used in variational inference as it yields more accurate estimates of the gradient of the variational objective than alternative approaches such as the score function method. Although there is overwhelming empirical evidence in the literature showing its success, there is relative…
New algorithm reduces variance in stochastic gradient estimation.
problem Optimizing the variance of stochastic gradient algorithms for non-log-concave distributions.
method Developed a Multi-index Antithetic Stochastic Gradient Algorithm (MASGA) that is independent of the distribution's structure.
result MASGA achieves performance comparable to Monte Carlo estimators with unbiased samples.
ASVGD accelerates SVGD for efficient sampling.
problem Slow SVGD in high-dimensional sampling.
method Accelerated gradient flow in a metric space of probability densities, using Nesterov's method and momentum-based updates.
result ASVGD outperforms SVGD and other methods in sampling efficiency.
Study online monotone density estimation with expert aggregation and log-optimal calibration.
problem Online monotone density estimation and log-optimal calibration.
method Proposed two online estimators: Grenander estimator and expert aggregation estimator.
result Online estimators achieve cumulative log-likelihood gap and pathwise regret bound.
Pathfinder uses quasi-Newton optimization for variational inference.
problem Approximating complex posterior distributions efficiently.
method Pathfinder combines quasi-Newton optimization with variational methods to approximate log densities.
result Pathfinder produces draws with lower KL divergence than ADVI and comparable to HMC, requiring fewer evaluations.
Bidirectional bounds stabilize training of energy-based models.
problem Training energy-based models is difficult and prone to instability.
method Propose bidirectional bounds linking to gradient penalty and Jacobi-determinant estimator.
result Significant stabilization and high-quality density estimation achieved.
New research shows input-gradients can be manipulated without changing model's core function, challenging their use for model interpretation.
problem Current methods for model interpretability using input-gradients are flawed due to their arbitrary manipulability.
method Investigated by reinterpreting logits as unnormalized log-densities, proposing novel approximations for score-matching.
result Improving alignment between implicit density model and data distribution enhances gradient structure and explanatory power.
As an important Markov Chain Monte Carlo (MCMC) method, stochastic gradient Langevin dynamics (SGLD) algorithm has achieved great success in Bayesian learning and posterior sampling. However, SGLD typically suffers from slow convergence rate due to its large variance caused by the stochastic gradient. In order to allev…
Parallel sampling for smooth distributions with fast convergence.
problem Efficiently sampling from distributions with smooth densities.
method Parallelization of Langevin algorithms under log-Sobolev inequalities.
result Samples close to target distribution with low KL divergence or TV distance.
Estimates log-concave densities in graphical models using tent functions.
problem Maximum likelihood estimation of log-concave densities in undirected graphs.
method MLE as product of tent functions corresponding to maximal cliques.
result MLE can be found via convex optimization.
Geodesic balls are isoperimetric in hyperbolic spaces with certain densities.
problem Proving isoperimetric properties in hyperbolic spaces with specific densities.
method Using geodesic balls and radial, strictly log-convex densities.
result Geodesic balls are isoperimetric in real hyperbolic space .
We examine the vertical component of surface area in the warped product of a Euclidean interval and a fiber manifold with product density. We determine general conditions under which vertical fibers minimize vertical surface area among regions bounding the same volume and use these results to conclude that in many such…
The kernel exponential family is a rich class of distributions, which can be fit efficiently and with statistical guarantees by score matching. Being required to choose a priori a simple kernel such as the Gaussian, however, limits its practical applicability. We provide a scheme for learning a kernel parameterized by …
Noise-corrected Langevin algorithm improves sampling from noisy data.
problem Sampling from noisy data with biased score function.
method Noise-corrected Langevin algorithm using noisy score function.
result Bias due to noisy data is removed, improving sampling accuracy.
Algorithm samples composite logconcave densities efficiently.
problem Sampling from composite logconcave densities efficiently.
method Uses a restricted Gaussian oracle and gradient queries.
result Achieves strong total variation distance guarantees.
Learning the Stein Discrepancy for Training and Evaluating Energy-Based Models without Samplingstat.ML
New method estimates model discrepancy without sampling for unnormalized models.
problem Evaluating and training unnormalized density models efficiently.
method Estimate Stein discrepancy using neural network parameterized vector function.
result Method outperforms existing goodness-of-fit tests and training methods.
We completely characterize isoperimetric regions in R^n with density e^h, where h is convex, smooth, and radially symmetric. In particular, balls around the origin constitute isoperimetric regions of any given volume, proving the Log-Convex Density Conjecture due to Kenneth Brakke.
New method trains EBMs using NFs for more accurate likelihood estimation.
problem Lack of statistical accuracy in EBMs likelihood estimation.
method Uses normalizing flows (NF) to fit an NF to an EBM during training.
result Accurate gradient for EBMs at all times, leading to a fast sampler.
New method improves sampling from high-dimensional target densities.
problem Sampling from high-dimensional target densities using Monte Carlo algorithms.
method Extends Metropolis-Adjusted Langevin Diffusion algorithm with random precondition matrix modeling.
result Significantly improves performance and computational efficiency over standard MCMC methods.
Novel criterion identifies heteroscedastic noise in causal discovery.
problem Heteroscedastic noise violates equal-variance assumption in causal discovery.
method Skewness-based criterion for identifying HSNMs.
result Skewness-based criterion distinguishes causal from anticausal directions.
ASVGD accelerates SVGD for efficient sampling from Gaussian targets.
problem Efficient sampling from Gaussian distributions using SVGD.
method Accelerated gradient flow in a metric space of probability densities, including momentum and Wasserstein regularization.
result ASVGD achieves optimal convergence rate for Gaussian targets, independent of covariance.
Generative adversarial networks (GANs) are a powerful approach to unsupervised learning. They have achieved state-of-the-art performance in the image domain. However, GANs are limited in two ways. They often learn distributions with low support---a phenomenon known as mode collapse---and they do not guarantee the exist…
Guarantees convergence for black-box variational inference without modifications.
problem Convergence guarantees for black-box variational inference.
method Analysis of log-smooth posterior densities, location-scale variational family, and convergence rates of algorithm design choices.
result Proximal stochastic gradient descent fixes suboptimal convergence rates and achieves strongest known guarantees.
Proposes PGPS for efficient Bayesian inference.
problem Efficient sampling from complex posterior distributions.
method Path-guided particle-based sampling with Log-weighted Shrinkage.
result PGPS generates samples closer to target distribution.