Log-density gradient estimation is a fundamental statistical problem and possesses various practical applications such as clustering and measuring non-Gaussianity. A naive two-step approach of first estimating the density and then taking its log-gradient is unreliable because an accurate density estimate does not neces…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,341 papers · 148 categories
Trend · papers per month
48 results for “log-density gradients”
Proposes log density gradient to improve reinforcement learning sample complexity.
problem Residual error in gradient estimation in policy gradient methods.
method Log density gradient method to correct residual error, using state-action discounted distributional formulation.
result Min-max optimization method to approximate log density gradient with on-policy samples, achieving sample complexity of .
Improved VI with Price's gradient estimator for target log-density.
problem Approximating target distributions from unnormalized log-densities.
method Stochastic gradient-based variational inference with Price's gradient estimator.
result Identifies Price's gradient as the key to WVI's superior performance.
Paper proposes a new method for robust modal regression.
problem Estimating the global mode of conditional density functions robustly.
method Directly approximates the gradient of modal regression risk using kernelized and neural-network-based log-density derivative estimators.
result Proposed methods achieve superior performance on various datasets.
Non-Gaussian component analysis (NGCA) is aimed at identifying a linear subspace such that the projected data follows a non-Gaussian distribution. In this paper, we propose a novel NGCA algorithm based on log-density gradient estimation. Unlike existing methods, the proposed NGCA algorithm identifies the linear subspac…
A new sampling method, RC-LMC, reduces computational cost for high-dimensional log-concave distributions.
problem High computational cost of LMC in high dimensions.
method RC-LMC updates only one coordinate at a time, adding noise.
result RC-LMC is more efficient than LMC in high dimensions, especially for skewed distributions.
A new metric tensor improves Riemann manifold Monte Carlo for Bayesian models.
problem Improving sampling efficiency in Bayesian hierarchical models.
method Metric tensor derived from log-density gradient covariance matrices.
result Metric tensors enhance sampling for complex Bayesian models.
Lower bounds show many sampling algorithms need many gradient queries.
problem Sampling from strongly log-concave densities in high dimensions.
method Information theory and stochastic gradient methods.
result Lower bound on number of gradient queries needed.
Mean shift clustering finds the modes of the data probability density by identifying the zero points of the density gradient. Since it does not require to fix the number of clusters in advance, the mean shift has been a popular clustering algorithm in various application fields. A typical implementation of the mean shi…
Residual Flows improve flow-based models for density estimation.
problem Density estimation using flow-based models with biased log-density estimates.
method Proposed a Russian roulette estimator for unbiased log-density estimation and used an alternative infinite series for gradient calculation. Improved invertible residual blocks with activation functions avoiding derivative saturation and generalized Lipschitz condition to induced mixed norms.
result Residual Flows achieve state-of-the-art performance on density estimation and outperform coupling block networks in joint generative and discriminative modeling.
BBVI converges nearly dimensionally independent for log-concave targets.
problem Efficiently optimizing variational parameters in high-dimensional spaces.
method Proved convergence rate of BBVI with reparametrization gradient for log-concave targets.
result BBVI converges with nearly independent dimension dependence for log-concave targets.
A new method removes whitening for better non-Gaussian component analysis.
problem Data covariance matrix ill-conditioning hinders LSNGCA performance.
method Developed a whitening-free least-squares NGCA method.
result Demonstrated superior performance compared to whitened LSNGCA.
LMC algorithm improved for nonsmooth distributions.
problem LMC's limitations in nonsmooth settings.
method Adding a small Gaussian perturbation to iterates, controlling bias and variance.
result Polynomial-time convergence guarantees for nonsmooth log-concave distributions.
Noise-corrected Langevin algorithm improves sampling from noisy data.
problem Sampling from noisy data with biased score function.
method Noise-corrected Langevin algorithm using noisy score function.
result Bias due to noisy data is removed, improving sampling accuracy.
Pathfinder uses quasi-Newton optimization for variational inference.
problem Approximating complex posterior distributions efficiently.
method Pathfinder combines quasi-Newton optimization with variational methods to approximate log densities.
result Pathfinder produces draws with lower KL divergence than ADVI and comparable to HMC, requiring fewer evaluations.
Paper improves sampling from smooth, log-concave densities with inaccurate gradients.
problem Sampling from smooth, log-concave densities with inaccurate gradient evaluations.
method Approximate sampling using discretizations of the Langevin diffusion with optimized step sizes and various gradient approximations.
result Improved guarantees on sampling error, including logarithmic improvements and nonasymptotic bounds for second-order methods.
Novel criterion identifies heteroscedastic noise in causal discovery.
problem Heteroscedastic noise violates equal-variance assumption in causal discovery.
method Skewness-based criterion for identifying HSNMs.
result Skewness-based criterion distinguishes causal from anticausal directions.
AR-DAE approximates entropy gradient for machine learning models.
problem Intractable computation of entropy gradient for continuous distributions.
method Amortized residual denoising autoencoder (AR-DAE) to approximate entropy gradient.
result AR-DAE provides an unbiased gradient approximation for entropy.
Proximal Diffusion Models improve generative model efficiency.
problem Improving generative model efficiency and accuracy.
method Developed Proximal Diffusion Models using proximal maps instead of scores.
result Proximal Diffusion Models achieve faster convergence and higher accuracy.
Reparameterization trick yields more accurate gradient estimates in variational inference.
problem Improving gradient estimates in variational inference.
method Idealized analysis of mean-field Gaussian approximations and quadratic log densities.
result Marginal variances of reparameterization gradient are smaller than score function gradient.
ASVGD accelerates SVGD for efficient sampling.
problem Slow SVGD in high-dimensional sampling.
method Accelerated gradient flow in a metric space of probability densities, using Nesterov's method and momentum-based updates.
result ASVGD outperforms SVGD and other methods in sampling efficiency.
Diffusion-PINN Samplerstat.ML
DPS uses PINNs to estimate drift in diffusion models for sampling.
problem Accurately estimating drift term in reverse SDE from unnormalized density.
method Diffusion-PINN Sampler (DPS) solves PINN for log-density of SDE marginals.
result DPS achieves convergence guarantees and accurately samples complex distributions.
Discriminative classification with generative features: bridging Naive Bayes and logistic regressionstat.ML
Smart Bayes integrates generative and discriminative features for improved classification.
problem Improving classification performance by combining generative and discriminative modeling.
method Integrates generative likelihood-ratio features into a logistic-regression-style classifier.
result Often outperforms logistic regression and Naive Bayes in simulations and real data.
Improved convergence for non-log-concave sampling.
problem Sampling from non-log-concave distributions.
method Novel conductance analysis of SGLD with auxiliary Markov Chain.
result SGLD achieves ε-sampling error with fewer evaluations.
We establish general conditions under which Markov chains produced by the Hamiltonian Monte Carlo method will and will not be geometrically ergodic. We consider implementations with both position-independent and position-dependent integration times. In the former case we find that the conditions for geometric ergodicit…
Empirical Bayes Estimation with Side Information: A Nonparametric Integrative Tweedie Approachstat.ME
Enhances normal mean estimation with side info using NIT approach.
problem Compound estimation of normal means with side information.
method Empirical Bayes, nonparametric integrative Tweedie (NIT) approach.
result NIT approach improves estimation risk and convergence rate with increasing auxiliary data.
Study sampling from logconcave distributions with dependent data streams.
problem Sampling from logconcave distributions with biased gradient estimates.
method Euler discretization of Langevin SDEs with dependent data.
result Upper bound on Wasserstein-2 distance between iterates and target distribution.
New analysis of Langevin Monte Carlo via convex optimization.
problem Sampling from logconcave smooth and non-smooth target distributions.
method Formulation as a convex optimization problem, analysis using convex optimization techniques.
result Non-asymptotic analysis of Unadjusted Langevin Algorithm and new sampling methods.
Proposes a deep neural network for multi-dimensional functional data classification.
problem Classifying multi-dimensional functional data with non-Gaussian distributions.
method Trains a deep neural network on the principle components of the training data.
result FDNN achieves minimax optimality when log density ratio has a locally connected modular structure.
New method for estimating diffusion model densities without solving flows.
problem Estimating log densities from diffusion models efficiently.
method Monte Carlo path integral estimation, avoiding flow solving.
result Significantly more scalable and efficient density estimation.
Normalizing flow regression approximates posterior distributions without additional sampling.
problem Bayesian inference with computationally expensive likelihood evaluations.
method Normalizing flow regression (NFR) for offline inference.
result NFR yields a tractable posterior approximation through regression on existing log-density evaluations.
Improved score matching methods for estimating score functions and Hessians without high dimensionality.
problem Estimating score functions and Hessians efficiently in high-dimensional data.
method Implicit score matching and denoising score matching, leveraging Gagliardo-Nirenberg inequalities.
result Achieves convergence rates similar to denoising score matching and estimates Hessians without dimensionality issues.
New method trains EBMs using NFs for more accurate likelihood estimation.
problem Lack of statistical accuracy in EBMs likelihood estimation.
method Uses normalizing flows (NF) to fit an NF to an EBM during training.
result Accurate gradient for EBMs at all times, leading to a fast sampler.
Regularized Variational and Spectral Log-Density-Ratio Estimation in the Gaussian Location Modelcs.LG
Estimates Gaussian location model with ridge regularization, comparing variational and spectral methods.
problem Estimating parameters in Gaussian location model with regularization.
method Ridge-regularized log-density-ratio estimation, variational and spectral approaches.
result Regularized variational estimator has lower risk with many observations, spectral estimator with fewer observations.
Unified view of score estimators for flexible densities.
problem Estimating the score from unknown distributions.
method Regularized nonparametric regression framework.
result Unified convergence analysis and new estimators with desirable properties.
Statistical analysis of algorithm unrolling for inverse problems.
problem Designing deep neural networks to solve inverse problems efficiently.
method Analysis of gradient descent network (GDN) unrolling depth and statistical performance.
result The optimal statistical performance of GDNs requires unrolling depth of order log(n)/log(ρ_n^-1), where ρ_n is the convergence rate.
MonoFlow rethinks GANs using Wasserstein gradient flows.
problem Inconsistencies between GAN theory and practice.
method Unified generative modeling framework based on Wasserstein gradient flows.
result Adversarial training can be seen as particle flow optimization.
Learning the Stein Discrepancy for Training and Evaluating Energy-Based Models without Samplingstat.ML
New method estimates model discrepancy without sampling for unnormalized models.
problem Evaluating and training unnormalized density models efficiently.
method Estimate Stein discrepancy using neural network parameterized vector function.
result Method outperforms existing goodness-of-fit tests and training methods.
A deep learning approach for fitting complex distributions.
problem Limited applicability of simple kernels in fitting complex distributions.
method Learning a deep network to parameterize the kernel of the exponential family.
result The method can fit complex structures on moderate-dimensional problems.
A new sampling method reduces computational cost for high-dimensional log-concave distributions.
problem High computational cost of ULMC in high dimensions.
method Random Coordinate ULMC (RC-ULMC) selects a single coordinate per iteration.
result RC-ULMC is cheaper than classical ULMC, especially in highly skewed and high-dimensional problems.
ASVGD accelerates SVGD for efficient sampling from Gaussian targets.
problem Efficient sampling from Gaussian distributions using SVGD.
method Accelerated gradient flow in a metric space of probability densities, including momentum and Wasserstein regularization.
result ASVGD achieves optimal convergence rate for Gaussian targets, independent of covariance.
New research shows input-gradients can be manipulated without changing model's core function, challenging their use for model interpretation.
problem Current methods for model interpretability using input-gradients are flawed due to their arbitrary manipulability.
method Investigated by reinterpreting logits as unnormalized log-densities, proposing novel approximations for score-matching.
result Improving alignment between implicit density model and data distribution enhances gradient structure and explanatory power.
New algorithm tames non-linear growth in stochastic optimization.
problem Computational challenges in E-step of EM framework.
method Employing interacting particle systems and taming techniques to create tIPLA.
result Non-asymptotic convergence error estimates in Wasserstein-2 distance for tIPLA.
A new diffusion method approximates Schrödinger bridge with improved convergence.
problem Approximating Schrödinger bridge with Langevin diffusion.
method Leveraging Langevin diffusion to approximate Schrödinger bridge.
result The difference between the two approximations is proportional to the score function.
Novel algorithm PSO improves density estimation for multimodal data.
problem Data log-density estimation for multimodal distributions.
method Probabilistic Surface Optimization (PSO) using virtual stochastic forces.
result PSO-LDE achieves superior log-density estimation accuracy.
DEEN learns energy and score functions from complex data.
problem Challenges in density estimation for high-dimensional data.
method Inference-free hierarchical framework using score matching and multilayer perceptrons.
result DEEN successfully learns energy and score functions from synthetic and high-dimensional data.
The Riemannian Langevin Algorithm samples from manifolds efficiently.
problem Sampling from distributions on manifolds with log-Sobolev inequality.
method Riemannian Langevin Algorithm, log-Sobolev inequality, self-concordance extension, stochastic smoothness bounding.
result The Riemannian Langevin Algorithm converges rapidly to the target density.
New method uses TT approximations to solve HJB equations for efficient sampling.
problem Efficiently sampling from complex probability densities.
method Direct time integration of HJB equations using Tensor Train compression.
result Sample-free, dimensionality-avoiding integration method.