Log-density gradient estimation is a fundamental statistical problem and possesses various practical applications such as clustering and measuring non-Gaussianity. A naive two-step approach of first estimating the density and then taking its log-gradient is unreliable because an accurate density estimate does not neces…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,982 papers · 148 categories
Trend · papers per month
48 results for “Log Density Estimation”
Study online monotone density estimation with expert aggregation and log-optimal calibration.
problem Online monotone density estimation and log-optimal calibration.
method Proposed two online estimators: Grenander estimator and expert aggregation estimator.
result Online estimators achieve cumulative log-likelihood gap and pathwise regret bound.
Estimates log-concave densities in graphical models using tent functions.
problem Maximum likelihood estimation of log-concave densities in undirected graphs.
method MLE as product of tent functions corresponding to maximal cliques.
result MLE can be found via convex optimization.
Study minimax risk of score estimation for log-concave distributions.
problem Minimizing risk in score estimation for log-concave distributions.
method Developed subclasses of log-concave densities and constructed a locally adaptive, multiscale estimator.
result Established minimax rates for score estimation over specific subclasses of log-concave densities.
Residual Flows improve flow-based models for density estimation.
problem Density estimation using flow-based models with biased log-density estimates.
method Proposed a Russian roulette estimator for unbiased log-density estimation and used an alternative infinite series for gradient calculation. Improved invertible residual blocks with activation functions avoiding derivative saturation and generalized Lipschitz condition to induced mixed norms.
result Residual Flows achieve state-of-the-art performance on density estimation and outperform coupling block networks in joint generative and discriminative modeling.
New method for estimating diffusion model densities without solving flows.
problem Estimating log densities from diffusion models efficiently.
method Monte Carlo path integral estimation, avoiding flow solving.
result Significantly more scalable and efficient density estimation.
Proposes log density gradient to improve reinforcement learning sample complexity.
problem Residual error in gradient estimation in policy gradient methods.
method Log density gradient method to correct residual error, using state-action discounted distributional formulation.
result Min-max optimization method to approximate log density gradient with on-policy samples, achieving sample complexity of .
Partition Tree estimates conditional densities for mixed continuous and categorical variables.
problem Estimating conditional densities for mixed data types.
method Tree-based framework modeling conditional distributions as piecewise-constant densities on adaptive partitions, minimizing conditional negative log-likelihood.
result Improved probabilistic prediction compared to CART-style trees and state-of-the-art methods.
Mean shift clustering finds the modes of the data probability density by identifying the zero points of the density gradient. Since it does not require to fix the number of clusters in advance, the mean shift has been a popular clustering algorithm in various application fields. A typical implementation of the mean shi…
Least Squares EM converges globally for log-concave mixtures.
problem Location estimation in mixtures of two log-concave densities.
method Least Squares EM algorithm applied to log-concave mixtures.
result Least Squares EM converges globally to the true location parameter.
This paper compares log-likelihood and BLEU scores for sequence generation tasks.
problem The discrepancy between density estimation and sequence generation performance.
method Comparing several density estimators on five machine translation tasks.
result The correlation between log-likelihood and BLEU varies depending on model families.
Paper proposes a new method for robust modal regression.
problem Estimating the global mode of conditional density functions robustly.
method Directly approximates the gradient of modal regression risk using kernelized and neural-network-based log-density derivative estimators.
result Proposed methods achieve superior performance on various datasets.
GBHT uses gradient boosting for density estimation with theoretical guarantees.
problem Density estimation for unsupervised learning.
method Gradient Boosting Histogram Transform (GBHT) with Negative Log Likelihood loss.
result GBHT achieves faster convergence rates and better performance than base learners in density estimation.
Diffusion-PINN Samplerstat.ML
DPS uses PINNs to estimate drift in diffusion models for sampling.
problem Accurately estimating drift term in reverse SDE from unnormalized density.
method Diffusion-PINN Sampler (DPS) solves PINN for log-density of SDE marginals.
result DPS achieves convergence guarantees and accurately samples complex distributions.
Several classification methods assume that the underlying distributions follow tree-structured graphical models. Indeed, trees capture statistical dependencies between pairs of variables, which may be crucial to attain low classification errors. The resulting classifier is linear in the log-transformed univariate and b…
TAKDE optimizes kernel density estimation for real-time dynamic processes.
problem Real-time density estimation in applications like computer vision and signal processing.
method Derives asymptotic mean integrated squared error (AMISE) upper bound for 'sliding window' kernel density estimator and proposes TAKDE as a novel, theoretically optimal estimator.
result TAKDE outperforms other dynamic density estimators in terms of test log-likelihood and runtime.
Improved VI with Price's gradient estimator for target log-density.
problem Approximating target distributions from unnormalized log-densities.
method Stochastic gradient-based variational inference with Price's gradient estimator.
result Identifies Price's gradient as the key to WVI's superior performance.
Novel algorithm PSO improves density estimation for multimodal data.
problem Data log-density estimation for multimodal distributions.
method Probabilistic Surface Optimization (PSO) using virtual stochastic forces.
result PSO-LDE achieves superior log-density estimation accuracy.
AR models are unreliable for density estimation, especially for image translation and outlier detection.
problem The reliability of autoregressive models as density estimators in image translation and outlier detection.
method Investigation into AR models' use as density estimators in two settings and examination of underlying optimization issues.
result Density estimates from AR models are unreliable and do not correlate with perceptual quality.
Improved GANs estimate convergence rate for density estimation.
problem Improving the accuracy of density estimation with GANs.
method Proved an oracle inequality for JS divergence between GAN estimate and true density.
result JS-divergence rate of convergence is .
Pathfinder uses quasi-Newton optimization for variational inference.
problem Approximating complex posterior distributions efficiently.
method Pathfinder combines quasi-Newton optimization with variational methods to approximate log densities.
result Pathfinder produces draws with lower KL divergence than ADVI and comparable to HMC, requiring fewer evaluations.
Regularized Variational and Spectral Log-Density-Ratio Estimation in the Gaussian Location Modelcs.LG
Estimates Gaussian location model with ridge regularization, comparing variational and spectral methods.
problem Estimating parameters in Gaussian location model with regularization.
method Ridge-regularized log-density-ratio estimation, variational and spectral approaches.
result Regularized variational estimator has lower risk with many observations, spectral estimator with fewer observations.
Proposes a neural density estimator for anomaly detection using labeled data.
problem Improving anomaly detection performance with limited labeled data.
method Uses deep autoregressive neural density estimators trained with anomaly labels to maximize normal likelihood and minimize anomalous likelihood.
result Significantly improves anomaly detection performance with few labeled instances compared to existing methods.
Non-Gaussian component analysis (NGCA) is aimed at identifying a linear subspace such that the projected data follows a non-Gaussian distribution. In this paper, we propose a novel NGCA algorithm based on log-density gradient estimation. Unlike existing methods, the proposed NGCA algorithm identifies the linear subspac…
In recent years, kernel density estimation has been exploited by computer scientists to model machine learning problems. The kernel density estimation based approaches are of interest due to the low time complexity of either O(n) or O(n*log(n)) for constructing a classifier, where n is the number of sampling instances.…
TDX predicts evolving distributions from streaming data.
problem Predicting density at future time points in evolving data.
method Dynamic basis expansion with isometric log-ratio transformation.
result TDX accurately captures monotonous drift patterns.
Study finds the number of modes in Gaussian kernel density estimators scales with sqrt(β log β).
problem Determining the number of clusters in Transformers.
method Used Kac-Rice formula and Edgeworth expansion to prove scaling.
result The expected number of modes scales as Θ(√(β log β)).
TraDE uses self-attention for better density estimation of tabular and image data.
problem Improving density estimation for tabular and image data.
method Self-attention-based architecture trained with a penalized maximum likelihood objective.
result TraDE produces significantly better density estimates than existing methods.
Lower bounds show density estimation requires linear samples or query time.
problem Statistical-computational trade-offs in density estimation.
method Lower bound analysis on data structures.
result Lower bounds demonstrate statistical-computational trade-offs for density estimation.
We study the construction of coresets for kernel density estimates. That is we show how to approximate the kernel density estimate described by a large point set with another kernel density estimate with a much smaller point set. For characteristic kernels (including Gaussian and Laplace kernels), our approximation pre…
A boosting method improves nonparametric density estimation without smoothing assumptions.
problem Overfitting in nonparametric data fitting.
method Introduces a boosting algorithm for univariate nonparametric maximum likelihood estimation.
result Demonstrates the effectiveness of the boosting approach through simulations and real data experiments.
BMTI method estimates densities without bins, outperforming traditional estimators.
problem Nonparametric, robust, and data-efficient density estimation in high-dimensional spaces.
method BMTI integrates log-density differences between neighboring points, weighted by uncertainties, using a maximum-likelihood formulation.
result BMTI reconstructs smooth profiles in high-dimensional spaces, outperforming traditional estimators.
Improved Robust Estimation for Erdős-Rényi Graphs: The Sparse Regime and Optimal Breakdown Pointcs.DS
New algorithm estimates edge density of random graphs robustly, achieving optimal breakdown point.
problem Estimating edge density of Erdős-Rényi graphs under adversarial edge manipulation.
method Sum-of-Squares (SoS) hierarchy, constructing constant-degree certificates for concentration.
result First polynomial-time algorithm with optimal breakdown point and matching error guarantees.
This paper improves Green's function estimates for compact Kähler manifolds.
problem Estimating Green's function norms for compact Kähler manifolds without curvature bounds.
method Proves an improved integral estimate for Green's function under volume density condition.
result Improved global geometric estimates, including eigenvalue bounds for Laplacian.
Paper refines InfoNCE for accurate mutual information estimation.
problem Indirect connection of InfoNCE to mutual information estimation.
method Introduces InfoNCE-anchor with an auxiliary anchor class for consistent density ratio estimation.
result InfoNCE-anchor yields a plug-in MI estimator with significantly reduced bias.
Discriminative classification with generative features: bridging Naive Bayes and logistic regressionstat.ML
Smart Bayes integrates generative and discriminative features for improved classification.
problem Improving classification performance by combining generative and discriminative modeling.
method Integrates generative likelihood-ratio features into a logistic-regression-style classifier.
result Often outperforms logistic regression and Naive Bayes in simulations and real data.
A new sampling method, RC-LMC, reduces computational cost for high-dimensional log-concave distributions.
problem High computational cost of LMC in high dimensions.
method RC-LMC updates only one coordinate at a time, adding noise.
result RC-LMC is more efficient than LMC in high dimensions, especially for skewed distributions.
Spectral density matrix estimation of multivariate time series is a classical problem in time series and signal processing. In modern neuroscience, spectral density based metrics are commonly used for analyzing functional connectivity among brain regions. In this paper, we develop a non-asymptotic theory for regularize…
PresGANs improve GANs by mitigating mode collapse and enhancing log-likelihood.
problem GANs struggle with mode collapse and lack a reliable way to evaluate generalization.
method PresGANs add noise to density networks and use entropy regularization to stabilize training and capture all modes.
result PresGANs reduce the gap in predictive log-likelihood between GANs and VAEs.
Geodesic balls are isoperimetric in hyperbolic spaces with certain densities.
problem Proving isoperimetric properties in hyperbolic spaces with specific densities.
method Using geodesic balls and radial, strictly log-convex densities.
result Geodesic balls are isoperimetric in real hyperbolic space .
Develops a local Fokker--Planck geometric framework for more accurate score estimation.
problem Inaccurate estimation of score function in non-linear, state-dependent drifts.
method Local Fokker--Planck geometric framework, time change to cumulative-variance coordinate, heat-ball mean-value representations, exact high-dimensional sampling.
result Exact local mean-value representations for the score and density, improved accuracy in low-density regions.
A new probabilistic mixup framework improves deep learning generalization.
problem Improving deep learning generalization performance.
method Introduces a novel probabilistic mixup framework for conditional density estimation.
result Empirical results show benefits over existing mixup variants.
We examine the vertical component of surface area in the warped product of a Euclidean interval and a fiber manifold with product density. We determine general conditions under which vertical fibers minimize vertical surface area among regions bounding the same volume and use these results to conclude that in many such…
New method estimates density ratio for well-separated distributions using multi-class logistic regression.
problem Challenges in estimating density ratio for well-separated distributions.
method Uses multi-class logistic regression with auxiliary densities to estimate log(p/q).
result Demonstrates superior performance on density ratio estimation, mutual information, and representation learning tasks.
Improved score matching methods for estimating score functions and Hessians without high dimensionality.
problem Estimating score functions and Hessians efficiently in high-dimensional data.
method Implicit score matching and denoising score matching, leveraging Gagliardo-Nirenberg inequalities.
result Achieves convergence rates similar to denoising score matching and estimates Hessians without dimensionality issues.
Transfer Learning (Il)liquidityq-fin.MF
Framework improves risk neutral density estimation in illiquid markets.
problem Challenges in estimating Risk Neutral Density in illiquid markets.
method Introduces Deep Log-Sum-Exp Neural Network leveraging Deep and Transfer learning.
result Framework recovers Risk Neutral Density with few option quotes in severe illiquidity.
Learning the Stein Discrepancy for Training and Evaluating Energy-Based Models without Samplingstat.ML
New method estimates model discrepancy without sampling for unnormalized models.
problem Evaluating and training unnormalized density models efficiently.
method Estimate Stein discrepancy using neural network parameterized vector function.
result Method outperforms existing goodness-of-fit tests and training methods.
Unified view of score estimators for flexible densities.
problem Estimating the score from unknown distributions.
method Regularized nonparametric regression framework.
result Unified convergence analysis and new estimators with desirable properties.