Reparameterization trick yields more accurate gradient estimates in variational inference.
problem Improving gradient estimates in variational inference.
method Idealized analysis of mean-field Gaussian approximations and quadratic log densities.
result Marginal variances of reparameterization gradient are smaller than score function gradient.
The paper classifies biharmonic quadratic maps between spheres, proving their energy density properties.
problem Classifying non-harmonic biharmonic quadratic forms between spheres.
method Proving non-harmonic biharmonic quadratic forms have constant energy density and classifying them.
result Non-harmonic biharmonic quadratic forms have constant energy density ( m + 1 ) / 2 (m+1)/2 ( m + 1 ) /2 . Entire area-minimizing surfaces of density 2 are planar or quadratic
problem Classifying entire area-minimizing surfaces
method Using density and algebraic properties
result All such surfaces are planar or algebraic
Log-density gradient estimation is a fundamental statistical problem and possesses various practical applications such as clustering and measuring non-Gaussianity. A naive two-step approach of first estimating the density and then taking its log-gradient is unreliable because an accurate density estimate does not neces…
We introduce a class of quadratic support (QS) functions, many of which play a crucial role in a variety of applications, including machine learning, robust statistical inference, sparsity promotion, and Kalman smoothing. Well known examples include the l2, Huber, l1 and Vapnik losses. We build on a dual representation…
Neural spline flows enhance flow models with rational-quadratic splines.
problem Improving flexibility and density estimation in flow models.
method Proposes a new differentiable module based on monotonic rational-quadratic splines.
result Demonstrates improved performance in density estimation, variational inference, and generative modeling of images.
A new algorithm for solving constrained convex optimization problems efficiently.
problem Constrained convex optimization problems requiring high accuracy solutions.
method Second-Order Conditional Gradient Sliding (SOCGS) algorithm, using projection-free methods to solve quadratic subproblems inexactly.
result Converges quadratically in primal gap after a finite number of linearly convergent iterations.
Random scan CAVI converges linearly under log-concave assumptions.
problem Analyzing the convergence rate of random scan Coordinate Ascent Variational Inference (CAVI) under log-concave conditions.
method Building on previous work, we analyze the random scan version of CAVI using optimal transport geometry.
result We obtain tight linear convergence rates for the random scan version of CAVI.
Algorithm samples composite logconcave densities efficiently.
problem Sampling from composite logconcave densities efficiently.
method Uses a restricted Gaussian oracle and gradient queries.
result Achieves strong total variation distance guarantees.
Study online monotone density estimation with expert aggregation and log-optimal calibration.
problem Online monotone density estimation and log-optimal calibration.
method Proposed two online estimators: Grenander estimator and expert aggregation estimator.
result Online estimators achieve O ( n 1 / 3 ) O(n^{1/3}) O ( n 1/3 ) cumulative log-likelihood gap and n log n \sqrt{n\log{n}} n log n pathwise regret bound. Poisson Midpoint Method improves Langevin Dynamics for diffusion models.
problem Slow convergence of LMC in diffusion models requiring many small steps.
method Poisson Midpoint Method approximates LMC with larger steps, proving quadratic speed up.
result Poisson Midpoint Method maintains quality of DDPM with fewer calls.
Proposes log density gradient to improve reinforcement learning sample complexity.
problem Residual error in gradient estimation in policy gradient methods.
method Log density gradient method to correct residual error, using state-action discounted distributional formulation.
result Min-max optimization method to approximate log density gradient with on-policy samples, achieving sample complexity of m − 1 / 2 m^{-1/2} m − 1/2 . Estimates log-concave densities in graphical models using tent functions.
problem Maximum likelihood estimation of log-concave densities in undirected graphs.
method MLE as product of tent functions corresponding to maximal cliques.
result MLE can be found via convex optimization.
Geodesic balls are isoperimetric in hyperbolic spaces with certain densities.
problem Proving isoperimetric properties in hyperbolic spaces with specific densities.
method Using geodesic balls and radial, strictly log-convex densities.
result Geodesic balls are isoperimetric in real hyperbolic space H R n H_{\mathbb R}^n H R n . We examine the vertical component of surface area in the warped product of a Euclidean interval and a fiber manifold with product density. We determine general conditions under which vertical fibers minimize vertical surface area among regions bounding the same volume and use these results to conclude that in many such…
We present sharp tail asymptotics for the density and the distribution function of linear combinations of correlated log-normal random variables, that is, exponentials of components of a correlated Gaussian vector. The asymptotic behavior turns out to depend on the correlation between the components, and the explicit s…
We completely characterize isoperimetric regions in R^n with density e^h, where h is convex, smooth, and radially symmetric. In particular, balls around the origin constitute isoperimetric regions of any given volume, proving the Log-Convex Density Conjecture due to Kenneth Brakke.
This work introduces a fixed-point optimization for variational inference.
problem Improving quantified uncertainty in predictions by optimizing a simplified distribution over parameters.
method Projective integral updates for high-dimensional variational inference.
result Efficient quasirandom quadrature sequence for mean-field distributions, leading to quasi-Newton variational Bayes (QNVB).
New algorithms sample structured logconcave families with improved efficiency.
problem Sampling structured logconcave families to high accuracy.
method Reduction framework inspired by proximal point methods, combined with restricted Gaussian oracles.
result Improved bounds for sampling structured distributions, matching or surpassing state-of-the-art results.
New lower bounds improve logistic log-likelihood optimization and inference.
problem Designing computationally tractable lower bounds for logistic log-likelihoods.
method Developed a piece-wise quadratic lower bound that uniformly improves tangent quadratic minorizers.
result Improves the speed of convergence and accuracy of variational Bayes approximations.
New sampling methods for log-concave densities using implicit integrators.
problem Sampling from log-concave densities efficiently.
method θ-method discretization of the overdamped Langevin diffusion.
result Geometric ergodicity and stability for θ ≥ 1 / 2 θ\ge1/2 θ ≥ 1/2 . Several classification methods assume that the underlying distributions follow tree-structured graphical models. Indeed, trees capture statistical dependencies between pairs of variables, which may be crucial to attain low classification errors. The resulting classifier is linear in the log-transformed univariate and b…
Study minimax risk of score estimation for log-concave distributions.
problem Minimizing risk in score estimation for log-concave distributions.
method Developed subclasses of log-concave densities and constructed a locally adaptive, multiscale estimator.
result Established minimax rates for score estimation over specific subclasses of log-concave densities.
Paper proves integrability and entropy compactness for Kähler potentials with uniform log-log threshold.
problem Integrability and entropy compactness for Kähler potentials with specific density.
method Skoda-Zeriahi type integrability theorem and log-log threshold detection.
result Positivity of integrability threshold and entropy compactness for uniform log-log threshold.
Least Squares EM converges globally for log-concave mixtures.
problem Location estimation in mixtures of two log-concave densities.
method Least Squares EM algorithm applied to log-concave mixtures.
result Least Squares EM converges globally to the true location parameter.
Residual Flows improve flow-based models for density estimation.
problem Density estimation using flow-based models with biased log-density estimates.
method Proposed a Russian roulette estimator for unbiased log-density estimation and used an alternative infinite series for gradient calculation. Improved invertible residual blocks with activation functions avoiding derivative saturation and generalized Lipschitz condition to induced mixed norms.
result Residual Flows achieve state-of-the-art performance on density estimation and outperform coupling block networks in joint generative and discriminative modeling.
DBSCAN++ speeds up density clustering for large datasets.
problem Slow runtime of DBSCAN for large datasets.
method DBSCAN++ computes densities for a subset of points instead of all.
result DBSCAN++ provides competitive performance and robustness.
In this paper, we study the Edgeworth expansion for a pre-averaging estimator of quadratic variation in the framework of continuous diffusion models observed with noise. More specifically, we obtain a second order expansion for the joint density of the estimators of quadratic variation and its asymptotic variance. Our …
Improves sequence modeling with a flow-based recurrent mixture density network.
problem Sequence modeling and sequence-to-sequence mapping applications.
method Generalized recurrent mixture density networks using normalized flow transformations.
result Significantly improved fit to image sequences measured by log-likelihood.
A new sampling method using log-concave Markov chains.
problem Sampling from unnormalized densities efficiently.
method Decomposes sampling into log-concave Markov chains with noisy measurements.
result Shows remarkable capacity to 'tunnel' between modes of a distribution.
A new method speeds up training of deep models by avoiding Jacobian determinant computation.
problem Efficiently training deep neural networks with complex log-determinant terms.
method Relative gradients to compute Jacobian updates efficiently.
result Training neural networks with Jacobian log-determinant objectives becomes feasible.
The Riemannian Langevin Algorithm samples from manifolds efficiently.
problem Sampling from distributions on manifolds with log-Sobolev inequality.
method Riemannian Langevin Algorithm, log-Sobolev inequality, self-concordance extension, stochastic smoothness bounding.
result The Riemannian Langevin Algorithm converges rapidly to the target density.
New method for estimating diffusion model densities without solving flows.
problem Estimating log densities from diffusion models efficiently.
method Monte Carlo path integral estimation, avoiding flow solving.
result Significantly more scalable and efficient density estimation.
New study shows low-degree polynomial algorithms struggle at clause densities close to Fix's.
problem Finding satisfying assignments in random k-SAT formulas at high clause densities.
method Analysis of low-degree polynomial algorithms and a new many-way overlap gap property.
result No efficient algorithms can find satisfying assignments at clause densities close to Fix's.
Partition Tree estimates conditional densities for mixed continuous and categorical variables.
problem Estimating conditional densities for mixed data types.
method Tree-based framework modeling conditional distributions as piecewise-constant densities on adaptive partitions, minimizing conditional negative log-likelihood.
result Improved probabilistic prediction compared to CART-style trees and state-of-the-art methods.
Compress++ speeds up distribution compression to near-linear time.
problem Accurately summarize a probability distribution using a small number of points efficiently.
method Introduces Compress++, a meta-procedure to speed up any thinning algorithm.
result Achieves n \sqrt{n} n points with O ( log n / n ) \mathcal{O}(\sqrt{\log n/n}) O ( log n / n ) integration error in O ( n log 3 n ) \mathcal{O}(n \log^3 n) O ( n log 3 n ) time and O ( n log 2 n ) \mathcal{O}( \sqrt{n} \log^2 n ) O ( n log 2 n ) space. We give a lower and an upper bound for the conformal dimension of the boundaries of certain small cancellation groups. We apply these bounds to the few relator and density models for random groups. This gives generic bounds of the following form, where l l l is the relator length, going to infinity. (a) $1 + 1/C < \Cdim(…
Faster algorithms for structured SVMs reduce computation time.
problem Efficiently solving quadratic programming problems with specific structures.
method Designing nearly-linear time algorithms for quadratic programs with low-rank factorizations and few linear constraints.
result First nearly-linear time algorithms for solving quadratic programs with specific structures.
Lower bounds show many sampling algorithms need many gradient queries.
problem Sampling from strongly log-concave densities in high dimensions.
method Information theory and stochastic gradient methods.
result Lower bound on number of gradient queries needed.
A new sampler for complex discrete distributions efficiently updates all variables in parallel.
problem Sampling complex high-dimensional discrete distributions efficiently and accurately.
method Discrete Langevin proposal (DLP) for parallel coordinate updates with controlled stepsize.
result DLP efficiently explores high-dimensional and strongly correlated variables with asymptotic bias of zero for log-quadratic distributions.
This paper compares log-likelihood and BLEU scores for sequence generation tasks.
problem The discrepancy between density estimation and sequence generation performance.
method Comparing several density estimators on five machine translation tasks.
result The correlation between log-likelihood and BLEU varies depending on model families.
Most of the empirical studies on stochastic volatility dynamics favor the 3/2 specification over the square-root (CIR) process in the Heston model. In the context of option pricing, the 3/2 stochastic volatility model is reported to be able to capture the volatility skew evolution better than the Heston model. In this …
MALA mixes efficiently under smoothness and isoperimetry assumptions.
problem Sampling from target densities efficiently.
method Metropolis-Adjusted Langevin algorithm (MALA) with smoothness and isoperimetry assumptions.
result MALA mixes in $O\left(\frac{(LΥ)^{\frac12}}{ψ_μ^2} \log\left(\frac{1}ε
ight)
ight)$ iterations.
Paper proposes a new method for robust modal regression.
problem Estimating the global mode of conditional density functions robustly.
method Directly approximates the gradient of modal regression risk using kernelized and neural-network-based log-density derivative estimators.
result Proposed methods achieve superior performance on various datasets.
Proposes a new Huber loss combining absolute and quadratic properties.
problem Improving robustness in learning models.
method Introduces a generalized Huber loss with a log-exp transform and provides an efficient minimization algorithm.
result Shows that the new loss function can be minimized efficiently.
This paper solves quadratic systems with sparse or generative priors.
problem Recovering signals from quadratic systems with full-rank matrices.
method Thresholded Wirtinger flow (TWF) and projected gradient descent (PGD) algorithms.
result The proposed methods significantly outperform existing algorithms in signal recovery.
DPS uses PINNs to estimate drift in diffusion models for sampling.
problem Accurately estimating drift term in reverse SDE from unnormalized density.
method Diffusion-PINN Sampler (DPS) solves PINN for log-density of SDE marginals.
result DPS achieves convergence guarantees and accurately samples complex distributions.
A new sampling method, RC-LMC, reduces computational cost for high-dimensional log-concave distributions.
problem High computational cost of LMC in high dimensions.
method RC-LMC updates only one coordinate at a time, adding noise.
result RC-LMC is more efficient than LMC in high dimensions, especially for skewed distributions.