Study minimax rates for density estimation under Huber contamination and Besov IPM losses.
problem Minimax convergence rates of nonparametric density estimation under Huber contamination model with outliers.
method Re-scaled thresholding wavelet series estimator and GAN architectures.
result Achieves minimax optimal convergence rates under Besov IPM losses.
We study the problem of estimating a nonparametric probability density under a large family of losses called Besov IPMs, which include, for example, L p \mathcal{L}^p L p distances, total variation distance, and generalizations of both Wasserstein and Kolmogorov-Smirnov distances. For a wide variety of settings, we provide b…
Novel algorithm speeds up computation of Sobolev IPM for graph-based probability measures.
problem Efficient computation of Sobolev IPM for graph-based probability measures.
method Established relation between Sobolev norm and weighted L p L^p L p -norm, proposed novel regularization, leveraged graph structure. result Proposed regularized Sobolev IPM provides closed-form expression for fast computation.
We study the minimax optimal rates for estimating a range of Integral Probability Metrics (IPMs) between two unknown probability measures, based on n n n independent samples from them. Curiously, we show that estimating the IPM itself between probability measures, is not significantly easier than estimating the probabili…
We propose a new Integral Probability Metric (IPM) between distributions: the Sobolev IPM. The Sobolev IPM compares the mean discrepancy of two distributions for functions (critic) restricted to a Sobolev ball defined with respect to a dominant measure μ μ μ . We show that the Sobolev IPM compares two distributions in hig…
Unified view of KL-divergence and IPMs via DRE, with new DRM metrics.
problem Unified understanding of KL-divergence and IPMs.
method Unified representation via maximum likelihood density-ratio estimation (DRE).
result Unified form of IPMs and novel DRM metrics.
We introduce new families of Integral Probability Metrics (IPM) for training Generative Adversarial Networks (GAN). Our IPMs are based on matching statistics of distributions embedded in a finite dimensional feature space. Mean and covariance feature matching IPMs allow for stable training of GANs, which we will call M…
A new IPM uses ReLU networks to measure probability discrepancies.
problem Measuring the difference between two probability distributions in high dimensions.
method Proposes a new parametric IPM using ReLU neural networks to optimize and distinguish between distributions.
result The proposed IPM has good convergence rates and can be used as a surrogate for other IPMs.
Generalizes Sobolev IPM for graph-based measures using Orlicz geometric structure.
problem Limitation of Le et al. (2025) framework to L p L^p L p geometry. method Generalizes Sobolev IPM through Orlicz geometric structure, employing convex functions to capture nuanced geometric relationships.
result GSI-M reduces to a simple univariate optimization problem, achieving remarkable computational efficiency.
Bayesian nonparametric LABS model adapts to function smoothness in Besov spaces.
problem Estimating functions with unknown smoothness in Besov spaces.
method Lévy Adaptive B-spline (LABS) regression model with automatic smoothness adaptation.
result LABS posterior contracts around true function in Besov classes at nearly minimax-optimal rates.
Kolmogorov-Arnold Networks improve deep learning adaptivity and can approximate Besov functions optimally.
problem Improving deep learning adaptivity and understanding approximation rates.
method Analyzing Besov norms and using Res-KANs for approximation.
result KANs can optimally approximate Besov functions at the optimal rate.
This paper studies continuum-armed bandits under Besov smoothness conditions and derives minimax rates.
problem Optimizing an unknown function with limited evaluations.
method Studies continuum-armed bandits under Besov smoothness conditions and derives minimax rates.
result Minimax rates over Besov spaces are identical to those over the smallest Hölder space into which Besov spaces embed.
Deep learning has shown high performances in various types of tasks from visual recognition to natural language processing, which indicates superior flexibility and adaptivity of deep learning. To understand this phenomenon theoretically, we develop a new approximation and estimation error analysis of deep learning wit…
Bayesian neural networks achieve optimal posterior contraction rates in Besov spaces with intrinsic dimensionality.
problem High-dimensional structured estimation problems with unknown smoothness levels.
method Sparse Bayesian neural networks with either sparse or continuous shrinkage priors.
result Optimal posterior contraction rates are achieved, adapting to the unknown smoothness level of the true function.
Wavelet-based online learning adapts to noisy Besov spaces with high probability.
problem Minimizing integrated squared error in Besov spaces with noisy observations.
method Adaptive wavelet-based online learning algorithm that dynamically adjusts to gradient noise.
result Achieves minimax-optimal integrated squared error with high probability.
New cyclicity measures defined in weighted Besov spaces, with stability and geometric analysis.
problem Characterizing cyclicity in weighted Besov spaces.
method Defining cyclicity indices based on potential theory and capacity, studying stability under perturbations, and linking zero set structure to cyclicity.
result Novel invariants and conditions for cyclicity in various function spaces.
Study analyzes deep learning's performance on variable exponent Besov space, highlighting adaptivity benefits.
problem Estimation error analysis of deep learning in variable exponent Besov space.
method Analysis of general approximation error and estimation errors of deep learning.
result Adaptivity of deep learning leads to significant improvement in estimation error, especially in high-dimensional spaces.
Toda flow explained as a porous medium equation.
problem Understanding the Toda flow through the lens of porous medium equations.
method Analyzing the geometry and dynamics of the porous medium equation and comparing it to the Toda flow.
result The Toda flow can be represented as a specific porous medium equation, revealing its gradient and Hamiltonian nature.
Proposes tests for comparing high-dimensional manifold samples.
problem Determining if two manifold samples come from the same distribution.
method Integral Probability Metric (IPM) with neural network approximations.
result Tests achieve type-II risk in specific orders of n n n . New framework improves experimental design using integral probability metrics.
problem Challenges in Bayesian Optimal Experimental Design (BOED) with KL divergence.
method Integrates integral probability metrics (IPMs) for stability and flexibility.
result IPM-based designs yield more robust and accurate credible sets.
The paper explores fair predictors in supervised learning using IPMs and Kolmogorov distance.
problem Achieving fairness in supervised learning with significant demographic effects.
method Identifying conditions for SP-fair predictors and using IPMs to measure unfairness.
result Fair predictors can improve accuracy and are computationally efficient.
New bounds use IPMs to improve generalization in machine learning.
problem Improving generalization bounds in machine learning.
method PAC-Bayes bounds with Integral Probability Metrics (IPM).
result Natural interpolation between worst-case and favorable cases.
IPMs struggle with hyperbolic spaces due to polynomially growing barrier parameters.
problem IPMs' efficiency is hindered in hyperbolic spaces.
method Analyzing the barrier parameter growth in hyperbolic and Hadamard spaces.
result The barrier parameter grows polynomially with the domain's diameter in hyperbolic spaces.
Bayesian neural network achieves nearly optimal performance in Besov space.
problem Bayesian neural networks in Besov space.
method Spike-and-slab prior and shrinkage prior for posterior convergence rate.
result The posterior convergence rate is nearly minimax and adaptive to unknown smoothness.
New spatiotemporal Besov process improves CT image reconstruction and other inverse problems.
problem Handling abrupt changes and sharp contrasts in spatiotemporal data.
method Generalized Besov process (STBP) with Q-exponential process for temporal correlation.
result STBP outperforms traditional methods in dynamic reconstruction and inverse problems.
Deep ReLU networks approximate functions in Sobolev and Besov spaces efficiently.
problem Efficiently approximating functions in Sobolev and Besov spaces using deep ReLU networks.
method Novel bit-extraction technique and VC-dimension method for deriving approximation bounds.
result Sharp upper and lower bounds for L p L_p L p -approximation of functions in Sobolev and Besov spaces. Bayesian KANs achieve near-minimax posterior contraction rates in anisotropic Besov spaces.
problem Statistical foundation for Bayesian Kolmogorov-Arnold networks in anisotropic Besov spaces.
method Sparse Bayesian KANs with spike-and-slab priors, hyperprior on model size, and approximation complexity bounds.
result Posterior contraction rates depend on intrinsic anisotropic smoothness and effective dimension of the compositional structure.
Study extends DRO with IPMs, linking robustness to regularization and GANs.
problem Addressing robustness of deep neural networks to adversarial attacks.
method Distributionally Robust Optimization (DRO) with Integral Probability Metrics (IPMs).
result DRO under any IPM corresponds to a family of regularization penalties.
Characterizes when differential forms have weak exterior derivatives based on limiting behavior of integration over simplices.
problem Characterizing differential forms with weak exterior derivatives.
method Uses integration over simplices to characterize the limiting behavior of differential forms.
result Proves a direct analogue of the Bourgain-Brezis-Mironescu characterization for differential forms.
Improved image generation quality using closed-form discriminator guidance in diffusion models.
problem Enhancing the quality of images generated by diffusion models.
method Theoretical framework to analyze GAN discriminator's effect on Langevin sampling, proposing IPM-GAN optimization as smoothed score-matching.
result Closed-form kernel-based discriminator guidance improves metrics like CLIP-FID and KID.
Deep ReLU networks can efficiently approximate Sobolev and Besov functions.
problem Approximating functions in Sobolev and Besov spaces using deep neural networks.
method Used deep ReLU neural networks with varied width and depth to approximate functions in Sobolev and Besov spaces.
result Generalized the approximation rate to hold under the Sobolev embedding condition.
ConvResNets approximate Besov functions and classify on low-dimensional manifolds.
problem Lack of statistical theories for deep learning on high-dimensional data.
method Exploits low-dimensional geometric structures of real-world data sets using ConvResNets.
result ConvResNets can approximate Besov functions and learn classifiers with optimal excess risk.
Adaptive algorithm minimizes online prediction errors for irregular data.
problem Online adversarial regression with highly irregular prediction rules.
method Adaptive wavelet-based algorithm for Besov space regression.
result Minimax-optimal regret bounds in adversarial settings.
New error bounds for GANs with nonlinear objective functions derived.
problem Statistical consistency of GANs with nonlinear objective functions.
method Derivation of statistical error bounds for ( f , Γ ) (f,Γ) ( f , Γ ) -GANs using Rademacher complexity. result Proves the statistical consistency of ( f , Γ ) (f,Γ) ( f , Γ ) -GANs. A nonparametric two-sample test using a parametric integral probability metric
problem Detecting distributional differences between two independent samples
method Propose a new two-sample test statistic based on a newly introduced integral probability metric (IPM)
result Establish theoretical guarantees for the associated two-sample testing procedure
In this paper we propose a function space approach to Representation Learning and the analysis of the representation layers in deep learning architectures. We show how to compute a weak-type Besov smoothness index that quantifies the geometry of the clustering in the feature space. This approach was already applied suc…
Study on distributed nonparametric function estimation with optimal rate and cost of adaptation.
problem Optimal rate of convergence and cost of adaptation in distributed nonparametric function estimation.
method Distributed minimax estimation and adaptive estimation under communication constraints for Gaussian sequence model and white noise model.
result Established minimax rate of convergence and exact communication cost for adaptation.
The paper proposes a new method for covariate balancing using IPM to improve causal inference.
problem Covariate imbalance in causal inference weighting methods, especially when models are not correctly specified.
method The integral probability metric (IPM) is used to determine optimal weights for treated and control groups.
result The proposed method can be consistent without specifying either the propensity score or outcome regression model.
Extends Onsager's conjecture to Besov spaces on manifolds with boundary.
problem Proving Onsager's conjecture on Riemannian manifolds with boundary.
method Constructing Hodge-Neumann heat kernel, obtaining off-diagonal decay and local Bernstein estimates.
result Extends Onsager's conjecture to Besov spaces B ^ 3 , V 1 3 \widehat{B}_{3,V}^{\frac{1}{3}} B 3 , V 3 1 . Develops a new divergence framework that combines f f f -divergences and IPMs.
problem Comparing distributions that are not absolutely continuous.
method Introduces ( f , Γ ) (f,Γ) ( f , Γ ) -divergences as a two-stage mass-redistribution/mass-transport process. result Improves estimation, learning, and uncertainty quantification in GANs for heavy-tailed distributions.
This paper is concerned with sequential filtering based stochastic optimization (FSO) approaches that leverage a probabilistic perspective to implement the incremental proximity method (IPM). The present FSO methods are derived based on the Kalman filter (KF) and the extended KF (EKF). In contrast with typical methods …
Deep learning has exhibited superior performance for various tasks, especially for high-dimensional datasets, such as images. To understand this property, we investigate the approximation and estimation ability of deep learning on anisotropic Besov spaces. The anisotropic Besov space is characterized by direction-depen…
Deep ReLU networks can approximate and learn smooth functions efficiently.
problem Efficiently approximating and learning smooth functions using deep ReLU neural networks.
method Extending recent results to anisotropic and mixed smooth function classes, establishing approximation rates.
result Deep ReLU networks achieve minimax optimal rates up to logarithmic factors for various smooth function classes.
Paper analyzes sample complexity for offline RL with deep ReLU networks.
problem Theoretical analysis of sample complexity for offline RL with deep ReLU networks.
method Establishes sample complexity for offline RL with deep ReLU networks, considering Besov dynamic closure and correlated structure.
result First theoretical characterization of sample complexity for offline RL with deep neural network function approximation.
Generative Adversarial Networks (GANs) are powerful models for learning complex distributions. Stable training of GANs has been addressed in many recent works which explore different metrics between distributions. In this paper we introduce Fisher GAN which fits within the Integral Probability Metrics (IPM) framework f…
New divergences improve estimation and GAN training performance.
problem Improving estimation and training in machine learning models.
method Function-space regularized Rényi divergences.
result New divergences reduce variance and improve training performance.
TMLE improves IPM estimation for ecological population dynamics.
problem Estimating key demographic properties from IPM data.
method Targeted Maximum Likelihood Estimation (TMLE) for IPMs.
result Robust and efficient estimators for IPM properties.
Deep learning models can adaptively estimate functions with varying smoothness using regularization.
problem Estimating functions with heterogeneous smoothness in Besov or BV classes.
method Introduced a Parallel NN variant of deep ReLU networks with ℓ 2 \ell_2 ℓ 2 regularization equivalent to promoting ℓ p \ell_p ℓ p -sparsity. result Achieves minimax rates for Besov and BV classes with exponentially closer performance as depth increases.