A boosting method improves nonparametric density estimation without smoothing assumptions.
problem Overfitting in nonparametric data fitting.
method Introduces a boosting algorithm for univariate nonparametric maximum likelihood estimation.
result Demonstrates the effectiveness of the boosting approach through simulations and real data experiments.
We propose to use nonparametric Bernstein copulas as bivariate pair-copulas in high-dimensional vine models. The resulting smooth and nonparametric vine copulas completely obviate the error-prone need for choosing the pair-copulas from parametric copula families. By means of a simulation study and an empirical analysis…
Optimal nonparametric regression estimator adapts to unknown smoothness.
problem Nonparametric regression with unknown smoothness.
method Constructs an interpolating estimator that adapts to unknown smoothness.
result Minimax optimal rates achieved on Hölder classes.
A new clustering method uses nonparametric smoothing to estimate cluster membership functions.
problem Clustering with flexible, nonparametric estimation.
method Nonparametric smoothing to estimate cluster membership functions without explicit modelling assumptions.
result The method automatically determines the number of clusters and level of flexibility.
A debiasing method improves nonparametric regression's statistical properties.
problem Lack of theoretical guarantees for modern nonparametric regression methods.
method Model-free debiasing method incorporating a correction term.
result Debiased estimator satisfies pointwise and uniform risk convergence, asymptotic normality.
Bayesian methods estimate regression functions on submanifolds using graph Laplacian eigenbasis.
problem Estimating regression functions on unknown smooth submanifolds.
method Random geometric graph structure, Bayesian priors based on random basis expansion in graph Laplacian eigenbasis.
result Posterior contraction rates are minimax optimal for any positive smoothness index.
We propose a flexible nonparametric regression method for ultrahigh-dimensional data. As a first step, we propose a fast screening method based on the favored smoothing bandwidth of the marginal local constant regression. Then, an iterative procedure is developed to recover both the important covariates and the regress…
New algorithm adapts to unknown demand smoothness for dynamic pricing.
problem Dynamic pricing with unknown Hölder smoothness of demand function.
method Self-similarity condition and adaptive algorithm.
result Adaptive algorithm achieves minimax optimal regret without prior knowledge of smoothness.
Develops nonparametric regression for non-smooth functions using fractional Laplacian.
problem Non-smooth regression functions in high dimensions.
method Fractional Laplacian eigenmaps for L2-fractional Sobolev spaces. result Upper bound on estimation error of $n^{- rac{2s}{2s+d}}$.
Study nonparametric covariance function estimation for noisy data.
problem Estimating covariance function from discrete noisy data in high dimensions.
method Adaptive learning-based estimators, including deep learning.
result Established oracle inequality and convergence rates for deep learning estimators.
Bayesian nonparametric LABS model adapts to function smoothness in Besov spaces.
problem Estimating functions with unknown smoothness in Besov spaces.
method Lévy Adaptive B-spline (LABS) regression model with automatic smoothness adaptation.
result LABS posterior contracts around true function in Besov classes at nearly minimax-optimal rates.
New estimator achieves minimax optimal risk in transfer learning.
problem Nonparametric regression with transfer learning.
method Confidence thresholding estimator and data-driven adaptive algorithm.
result Adaptive algorithm achieves minimax risk up to a logarithmic factor.
New method for causal inference with observed covariates improves learning rates.
problem Causal inference with observed covariates in nonparametric instrumental variable regression.
method Introduces novel Fourier measure for partial smoothing and adapts kernel lengthscales for anisotropic smoothness.
result Upper and lower learning rates for KIV-O show interpolation between NPIV and NPR rates.
GES algorithm improves consistency for nonparametric DAG models.
problem Consistent estimation of nonparametric DAG models.
method Greedy equivalence search with new consistency proof for nonparametric families.
result Consistency of GES for general nonparametric DAG models with smooth factorization.
Proposes a nonparametric model for dynamic team rankings.
problem Dynamic ranking of distinct teams over time.
method Kernel smoothing for nonparametric estimation in sparse settings.
result Time-varying oracle bounds for estimation and excess risk.
Paper introduces a new histogram estimator for nonparametric density estimation that improves performance.
problem Smoothness-based nonparametric density estimators are not optimal for all types of data.
method Incorporates a multi-view latent variable model into histogram-style estimators.
result A new histogram estimator converges faster to multi-view models in L1 error. Study nonparametric contextual bandits with batched updates, achieving optimal regret.
problem Optimal regret in nonparametric contextual bandits with batch constraints.
method Dynamic binning of covariate space, optimal regret achieved.
result Achieves optimal regret (up to logarithmic factors) for nonparametric contextual bandits.
The paper introduces Shapley curves for measuring variable importance in nonparametric settings.
problem Limited statistical understanding of Shapley values as variable importance measures.
method Introduces Shapley curves based on conditional expectation and covariate distribution; derives convergence rates and normality; proposes a novel bootstrap procedure.
result Validates theoretical findings with numerical studies and analyzes vehicle prices determinants.
Bayesian nonparametrics improves data-driven risk optimization under distributional uncertainty.
problem Improving out-of-sample performance in machine learning models due to distributional uncertainty.
method Combining Bayesian nonparametric theory and decision-theoretic preferences to propose a robust optimization criterion.
result The proposed robust optimization procedure provides favorable statistical guarantees and tractable approximations.
Optimal rates for shallow ReLU networks in nonparametric regression.
problem Approximating smooth and non-smooth functions with shallow ReLU networks.
method Analysis of shallow ReLUk neural networks, using variation norms and deep learning theory. result Optimal approximation rates for shallow ReLU networks in nonparametric regression.
New method efficiently interpolates nonparametric density estimators.
problem Efficient evaluation of nonparametric density estimators.
method Piecewise multivariate polynomial interpolation scheme.
result New estimator with low space requirements and efficient querying.
Study on discrepancy principle for learning algorithms in nonparametric regression.
problem Determining optimal iteration number in nonparametric regression with unknown optimal iteration.
method Investigates discrepancy principle and modified principles for kernelized spectral filters, using deviation inequalities and change-of-norm arguments.
result Classical discrepancy principle is adaptive for slow rates, while modified principles are adaptive for faster rates.
Nonparametric density deconvolution and denoising using simulation-based inference
problem Learning latent signals and their distributions in the presence of measurement noise
method Convolutional maximum mean discrepancy (convMMD) loss and likelihood-free framework
result Learn a latent generative model matching observed data distribution
Deep neural networks can learn smooth functions without parameters.
problem Learning smooth functions from shallow ReLU neural networks.
method Using over-parameterized shallow ReLU neural networks with norm constraints.
result Least squares estimators based on shallow neural networks are minimax optimal.
Paper tackles adversarial attacks on nonparametric regression models.
problem Vulnerability of machine learning models to adversarial attacks in nonparametric regression.
method Establishes minimax rate and proposes adaptive estimators for robust nonparametric regression under adversarial Lq-risks. result Achieves minimax optimality and provides adaptive estimators for robust nonparametric regression.
New active learning algorithm adapts to data without strict assumptions.
problem Efficiently label data with expensive labeling costs.
method Nonparametric adaptive active learning under local smoothness condition.
result Achieves minimax rate of convergence, performs almost as well as best non-adaptive algorithms.
New algorithm improves matrix estimation with one-sided covariates.
problem Estimating matrix means with unobserved row covariates.
method Proposes an algorithm for nonparametric matrix estimation with observed column covariates.
result Achieves minimax optimal nonparametric rate in moderately proportioned matrices.
Estimates nonparametric densities from mixed samples.
problem Unmixing convex combinations of nonparametric densities from observed groups.
method Proposes an estimator using topic modeling and U-statistics.
result Rate-optimal estimator for nonparametric density estimation.
This work addresses various open questions in the theory of active learning for nonparametric classification. Our contributions are both statistical and algorithmic: -We establish new minimax-rates for active learning under common \textit{noise conditions}. These rates display interesting transitions -- due to the inte…
Modeling dynamical systems with ordinary differential equations implies a mechanistic view of the process underlying the dynamics. However in many cases, this knowledge is not available. To overcome this issue, we introduce a general framework for nonparametric ODE models using penalized regression in Reproducing Kerne…
Nonparametric neural-network estimation of current-status data
problem Estimation of conditional cumulative distribution function with current-status data
method Neural-network sieve maximum likelihood estimator
result Explicit convergence rate for Hölder smoothness
BaNk-UCB tackles batched nonparametric bandits with k-NN regression and UCB.
problem Sequential decision-making with limited online feedback in domains like medicine and marketing.
method Combines k-NN regression with UCB principle for fully nonparametric, adaptive, and simple implementation.
result Near-optimal regret guarantees under Lipschitz smoothness and margin assumptions, with minimax-optimal rates.
New bounds for agnostic learning with average smoothness.
problem Distribution-free nonparametric regression with average smoothness.
method Distribution-free uniform convergence bounds and agnostic learning algorithm.
result Distribution-free uniform convergence bounds for average-smoothness classes in the agnostic setting.
Adversarial online nonparametric regression achieves optimal rates with locally adaptive learning.
problem Adversarial online nonparametric regression with general convex losses.
method Parameter-free learning algorithm leveraging chaining trees to compete against H{ö}lder functions, dynamically tracking and adapting to local smoothness variations.
result First computationally efficient algorithm with locally adaptive optimal rates for online regression in an adversarial setting.
Paper develops methods to estimate derivative of dose-response curve for continuous treatments.
problem Estimating the derivative of the dose-response curve for continuous treatments.
method Doubly robust (DR) inference method using kernel smoothing, bias-corrected IPW and DR estimators.
result Proposes novel bias-corrected IPW and DR estimators for continuous treatments.
Estimates personalized treatment response curves using covariates.
problem Flexible estimation of personalized treatment response curves.
method Sieve based nonparametric estimator of smoothed regimen-response curve function.
result Asymptotic linearity and undersmoothing criteria for efficient estimation.
Study non-stationary distributions, proving risk bounds for density estimation.
problem Estimating current distribution under gradual changes.
method Proves tight minimax risk bounds for nonparametric density estimation under drift.
result Generalizes previous results on agnostic learning under drift.
Deep neural networks with specific parameter sets can approximate smooth functions efficiently.
problem Approximating smooth functions with deep neural networks.
method Deep neural networks with ReLU activation and specific parameter sets {0,±21,±1,2} are used to approximate Cβ-smooth functions. result The constructed networks can approximate Cβ-smooth functions with parameters {0,±21,±1,2} efficiently, achieving the same convergence rate as sparse networks with parameters in [−1,1]. Gradient-free optimization for additive models achieves optimal error.
problem Optimizing noisy functions with zero-order information.
method Proposed a randomized gradient estimator for gradient-free optimization.
result Achieves minimax optimal error of order dT−(β−1)/β. Transformers can efficiently approximate nonparametric regression with minimal parameters and sequences.
problem Efficiently approximating nonparametric regression functions with transformers.
method Kernel-weighted polynomial basis and gradient descent.
result Achieves minimax optimal rate of convergence with fewer parameters and sequences.
Study shows rates for Laplacian-eigenmap methods in nonparametric regression.
problem Minimizing error in nonparametric regression using Laplacian-eigenmap.
method Adaptive and non-adaptive minimax rates using Sobolev space constraints.
result Extends minimax rates to various weighted Laplacian matrices.
Improves RL efficiency with nonparametric methods.
problem High sample complexity in policy gradient algorithms.
method Nonparametric regression and density estimation for value function estimation.
result Better sample efficiency compared to existing methods.
Consider a nonparametric contextual multi-arm bandit problem where each arm a∈[K] is associated to a nonparametric reward function fa:[0,1]→R mapping from contexts to the expected reward. Suppose that there is a large set of arms, yet there is a simple but unknown structure amongst the arm reward…
Improved model for non-smooth signals with complex spectra.
problem Current models struggle with non-smooth signals and complex spectral structures.
method CGPCM and RGPCM models with causality and Bayesian nonparametric interpretations, improved variational inference.
result Proposed models show better performance on synthetic and real-world data.
We consider the problem of comparing probability densities between two groups. A new probabilistic tensor product smoothing spline framework is developed to model the joint density of two variables. Under such a framework, the probability density comparison is equivalent to testing the presence/absence of interactions.…
Signal processing tasks as fundamental as sampling, reconstruction, minimum mean-square error interpolation and prediction can be viewed under the prism of reproducing kernel Hilbert spaces. Endowing this vantage point with contemporary advances in sparsity-aware modeling and processing, promotes the nonparametric basi…
SPARKLE handles high-dimensional covariates for online decision-making.
problem Complex reward-covariate relationships in high-dimensional settings.
method SPARKLE uses a sparse additive reward model with doubly penalized estimator and adaptive screening.
result SPARKLE achieves sublinear regret bound logarithmic in covariate dimensionality.
Nonparametric tests via kernel embedding of distributions have witnessed a great deal of practical successes in recent years. However, statistical properties of these tests are largely unknown beyond consistency against a fixed alternative. To fill in this void, we study here the asymptotic properties of goodness-of-fi…