Optimal nonparametric regression estimator adapts to unknown smoothness.
problem Nonparametric regression with unknown smoothness.
method Constructs an interpolating estimator that adapts to unknown smoothness.
result Minimax optimal rates achieved on Hölder classes.
Study uniform consistency in nonparametric mixture models and mixed regression.
problem Uniform consistency in nonparametric mixture models and mixed regression models.
method Construct uniformly consistent estimators under general conditions, develop novel technical tools.
result Prove uniform consistency results for nonparametric mixtures and mixed regression models.
A debiasing method improves nonparametric regression's statistical properties.
problem Lack of theoretical guarantees for modern nonparametric regression methods.
method Model-free debiasing method incorporating a correction term.
result Debiased estimator satisfies pointwise and uniform risk convergence, asymptotic normality.
Study on online nonparametric regression using Sobolev kernel methods.
problem Adversarial nonparametric regression in high dimensions.
method Online kernelized ridge regression with Sobolev kernel analysis.
result Upper bounds on regret for Sobolev space classes, revealing optimality in certain cases.
Improved nonparametric regression with debiasing for root-n consistency.
problem Challenges in achieving root-n consistency and normal distribution for nonparametric estimators.
method Debiasing technique by adding a correction term to nonparametric estimators.
result Achieves root-n consistency and asymptotic normality.
Transformers can efficiently approximate nonparametric regression with minimal parameters and sequences.
problem Efficiently approximating nonparametric regression functions with transformers.
method Kernel-weighted polynomial basis and gradient descent.
result Achieves minimax optimal rate of convergence with fewer parameters and sequences.
A novel deep bootstrap framework for nonparametric regression using conditional diffusion models.
problem Nonparametric regression with efficient sampling and accurate estimation.
method Conditional diffusion model for learning conditional distributions, integrating sampling and regression into a unified generative framework.
result Established optimal convergence rates in the Wasserstein distance and convergence guarantees for the bootstrap procedure.
Adversarial online nonparametric regression achieves optimal rates with locally adaptive learning.
problem Adversarial online nonparametric regression with general convex losses.
method Parameter-free learning algorithm leveraging chaining trees to compete against H{ö}lder functions, dynamically tracking and adapting to local smoothness variations.
result First computationally efficient algorithm with locally adaptive optimal rates for online regression in an adversarial setting.
Procedure groups nonparametric regression curves automatically.
problem Determining groups of nonparametric regression curves when curves are numerous.
method Automatic selection of group number through testing procedure.
result Groups of nonparametric regression curves exist in tunnel geometry.
Method constructs nonparametric prediction intervals with finite-sample guarantees.
problem Nonparametric instrumental variable regression with finite-sample coverage.
method Conformal inference framework applied to NPIV, combining with various estimators.
result Distribution-free, finite-sample coverage over chosen IV shifts.
Study shows rates for Laplacian-eigenmap methods in nonparametric regression.
problem Minimizing error in nonparametric regression using Laplacian-eigenmap.
method Adaptive and non-adaptive minimax rates using Sobolev space constraints.
result Extends minimax rates to various weighted Laplacian matrices.
Develops abstention procedure for nonparametric regression via variance testing.
problem Prediction with selective abstention in error-critical machine learning.
method Nonparametric heteroskedastic regression via testing hypothesis on conditional variance.
result Non-asymptotic risk bounds and convergence regimes for the estimator.
New similarity measure for covariate shift improves nonparametric regression rates.
problem Improving nonparametric regression under covariate shift.
method Introducing a new similarity measure based on probability ratios.
result Shows a sharper rate of convergence compared to transfer exponent.
Neural networks improve nonparametric regression with measurement errors.
problem Nonparametric regression with measurement errors.
method Proposes a neural network design using FNN, normalizing flow, and inference network.
result Neural network approach is more flexible and superior or comparable to classical methods.
NTK neural networks are robust to adversarial attacks in nonparametric regression.
problem Adversarial robustness of neural networks in nonparametric regression.
method Gradient flow with early stopping for NTK neural networks, proving robustness in Sobolev spaces.
result NTK neural networks achieve optimal adversarial robustness rates in Sobolev spaces.
Develops nonparametric regression for non-smooth functions using fractional Laplacian.
problem Non-smooth regression functions in high dimensions.
method Fractional Laplacian eigenmaps for L2-fractional Sobolev spaces. result Upper bound on estimation error of $n^{- rac{2s}{2s+d}}$.
Localized transfer learning improves nonparametric regression performance.
problem Improving nonparametric regression performance on target tasks.
method Localized transfer learning framework that models heterogeneity and partition covariate space into cells.
result Sharp minimax rates show local transfer mitigates the curse of dimensionality.
Improves MARS for nonparametric multivariate regression with dimension reduction.
problem High number of basis functions in MARS for high-order interactions.
method Linear combinations of covariates for dimension reduction, facilitating gradient calculation and eigen-analysis for estimation.
result Asymptotic theory and numerical studies show improved performance over MARS.
Bayesian methods estimate regression functions on submanifolds using graph Laplacian eigenbasis.
problem Estimating regression functions on unknown smooth submanifolds.
method Random geometric graph structure, Bayesian priors based on random basis expansion in graph Laplacian eigenbasis.
result Posterior contraction rates are minimax optimal for any positive smoothness index.
Paper extends nonparametric regression bounds for dependent β-mixing samples.
problem Analyzing error in nonparametric regression with dependent data.
method Extends uniform deviation inequalities from independent to dependent β-mixing samples. result Derives generalization bounds for nonparametric regression with dependent data.
New approach to adaptively select bandwidths in nonparametric regression.
problem Adaptive bandwidth selection in nonparametric regression.
method Inspired by ℓ2-norms of interval projections, introduces a new bandwidth selection procedure. result Obtains non-asymptotic risk bounds for local polynomial regression methods that adapt to local Hölder exponent.
Paper develops a privacy-preserving nonparametric regression method.
problem Nonparametric regression with local differential privacy constraints.
method Privatised discretisation and Laplace noise applied to feature vectors and responses.
result Strongly universally consistent estimator for regression and classification.
Optimal rates for shallow ReLU networks in nonparametric regression.
problem Approximating smooth and non-smooth functions with shallow ReLU networks.
method Analysis of shallow ReLUk neural networks, using variation norms and deep learning theory. result Optimal approximation rates for shallow ReLU networks in nonparametric regression.
New estimator achieves minimax optimal risk in transfer learning.
problem Nonparametric regression with transfer learning.
method Confidence thresholding estimator and data-driven adaptive algorithm.
result Adaptive algorithm achieves minimax risk up to a logarithmic factor.
Fast nonparametric conditional independence testing via two-stage regression
problem Fast nonparametric conditional independence testing
method BLITZ (Broad-to-Local Independence Testing via residualiZation)
result Better null calibration than fast kernel, random-feature, and regression-based competitors
Gradient descent trains neural networks to match kernel regression's sharp generalization rate.
problem Training over-parameterized neural networks for nonparametric regression.
method Gradient descent with early stopping on over-parameterized two-layer neural networks.
result Trained neural networks achieve sharp generalization rate of O(εn2). We propose nonparametric methods for individual calibration in regression models.
problem Uncertainty quantification and individual calibration for regression models.
method Nonparametric methods agnostic of the underlying model, combining nonparametric and covering number arguments.
result Established matching upper and lower bounds for calibration error.
Transforms conditional density estimation into a nonparametric regression problem.
problem Conditional density estimation in high dimensions.
method Introduces auxiliary samples to transform into nonparametric regression.
result Estimator converges to true conditional density in data limit.
BaNk-UCB tackles batched nonparametric bandits with k-NN regression and UCB.
problem Sequential decision-making with limited online feedback in domains like medicine and marketing.
method Combines k-NN regression with UCB principle for fully nonparametric, adaptive, and simple implementation.
result Near-optimal regret guarantees under Lipschitz smoothness and margin assumptions, with minimax-optimal rates.
Modeling dynamical systems with ordinary differential equations implies a mechanistic view of the process underlying the dynamics. However in many cases, this knowledge is not available. To overcome this issue, we introduce a general framework for nonparametric ODE models using penalized regression in Reproducing Kerne…
The paper introduces a method for learning nonparametric Volterra kernels using Gaussian processes.
problem Learning nonparametric nonlinear operators from data.
method NVKM model using Volterra series and Gaussian processes for unobserved and observed input functions.
result The NVKM model can perform both single and multiple output regression and system identification.
PAN+SR tackles scalable symbolic regression for large p datasets.
problem Symbolic regression struggles with large number of input variables and measurement error.
method Combines ab initio nonparametric variable selection with SR to pre-screen and reduce search complexity.
result PAN+SR consistently enhances 19 SR methods' performance on challenging datasets.
Proposes a non-crossing deep neural network quantile regression method.
problem Quantile crossing in nonparametric quantile regression.
method Non-crossing constraints via rectified linear unit penalty function.
result Established non-asymptotic upper bounds for excess risk.
This paper analyzes how machine learning models resist adversarial attacks in nonparametric regression.
problem Adversarial attacks on machine learning models in nonparametric regression.
method Theoretical analysis of minimax rates of convergence under adversarial sup-norm.
result The minimax rate under adversarial attacks is the sum of two terms: standard rate and deviation of true function.
A new method for nonparametric regression using mesh-based solutions.
problem Estimating regression functions non-parametrically with computational tractability.
method Mesh-based approximate solution (MBS) for penalized regression problems.
result MBS transforms NPR to a discrete convex minimization problem, making it computationally feasible.
Paper tackles adversarial attacks on nonparametric regression models.
problem Vulnerability of machine learning models to adversarial attacks in nonparametric regression.
method Establishes minimax rate and proposes adaptive estimators for robust nonparametric regression under adversarial Lq-risks. result Achieves minimax optimality and provides adaptive estimators for robust nonparametric regression.
We propose an approach to multivariate nonparametric regression that generalizes reduced rank regression for linear models. An additive model is estimated for each dimension of a q-dimensional response, with a shared p-dimensional predictor variable. To control the complexity of the model, we employ a functional fo…
Develops a new nonparametric trace regression model for high-dimensional data.
problem Violation of known functional form and global low-rank structure assumptions in trace regression.
method Structured sign series representations for nonparametric trace regression models.
result Establishes excess risk bounds and sample complexities for the proposed model.
Additive nonparametric regression models provide an attractive tool for variable selection in high dimensions when the relationship between the response and predictors is complex. They offer greater flexibility compared to parametric non-linear regression models and better interpretability and scalability than the non-…
Paper derives convergence rates for NPMLE in Hellinger distance using deep neural networks.
problem Difficulty in proving convergence of excess risk in nonparametric logistic regression.
method Unified approach for analyzing NPMLE, deriving convergence rates in Hellinger distance.
result Derives nearly optimal convergence rates for NPMLE with deep neural networks.
Nonparametric extension of tensor regression is proposed. Nonlinearity in a high-dimensional tensor space is broken into simple local functions by incorporating low-rank tensor decomposition. Compared to naive nonparametric approaches, our formulation considerably improves the convergence rate of estimation while maint…
A new graph-based approach for estimating complex data with manifold structure.
problem Regression of large-scale, complex data with underlying geometric structure and noises.
method Constructing a skeleton graph to capture geometric structure, defining metrics, and applying nonparametric regression.
result Statistical guarantees and effectiveness demonstrated through simulations and real data examples.
Deep-HGP uses Bayesian nonparametric approach for complex data regression.
problem Complex data regression with compositional structures.
method Deep Gaussian processes with a squared-exponential kernel, data-driven lengthscale parameters.
result Posterior distribution optimally recovers unknown true regression curve in terms of quadratic loss.
Two EM algorithms estimate prior distributions in mixture of linear regressions.
problem Estimating prior distributions in mixture of linear regressions.
method Two EM algorithms: one for continuous priors, one for discrete priors.
result Both algorithms accurately estimate prior distributions and the number of clusters.
Improves regression efficiency by separating material and immaterial parts of responses.
problem Improving estimation efficiency in nonlinear multivariate regressions.
method Kernel envelope (KENV) estimator for nonparametric response envelopes in reproducing kernel Hilbert space.
result KENV achieves lower in-sample prediction risk than kernel ridge regression in non-trivial immaterial components.
Modal regression estimates the local modes of the distribution of Y given X=x, instead of the mean, as in the usual regression sense, and can hence reveal important structure missed by usual regression methods. We study a simple nonparametric method for modal regression, based on a kernel density estimate (KDE) of …
Proposes a lasso variant of MARS for nonparametric regression.
problem Nonparametric regression with MARS.
method Least squares estimation over convex function combinations with a complexity constraint.
result Achieves logarithmic convergence rate in dimensionality.
The study establishes risk bounds for distributional regression estimators.
problem Estimating distributional regression models with nonparametric methods.
method Theoretical bounds for CRPS and MSE are derived for convex and non-convex constraints.
result Theoretical risk bounds are validated through experiments on simulated and real data.