Study uniform consistency in nonparametric mixture models and mixed regression.
problem Uniform consistency in nonparametric mixture models and mixed regression models.
method Construct uniformly consistent estimators under general conditions, develop novel technical tools.
result Prove uniform consistency results for nonparametric mixtures and mixed regression models.
Optimal nonparametric regression estimator adapts to unknown smoothness.
problem Nonparametric regression with unknown smoothness.
method Constructs an interpolating estimator that adapts to unknown smoothness.
result Minimax optimal rates achieved on Hölder classes.
A novel deep bootstrap framework for nonparametric regression using conditional diffusion models.
problem Nonparametric regression with efficient sampling and accurate estimation.
method Conditional diffusion model for learning conditional distributions, integrating sampling and regression into a unified generative framework.
result Established optimal convergence rates in the Wasserstein distance and convergence guarantees for the bootstrap procedure.
Paper extends nonparametric regression bounds for dependent β-mixing samples.
problem Analyzing error in nonparametric regression with dependent data.
method Extends uniform deviation inequalities from independent to dependent β-mixing samples. result Derives generalization bounds for nonparametric regression with dependent data.
Adversarial online nonparametric regression achieves optimal rates with locally adaptive learning.
problem Adversarial online nonparametric regression with general convex losses.
method Parameter-free learning algorithm leveraging chaining trees to compete against H{ö}lder functions, dynamically tracking and adapting to local smoothness variations.
result First computationally efficient algorithm with locally adaptive optimal rates for online regression in an adversarial setting.
A debiasing method improves nonparametric regression's statistical properties.
problem Lack of theoretical guarantees for modern nonparametric regression methods.
method Model-free debiasing method incorporating a correction term.
result Debiased estimator satisfies pointwise and uniform risk convergence, asymptotic normality.
Study on online nonparametric regression using Sobolev kernel methods.
problem Adversarial nonparametric regression in high dimensions.
method Online kernelized ridge regression with Sobolev kernel analysis.
result Upper bounds on regret for Sobolev space classes, revealing optimality in certain cases.
Gradient descent trains neural networks to match kernel regression's sharp generalization rate.
problem Training over-parameterized neural networks for nonparametric regression.
method Gradient descent with early stopping on over-parameterized two-layer neural networks.
result Trained neural networks achieve sharp generalization rate of O(εn2). Improved nonparametric regression with debiasing for root-n consistency.
problem Challenges in achieving root-n consistency and normal distribution for nonparametric estimators.
method Debiasing technique by adding a correction term to nonparametric estimators.
result Achieves root-n consistency and asymptotic normality.
Modeling dynamical systems with ordinary differential equations implies a mechanistic view of the process underlying the dynamics. However in many cases, this knowledge is not available. To overcome this issue, we introduce a general framework for nonparametric ODE models using penalized regression in Reproducing Kerne…
The study establishes risk bounds for distributional regression estimators.
problem Estimating distributional regression models with nonparametric methods.
method Theoretical bounds for CRPS and MSE are derived for convex and non-convex constraints.
result Theoretical risk bounds are validated through experiments on simulated and real data.
We propose an approach to multivariate nonparametric regression that generalizes reduced rank regression for linear models. An additive model is estimated for each dimension of a q-dimensional response, with a shared p-dimensional predictor variable. To control the complexity of the model, we employ a functional fo…
Normalized compound random measures are flexible nonparametric priors for related distributions. We consider building general nonparametric regression models using normalized compound random measure mixture models. Posterior inference is made using a novel pseudo-marginal Metropolis-Hastings sampler for normalized comp…
Transformers can efficiently approximate nonparametric regression with minimal parameters and sequences.
problem Efficiently approximating nonparametric regression functions with transformers.
method Kernel-weighted polynomial basis and gradient descent.
result Achieves minimax optimal rate of convergence with fewer parameters and sequences.
Deep neural networks with adversarial training achieve sup-norm convergence for nonparametric regression.
problem Achieving sup-norm convergence for deep neural network estimators in nonparametric regression.
method Developed an adversarial training scheme to address the sup-norm convergence issue.
result Deep neural network estimators achieve optimal sup-norm convergence with the proposed adversarial training.
Procedure groups nonparametric regression curves automatically.
problem Determining groups of nonparametric regression curves when curves are numerous.
method Automatic selection of group number through testing procedure.
result Groups of nonparametric regression curves exist in tunnel geometry.
A new graph-based approach for estimating complex data with manifold structure.
problem Regression of large-scale, complex data with underlying geometric structure and noises.
method Constructing a skeleton graph to capture geometric structure, defining metrics, and applying nonparametric regression.
result Statistical guarantees and effectiveness demonstrated through simulations and real data examples.
Inference in popular nonparametric Bayesian models typically relies on sampling or other approximations. This paper presents a general methodology for constructing novel tractable nonparametric Bayesian methods by applying the kernel trick to inference in a parametric Bayesian model. For example, Gaussian process regre…
Method constructs nonparametric prediction intervals with finite-sample guarantees.
problem Nonparametric instrumental variable regression with finite-sample coverage.
method Conformal inference framework applied to NPIV, combining with various estimators.
result Distribution-free, finite-sample coverage over chosen IV shifts.
Study shows rates for Laplacian-eigenmap methods in nonparametric regression.
problem Minimizing error in nonparametric regression using Laplacian-eigenmap.
method Adaptive and non-adaptive minimax rates using Sobolev space constraints.
result Extends minimax rates to various weighted Laplacian matrices.
Bayesian method with Gaussian process priors achieves optimal convergence rates for regression function and its derivatives.
problem Estimating the regression function and its derivatives in nonparametric regression.
method Bayesian approach with Gaussian process priors, focusing on convergence rates and plug-in property.
result Equivalence of convergence rates of posterior distributions and Bayes estimators for regression function and its derivatives.
Develops a new nonparametric trace regression model for high-dimensional data.
problem Violation of known functional form and global low-rank structure assumptions in trace regression.
method Structured sign series representations for nonparametric trace regression models.
result Establishes excess risk bounds and sample complexities for the proposed model.
The paper provides bounds for regression schemes using nonstationary training samples.
problem Developing confidence intervals for nonparametric regression with nonstationary data.
method The approach involves Rademacher and Vapnik-Chervonenkis theories to analyze the cost and optimality of regression schemes.
result The paper establishes nonasymptotic bounds for regression schemes and optimality in L2-distance. Develops abstention procedure for nonparametric regression via variance testing.
problem Prediction with selective abstention in error-critical machine learning.
method Nonparametric heteroskedastic regression via testing hypothesis on conditional variance.
result Non-asymptotic risk bounds and convergence regimes for the estimator.
The paper analyzes covariate shift in nonparametric regression with Markovian data.
problem Covariate shift in regression problems with Markovian data.
method Extension of nonparametric convergence rates to Markovian dependence structures, using Hölder smoothness assumptions and similarity measures.
result Precise convergence rates for Nadaraya-Watson kernel estimators under specific Markovian conditions.
New similarity measure for covariate shift improves nonparametric regression rates.
problem Improving nonparametric regression under covariate shift.
method Introducing a new similarity measure based on probability ratios.
result Shows a sharper rate of convergence compared to transfer exponent.
New framework for distributed nonparametric estimation under slow communication.
problem Efficiently estimate nonparametric models across multiple nodes with limited communication.
method Developed a general framework for nonparametric estimation under communication constraints.
result Derived minimax lower and upper bounds for various models.
Neural networks improve nonparametric regression with measurement errors.
problem Nonparametric regression with measurement errors.
method Proposes a neural network design using FNN, normalizing flow, and inference network.
result Neural network approach is more flexible and superior or comparable to classical methods.
NTK neural networks are robust to adversarial attacks in nonparametric regression.
problem Adversarial robustness of neural networks in nonparametric regression.
method Gradient flow with early stopping for NTK neural networks, proving robustness in Sobolev spaces.
result NTK neural networks achieve optimal adversarial robustness rates in Sobolev spaces.
New method tackles concept shifts in nonparametric regression using robust and adaptive transfer learning.
problem Concept shifts and sample scarcity in target domains hinder nonparametric regression.
method Robust and adaptive transfer learning procedure leveraging fixed bandwidth Gaussian kernels.
result Spectral algorithms with fixed bandwidth Gaussian kernels attain minimax convergence rates for nonparametric regression.
Develops nonparametric regression for non-smooth functions using fractional Laplacian.
problem Non-smooth regression functions in high dimensions.
method Fractional Laplacian eigenmaps for L2-fractional Sobolev spaces. result Upper bound on estimation error of $n^{- rac{2s}{2s+d}}$.
Unified approach for nonparametric regression and conditional distribution learning.
problem Nonparametric regression and conditional distribution learning problems.
method Generative learning framework with deep neural networks to estimate a conditional generator.
result The approach estimates a regression function and a conditional generator simultaneously, providing good prediction intervals.
Localized transfer learning improves nonparametric regression performance.
problem Improving nonparametric regression performance on target tasks.
method Localized transfer learning framework that models heterogeneity and partition covariate space into cells.
result Sharp minimax rates show local transfer mitigates the curse of dimensionality.
Improves MARS for nonparametric multivariate regression with dimension reduction.
problem High number of basis functions in MARS for high-order interactions.
method Linear combinations of covariates for dimension reduction, facilitating gradient calculation and eigen-analysis for estimation.
result Asymptotic theory and numerical studies show improved performance over MARS.
Bayesian methods estimate regression functions on submanifolds using graph Laplacian eigenbasis.
problem Estimating regression functions on unknown smooth submanifolds.
method Random geometric graph structure, Bayesian priors based on random basis expansion in graph Laplacian eigenbasis.
result Posterior contraction rates are minimax optimal for any positive smoothness index.
Consider the multivariate nonparametric regression model. It is shown that estimators based on sparsely connected deep neural networks with ReLU activation function and properly chosen network architecture achieve the minimax rates of convergence (up to logn-factors) under a general composition assumption on the re…
This paper investigates WDRO for nonparametric regression, achieving robustness against distributional uncertainty.
problem Addressing model misspecification in nonparametric regression under distributional uncertainty.
method Wasserstein distributionally robust optimization (WDRO) with structural distinction based on Wasserstein distance order.
result Achieves a convergence rate of n−2β/(d+2β) up to logarithmic factors, showing minimax optimality. The paper tightens bounds on covering numbers for deep ReLU networks.
problem Characterizing the capacity and performance of deep ReLU networks.
method Derives tight lower and upper bounds on metric entropy of ReLU networks.
result Establishes optimality in nonparametric regression via deep networks.
New approach to adaptively select bandwidths in nonparametric regression.
problem Adaptive bandwidth selection in nonparametric regression.
method Inspired by ℓ2-norms of interval projections, introduces a new bandwidth selection procedure. result Obtains non-asymptotic risk bounds for local polynomial regression methods that adapt to local Hölder exponent.
Paper develops a privacy-preserving nonparametric regression method.
problem Nonparametric regression with local differential privacy constraints.
method Privatised discretisation and Laplace noise applied to feature vectors and responses.
result Strongly universally consistent estimator for regression and classification.
We analyze the problem of regression when both input covariates and output responses are functions from a nonparametric function class. Function to function regression (FFR) covers a large range of interesting applications including time-series prediction problems, and also more general tasks like studying a mapping be…
Optimal rates for shallow ReLU networks in nonparametric regression.
problem Approximating smooth and non-smooth functions with shallow ReLU networks.
method Analysis of shallow ReLUk neural networks, using variation norms and deep learning theory. result Optimal approximation rates for shallow ReLU networks in nonparametric regression.
New estimator achieves minimax optimal risk in transfer learning.
problem Nonparametric regression with transfer learning.
method Confidence thresholding estimator and data-driven adaptive algorithm.
result Adaptive algorithm achieves minimax risk up to a logarithmic factor.
We propose Dirichlet Process mixtures of Generalized Linear Models (DP-GLM), a new method of nonparametric regression that accommodates continuous and categorical inputs, and responses that can be modeled by a generalized linear model. We prove conditions for the asymptotic unbiasedness of the DP-GLM regression mean fu…
Fast nonparametric conditional independence testing via two-stage regression
problem Fast nonparametric conditional independence testing
method BLITZ (Broad-to-Local Independence Testing via residualiZation)
result Better null calibration than fast kernel, random-feature, and regression-based competitors
We propose nonparametric methods for individual calibration in regression models.
problem Uncertainty quantification and individual calibration for regression models.
method Nonparametric methods agnostic of the underlying model, combining nonparametric and covering number arguments.
result Established matching upper and lower bounds for calibration error.
New algorithm for nonparametric IV regression using stochastic gradients.
problem Identifying causal effects in the presence of unobservable confounders.
method Functional stochastic gradient descent for NPIV regression.
result Superior stability and competitive performance compared to existing methods.
Transforms conditional density estimation into a nonparametric regression problem.
problem Conditional density estimation in high dimensions.
method Introduces auxiliary samples to transform into nonparametric regression.
result Estimator converges to true conditional density in data limit.