Paper provides an upper bound for bias of Nadaraya-Watson kernel regression.
problem Estimating bias of Nadaraya-Watson kernel regression for finite bandwidths.
method Proposes an upper bound for bias under Lipschitz assumptions, extending to discontinuous derivatives and multidimensional domains.
result Upper bound on bias for finite bandwidths, tighter than previous infinitesimal bandwidth analysis.
A new method TNW-CATE estimates treatment effects using neural networks.
problem Estimating heterogeneous treatment effects with limited controls and many treatments.
method Trainable Nadaraya-Watson regression with shared parameters neural network.
result TNW-CATE outperforms traditional methods in various simulation experiments.
New insights into simple kernel smoothing reveal surprising asymptotics.
problem Understanding precise asymptotics of Nadaraya-Watson kernel smoothing.
method Using ideas from the random energy model in statistical physics.
result Sharp asymptotics for the NW predictor on the sphere.
Study on predicting graph labels at nodes using local averaging and distance estimation.
problem Predicting graph labels at nodes given observations at other nodes.
method Local averaging and distance estimation methods for graph regression.
result Alternative methods can achieve standard nonparametric rates even when graph neighborhoods are too large or small.
The paper analyzes covariate shift in nonparametric regression with Markovian data.
problem Covariate shift in regression problems with Markovian data.
method Extension of nonparametric convergence rates to Markovian dependence structures, using Hölder smoothness assumptions and similarity measures.
result Precise convergence rates for Nadaraya-Watson kernel estimators under specific Markovian conditions.
Enhances supervised learning speed with KT algorithm.
problem Speed up supervised learning tasks with minimal loss.
method Generalizes kernel thinning to supervised learning, combining NW and KRR with KT.
result KT-based estimators offer superior computational and statistical efficiency.
ABIForest improves anomaly detection using attention weights.
problem Anomaly detection in datasets.
method Attention mechanism integrated into Isolation Forest.
result ABIForest outperforms standard Isolation Forest on synthetic and real datasets.
The most direct way to express arbitrary dependencies in datasets is to estimate the joint distribution and to apply afterwards the argmax-function to obtain the mode of the corresponding conditional distribution. This method is in practice difficult, because it requires a global optimization of a complicated function,…
New insights into Nadaraya-Watson interpolators show varied generalization behaviors.
problem Understanding generalization of interpolating predictors, especially in noisy data.
method Revisiting Nadaraya-Watson estimator with a single hyperparameter.
result Multiple overfitting behaviors exist, ranging from catastrophic to tempered.
New theory shows how multi-head attention reduces variance and decorrelates outputs.
problem Understanding and optimizing multi-head attention in neural networks.
method Developed a statistical theory linking multi-head attention to ensemble Nadaraya-Watson estimators.
result MHA variance reduction depends on head decorrelation, not just head count.
TAP transfers knowledge from unlabeled data to improve cross-modal learning.
problem Improving supervised learning performance using unlabeled data from a different modality.
method Probabilistic approach for missing information estimation, kernel regression, cross-attention module, TAP neural network.
result TAP significantly improves generalization across different domains and neural network architectures.
Novel method for time-series prediction with tighter confidence intervals.
problem Improving prediction intervals for time-series data.
method Kernel-based Optimally Weighted Conformal Prediction Intervals (KOWCPI) using adaptive weights.
result KOWCPI achieves narrower confidence intervals with guaranteed coverage.
A debiasing method improves nonparametric regression's statistical properties.
problem Lack of theoretical guarantees for modern nonparametric regression methods.
method Model-free debiasing method incorporating a correction term.
result Debiased estimator satisfies pointwise and uniform risk convergence, asymptotic normality.
Estimates path-valued data using signature metrics and local kernels.
problem Nonparametric regression and classification for path-valued data.
method Combines signature transform and local kernel regression.
result Establishes convergence bounds and demonstrates competitive accuracy.
ABRF uses attention weights to improve RF performance.
problem Improving the performance of random forest models.
method Attention mechanism applied to random forest with optimization and gradient-based methods.
result The proposed ABRF models outperform standard RF models on various datasets.
Estimates time-series drifts from i.i.d. data using a direct Nadaraya-Watson plug-in method.
problem Nonparametric estimation of Schrödinger bridge drifts from single time interval data.
method Direct Nadaraya-Watson plug-in estimator based on kernelized numerator and denominator terms.
result Uniform non-asymptotic bound, CLT under undersmoothing, and adaptive bandwidth selector.
Improved nonparametric regression with debiasing for root-n consistency.
problem Challenges in achieving root-n consistency and normal distribution for nonparametric estimators.
method Debiasing technique by adding a correction term to nonparametric estimators.
result Achieves root-n consistency and asymptotic normality.
Random forests with attention and self-attention improve regression performance.
problem Improving regression model performance on various datasets.
method Proposes new models using attention and self-attention mechanisms to solve regression problems.
result The models improve model performance on many datasets.
Improved local feature attributions using neighbourhood reference distributions.
problem Misleading results from global population in local model behaviour.
method Formulation of neighbourhood reference distributions and self-normalised importance sampling.
result Neighbourhood Shapley values provide meaningful sparse feature attributions.
The paper introduces a method to quantify uncertainty in neural networks without parametric assumptions.
problem Uncertainty quantification for neural network predictions.
method Nonparametric estimation of conditional label distribution using Nadaraya-Watson kernel.
result The method effectively disentangles aleatoric and epistemic uncertainties.
Generative models use kernel smoothing for conditioning on small example sets.
problem Improving generative models' performance with limited conditioning examples.
method Showed that cross-attention conditioning is equivalent to kernel smoothing, specifically a Nadaraya--Watson kernel smoother.
result The approach predicts and confirms three failure regimes for kernel-based conditioning.
Efficient classifier with uncertainty bounds for safety-critical applications.
problem Lack of uncertainty bounds in high-accuracy classifiers for safety-critical tasks.
method Nadaraya-Watson estimator with frequentist bounds.
result Competitive accuracy and uncertainty bounds at reduced computational cost.
Quantum Graphical Models (QGMs) generalize classical graphical models by adopting the formalism for reasoning about uncertainty from quantum mechanics. Unlike classical graphical models, QGMs represent uncertainty with density matrices in complex Hilbert spaces. Hilbert space embeddings (HSEs) also generalize Bayesian …
This study improves estimation of locally stationary functional time series using NW method.
problem Accurately capturing time-dependence in locally stationary functional time series with time-varying covariates.
method Nadaraya-Watson (NW) estimation procedure for the conditional distribution of LSFTS.
result Established convergence rates of NW estimator for LSFTS with respect to Wasserstein distance.
Transformers can approximate Kalman Filtering in linear systems with small error.
problem Approximating Kalman Filtering using Transformers for linear dynamical systems.
method Two-step reduction: 1) Softmax self-attention block approximates Nadaraya-Watson kernel smoothing, 2) This estimator approximates Kalman Filter.
result Constructs a Transformer that implements the Kalman Filter with small additive error, uniformly bounded in time.
The paper develops predictors for functional data on manifolds.
problem Functional data prediction on time-varying manifolds.
method Least-squares local linear Fréchet curve predictor and weighted Fréchet mean approach.
result Asymptotical optimality of the proposed predictors.
Paper develops NW kernel estimator for LSPs with Wasserstein bounds.
problem Capturing nuanced dynamics in time series data with evolving statistical characteristics.
method Nadaraya-Watson kernel smoothing for conditional probability estimation, using Wasserstein and sliced Wasserstein distances.
result Established convergence rates and bounds for NW-based conditional probability estimator in LSPs.
Paper estimates FPR of Bayes classifier using soft labels.
problem Determining optimal classifier performance.
method Uses soft labels and denoising technique.
result Consistent and unbiased FPR estimator developed.
Paper introduces novel survival models for handling censored data.
problem Complex data structures and heavy censoring in survival analysis.
method Combines imprecise probability theory with attention mechanisms.
result Proposed models, especially iSurvJ, outperform traditional methods.
Study the averaging estimator on graphs with labeled nodes.
problem Understanding the quality of averaging estimators on graph data.
method Rigorously study concentration properties, variance bounds, and risk bounds.
result Contributes to theoretical understanding of graph learning.
BENK estimates treatment effects with neural kernels for censored data.
problem Estimating heterogeneous treatment effects with censored time-to-event data.
method Proposes a method using the Beran estimator with neural kernels for survival functions.
result Shows improved accuracy compared to existing methods in various scenarios.
We analyze the empirical performance of several non-parametric estimators of the pricing functional for European options, using historical put and call prices on the S&P500 during the year 2012. Two main families of estimators are considered, obtained by estimating the pricing functional directly, and by estimating the…
Active local learning uses fewer labels to predict near-optimal functions.
problem Efficiently predicting near-optimal functions with fewer labels.
method Active local learning algorithm for estimating functions with fewer labels.
result Algorithm makes significantly fewer label queries than traditional methods.
We address the problem of prescribing an optimal decision in a framework where the cost function depends on uncertain problem parameters that need to be learned from data. Earlier work proposed prescriptive formulations based on supervised machine learning methods. These prescriptive methods can factor in contextual in…
New method generates synthetic time series paths with more flexibility.
problem Restrictions in generating synthetic paths using Brownian reference.
method Introduces Triangular-Reference Schrödinger Bridges (TR-SBTS) for time series generation.
result Generates synthetic paths with more flexibility in stochastic volatility and correlated noise.
Ability for accurate hospital case cost modelling and prediction is critical for efficient health care financial management and budgetary planning. A variety of regression machine learning algorithms are known to be effective for health care cost predictions. The purpose of this experiment was to build an Azure Machine…
Efficiently private regression for unbounded data.
problem Privacy constraints in regression settings with unbounded covariates.
method Differential privacy techniques on mean and covariance estimation extended to sub-gaussian regime.
result Unbiased estimate of true regression vector learned up to a scaling factor.
This paper studies robust regression in the settings of Huber's ε ε ε -contamination models. We consider estimators that are maximizers of multivariate regression depth functions. These estimators are shown to achieve minimax rates in the settings of ε ε ε -contamination models for various regression problems including nonpa…
This paper studies the nonparametric modal regression problem systematically from a statistical learning view. Originally motivated by pursuing a theoretical understanding of the maximum correntropy criterion based regression (MCCR), our study reveals that MCCR with a tending-to-zero scale parameter is essentially moda…
Proposes FARM model combining latent factor and sparse regression.
problem Testing adequacy of latent factor and sparse regression models.
method Factor Augmented sparse linear Regression Model (FARM) with FabTest and ANOVA type tests.
result Model robustness and effectiveness validated through experiments.
Survey of SDR methods for high-dimensional regression and embedding.
problem Reducing dimensionality in high-dimensional data.
method Involves both statistical and machine learning approaches, covering inverse and forward regression methods.
result Supervised Kernel Dimension Reduction is equivalent to supervised PCA.
This paper reviews SDR methods for multivariate response regression.
problem Handling sufficient dimension reduction for multivariate response regression.
method Characterizes SDR estimators as inverse or forward regression methods.
result Pooled marginal, projective resampling, distance-based, ordinary least squares, partial least squares, and semiparametric SDR estimators are discussed.
Paper introduces semi-supervised linear extremile regression for high-dimensional data.
problem Challenges in high-dimensional extremile regression due to data sparsity and overfitting.
method Proposes semi-supervised learning for linear extremile regression, achieving n \sqrt{n} n -consistency. result Demonstrates improved estimation efficiency and performance in high-dimensional settings.
Improves logistic regression performance with nonconvex programming.
problem Stochastic generalized linear regression with chance constraints.
method Nonconvex programming techniques, clustering, quantile estimation.
result Over 1 to 2 percent improvement in model performance.
Study improves H H H -consistency bounds for regression analysis.
problem Improving H H H -consistency bounds for regression analysis. method Generalized theorems and novel H H H -consistency bounds for various surrogate loss functions. result Derives principled surrogate losses for adversarial regression.
Prevalidated ridge regression simplifies logistic regression for high-dimensional data.
problem Efficient probabilistic classification in high-dimensional data with logistic regression.
method Developed a prevalidated ridge regression model that matches logistic regression's performance but is more computationally efficient.
result Prevalidated ridge regression achieves similar classification error and log-loss to logistic regression for high-dimensional data.
Introduces a new model for mapping matrices to matrices, subsuming linear regression.
problem Learning matrix-to-matrix mappings from data.
method Partial trace regression model, leveraging quantum information theory.
result Relevance demonstrated in matrix-to-matrix regression and positive semidefinite matrix completion.
Meta-theorems validate fair regression algorithms under demographic parity constraints.
problem Regression under demographic parity constraints.
method Meta-theorems and post-processing methods.
result Fair minimax optimal regression can be achieved through post-processing.