arXiv research

Kernel ridgeless regression with random features shows good generalization without explicit regularization.

Paper explores how DPP sampling can implicitly regularize kernel regression.

Study shows that ridgeless Gaussian kernel regression overfits even with varying bandwidth or dimensionality.

Enhanced kernel ridgeless regression improves performance with LAB RBF kernels.

Lower bound proves ridgeless regression performs poorly near interpolation threshold.

The paper examines prediction and estimation risks of ridgeless least squares under general error assumptions.

Data splitting enhances model performance in overparametrized ridgeless regression.

Downsampling can improve generalization in ridgeless linear regression, especially with optimal sketching size.

New bounds for KRR condition number reveal overfitting phenomena.

Interpolating models can have heavy-tailed risk, leading to rare but severe errors.

In the absence of explicit regularization, Kernel "Ridgeless" Regression with nonlinear kernels has the potential to fit the training data perfectly. It has been observed empirically, however, that such interpolated solutions can still generalize well on test data. We isolate a phenomenon of implicit regularization for…

Sharp analysis of knowledge distillation for high-dimensional regression.

Lecture notes on advanced linear regression methods.

Knoop enhances variable selection with over-parameterization and knockoffs.

New method estimates precision matrices without models, achieving dense, consistent, and model-free properties.

Locality helps in learning from high-dimensional data.

Optimal recovery framework for non-IID data in Hilbert spaces.

W2S FT often outperforms weak teachers due to low intrinsic dimensionality.

The paper analyzes how generated data improves adversarial training in high-dimensional regression.

Ridgeless ReLU networks interpolate datasets and extrapolate based on curvature signs.

This work analyzes how preconditioning affects generalization in machine learning models.

Interpolators -- estimators that achieve zero training error -- have attracted growing attention in machine learning, mainly because state-of-the art neural networks appear to be models of this type. In this paper, we study minimum $\ell_2$ norm ("ridgeless") interpolation in high-dimensional least squares regression. …

Study the cost of overfitting in noisy KRR models.

The paper analyzes bagging in overparameterized learning, deriving risk properties and optimal subsample sizes.

Paper analyzes SGD in kernel regression, showing it outperforms offline methods.

New method stabilizes machine learning for physics-informed inverse problems.

We analyze optimal weighted ridge regression in overparameterized linear models.

Study ridge ensembles in proportional feature-to-sample size regime, proving risk equivalence and GCV consistency.

Linear models can be poisoned by shifting a fraction of one class's data, revealing scaling laws and weight alignment.

Overparameterized ensembles don't offer generalization benefits over single large models.

Consider the problem: given the data pair $(\mathbf{x}, \mathbf{y})$ drawn from a population with $f_*(x) = \mathbf{E}[\mathbf{y} | \mathbf{x} = x]$, specify a neural network model and run gradient flow on the weights over time until reaching any stationarity. How does $f_t$, the function computed by the neural network…

Complexity helps identify sparse risk factors in asset pricing.

Linear memory stores associations up to a logarithmic scale, but listwise retrieval can handle a quadratic scale.

Ability for accurate hospital case cost modelling and prediction is critical for efficient health care financial management and budgetary planning. A variety of regression machine learning algorithms are known to be effective for health care cost predictions. The purpose of this experiment was to build an Azure Machine…

Efficiently private regression for unbounded data.

This paper studies robust regression in the settings of Huber's $ε$-contamination models. We consider estimators that are maximizers of multivariate regression depth functions. These estimators are shown to achieve minimax rates in the settings of $ε$-contamination models for various regression problems including nonpa…

This paper studies the nonparametric modal regression problem systematically from a statistical learning view. Originally motivated by pursuing a theoretical understanding of the maximum correntropy criterion based regression (MCCR), our study reveals that MCCR with a tending-to-zero scale parameter is essentially moda…

Proposes FARM model combining latent factor and sparse regression.

Survey of SDR methods for high-dimensional regression and embedding.

This paper reviews SDR methods for multivariate response regression.

Paper introduces semi-supervised linear extremile regression for high-dimensional data.

Improves logistic regression performance with nonconvex programming.

Study improves $H$-consistency bounds for regression analysis.

Prevalidated ridge regression simplifies logistic regression for high-dimensional data.

Introduces a new model for mapping matrices to matrices, subsuming linear regression.

Meta-theorems validate fair regression algorithms under demographic parity constraints.

We analyzed optimism in linear and kernel regression models.

A new optimizer, MVO, improves nonlinear regression performance.