arXiv research

Kernel ridgeless regression with random features shows good generalization without explicit regularization.

HARFE approximates sparse additive functions using random features and ridge regression.

DRE combines DNN with random feature regression for efficient neural network design.

We analyze the condition number of random feature matrices and prove their well-conditioned nature.

Random features and KRR generalize similarly when N is large enough.

Paper introduces a method for operator learning using random features.

This paper improves kernel quantile regression with random features for handling heavy-tailed noises.

New decentralized KRR algorithm adapts to node-specific data.

We find a deterministic equivalent for random feature regression's test error, independent of feature map dimension.

This paper presents Correlated Nystrom Views (XNV), a fast semi-supervised algorithm for regression and classification. The algorithm draws on two main ideas. First, it generates two views consisting of computationally inexpensive random features. Second, XNV applies multiview regression using Canonical Correlation Ana…

We propose a new method for input variable selection in nonlinear regression. The method is embedded into a kernel regression machine that can model general nonlinear functions, not being a priori limited to additive models. This is the first kernel-based variable selection method applicable to large datasets. It sides…

Random feature matrices' singular values concentrate near their full expectation in high dimensions.

The paper analyzes ridge regression with random features for non-identically distributed data.

The paper analyzes the performance of random feature regression in high dimensions.

The paper compares Bayesian uncertainty to MAP estimator in random features regression.

The paper studies multiple descent in multi-component prediction models.

We study the generalization properties of ridge regression with random features in the statistical learning framework. We show for the first time that $O(1/\sqrt{n})$ learning bounds can be achieved with only $O(\sqrt{n}\log n)$ random features rather than $O({n})$ as suggested by previous results. Further, we prove fa…

Optimal AFs minimize RFR test error and sensitivity.

Unified derivation of high-dimensional linear models using stochastic gradient descent.

RFRBoost uses random features to boost deep residual neural networks, improving performance and computational efficiency.

New method improves Gaussian kernel approximations for high-frequency data.

Paper analyzes error bounds for learning with vector-valued RF, improving existing analyses.

Analyzes neural networks using linear models to understand their behavior.

Study shows RFRR's effectiveness with nearly orthogonal data in overparameterized settings.

The study proves Gaussian universality of deep random features learning.

New method optimizes hyperparameters for randomized algorithms like random feature regression.

SRF improves kernel approximation and GP regression performance.

New algorithm reduces online regression error in RKHS.

Random feature maps improve forecasting with cheaper computation.

Study on RF regression with SGD shows double descent phenomenon.

More features and data lead to better model performance in random feature regression.

Study compares random and learned features in deep Bayesian linear models.

Lower bound proves ridgeless regression performs poorly near interpolation threshold.

RFAD uses random features to speed up dataset distillation.

Distributed learning with random features and gradient descent improves performance and reduces memory usage.

Deep random feature models are analyzed for their performance with exact asymptotic expressions.

New insights into how high-dimensional models handle covariate shifts.

Randomized feature models learn interaction kernels from agent paths.

We study generalised linear regression and classification for a synthetically generated dataset encompassing different problems of interest, such as learning with random features, neural networks in the lazy training regime, and the hidden manifold model. We consider the high-dimensional regime and using the replica me…

Bayesian Additive Distribution Regression (DistBART) predicts distributions from grouped data.

A novel model uses ODE-based random features to model nonlinear dynamical systems.

Quantum algorithm speeds up learning from big data exponentially.

We demonstrate that distributed block coordinate descent can quickly solve kernel regression and classification problems with millions of data points. Armed with this capability, we conduct a thorough comparison between the full kernel, the Nyström method, and random features on three large classification tasks from va…

Ensembles of random-feature models can't outperform a single large model.

Random feature model approximates PDE solutions efficiently.

New RFs reduce kernel approximation variance and improve Transformer performance.

Improved machine learning models outperform their simpler counterparts by using imperfect labels.

Ridge regression reveals surprising high-dimensional behaviors via random matrix theory.