arXiv research

Kernel ridgeless regression with random features shows good generalization without explicit regularization.

Improved kernel ridge regression for large datasets using weighted random binning.

HARFE approximates sparse additive functions using random features and ridge regression.

This paper presents Correlated Nystrom Views (XNV), a fast semi-supervised algorithm for regression and classification. The algorithm draws on two main ideas. First, it generates two views consisting of computationally inexpensive random features. Second, XNV applies multiview regression using Canonical Correlation Ana…

DRE combines DNN with random feature regression for efficient neural network design.

We find a deterministic equivalent for random feature regression's test error, independent of feature map dimension.

Random Fourier features is one of the most popular techniques for scaling up kernel methods, such as kernel ridge regression. However, despite impressive empirical results, the statistical properties of random Fourier features are still not well understood. In this paper we take steps toward filling this gap. Specifica…

Lower bound proves ridgeless regression performs poorly near interpolation threshold.

We analyze the condition number of random feature matrices and prove their well-conditioned nature.

Paper introduces a method for operator learning using random features.

Random features and KRR generalize similarly when N is large enough.

New decentralized KRR algorithm adapts to node-specific data.

Unified derivation of high-dimensional linear models using stochastic gradient descent.

This paper improves kernel quantile regression with random features for handling heavy-tailed noises.

New method optimizes hyperparameters for randomized algorithms like random feature regression.

SRF improves kernel approximation and GP regression performance.

Aggregates predictions from multiple regression models using random projections and kernel methods.

Random feature matrices' singular values concentrate near their full expectation in high dimensions.

Paper analyzes error bounds for learning with vector-valued RF, improving existing analyses.

Analyzes neural networks using linear models to understand their behavior.

Study compares random and learned features in deep Bayesian linear models.

We propose a new method for input variable selection in nonlinear regression. The method is embedded into a kernel regression machine that can model general nonlinear functions, not being a priori limited to additive models. This is the first kernel-based variable selection method applicable to large datasets. It sides…

The paper analyzes ridge regression with random features for non-identically distributed data.

To improve accuracy and speed of regressions and classifications, we present a data-based prediction method, Random Bits Regression (RBR). This method first generates a large number of random binary intermediate/derived features based on the original input matrix, and then performs regularized linear/logistic regressio…

The paper analyzes the performance of random feature regression in high dimensions.

Introduces a new feature importance measure using Gram-Schmidt decorrelation.

Characterizes RFF regression in large $n,p,N$ setting, providing precise learning phases and double descent curve.

The paper compares Bayesian uncertainty to MAP estimator in random features regression.

Enhances random forest performance with exogenous randomness.

Analyzes bias-variance in overparameterized linear models using random features.

New method reduces variance and bias in approximating indefinite kernels.

Optimal AFs minimize RFR test error and sensitivity.

A new feature selection method using random forest and Kolmogorov filter.

ParK efficiently solves kernel ridge regression for large datasets.

Study shows RFRR's effectiveness with nearly orthogonal data in overparameterized settings.

Study shows how mini-batch GD with random reshuffling affects least squares regression dynamics.

The paper studies multiple descent in multi-component prediction models.

Ridge regression reveals surprising high-dimensional behaviors via random matrix theory.

Study on RF regression with SGD shows double descent phenomenon.

We study the generalization properties of ridge regression with random features in the statistical learning framework. We show for the first time that $O(1/\sqrt{n})$ learning bounds can be achieved with only $O(\sqrt{n}\log n)$ random features rather than $O({n})$ as suggested by previous results. Further, we prove fa…

New method improves Gaussian kernel approximations for high-frequency data.

The study proves Gaussian universality of deep random features learning.

More features and data lead to better model performance in random feature regression.

Quantum-assisted Gaussian process speeds up data regression.

Logistic regression for brain imaging without p-values.

Unified approach for interpretable regression with flexible modeling.

New algorithm reduces online regression error in RKHS.

RFRBoost uses random features to boost deep residual neural networks, improving performance and computational efficiency.