arXiv research

HARFE approximates sparse additive functions using random features and ridge regression.

Improved kernel ridge regression for large datasets using weighted random binning.

DRE combines DNN with random feature regression for efficient neural network design.

Random Fourier features is one of the most popular techniques for scaling up kernel methods, such as kernel ridge regression. However, despite impressive empirical results, the statistical properties of random Fourier features are still not well understood. In this paper we take steps toward filling this gap. Specifica…

The paper analyzes ridge regression with random features for non-identically distributed data.

ParK efficiently solves kernel ridge regression for large datasets.

Random features and KRR generalize similarly when N is large enough.

RF models implicitly regularize kernel methods as feature count increases.

Ridge regression reveals surprising high-dimensional behaviors via random matrix theory.

Paper analyzes error bounds for learning with vector-valued RF, improving existing analyses.

We introduce single-set spectral sparsification as a deterministic sampling based feature selection technique for regularized least squares classification, which is the classification analogue to ridge regression. The method is unsupervised and gives worst-case guarantees of the generalization power of the classificati…

Analyzes neural networks using linear models to understand their behavior.

Bayes-optimal learning of deep random networks with Gaussian weights is studied.

We propose LOCO, an algorithm for large-scale ridge regression which distributes the features across workers on a cluster. Important dependencies between variables are preserved using structured random projections which are cheap to compute and must only be communicated once. We show that LOCO obtains a solution which …

New decentralized KRR algorithm adapts to node-specific data.

Study ridge regression for non-identically distributed data with varying variances.

Short proof shows how ridge regression works with random data.

The paper analyzes the performance of random feature regression in high dimensions.

Deterministic algorithm estimates ridge regression with minimal space.

We analyze generalization in deep learning models using random matrix theory.

Paper proposes methods for transfer learning with random coefficient ridge regression.

New ridge regression bounds for high-dimensional data without proportional growth.

The study proves Gaussian universality of deep random features learning.

Study shows RFRR's effectiveness with nearly orthogonal data in overparameterized settings.

The paper studies multiple descent in multi-component prediction models.

Random Fourier features is a widely used, simple, and effective technique for scaling up kernel methods. The existing theoretical analysis of the approach, however, remains focused on specific learning tasks and typically gives pessimistic bounds which are at odds with the empirical results. We tackle these problems an…

We provide a unified analysis of the predictive risk of ridge regression and regularized discriminant analysis in a dense random effects model. We work in a high-dimensional asymptotic regime where $p, n \to \infty$ and $p/n \to γ\in (0, \, \infty)$, and allow for arbitrary covariance among the features. For both metho…

Conventional seismic techniques for detecting the subsurface geologic features are challenged by limited data coverage, computational inefficiency, and subjective human factors. We developed a novel data-driven geological feature detection approach based on pre-stack seismic measurements. Our detection method employs a…

Ensembles of random-feature models can't outperform a single large model.

Improved machine learning models outperform their simpler counterparts by using imperfect labels.

Improved ridge regression with Frequent Directions for large-scale tasks.

Study predictive performance of linear regression with random functional covariates.

The study identifies spurious correlations in high-dimensional regression and quantifies their impact.

We find a deterministic equivalent for random feature regression's test error, independent of feature map dimension.

Ridge leverage scores provide a balance between low-rank approximation and regularization, and are ubiquitous in randomized linear algebra and machine learning. Deterministic algorithms are also of interest in the moderately big data regime, because deterministic algorithms provide interpretability to the practitioner …

Meta-learning improves predictions with generalized ridge regression in high-dimensional settings.

We study the generalization properties of ridge regression with random features in the statistical learning framework. We show for the first time that $O(1/\sqrt{n})$ learning bounds can be achieved with only $O(\sqrt{n}\log n)$ random features rather than $O({n})$ as suggested by previous results. Further, we prove fa…

Improves kernel ridge regression by optimizing scale and feature parameters.

Study precise estimators for correlated data using RDT.

We improve prediction risk estimation for large datasets using sketching and ridge regression.

Efficiently predict LLM benchmarks using feature selection and regression.

Kernel ridge regression for causal inference with missing data.

We show that the error probability of reconstructing kernel matrices from Random Fourier Features for the Gaussian kernel function is at most $\mathcal{O}(R^{2/3} \exp(-D))$, where $D$ is the number of random features and $R$ is the diameter of the data domain. We also provide an information-theoretic method-independen…

New insights into how randomization affects greedy model selection.

New method detects biomarker-treatment interactions in clinical trials.

This paper improves kernel quantile regression with random features for handling heavy-tailed noises.

New framework investigates how fake features affect model generalization.

Paper introduces a method for operator learning using random features.