arXiv research

Orthogonal random features approximate a Bessel kernel, offering sharper bounds than random Fourier features.

Sparse random features improve accuracy in data-scarce settings.

Random feature maps improve forecasting with cheaper computation.

We study the approximation properties of random ReLU features through their reproducing kernel Hilbert space (RKHS). We first prove a universality theorem for the RKHS induced by random features whose feature maps are of the form of nodes in neural networks. The universality result implies that the random ReLU features…

Kernel ridgeless regression with random features shows good generalization without explicit regularization.

RFRBoost uses random features to boost deep residual neural networks, improving performance and computational efficiency.

We propose semi-random features for nonlinear function approximation. The flexibility of semi-random feature lies between the fully adjustable units in deep learning and the random features used in kernel methods. For one hidden layer models with semi-random features, we prove with no unrealistic assumptions that the m…

The randomized-feature approach has been successfully employed in large-scale kernel approximation and supervised learning. The distribution from which the random features are drawn impacts the number of features required to efficiently perform a learning task. Recently, it has been shown that employing data-dependent …

A faster graph kernel using optical random features.

Gradient descent amplifies random features in neural networks to useful ones.

New method improves feature importance assessment in random forests.

A new method for approximating softmax and Gaussian kernels with reduced error.

Kernel method has been developed as one of the standard approaches for nonlinear learning, which however, does not scale to large data set due to its quadratic complexity in the number of samples. A number of kernel approximation methods have thus been proposed in the recent years, among which the random features metho…

Hermite polynomials improve private data generation by reducing feature count.

Random feature matrices' singular values concentrate near their full expectation in high dimensions.

HARFE approximates sparse additive functions using random features and ridge regression.

Enhances random forest performance with exogenous randomness.

Random Fourier features classification achieves fast learning rates with fewer features.

New random feature maps for Laplacian and related kernels.

Random feature maps are ubiquitous in modern statistical machine learning, where they generalize random projections by means of powerful, yet often difficult to analyze nonlinear operators. In this paper, we leverage the "concentration" phenomenon induced by random matrix theory to perform a spectral analysis on the Gr…

Improves efficiency of random feature approximations for dot product kernels.

One-bit quantization improves inference speed for Random Features models.

We study the generalization properties of ridge regression with random features in the statistical learning framework. We show for the first time that $O(1/\sqrt{n})$ learning bounds can be achieved with only $O(\sqrt{n}\log n)$ random features rather than $O({n})$ as suggested by previous results. Further, we prove fa…

TRF uses ternary random features to improve ML performance without extra computation.

Survey on random features for kernel approximation, focusing on algorithms, theory, and practical applications.

Random feature models approximate functions in Banach spaces efficiently.

Improved kernel ridge regression for large datasets using weighted random binning.

A stability metric compares feature selection algorithms in machine learning.

This study examines a single attention layer's capabilities using random features.

We analyze the condition number of random feature matrices and prove their well-conditioned nature.

A new model combines diffusion and random features for better interpretability and comparable performance.

Generative model uses random convolutional features to create financial time series.

Kernel methods are powerful and flexible approach to solve many problems in machine learning. Due to the pairwise evaluations in kernel methods, the complexity of kernel computation grows as the data size increases; thus the applicability of kernel methods is limited for large scale datasets. Random Fourier Features (R…

A novel model uses ODE-based random features to model nonlinear dynamical systems.

This paper surveys various methods for dimensionality reduction and nearest neighbor search.

Random feature models can outperform a weak teacher with early stopping.

HRFs adaptively linearize kernels for accurate approximations.

Random feature maps improve forecasting of chaotic dynamical systems.

Random forests are a type of ensemble method which makes predictions by combining the results of several independent trees. However, the theory of random forests has long been outpaced by their application. In this paper, we propose a novel random forests algorithm based on cooperative game theory. Banzhaf power index …

New method optimizes hyperparameters for randomized algorithms like random feature regression.

Although kernel methods are widely used in many learning problems, they have poor scalability to large datasets. To address this problem, sketching and stochastic gradient methods are the most commonly used techniques to derive efficient large-scale learning algorithms. In this study, we consider solving a binary class…

Attention layers are sensitive to single words, improving generalization over random features.

Random feature approximation speeds up spectral methods and improves learning rates.

Study loop corrections in random feature models affecting training and test errors.

Multivariate splines linked to infinitely-wide neural networks with improved numerical performance.

Random matrix theory explains how neural networks adapt to data.

Random features and KRR generalize similarly when N is large enough.

The paper studies multiple descent in multi-component prediction models.