arXiv research

Random learning rate improves neural network training without extra cost.

RandQL is a new model-free algorithm for MDPs with a novel learning rate randomization approach.

Rating prediction is an important application, and a popular research topic in collaborative filtering. However, both the validity of learning algorithms, and the validity of standard testing procedures rest on the assumption that missing ratings are missing at random (MAR). In this paper we present the results of a us…

Random feature approximation speeds up spectral methods and improves learning rates.

New random forest variants achieve optimal performance in high dimensions.

A single slow-growing tree matches Random Forest's performance.

Study reveals convergence properties of SGD with random learning rate.

Random forests remain among the most popular off-the-shelf supervised learning algorithms. Despite their well-documented empirical success, however, until recently, few theoretical results were available to describe their performance and behavior. In this work we push beyond recent work on consistency and asymptotic no…

Random Fourier features classification achieves fast learning rates with fewer features.

Improved rates for continual learning using SGD and last-iterate analysis.

Extends random feature analysis to spectral methods and improves learning rates.

Hyperparameter tuning is a bothersome step in the training of deep learning models. One of the most sensitive hyperparameters is the learning rate of the gradient descent. We present the 'All Learning Rates At Once' (Alrao) optimization method for neural networks: each unit or feature in the network gets its own learni…

Although kernel methods are widely used in many learning problems, they have poor scalability to large datasets. To address this problem, sketching and stochastic gradient methods are the most commonly used techniques to derive efficient large-scale learning algorithms. In this study, we consider solving a binary class…

Gradient descent and SGD solve nonlinear inverse problems efficiently.

New insights into learning rates and batch sizes for neural networks using random matrix theory.

The paper analyzes Random Search and introduces BLiN-MOS for bandit learning.

New methods tackle statistical inverse problems with random data.

The paper analyzes learning rates for non-irreducible Markov chains.

New random forest method provides optimal rates and confidence bands.

Infinitesimal gradient boosting is a new algorithm derived from gradient boosting.

Study shows how Laplacian semi-supervised learning behaves at low labeling rates.

In this paper, we propose a unified convergence analysis for a class of generic shuffling-type gradient methods for solving finite-sum optimization problems. Our analysis works with any sampling without replacement strategy and covers many known variants such as randomized reshuffling, deterministic or randomized singl…

Paper analyzes error bounds for learning with vector-valued RF, improving existing analyses.

We propose accelerated randomized coordinate descent algorithms for stochastic optimization and online learning. Our algorithms have significantly less per-iteration complexity than the known accelerated gradient algorithms. The proposed algorithms for online learning have better regret performance than the known rando…

We prove that, under low noise assumptions, the support vector machine with $N\ll m$ random features (RFSVM) can achieve the learning rate faster than $O(1/\sqrt{m})$ on a training set with $m$ samples when an optimized feature map is used. Our work extends the previous fast rate analysis of random features method from…

Random covers of hyperbolic surfaces have a spectral gap with polynomial rate.

We study the generalization properties of ridge regression with random features in the statistical learning framework. We show for the first time that $O(1/\sqrt{n})$ learning bounds can be achieved with only $O(\sqrt{n}\log n)$ random features rather than $O({n})$ as suggested by previous results. Further, we prove fa…

Random feature models approximate functions in Banach spaces efficiently.

We enhance short-rate models to control implied volatility analytically.

Deep nets learn structured densities without dimensionality issues.

Develops a new model for cross-currency derivatives pricing.

This paper investigates the grant-free random access with massive IoT devices. By embedding the data symbols in the signature sequences, joint device activity detection and data decoding can be achieved, which, however, significantly increases the computational complexity. Coordinate descent algorithms that enjoy a low…

We find optimal learning rate schedules for a random feature model.

Paper introduces a new regularization method for kernel gradient descent learning.

Proposes reducing random error in stochastic optimization by variance regularization.

A new random forest algorithm improves tree construction for optimal performance.

TLRF improves timely COVID-19 outbreak detection with small sample size counties.

HF-opt uses Hamiltonian dynamics to optimize functions, achieving accelerated rates with randomized integration time.

New insights into image compression trade-offs with private randomness.

Endogenous randomness emerges from adversarial market learning.

We study the convergence rate of stochastic optimization of exact (NP-hard) objectives, for which only biased estimates of the gradient are available. We motivate this problem in the context of learning the structure and parameters of Ising models. We first provide a convergence-rate analysis of deterministic errors fo…

Study confirms optimal minimax rate for nonlocal interaction kernel estimation.

Paper analyzes convergence rates of compressed LSR algorithms in federated learning.

Random Forests provide interpretable prediction intervals with theoretical guarantees.

This paper improves kernel quantile regression with random features for handling heavy-tailed noises.

RQMC improves kernel-based learning by reducing deterministic error and offering computational advantages.

A new method reduces computational cost for gene expression inference in large microarray data sets.

The paper relaxes the stability condition to boost confidence in generalization for randomized learning algorithms.