arXiv research

We improve private training accuracy with learning rate schedules and matrix factorizations.

SALR improves deep learning generalization by dynamically adjusting learning rates.

Paper improves learning rates for SGD and NAG.

Variational autoencoders optimize an objective that combines a reconstruction loss (the distortion) and a KL term (the rate). The rate is an upper bound on the mutual information, which is often interpreted as a regularizer that controls the degree of compression. We here examine whether inclusion of the rate also acts…

Estimates yearly improvement rates for nearly all technologies using US patent data.

Improved first-order algorithm for entropy regularized OT with faster convergence.

Paper analyzes convergence rate of noisy Bayesian Optimization with Expected Improvement.

Ensembling improves performance when classifiers disagree more than average.

Rate-In dynamically adjusts dropout rates during inference to improve uncertainty estimation in neural networks.

Proposes a sparsity algorithm to improve corporate credit ratings.

CEI achieves convergence rates for constrained Bayesian optimization.

FIRE PBT improves neural network training by focusing on long-term performance.

Large learning rates improve generalization, but optimal ranges are narrower than previously thought.

Framework integrates financial and annual report data for better corporate credit ratings.

Adaptive algorithm improves convergence rate of Langevin dynamics.

Demographic projections of future mortality rates involve a high level of uncertainty and require stochastic mortality models. The current paper investigates forward mortality models driven by a (possibly infinite dimensional) Wiener process and a compensated Poisson random measure. A major innovation of the paper is t…

Warm-up improves training by adapting learning rate based on curvature.

We study two time-scale linear stochastic approximation algorithms, which can be used to model well-known reinforcement learning algorithms such as GTD, GTD2, and TDC. We present finite-time performance bounds for the case where the learning rate is fixed. The key idea in obtaining these bounds is to use a Lyapunov fun…

This paper introduces and evaluates a novel training method for neural networks: Dual Variable Learning Rates (DVLR). Building on insights from behavioral psychology, the dual learning rates are used to emphasize correct and incorrect responses differently, thereby making the feedback to the network more specific. Furt…

An expanding literature articulates the view that Taylor rules are helpful in predicting exchange rates. In a changing world however, Taylor rule parameters may be subject to structural instabilities, for example during the Global Financial Crisis. This paper forecasts exchange rates using such Taylor rules with Time V…

The paper uses machine learning and Lie groups to improve rating transitions and XVA calculations.

New factorial power constants improve optimization convergence rates.

Conditional expectiles are becoming an increasingly important tool in finance as well as in other areas of applications. We analyse a support vector machine type approach for estimating conditional expectiles and establish learning rates that are minimax optimal modulo a logarithmic factor if Gaussian RBF kernels are u…

Efficient global optimization is the problem of minimizing an unknown function f, using as few evaluations f(x) as possible. It can be considered as a continuum-armed bandit problem, with noiseless data and simple regret. Expected improvement is perhaps the most popular method for solving this problem; the algorithm pe…

We propose the stochastic average gradient (SAG) method for optimizing the sum of a finite number of smooth convex functions. Like stochastic gradient (SG) methods, the SAG method's iteration cost is independent of the number of terms in the sum. However, by incorporating a memory of previous gradient values the SAG me…

AdaDetectGPT improves text authorship detection with statistical guarantees.

Learning rate annealing improves robustness in stochastic optimization.

A new algorithm improves convergence rates for convex optimization problems.

This paper improves deep neural network approximation for fully connected networks, achieving optimal convergence rates.

In this paper we improve the spectral convergence rates for graph-based approximations of Laplace-Beltrami operators constructed from random data. We utilize regularity of the continuum eigenfunctions and strong pointwise consistency results to prove that spectral convergence rates are the same as the pointwise consist…

We present an improved analysis of the Euler-Maruyama discretization of the Langevin diffusion. Our analysis does not require global contractivity, and yields polynomial dependence on the time horizon. Compared to existing approaches, we make an additional smoothness assumption, and improve the existing rate from $O(η)…

Improved convergence rates for Stein Variational Gradient Descent in finite-particle settings.

Active learning can't improve over passive in certain settings.

This paper uses LLMs to improve equity stock ratings by ingesting diverse financial and news data.

New ensemble method improves model stability exponentially.

The paper analyzes and improves the learning rates of distributed kernel ridge regression.

Prodigy estimates learning rate without tuning, improving convergence.

Adaptive learning rates improve FTPL's BOBW guarantees in bandit problems.

In semi-supervised classification, one is given access both to labeled and unlabeled data. As unlabeled data is typically cheaper to acquire than labeled data, this setup becomes advantageous as soon as one can exploit the unlabeled data in order to produce a better classifier than with labeled data alone. However, the…

FastAdaBelief improves convergence rate of AdaBelief by exploiting strong convexity.

Improved LoRA+ adapts large models more efficiently.

Copula models for sovereign ratings improved by incorporating climate risk.

This paper studies first order methods for solving smooth minimax optimization problems $\min_x \max_y g(x,y)$ where $g(\cdot,\cdot)$ is smooth and $g(x,\cdot)$ is concave for each $x$. In terms of $g(\cdot,y)$, we consider two settings -- strongly convex and nonconvex -- and improve upon the best known rates in both. …

Cross-domain recommendation has been proposed to transfer user behavior pattern by pooling together the rating data from multiple domains to alleviate the sparsity problem appearing in single rating domains. However, previous models only assume that multiple domains share a latent common rating pattern based on the use…

This work improves wireless network learning by using side-information about interference.

We revisit the stochastic limited-memory BFGS (L-BFGS) algorithm. By proposing a new framework for the convergence analysis, we prove improved convergence rates and computational complexities of the stochastic L-BFGS algorithms compared to previous works. In addition, we propose several practical acceleration strategie…

Improved Frank-Wolfe algorithm for generalized self-concordant functions converges quickly.

This paper investigates the efficacy of a regularized multi-task learning (MTL) framework based on SVM (M-SVM) to answer whether MTL always provides reliable results and how MTL outperforms independent learning. We first find that M-SVM is Bayes risk consistent in the limit of large sample size. This implies that despi…