arXiv research

Optimizes exp-concave losses with a new risk bound.

We derive optimal statistical and computational complexity bounds for exp-concave stochastic minimization in terms of the effective dimension. For common eigendecay patterns of the population covariance matrix, this quantity is significantly smaller than the ambient dimension. Our results reveal interesting connections…

A new algorithm reduces online exp-concave optimization runtime.

SA algorithms control dynamic regret in non-stationary settings with strong convexity or exp-concavity.

In this paper we extend the setting of the online prediction with expert advice to function-valued forecasts. At each step of the online game several experts predict a function, and the learner has to efficiently aggregate these functional forecasts into a single forecast. We adapt basic mixable (and exponentially conc…

An online learning framework for survival analysis with real-time adaptation.

New algorithm reduces dynamic regret for exp-concave losses.

Near-logarithmic regret per switch achieved for mixable/exp-concave losses.

The goal of online prediction with expert advice is to find a decision strategy which will perform almost as well as the best expert in a given pool of experts, on any sequence of outcomes. This problem has been widely studied and $O(\sqrt{T})$ and $O(\log{T})$ regret bounds can be achieved for convex losses (\cite{zin…

Improved online convex optimization with delayed feedback using curvature.

Information concentration of probability measures have important implications in learning theory. Recently, it is discovered that the information content of a log-concave distribution concentrates around their differential entropy, albeit with an unpleasant dependence on the ambient dimension. In this work, we prove th…

The overarching goal of this paper is to derive excess risk bounds for learning from exp-concave loss functions in passive and sequential learning settings. Exp-concave loss functions encompass several fundamental problems in machine learning such as squared loss in linear regression, logistic loss in classification, a…

We introduce several new black-box reductions that significantly improve the design of adaptive and parameter-free online learning algorithms by simplifying analysis, improving regret guarantees, and sometimes even improving runtime. We reduce parameter-free online learning to online exp-concave optimization, we reduce…

MetaGrad adapts multiple learning rates for faster online optimization.

Paper proposes an online learning method with multi-level adaptivity for diverse loss functions.

New algorithms minimize dynamic regret in non-stationary online learning.

Reduces dynamic regret to static problem in RKHS.

New method achieves both universality and adaptivity in online convex optimization.

Improved cumulative regret for sequence prediction with limited expert advice.

In this paper, we present a simple analysis of {\bf fast rates} with {\it high probability} of {\bf empirical minimization} for {\it stochastic composite optimization} over a finite-dimensional bounded convex set with exponential concave loss functions and an arbitrary convex regularization. To the best of our knowledg…

New algorithms minimize dynamic regret for strongly convex losses.

Recursive least-squares algorithms often use forgetting factors as a heuristic to adapt to non-stationary data streams. The first contribution of this paper rigorously characterizes the effect of forgetting factors for a class of online Newton algorithms. For exp-concave and strongly convex objectives, the algorithms a…

A standard introduction to online learning might place Online Gradient Descent at its center and then proceed to develop generalizations and extensions like Online Mirror Descent and second-order methods. Here we explore the alternative approach of putting Exponential Weights (EW) first. We show that many standard meth…

Gaptron algorithm reduces mistakes in online multiclass classification.

Improved regret bounds for online convex optimization under stochastic and adversarial settings.

Unified derivation of PAC-Bayes and MI bounds for general VC classes with fast rates.

The speed with which a learning algorithm converges as it is presented with more data is a central problem in machine learning --- a fast rate of convergence means less data is needed for the same level of performance. The pursuit of fast rates in online and statistical learning has led to the discovery of many conditi…

Bayesian optimization reduces computational effort in aircraft design optimization.

Bayesian optimization method tackles combinatorial spaces, scalable for large data.

New algorithm solves complex stopping problems with robust optimization.

L2O uses ML to optimize traditional optimization techniques.

When hyperparameter optimization of a machine learning algorithm is repeated for multiple datasets it is possible to transfer knowledge to an optimization run on a new dataset. We develop a new hyperparameter-free ensemble model for Bayesian optimization that is a generalization of two existing transfer learning extens…

Meta algorithm solves multivariate optimization using univariate optimizers.

A novel neural network approach for optimization problems.

New algorithms ensure reproducibility and optimal convergence in convex optimization.

New algorithm selects robust martingale for optimal stopping problems.

Numerical optimization is an important tool in the field of computational physics in general and in nano-optics in specific. It has attracted attention with the increase in complexity of structures that can be realized with nowadays nano-fabrication technologies for which a rational design is no longer feasible. Also, …

This paper shows how to combine optimal tests into log-optimal processes.

Learning optimal feedback control laws capable of executing optimal trajectories is essential for many robotic applications. Such policies can be learned using reinforcement learning or planned using optimal control. While reinforcement learning is sample inefficient, optimal control only plans an optimal trajectory fr…

Paper studies optimal control for a specific geometric problem.

New algorithm AG-OG optimizes separable convex-concave problems efficiently.

Adapts Bayesian optimization for mixed constraints in aircraft design.

Adam optimizer converges to zeros of a new vector field, not just gradient zeros.

Optimal crypto asset routing with CFMMs, including fixed costs.

BOSH optimizes functions with stochastic evaluations more efficiently and precisely.

VeLO learns versatile optimizers from deep learning tasks.

New learned optimizers outperform baselines by incorporating known and novel mechanisms.

PAGE optimizes nonconvex problems with optimal convergence rates.