arXiv research

Optimizes exp-concave losses with a new risk bound.

The goal of online prediction with expert advice is to find a decision strategy which will perform almost as well as the best expert in a given pool of experts, on any sequence of outcomes. This problem has been widely studied and $O(\sqrt{T})$ and $O(\log{T})$ regret bounds can be achieved for convex losses (\cite{zin…

In this paper we extend the setting of the online prediction with expert advice to function-valued forecasts. At each step of the online game several experts predict a function, and the learner has to efficiently aggregate these functional forecasts into a single forecast. We adapt basic mixable (and exponentially conc…

The overarching goal of this paper is to derive excess risk bounds for learning from exp-concave loss functions in passive and sequential learning settings. Exp-concave loss functions encompass several fundamental problems in machine learning such as squared loss in linear regression, logistic loss in classification, a…

New algorithm reduces dynamic regret for exp-concave losses.

SA algorithms control dynamic regret in non-stationary settings with strong convexity or exp-concavity.

Near-logarithmic regret per switch achieved for mixable/exp-concave losses.

Improved online convex optimization with delayed feedback using curvature.

A new algorithm reduces online exp-concave optimization runtime.

Reduces dynamic regret to static problem in RKHS.

New algorithms minimize dynamic regret in non-stationary online learning.

We derive optimal statistical and computational complexity bounds for exp-concave stochastic minimization in terms of the effective dimension. For common eigendecay patterns of the population covariance matrix, this quantity is significantly smaller than the ambient dimension. Our results reveal interesting connections…

Paper proposes an online learning method with multi-level adaptivity for diverse loss functions.

Information concentration of probability measures have important implications in learning theory. Recently, it is discovered that the information content of a log-concave distribution concentrates around their differential entropy, albeit with an unpleasant dependence on the ambient dimension. In this work, we prove th…

Gaptron algorithm reduces mistakes in online multiclass classification.

An online learning framework for survival analysis with real-time adaptation.

New algorithms minimize dynamic regret for strongly convex losses.

MetaGrad adapts multiple learning rates for faster online optimization.

Improved cumulative regret for sequence prediction with limited expert advice.

A standard introduction to online learning might place Online Gradient Descent at its center and then proceed to develop generalizations and extensions like Online Mirror Descent and second-order methods. Here we explore the alternative approach of putting Exponential Weights (EW) first. We show that many standard meth…

In this paper, we present a simple analysis of {\bf fast rates} with {\it high probability} of {\bf empirical minimization} for {\it stochastic composite optimization} over a finite-dimensional bounded convex set with exponential concave loss functions and an arbitrary convex regularization. To the best of our knowledg…

We introduce several new black-box reductions that significantly improve the design of adaptive and parameter-free online learning algorithms by simplifying analysis, improving regret guarantees, and sometimes even improving runtime. We reduce parameter-free online learning to online exp-concave optimization, we reduce…

The speed with which a learning algorithm converges as it is presented with more data is a central problem in machine learning --- a fast rate of convergence means less data is needed for the same level of performance. The pursuit of fast rates in online and statistical learning has led to the discovery of many conditi…

Unified derivation of PAC-Bayes and MI bounds for general VC classes with fast rates.

New method achieves both universality and adaptivity in online convex optimization.

Recursive least-squares algorithms often use forgetting factors as a heuristic to adapt to non-stationary data streams. The first contribution of this paper rigorously characterizes the effect of forgetting factors for a class of online Newton algorithms. For exp-concave and strongly convex objectives, the algorithms a…

Improved regret bounds for online convex optimization under stochastic and adversarial settings.

Introduces Fitzpatrick losses, tighter than Fenchel-Young losses.

We study losses for binary classification and class probability estimation and extend the understanding of them from margin losses to general composite losses which are the composition of a proper loss with a link function. We characterise when margin losses can be proper composite losses, explicitly show how to determ…

We present the Tamed Cross Entropy (TCE) loss function, a robust derivative of the standard Cross Entropy (CE) loss used in deep learning for classification tasks. However, unlike other robust losses, the TCE loss is designed to exhibit the same training properties than the CE loss in noiseless scenarios. Therefore, th…

Unified surrogate loss framework for multi-label learning with strong consistency guarantees.

This paper introduces new loss functions for balanced multi-class classification.

We present $α$-loss, $α\in [1,\infty]$, a tunable loss function for binary classification that bridges log-loss ($α=1$) and $0$-$1$ loss ($α= \infty$). We prove that $α$-loss has an equivalent margin-based form and is classification-calibrated, two desirable properties for a good surrogate loss function for the ideal y…

This work broadens calibeating to various proper losses using Bregman divergence.

This work generalizes calibeating for a broader range of proper losses using Bregman divergence.

Proposes squentropy loss for improved classification accuracy and model calibration.

New loss function calibrates WW-hinge loss for multiclass SVM.

The study analyzes a model for aggregate losses with dependent and overdispersed inter-losses times.

Symmetric losses improve classifier robustness from corrupted labels.

Two new algorithms improve performance in adversarial bandits with unbounded losses.

Paper explores connections between loss functions and consistency in binary classification and regression.

Paper introduces a new topological loss for better convergence.

Novel loss functions improve decision tree learning from noisy data.

Theoretical analysis of cross-entropy loss functions and their robustness.

This paper improves operational risk modeling by selecting better loss severity distributions.

This paper improves loss functions for deep learning with noisy labels.

Classification is the most important process in data analysis. However, due to the inherent non-convex and non-smooth structure of the zero-one loss function of the classification model, various convex surrogate loss functions such as hinge loss, squared hinge loss, logistic loss, and exponential loss are introduced. T…

We study cross-country GDP losses due to financial crises in terms of frequency (number of loss events per period) and severity (loss per occurrence). We perform the Loss Distribution Approach (LDA) to estimate a multi-country aggregate GDP loss probability density function and the percentiles associated to extreme eve…