arXiv research

Improves statistical learning bounds with self-concordant losses.

New bounds on minimax regret for sequential probability assignment using logarithmic loss.

Paper improves learning rates for GSC loss functions using iterated Tikhonov regularization.

The classical asymptotic theory for parametric $M$-estimators guarantees that, in the limit of infinite sample size, the excess risk has a chi-square type distribution, even in the misspecified case. We demonstrate how self-concordance of the loss allows to characterize the critical sample size sufficient to guarantee …

We consider distributed convex optimization problems originated from sample average approximation of stochastic optimization, or empirical risk minimization in machine learning. We assume that each machine in the distributed computing system has access to a local empirical loss function, constructed with i.i.d. data sa…

Unified analysis of online optimization with self-concordant barriers, improving regret bounds.

New algorithm reduces prediction errors across various loss functions.

The paper improves confidence set construction for statistical inference.

The self-concordant-like property of a smooth convex function is a new analytical structure that generalizes the self-concordant notion. While a wide variety of important applications feature the self-concordant-like property, this concept has heretofore remained unexploited in convex optimization. To this end, we deve…

Algorithm for online decision making with unknown dynamics and aggregate feedback.

We study the smooth structure of convex functions by generalizing a powerful concept so-called self-concordance introduced by Nesterov and Nemirovskii in the early 1990s to a broader class of convex functions, which we call generalized self-concordant functions. This notion allows us to develop a unified framework for …

Improved confidence bounds for linear logistic model with applications to bandits.

Unified algorithm for linear bandits with improved regret bound.

Improved Frank-Wolfe algorithm for generalized self-concordant functions converges quickly.

In this paper, we study large-scale convex optimization algorithms based on the Newton method applied to regularized generalized self-concordant losses, which include logistic regression and softmax regression. We first prove that our new simple scheme based on a sequence of problems with decreasing regularization para…

New approach for online learning with adaptive adversaries, simpler and more effective.

New insights into natural exponential families improve regret bounds for bandit problems.

Interior-point methods adapted for manifolds, achieving similar optimization results.

New method solves constrained self-concordant minimization problems efficiently.

New bounds for online portfolio selection without smoothness assumptions.

New algorithm tackles heterogeneous curvature in online convex optimization.

Adaptive-SGD method optimizes machine learning training with dynamic batch and step sizes.

Study on using random subspaces for ERM with various loss functions.

Many problems in statistical learning, imaging, and computer vision involve the optimization of a non-convex objective function with singularities at the boundary of the feasible set. For such challenging instances, we develop a new interior-point technique building on the Hessian-barrier algorithm recently introduced …

We propose a randomized second-order method for optimization known as the Newton Sketch: it is based on performing an approximate Newton step using a randomly projected or sub-sampled Hessian. For self-concordant functions, we prove that the algorithm has super-linear convergence with exponentially high probability, wi…

The Nyström method improves learning efficiency for convex losses.

We propose a variable metric framework for minimizing the sum of a self-concordant function and a possibly non-smooth convex function, endowed with an easily computable proximal operator. We theoretically establish the convergence of our framework without relying on the usual Lipschitz gradient assumption on the smooth…

Many scientific and engineering applications feature nonsmooth convex minimization problems over convex sets. In this paper, we address an important instance of this broad class where we assume that the nonsmooth objective is equipped with a tractable proximity operator and that the convex constraint set affords a self…

The paper generalizes offset Rademacher complexities to convex and non-convex problems.

Improved prediction algorithm for 'easy' sequences with reduced regret.

New method improves HMC for sampling on manifolds.

New sampling method improves accuracy for constrained spaces.

The goal of this note is to construct, on many manifolds, non-trivial concordances from the identity to itself. This produces counterexamples to a recent conjecture by Botvinnik.

We consider the class of convex minimization problems, composed of a self-concordant function, such as the $\log\det$ metric, a convex data fidelity term $h(\cdot)$ and, a regularizing -- possibly non-smooth -- function $g(\cdot)$. This type of problems have recently attracted a great deal of interest, mainly due to th…

We analyze the problem of sequential probability assignment for binary outcomes with side information and logarithmic loss, where regret---or, redundancy---is measured with respect to a (possibly infinite) class of experts. We provide upper and lower bounds for minimax regret in terms of sequential complexities of the …

The paper analyzes the excess risk of PCA and provides a precise characterization.

Unified CS for GLMs improves bandit regret bounds.

RHMC improves sampling polytopes defined by inequalities with barriers.

The Riemannian Langevin Algorithm samples from manifolds efficiently.

In this paper we consider the composite self-concordant (CSC) minimization problem, which minimizes the sum of a self-concordant function $f$ and a (possibly nonsmooth) proper closed convex function $g$. The CSC minimization is the cornerstone of the path-following interior point methods for solving a broad class of co…

We compute the effect of concordance surgery, a generalization of knot surgery defined using a self-concordance of a knot, on the Ozsváth-Szabó 4-manifold invariant. The formula involves the graded Lefschetz number of the concordance map on knot Floer homology. The proof uses the sutured Floer TQFT, and a version of su…

Study shows fast rates for inverse reinforcement learning with linear rewards.

Popular machine learning estimators involve regularization parameters that can be challenging to tune, and standard strategies rely on grid search for this task. In this paper, we revisit the techniques of approximating the regularization path up to predefined tolerance $ε$ in a unified framework and show that its comp…

New GLB algorithm handles non-stationary data with forgetting.

New algorithm improves sampling from constrained spaces.

This article introduces the concepts around Online Bandit Linear Optimization and explores an efficient setup called SCRiBLe (Self-Concordant Regularization in Bandit Learning) created by Abernethy et. al.\cite{abernethy}. The SCRiBLe setup and algorithm yield a $O(\sqrt{T})$ regret bound and polynomial run time comple…

Unified meta-algorithm improves average performance across similar tasks in adversarial bandits.

We propose an algorithmic framework for convex minimization problems of a composite function with two terms: a self-concordant function and a possibly nonsmooth regularization term. Our method is a new proximal Newton algorithm that features a local quadratic convergence rate. As a specific instance of our framework, w…