arXiv research

Gradient descent converges with arbitrary stepsize for separable data under Fenchel-Young losses.

The paper sets lower bounds for adversarial robustness in multiclass classification.

New algorithm handles bandit problems under translations and scales.

In this work we develop a new algorithm for regularized empirical risk minimization. Our method extends recent techniques of Shalev-Shwartz [02/2015], which enable a dual-free analysis of SDCA, to arbitrary mini-batching schemes. Moreover, our method is able to better utilize the information in the data defining the ER…

In the regression problem, L1 and L2 are the most commonly used loss functions, which produce mean predictions with different biases. However, the predictions are neither robust nor adequate enough since they only capture a few conditional distributions instead of the whole distribution, especially for small datasets. …

We consider deep linear networks with arbitrary convex differentiable loss. We provide a short and elementary proof of the fact that all local minima are global minima if the hidden layers are either 1) at least as wide as the input layer, or 2) at least as wide as the output layer. This result is the strongest possibl…

New algorithm tackles adversarial bandits with arbitrary strategies.

Deep neural networks' loss surfaces contain every low-dimensional pattern.

New algorithm tackles multi-armed bandit with arbitrary delays and general bounded losses.

Study predicts SGD test loss for structured features.

Algorithm learns arbitrary ReLU neurons under Gaussian inputs.

Proposes a new framework for learning from labeled and unlabeled data.

In contrast to the standard classification paradigm where the true class is given to each training pattern, complementary-label learning only uses training patterns each equipped with a complementary label, which only specifies one of the classes that the pattern does not belong to. The goal of this paper is to derive …

Two new algorithms improve performance in adversarial bandits with unbounded losses.

Most systems and learning algorithms optimize average performance or average loss -- one reason being computational complexity. However, many objectives of practical interest are more complex than simply average loss. This arises, for example, when balancing performance or loss with fairness across people. We prove tha…

We present surrogate regret bounds for arbitrary surrogate losses in the context of binary classification with label-dependent costs. Such bounds relate a classifier's risk, assessed with respect to a surrogate loss, to its cost-sensitive classification risk. Two approaches to surrogate regret bounds are developed. The…

New research shows many batch selection methods for training work just as well as full batch training.

Piecewise linear activations create many spurious local minima in neural networks.

This paper calculates worst-case target semi-variances for uncertain losses.

Global implicit function theorem for Fréchet spaces, solving derivative loss problems.

In recent literature, a general two step procedure has been formulated for solving the problem of phase retrieval. First, a spectral technique is used to obtain a constant-error initial estimate, following which, the estimate is refined to arbitrary precision by first-order optimization of a non-convex loss function. N…

Easyllp simplifies LLP, achieving low task loss at individual instance level.

The study establishes SQ lower bounds for learning halfspaces and ReLUs under Gaussian marginals.

Flexible tree ensemble learning framework supports arbitrary loss functions and multi-task learning.

We study proper losses for discrete generative models without knowing the target distribution.

Gradient descent implicitly follows regularization for general losses.

A novel gradient-based method optimizes decision trees for complex tasks.

Fine-grained analysis of gradient descent with momentum provides modified loss equations.

New bounds on minimax regret for sequential probability assignment using logarithmic loss.

In this paper we consider Fourier transform techniques to efficiently compute the Value-at-Risk and the Conditional Value-at-Risk of an arbitrary loss random variable, characterized by having a computable generalized characteristic function. We exploit the property of these risk measures of being the solution of an ele…

Unified approach for robust low rank matrix estimation with adversaries.

Reduces dynamic regret to static problem in RKHS.

The paper decomposes probabilistic scores into reliability, uncertainty, and information loss.

We consider on-line density estimation with a parameterized density from the exponential family. The on-line algorithm receives one example at a time and maintains a parameter that is essentially an average of the past examples. After receiving an example the algorithm incurs a loss which is the negative log-likelihood…

This paper introduces SRPR for robust phase retrieval with smoothed loss functions.

Active learning is a type of sequential design for supervised machine learning, in which the learning algorithm sequentially requests the labels of selected instances from a large pool of unlabeled data points. The objective is to produce a classifier of relatively low risk, as measured under the 0-1 loss, ideally usin…

SGD converges globally to logistic loss minima for two-layer nets.

New algorithm minimizes cumulative loss in dynamic linear bandits without prior knowledge of comparator switches.

This paper establishes minimax rates for online regression with arbitrary classes of functions and general losses. We show that below a certain threshold for the complexity of the function class, the minimax rates depend on both the curvature of the loss function and the sequential complexities of the class. Above this…

Dynamic regret minimization is shown equivalent to static regret minimization for linear losses.

We establish linear regret bounds for convex smooth losses using Fenchel-Young losses.

Study ruin probabilities in risk processes on stochastic networks.

The paper analyzes the implicit bias of SGD near loss manifold and provides new insights.

We consider an active learning setting where the algorithm has access to a large pool of unlabeled data and a small pool of labeled data. In each iteration, the algorithm chooses few unlabeled data points and obtains their labels from an oracle. In this paper, we consider a probabilistic querying procedure to choose th…

New method trains generative models without discriminators, improving stability and accuracy.

Efficient online learning with pairwise loss functions is a crucial component in building large-scale learning system that maximizes the area under the Receiver Operator Characteristic (ROC) curve. In this paper we investigate the generalization performance of online learning algorithms with pairwise loss functions. We…

This work studies applications and generalizations of a simple estimation technique that provides exponential concentration under heavy-tailed distributions, assuming only bounded low-order moments. We show that the technique can be used for approximate minimization of smooth and strongly convex losses, and specificall…

The study of a machine learning problem is in many ways is difficult to separate from the study of the loss function being used. One avenue of inquiry has been to look at these loss functions in terms of their properties as scoring rules via the proper-composite representation, in which predictions are mapped to probab…