arXiv research

Paper proposes a boosting method with fast learning rates and early stopping.

We study body-and-hinge and panel-and-hinge chains in R^d, with two marked points: one on the first body, the other on the last. For a general chain, the squared distance between the marked points gives a Morse-Bott function on a torus configuration space. Maximal configurations, when the distance between the two marke…

Support vector machines (SVMs) are an important tool in modern data analysis. Traditionally, support vector machines have been fitted via quadratic programming, either using purpose-built or off-the-shelf algorithms. We present an alternative approach to SVM fitting via the majorization--minimization (MM) paradigm. Alg…

New loss functions improve extreme classification with missing labels.

Classification is the most important process in data analysis. However, due to the inherent non-convex and non-smooth structure of the zero-one loss function of the classification model, various convex surrogate loss functions such as hinge loss, squared hinge loss, logistic loss, and exponential loss are introduced. T…

The Nyström method improves learning efficiency for convex losses.

Classification and regression tasks in overparameterized models show different generalization properties.

We present an efficient second-order algorithm with $\tilde{O}(\frac{1}η\sqrt{T})$ regret for the bandit online multiclass problem. The regret bound holds simultaneously with respect to a family of loss functions parameterized by $η$, for a range of $η$ restricted by the norm of the competitor. The family of loss funct…

Cyanure is an open-source C++ software package with a Python interface. The goal of Cyanure is to provide state-of-the-art solvers for learning linear models, based on stochastic variance-reduced stochastic optimization with acceleration mechanisms. Cyanure can handle a large variety of loss functions (logistic, square…

A popular heuristic for improved performance in Generative adversarial networks (GANs) is to use some form of gradient penalty on the discriminator. This gradient penalty was originally motivated by a Wasserstein distance formulation. However, the use of gradient penalty in other GAN formulations is not well motivated.…

We propose random hinge forests, a simple, efficient, and novel variant of decision forests. Importantly, random hinge forests can be readily incorporated as a general component within arbitrary computation graphs that are optimized end-to-end with stochastic gradient descent or variants thereof. We derive random hinge…

Motivated by the hinge structure present in protein chains and other molecular conformations, we study the singularities of certain maps associated to body-and-hinge and panel-and-hinge chains. These are sequentially articulated systems where two consecutive rigid pieces are connected by a hinge, that is, a codimension…

Hinge-FM2I fills missing data in time series with high accuracy.

Discrete time hedging in a complete diffusion market is considered. The hedge portfolio is rebalanced when the absolute difference between delta of the hedge portfolio and the derivative contract reaches a threshold level. The rate of convergence of the expected squared hedging error as the threshold level approaches z…

Study on using random subspaces for ERM with various loss functions.

Support vector machines (SVMs) naturally embody sparseness due to their use of hinge loss functions. However, SVMs can not directly estimate conditional class probabilities. In this paper we propose and study a family of coherence functions, which are convex and differentiable, as surrogates of the hinge function. The …

New loss function calibrates WW-hinge loss for multiclass SVM.

New loss function improves convergence rate for neural networks.

JoVA combines two VAEs to learn user and item representations for better recommendation.

Christoffel function characterizes the corruption a bounded-degree certificate cannot remove in robust halfspace learning.

Lasso is a seminal contribution to high-dimensional statistics, but it hinges on a tuning parameter that is difficult to calibrate in practice. A partial remedy for this problem is Square-Root Lasso, because it inherently calibrates to the noise variance. However, Square-Root Lasso still requires the calibration of a t…

Gradient descent is a simple and widely used optimization method for machine learning. For homogeneous linear classifiers applied to separable data, gradient descent has been shown to converge to the maximal margin (or equivalently, the minimal norm) solution for various smooth loss functions. The previous theory does …

We develop and analyze stochastic optimization algorithms for problems in which the expected loss is strongly convex, and the optimum is (approximately) sparse. Previous approaches are able to exploit only one of these two structures, yielding an $\order(\pdim/T)$ convergence rate for strongly convex objectives in $\pd…

New approach uses hinge loss for iterative regularization in classification.

We propose a new algorithm to incorporate class conditional information into the critic of GANs via a multi-class generalization of the commonly used Hinge loss that is compatible with both supervised and semi-supervised settings. We study the compromise between training a state of the art generator and an accurate cla…

We consider the problem of recovering a low-rank matrix from its clipped observations. Clipping is conceivable in many scientific areas that obstructs statistical analyses. On the other hand, matrix completion (MC) methods can recover a low-rank matrix from various information deficits by using the principle of low-ran…

The top-$k$ error is often employed to evaluate performance for challenging classification tasks in computer vision as it is designed to compensate for ambiguity in ground truth labels. This practical success motivates our theoretical analysis of consistent top-$k$ classification. Surprisingly, it is not rigorously und…

Proposes hinge-Wasserstein to improve uncertainty estimation in regression tasks.

The paper studies consistency of surrogate loss procedures under constrained classifiers.

Smooth, globally PŁ functions are essentially nonlinear least-squares.

We present surrogate regret bounds for arbitrary surrogate losses in the context of binary classification with label-dependent costs. Such bounds relate a classifier's risk, assessed with respect to a surrogate loss, to its cost-sensitive classification risk. Two approaches to surrogate regret bounds are developed. The…

New framework enhances neural network robustness against adversarial attacks.

A new procedure for learning cost-sensitive SVM(CS-SVM) classifiers is proposed. The SVM hinge loss is extended to the cost sensitive setting, and the CS-SVM is derived as the minimizer of the associated risk. The extension of the hinge loss draws on recent connections between risk minimization and probability elicitat…

Distance/Similarity learning is a fundamental problem in machine learning. For example, kNN classifier or clustering methods are based on a distance/similarity measure. Metric learning algorithms enhance the efficiency of these methods by learning an optimal distance function from data. Most metric learning methods nee…

A new method for high-dimensional classification using Bernstein polynomials.

Study efficient learning of halfspaces with constant noise tolerance.

New insights into when benign overfitting occurs in linear and classification tasks.

We prove a new generalization bound that shows for any class of linear predictors in Gaussian space, the Rademacher complexity of the class and the training error under any continuous loss $\ell$ can control the test error under all Moreau envelopes of the loss $\ell$. We use our finite-sample bound to directly recover…

Learning with non-modular losses is an important problem when sets of predictions are made simultaneously. The main tools for constructing convex surrogate loss functions for set prediction are margin rescaling and slack rescaling. In this work, we show that these strategies lead to tight convex surrogates iff the unde…

Introduces Soft-SVM for binary classification bridging logistic and SVM.

Partial monitoring is a general model for sequential learning with limited feedback formalized as a game between two players. In this game, the learner chooses an action and at the same time the opponent chooses an outcome, then the learner suffers a loss and receives a feedback signal. The goal of the learner is to mi…

We propose the Margin Adaptation for Generative Adversarial Networks (MAGANs) algorithm, a novel training procedure for GANs to improve stability and performance by using an adaptive hinge loss function. We estimate the appropriate hinge loss margin with the expected energy of the target distribution, and derive princi…

New method learns distances and similarities robustly from noisy data.

We give elementary constructions for Satake-Furstenberg, Martin and Karpelevich boundaries of symmetric spaces. We also consruct some "new" boundaries

AUC (area under ROC curve) is an important evaluation criterion, which has been popularly used in many learning tasks such as class-imbalance learning, cost-sensitive learning, learning to rank, etc. Many learning approaches try to optimize AUC, while owing to the non-convexity and discontinuousness of AUC, almost all …

The paper analyzes the dynamics of a simple neural network using a mean-field approach.

Discrete forms of the scalar, sectional and Ricci curvatures are constructed on simplicial piecewise flat triangulations of smooth manifolds, depending directly on the simplicial structure and a choice of dual tessellation. This is done by integrating over volumes which include appropriate samplings of hinges for each …

We derive the fast convergence rates of a deep neural network (DNN) classifier with the rectified linear unit (ReLU) activation function learned using the hinge loss. We consider three cases for a true model: (1) a smooth decision boundary, (2) smooth conditional class probability, and (3) the margin condition (i.e., t…