arXiv research

New approach uses hinge loss for iterative regularization in classification.

New framework enhances neural network robustness against adversarial attacks.

Support vector machines (SVMs) naturally embody sparseness due to their use of hinge loss functions. However, SVMs can not directly estimate conditional class probabilities. In this paper we propose and study a family of coherence functions, which are convex and differentiable, as surrogates of the hinge function. The …

A new procedure for learning cost-sensitive SVM(CS-SVM) classifiers is proposed. The SVM hinge loss is extended to the cost sensitive setting, and the CS-SVM is derived as the minimizer of the associated risk. The extension of the hinge loss draws on recent connections between risk minimization and probability elicitat…

Introduces Soft-SVM for binary classification bridging logistic and SVM.

We develop efficient algorithms to train $\ell_1$-regularized linear classifiers with large dimensionality $d$ of the feature space, number of classes $k$, and sample size $n$. Our focus is on a special class of losses that includes, in particular, the multiclass hinge and logistic losses. Our approach combines several…

The Support Vector Machine (SVM) has been used in a wide variety of classification problems. The original SVM uses the hinge loss function, which is non-differentiable and makes the problem difficult to solve in particular for regularized SVMs, such as with $\ell_1$-regularization. This paper considers the Huberized SV…

We propose random hinge forests, a simple, efficient, and novel variant of decision forests. Importantly, random hinge forests can be readily incorporated as a general component within arbitrary computation graphs that are optimized end-to-end with stochastic gradient descent or variants thereof. We derive random hinge…

Motivated by the hinge structure present in protein chains and other molecular conformations, we study the singularities of certain maps associated to body-and-hinge and panel-and-hinge chains. These are sequentially articulated systems where two consecutive rigid pieces are connected by a hinge, that is, a codimension…

Study shows how classifiers can approach Bayes error in high-dimensional settings.

Hinge-FM2I fills missing data in time series with high accuracy.

Improves graph recovery in Gaussian graphical modeling.

Regularization helps improve classification of noisy high-dimensional data.

New loss function calibrates WW-hinge loss for multiclass SVM.

New loss function improves convergence rate for neural networks.

Substantial evidence indicates that major psychiatric disorders are associated with distributed neural dysconnectivity, leading to strong interest in using neuroimaging methods to accurately predict disorder status. In this work, we are specifically interested in a multivariate approach that uses features derived from …

We study body-and-hinge and panel-and-hinge chains in R^d, with two marked points: one on the first body, the other on the last. For a general chain, the squared distance between the marked points gives a Morse-Bott function on a torus configuration space. Maximal configurations, when the distance between the two marke…

The regularity theory for pluriclosed flow hinges on obtaining $C^α$ regularity for the metric assuming uniform equivalence to a background metric. This estimate was established in \cite{StreetsPCFBI} by an adaptation of ideas from Evans-Krylov, the key input being a sharp differential inequality satisfied by the assoc…

JoVA combines two VAEs to learn user and item representations for better recommendation.

Classification is the most important process in data analysis. However, due to the inherent non-convex and non-smooth structure of the zero-one loss function of the classification model, various convex surrogate loss functions such as hinge loss, squared hinge loss, logistic loss, and exponential loss are introduced. T…

This work simplifies SVM parameter selection using S&S ratio.

Modern data analysis depends increasingly on estimating models via flexible high-dimensional or nonparametric machine learning methods, where the identification of structural parameters is often challenging and untestable. In linear settings, this identification hinges on the completeness condition, which requires the …

Gradient descent is a simple and widely used optimization method for machine learning. For homogeneous linear classifiers applied to separable data, gradient descent has been shown to converge to the maximal margin (or equivalently, the minimal norm) solution for various smooth loss functions. The previous theory does …

Paper proposes a boosting method with fast learning rates and early stopping.

Novel regularization for Vision Transformers improves model generalization and sparsity.

In this paper, we study randomized reduction methods, which reduce high-dimensional features into low-dimensional space by randomized methods (e.g., random projection, random hashing), for large-scale high-dimensional classification. Previous theoretical results on randomized reduction methods hinge on strong assumptio…

We present a max-margin nonparametric latent feature model, which unites the ideas of max-margin learning and Bayesian nonparametrics to discover discriminative latent features for link prediction and automatically infer the unknown latent social dimension. By minimizing a hinge-loss using the linear expectation operat…

Study on using random subspaces for ERM with various loss functions.

Feature selection is a standard approach to understanding and modeling high-dimensional classification data, but the corresponding statistical methods hinge on tuning parameters that are difficult to calibrate. In particular, existing calibration schemes in the logistic regression framework lack any finite sample guara…

We propose a new algorithm to incorporate class conditional information into the critic of GANs via a multi-class generalization of the commonly used Hinge loss that is compatible with both supervised and semi-supervised settings. We study the compromise between training a state of the art generator and an accurate cla…

The top-$k$ error is often employed to evaluate performance for challenging classification tasks in computer vision as it is designed to compensate for ambiguity in ground truth labels. This practical success motivates our theoretical analysis of consistent top-$k$ classification. Surprisingly, it is not rigorously und…

Proposes hinge-Wasserstein to improve uncertainty estimation in regression tasks.

The paper studies consistency of surrogate loss procedures under constrained classifiers.

Cyanure is an open-source C++ software package with a Python interface. The goal of Cyanure is to provide state-of-the-art solvers for learning linear models, based on stochastic variance-reduced stochastic optimization with acceleration mechanisms. Cyanure can handle a large variety of loss functions (logistic, square…

We present a discriminative nonparametric latent feature relational model (LFRM) for link prediction to automatically infer the dimensionality of latent features. Under the generic RegBayes (regularized Bayesian inference) framework, we handily incorporate the prediction loss with probabilistic inference of a Bayesian …

We present sparse topical coding (STC), a non-probabilistic formulation of topic models for discovering latent representations of large collections of data. Unlike probabilistic topic models, STC relaxes the normalization constraint of admixture proportions and the constraint of defining a normalized likelihood functio…

New bounds on optimal transport regularization show faster convergence rates than previously known.

Distance/Similarity learning is a fundamental problem in machine learning. For example, kNN classifier or clustering methods are based on a distance/similarity measure. Metric learning algorithms enhance the efficiency of these methods by learning an optimal distance function from data. Most metric learning methods nee…

For a market impact model, price manipulation and related notions play a role that is similar to the role of arbitrage in a derivatives pricing model. Here, we give a systematic investigation into such regularity issues when orders can be executed both at a traditional exchange and in a dark pool. To this end, we focus…

Study efficient learning of halfspaces with constant noise tolerance.

Optimal sampling bounds for various classification losses under different regularization terms.

Learning with non-modular losses is an important problem when sets of predictions are made simultaneously. The main tools for constructing convex surrogate loss functions for set prediction are margin rescaling and slack rescaling. In this work, we show that these strategies lead to tight convex surrogates iff the unde…

New loss functions improve extreme classification with missing labels.

Partial monitoring is a general model for sequential learning with limited feedback formalized as a game between two players. In this game, the learner chooses an action and at the same time the opponent chooses an outcome, then the learner suffers a loss and receives a feedback signal. The goal of the learner is to mi…

We propose the Margin Adaptation for Generative Adversarial Networks (MAGANs) algorithm, a novel training procedure for GANs to improve stability and performance by using an adaptive hinge loss function. We estimate the appropriate hinge loss margin with the expected energy of the target distribution, and derive princi…

New neural network approach solves Poisson equations efficiently.

New method learns distances and similarities robustly from noisy data.

We give elementary constructions for Satake-Furstenberg, Martin and Karpelevich boundaries of symmetric spaces. We also consruct some "new" boundaries