arXiv research

The paper explores symmetric losses for better learning from corrupted labels.

Logitron combines Perceptron and logistic loss for improved classification.

Learning with non-modular losses is an important problem when sets of predictions are made simultaneously. The main tools for constructing convex surrogate loss functions for set prediction are margin rescaling and slack rescaling. In this work, we show that these strategies lead to tight convex surrogates iff the unde…

The paper analyzes top-k classification and proposes consistent loss functions.

The Nyström method improves learning efficiency for convex losses.

We carefully study how well minimizing convex surrogate loss functions, corresponds to minimizing the misclassification error rate for the problem of binary classification with linear predictors. In particular, we show that amongst all convex surrogate losses, the hinge loss gives essentially the best possible bound, o…

Support vector machines (SVMs) naturally embody sparseness due to their use of hinge loss functions. However, SVMs can not directly estimate conditional class probabilities. In this paper we propose and study a family of coherence functions, which are convex and differentiable, as surrogates of the hinge function. The …

Study spectral learning for odeco tensors, addressing initialization bottlenecks.

AUC (area under ROC curve) is an important evaluation criterion, which has been popularly used in many learning tasks such as class-imbalance learning, cost-sensitive learning, learning to rank, etc. Many learning approaches try to optimize AUC, while owing to the non-convexity and discontinuousness of AUC, almost all …

Minimizing a convex risk function is the main step in many basic learning algorithms. We study protocols for convex optimization which provably leak very little about the individual data points that constitute the loss function. Specifically, we consider differentially private algorithms that operate in the local model…

Study on using random subspaces for ERM with various loss functions.

It is widely conjectured that the reason that training algorithms for neural networks are successful because all local minima lead to similar performance, for example, see (LeCun et al., 2015, Choromanska et al., 2015, Dauphin et al., 2014). Performance is typically measured in terms of two metrics: training performanc…

Introduces Soft-SVM for binary classification bridging logistic and SVM.

This work refines claims about neural network connectivity, showing that simultaneous linear connectivity is possible under certain conditions.

Gradient descent converges to max-margin solution for hinge loss.

New loss function calibrates WW-hinge loss for multiclass SVM.

Study shows how classifiers can approach Bayes error in high-dimensional settings.

New loss function improves convergence rate for neural networks.

Neural networks' optimization dynamics are confined to a single basin despite connected basins in the loss landscape.

Improves GANs by incorporating class information with a multi-hinge loss.

Empirical risk minimization frequently employs convex surrogates to underlying discrete loss functions in order to achieve computational tractability during optimization. However, classical convex surrogates can only tightly bound modular loss functions, sub-modular functions or supermodular functions separately while …

Graphical models for structured domains are powerful tools, but the computational complexities of combinatorial prediction spaces can force restrictions on models, or require approximate inference in order to be tractable. Instead of working in a combinatorial space, we use hinge-loss Markov random fields (HL-MRFs), an…

The paper studies how convex surfaces shrink under mean curvature flow with a free boundary.

The paper studies consistency of surrogate loss procedures under constrained classifiers.

New bounds for online portfolio selection without smoothness assumptions.

The paper explores transferability of adversarial examples between convex and 01 loss models, finding non-transferability due to different decision boundaries caused by outliers.

New approach uses hinge loss for iterative regularization in classification.

Modern machine learning focuses on highly expressive models that are able to fit or interpolate the data completely, resulting in zero training loss. For such models, we show that the stochastic gradients of common loss functions satisfy a strong growth condition. Under this condition, we prove that constant step-size …

New robust metric learning method improves performance in noisy data.

The Support Vector Machine (SVM) has been used in a wide variety of classification problems. The original SVM uses the hinge loss function, which is non-differentiable and makes the problem difficult to solve in particular for regularized SVMs, such as with $\ell_1$-regularization. This paper considers the Huberized SV…

Optimal sketching bounds for sparse linear regression under various loss functions are established.

New algorithm tackles heterogeneous curvature in online convex optimization.

We propose a projected semi-stochastic gradient descent method with mini-batch for improving both the theoretical complexity and practical performance of the general stochastic gradient descent method (SGD). We are able to prove linear convergence under weak strong convexity assumption. This requires no strong convexit…

New loss functions improve extreme classification with missing labels.

Efficient algorithms for large-scale multiclass classification with linear classifiers.

Paper proposes a boosting method with fast learning rates and early stopping.

New models improve classification model performance, especially robust to small training sets.

Proposes hinge-Wasserstein to improve uncertainty estimation in regression tasks.

Improved online learning for hidden-convex losses achieves optimal regret.

Unified analysis of online optimization with self-concordant barriers, improving regret bounds.

SGD's escape rate depends on log loss barrier, not linear loss barrier.

Coordinate descent methods employ random partial updates of decision variables in order to solve huge-scale convex optimization problems. In this work, we introduce new adaptive rules for the random selection of their updates. By adaptive, we mean that our selection rules are based on the dual residual or the primal-du…

We present a new machine learning approach to estimate personalized treatment effects in the classical potential outcomes framework with binary outcomes. To overcome the problem that both treatment and control outcomes for the same unit are required for supervised learning, we propose surrogate loss functions that inco…

Paper introduces a new loss function for deep imbalanced classification.

A mean-convex set can be regarded as a barrier for the construction of minimal surfaces. Namely, if we are given a mean-convex set and a null-homotopic Jordan curve on its boundary, then there exists an embedded minimal disk with boundary the given curve contained in the starting mean-convex set. Does a mean-convex set…

JoVA combines two VAEs to learn user and item representations for better recommendation.

A new procedure for learning cost-sensitive SVM(CS-SVM) classifiers is proposed. The SVM hinge loss is extended to the cost sensitive setting, and the CS-SVM is derived as the minimizer of the associated risk. The extension of the hinge loss draws on recent connections between risk minimization and probability elicitat…

DGKIP extends KIP for dataset distillation without bi-level optimization.