A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Proposes AML loss function for TransE to improve link prediction in knowledge graphs.
We propose a new learning to rank algorithm, named Weighted Margin-Rank Batch loss (WMRB), to extend the popular Weighted Approximate-Rank Pairwise loss (WARP). WMRB uses a new rank estimator and an efficient batch training algorithm. The approach allows more accurate item rank approximation and explicit utilization of…
Improved unsupervised probing for ranking tasks using Contrast-Consistent Ranking.
New method improves consistency in preference learning for neural networks.
Perceptron is a classic online algorithm for learning a classification function. In this paper, we provide a novel extension of the perceptron algorithm to the learning to rank problem in information retrieval. We consider popular listwise performance measures such as Normalized Discounted Cumulative Gain (NDCG) and Av…
The paper proposes a framework for structured prediction using projection oracles.
Learning to rank is a supervised learning problem where the output space is the space of rankings but the supervision space is the space of relevance scores. We make theoretical contributions to the learning to rank problem both in the online and batch settings. First, we propose a perceptron-like algorithm for learnin…
Deep neural networks trained using a softmax layer at the top and the cross-entropy loss are ubiquitous tools for image classification. Yet, this does not naturally enforce intra-class similarity nor inter-class margin of the learned deep representations. To simultaneously achieve these two goals, different solutions h…
State-of-the-art neural networks are vulnerable to adversarial examples; they can easily misclassify inputs that are imperceptibly different than their training and test data. In this work, we establish that the use of cross-entropy loss function and the low-rank features of the training data have responsibility for th…
Unified binary and multiclass margin-based classification methods.
We analyze bias-variance of margin losses.
Over the past decades, numerous loss functions have been been proposed for a variety of supervised learning tasks, including regression, classification, ranking, and more generally structured prediction. Understanding the core principles and theoretical properties underpinning these losses is key to choose the right lo…
Paper corrects Max-Margin loss for multi-label tasks.
Principal Component Analysis (PCA) is a very successful dimensionality reduction technique, widely used in predictive modeling. A key factor in its widespread use in this domain is the fact that the projection of a dataset onto its first principal components minimizes the sum of squared errors between the original …
New loss function improves convergence rate for neural networks.
Paper introduces negative margin loss for better few-shot classification accuracy.
Proposes SOVR loss to improve adversarial robustness by increasing logit margins.
New method improves calibration in multi-output probabilistic models.
Gradient descent converges to max-margin solution for hinge loss.
Paper establishes comparison theorems for large-margin learning.
Paper proposes adaptive margin loss to improve few-shot learning.
This paper establishes risk convergence and asymptotic weight matrix alignment --- a form of implicit regularization --- of gradient flow and gradient descent when applied to deep linear networks on linearly separable data. In more detail, for gradient flow applied to strictly decreasing loss functions (with similar re…
New method prevents class collapse in metric learning with margin-based losses.
We consider the problem of discriminative factor analysis for data that are in general non-Gaussian. A Bayesian model based on the ranks of the data is proposed. We first introduce a new {\em max-margin} version of the rank-likelihood. A discriminative factor model is then developed, integrating the max-margin rank-lik…
New margin-based learning guarantees improve generalization bounds.
Estimates joint probability distribution from 1-way marginals using low-rank tensors and random projections.
An econometric or statistical model may undergo a marginal gain if we admit a new variable to the model, and a marginal loss if we remove an existing variable from the model. Assuming equality of opportunity among all candidate variables, we derive a valuation framework by the expected marginal gain and marginal loss i…
Paper explores connections between loss functions and consistency in binary classification and regression.
New estimator GMIPS reduces variance in ranking policy evaluation.
Consider a classification problem where we have both labeled and unlabeled data available. We show that for linear classifiers defined by convex margin-based surrogate losses that are decreasing, it is impossible to construct any semi-supervised approach that is able to guarantee an improvement over the supervised clas…
Paper analyzes GMM for separable data with various parameter structures.
Identifies interpretable generative model for multivariate data.
This work improves adversarial robustness by boosting model ensembles with margin maximization.
The paper proposes effective margin regularization to improve adversarial robustness in deep neural networks.
We present a formulation of deep learning that aims at producing a large margin classifier. The notion of margin, minimum distance to a decision boundary, has served as the foundation of several theoretically profound and empirically successful results for both classification and regression tasks. However, most large m…
We analyze the semi-hard triplet loss using Edgeworth expansion for better understanding of its behavior.
PLD distills knowledge using choice-theoretic Plackett-Luce model.
Max-margin learning is a powerful approach to building classifiers and structured output predictors. Recent work on max-margin supervised topic models has successfully integrated it with Bayesian topic models to discover discriminative latent semantic structures and make accurate predictions for unseen testing data. Ho…
Research characterizes learnability of multilabel ranking problems.
This manuscript shows that AdaBoost and its immediate variants can produce approximate maximum margin classifiers simply by scaling step size choices with a fixed small constant. In this way, when the unscaled step size is an optimal choice, these results provide guarantees for Friedman's empirically successful "shrink…
Advances robustness of metric learning by adversarial margin in input space.
This paper studies Fenchel-Young losses, a generic way to construct convex loss functions from a regularization function. We analyze their properties in depth, showing that they unify many well-known loss functions and allow to create useful new ones easily. Fenchel-Young losses constructed from a generalized entropy, …
A fast method for training linear classifiers maximizes margins.
A new SVM classifier using soft-margin loss for improved performance.
Cross-entropy loss together with softmax is arguably one of the most common used supervision components in convolutional neural networks (CNNs). Despite its simplicity, popularity and excellent performance, the component does not explicitly encourage discriminative learning of features. In this paper, we propose a gene…
Embedding words in a vector space has gained a lot of attention in recent years. While state-of-the-art methods provide efficient computation of word similarities via a low-dimensional matrix embedding, their motivation is often left unclear. In this paper, we argue that word embedding can be naturally viewed as a rank…
Proposes a cross entropy loss for better ranking algorithms.
Gradient descent reveals the exact implicit bias via dual optimization for linearly separable data.