Efficient online learning with pairwise loss functions is a crucial component in building large-scale learning system that maximizes the area under the Receiver Operator Characteristic (ROC) curve. In this paper we investigate the generalization performance of online learning algorithms with pairwise loss functions. We…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “pairwise loss functions”
In this paper we study the stability and its trade-off with optimization error for stochastic gradient descent (SGD) algorithms in the pairwise learning setting. Pairwise learning refers to a learning task which involves a loss function depending on pairs of instances among which notable examples are bipartite ranking,…
Doing well with less! On Sampling Techniques for Empirical Pairwise Loss Estimation/Minimizationstat.ML
Sampling a fraction of pairs can match full evaluation in machine learning losses.
problem High computational cost of full pairwise loss evaluation.
method Survey sampling techniques targeting informative pairs.
result Performance close to full pairwise evaluation achieved with frugal sampling.
Pairwise learning usually refers to a learning task which involves a loss function depending on pairs of examples, among which most notable ones include ranking, metric learning and AUC maximization. In this paper, we study an online algorithm for pairwise learning with a least-square loss function in an unconstrained …
Regularized empirical risk minimization including support vector machines plays an important role in machine learning theory. In this paper regularized pairwise learning (RPL) methods based on kernels will be investigated. One example is regularized minimization of the error entropy loss which has recently attracted qu…
Cross-entropy loss linked to metric learning, outperforming complex pairwise losses.
problem Improving metric learning performance without complex optimization schemes.
method Theoretical analysis linking cross-entropy to pairwise losses, showing cross-entropy as an upper bound and equivalent to mutual information maximization.
result Minimizing cross-entropy is equivalent to maximizing mutual information, leading to state-of-the-art performance.
Introduces CHL, a new loss function for continuous similarity learning.
problem Binary similarity learning limitations.
method CHL is a novel loss function that generalizes histogram loss to continuous similarities.
result CHL solves a wider range of tasks including similarity learning, representation learning, and data visualization.
In this paper, we study the generalization properties of online learning based stochastic methods for supervised learning problems where the loss function is dependent on more than one training sample (e.g., metric learning, ranking). We present a generic decoupling technique that enables us to provide Rademacher compl…
Framework learns sentence order from paragraphs using attention and transformer networks.
problem Learning to order sentences from a paragraph.
method Bidirectional sentence encoder and self-attention transformer network for ranking.
result Framework outperforms state-of-the-art methods on sentence ordering and discrimination tasks.
We propose a new model for supervised learning to rank. In our model, the relevance labels are assumed to follow a categorical distribution whose probabilities are constructed based on a scoring function. We optimize the training objective with respect to the multivariate categorical variables with an unbiased and low-…
The problem of bipartite ranking, where instances are labeled positive or negative and the goal is to learn a scoring function that minimizes the probability of mis-ranking a pair of positive and negative instances (or equivalently, that maximizes the area under the ROC curve), has been widely studied in recent years. …
We decompose the squared price-of-risk premium into three components: intervention-stable premium, confounding wedge, and information loss.
problem Decomposing the squared price-of-risk premium into its components
method Identifying an order-three obstruction to aggregation across portfolios
result The decomposition is estimable and detectable with a permutation-calibrated screen
We formulate learning of a binary autoencoder as a biconvex optimization problem which learns from the pairwise correlations between encoded and decoded bits. Among all possible algorithms that use this information, ours finds the autoencoder that reconstructs its inputs with worst-case optimal loss. The optimal decode…
New hashing method improves document retrieval precision.
problem Efficiently retrieving similar documents from large text databases.
method Pairwise supervised hashing with Bernoulli VAE and unbiased gradient estimator.
result Superior performance compared to existing methods.
A new framework for clustering with uncertainty quantification.
problem Lack of uncertainty quantification in clustering methods.
method Generalized Bayes framework using Gibbs posteriors and loss functions.
result Efficient algorithms for clustering and uncertainty quantification.
Vector embedding is a foundational building block of many deep learning models, especially in natural language processing. In this paper, we present a theoretical framework for understanding the effect of dimensionality on vector embeddings. We observe that the distributional hypothesis, a governing principle of statis…
This paper improves multi-label ranking by reweighting univariate losses, enhancing consistency and performance.
problem Improving multi-label ranking performance while maintaining consistency.
method Systematic study of consistency and generalization error bounds for learning algorithms, proposing a reweighted univariate loss.
result Inconsistent pairwise losses can lead to better performance than consistent univariate losses in practice.
Proposes ConstraintMatch for semi-supervised clustering with unconstrained data.
problem Leveraging unconstrained data alongside constraints for clustering models.
method Semi-supervised context with pseudo-constraining and pseudo-labeling mechanisms.
result Demonstrates effectiveness of ConstraintMatch over baselines.
Ranking is a key aspect of many applications, such as information retrieval, question answering, ad placement and recommender systems. Learning to rank has the goal of estimating a ranking model automatically from training data. In practical settings, the task often reduces to estimating a rank functional of an object …
Pairwise quantile regression tackles similarity scoring in biometric systems.
problem Analyzing errors in similarity scoring for facial recognition.
method Established theoretical guarantees for pairwise quantile regression solutions, leveraging sharp concentration results for -processes.
result Proved generalization bounds and identified conditions for fast learning rates.
AUC (area under ROC curve) is an important evaluation criterion, which has been popularly used in many learning tasks such as class-imbalance learning, cost-sensitive learning, learning to rank, etc. Many learning approaches try to optimize AUC, while owing to the non-convexity and discontinuousness of AUC, almost all …
Low-rank matrix completion has achieved great success in many real-world data applications. A matrix factorization model that learns latent features is usually employed and, to improve prediction performance, the similarities between latent variables can be exploited by pairwise learning using the graph regularized mat…
The use of orthogonal projections on high-dimensional input and target data in learning frameworks is studied. First, we investigate the relations between two standard objectives in dimension reduction, preservation of variance and of pairwise relative distances. Investigations of their asymptotic correlation as well a…
We consider the problem of concurrent portfolio losses in two non-overlapping credit portfolios. In order to explore the full statistical dependence structure of such portfolio losses, we estimate their empirical pairwise copulas. Instead of a Gaussian dependence, we typically find a strong asymmetry in the copulas. Co…
In this paper, we consider unregularized online learning algorithms in a Reproducing Kernel Hilbert Spaces (RKHS). Firstly, we derive explicit convergence rates of the unregularized online learning algorithms for classification associated with a general gamma-activating loss (see Definition 1 in the paper). Our results…
Paper relaxes assumptions for non-parametric estimation in pairwise learning.
problem Generalization performance of non-parametric estimation for pairwise learning.
method Significantly relaxes restrictive assumptions, constructs structured deep ReLU neural network, and designs targeted hypothesis space.
result Establishes a sharp oracle inequality for empirical minimizer with general hypothesis space for Lipschitz continuous pairwise losses.
Unified framework for binary responses using AUC loss and low-rank constraint.
problem Statistical inefficiency and shared structure in fitting multiple binary responses.
method Pairwise AUC loss aggregation with low-rank constraint, scalable projected gradient descent.
result Unified framework outperforms likelihood-based approaches in challenging settings.
This paper evaluates various loss functions for Transformer models in stock ranking.
problem Evaluating loss functions for Transformer models in stock ranking.
method Systematic evaluation of advanced loss functions (pointwise, pairwise, listwise) on S&P 500 data.
result Different loss functions impact a model's ability to discern profitable relative orderings among assets.
AutoEmbedder clusters unlabeled data using semi-supervised DNN embedding.
problem Clustering unlabeled data efficiently and effectively.
method Semi-supervised DNN embedding system using Siamese network architecture.
result AutoEmbedder outperforms existing DNN-based semi-supervised methods.
Area under ROC (AUC) is an important metric for binary classification and bipartite ranking problems. However, it is difficult to directly optimizing AUC as a learning objective, so most existing algorithms are based on optimizing a surrogate loss to AUC. One significant drawback of these surrogate losses is that they …
In this work, we addressed the issue of applying a stochastic classifier and a local, fuzzy confusion matrix under the framework of multi-label classification. We proposed a novel solution to the problem of correcting label pairwise ensembles. The main step of the correction procedure is to compute classifier- specific…
We propose a novel autoencoding model called Pairwise Augmented GANs. We train a generator and an encoder jointly and in an adversarial manner. The generator network learns to sample realistic objects. In turn, the encoder network at the same time is trained to map the true data distribution to the prior in latent spac…
This paper extends Median-of-Means to new learning problems involving pairwise comparisons.
problem Learning from pairwise comparisons in machine learning.
method Segmenting data into blocks, comparing pairs of decision rules, and declaring the winner based on majority performance.
result The Median-of-Means approach maintains robustness and performance under various sampling schemes.
Paper introduces differential pairwise privacy for secure metric learning.
problem Securely measuring similarities of individuals given sensitive pairwise data.
method Develops differential pairwise privacy (DPP) to protect sensitive pairwise data.
result Achieves pairwise data privacy without significant performance loss.
This work presents a new strategy for multi-class classification that requires no class-specific labels, but instead leverages pairwise similarity between examples, which is a weaker form of annotation. The proposed method, meta classification learning, optimizes a binary classifier for pairwise similarity prediction a…
A simple framework improves deep metric learning performance.
problem Imbalanced data pairs in pairwise deep metric learning.
method Formulated a robust loss for balanced pairs over mini-batches, using distributionally robust optimization.
result Empirically outperforms state-of-the-art methods.
A new method uses vectorized summaries of persistence diagrams for efficient hypothesis testing.
problem Efficient hypothesis testing for large and complex persistence diagrams.
method Vectorized summaries of Betti functions and a new shuffling technique.
result The vectorized Betti function leads to competitive results compared to baseline methods.
Clustering using neural networks has recently demonstrated promising performance in machine learning and computer vision applications. However, the performance of current approaches is limited either by unsupervised learning or their dependence on large set of labeled data samples. In this paper, we propose ClusterNet …
Paper bridges ordinary-label and complementary-label learning frameworks.
problem Combining complementary-label learning with ordinary-label learning.
method Integrates loss functions for one-versus-all and pairwise classification.
result Derives classification risk and error bound for additivity and duality loss functions.
We propose a new class of convex penalty functions, called \emph{variational Gram functions} (VGFs), that can promote pairwise relations, such as orthogonality, among a set of vectors in a vector space. These functions can serve as regularizers in convex optimization problems arising from hierarchical classification, m…
This paper examines how different loss functions affect neural network features and performance.
problem Investigating which loss function is best for deep neural networks.
method Examining last-layer features of deep networks and drawing inspiration from the Neural Collapse phenomenon.
result All relevant loss functions (CE, LS, FL, MSE) produce equivalent features and similar performance.
Many machine learning problems can be formulated as predicting labels for a pair of objects. Problems of that kind are often referred to as pairwise learning, dyadic prediction or network inference problems. During the last decade kernel methods have played a dominant role in pairwise learning. They still obtain a stat…
CnGAN generates synthetic user preferences for non-overlapped users in cross-network recommender systems.
problem Cross-network recommender solutions ignore non-overlapped users, limiting their applicability.
method Multi-task learning, encoder-GAN architecture, user-based pairwise loss function.
result Generated user preferences improve recommendations for non-overlapped users, achieving superior performance.
Study learns linear utility functions from comparisons, showing learnability gaps between passive and active learning.
problem Learn linear utility functions from pairwise comparison queries.
method Analyzes passive and active learning settings, considering noise-free and noisy query responses.
result Efficient learnability of linear utilities in passive learning, but not for utility parameters without strong assumptions.
This paper provides a block coordinate descent algorithm to solve unconstrained optimization problems. In our algorithm, computation of function values or gradients is not required. Instead, pairwise comparison of function values is used. Our algorithm consists of two steps; one is the direction estimate step and the o…
The paper derives upper bounds on the MLE error for BTL model under general graphs.
problem Estimating the MLE of BTL model parameters with -loss under general graphs.
method Novel upper bounds on estimation error dependent on algebraic connectivity and graph topology.
result Upper bounds on error are sharp and match minimax lower bounds under certain graph topologies.
We consider the problem of rank loss minimization in the setting of multilabel classification, which is usually tackled by means of convex surrogate losses defined on pairs of labels. Very recently, this approach was put into question by a negative result showing that commonly used pairwise surrogate losses, such as ex…
Paper shows robustness of kernel-based pairwise learning without strict assumptions.
problem Statistical robustness of kernel-based pairwise learning under minimal conditions.
method No assumptions on input and output spaces; derives influence function and robustness.
result Qualitative robustness of kernel-based estimator established.