A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
This work shows that supervised contrastive learning achieves similar results to cross-entropy but requires more iterations.
problem The question of whether there are fundamental differences in representation geometry between supervised contrastive learning and cross-entropy.
method The authors prove that both losses attain their minimum when representations of each class collapse to the vertices of a regular simplex, and they empirically validate this finding.
result Supervised contrastive learning requires more iterations to reach a close-to-optimal state compared to cross-entropy, indicating different optimization behavior.
This paper shows cross-entropy can recover latent structures in supervised learning.
problem Understanding why supervised learning works well and how models learn interpretable factors of variation.
method Extending identifiability results to parametric instance discrimination, proving cross-entropy minimization can recover latent structures up to linear transformations.
result Models trained with cross-entropy can learn representations of ground-truth factors of variation up to a linear transformation.
In this paper, we propose a semi-supervised clustering method, CEC-IB, that models data with a set of Gaussian distributions and that retrieves clusters based on a partial labeling provided by the user (partition-level side information). By combining the ideas from cross-entropy clustering (CEC) with those from the inf…
Loss functions play a crucial role in deep metric learning thus a variety of them have been proposed. Some supervise the learning process by pairwise or tripletwise similarity constraints while others take advantage of structured similarity information among multiple data points. In this work, we approach deep metric l…
Inspired by the success of Convolutional Neural Networks (CNNs) for supervised prediction in images, we design the Deconvolutional Generative Model (DGM), a new probabilistic generative model whose inference calculations correspond to those in a given CNN architecture. The DGM uses a CNN to design the prior distributio…
Deep learning has been shown to achieve impressive results in several domains like computer vision and natural language processing. A key element of this success has been the development of new loss functions, like the popular cross-entropy loss, which has been shown to provide faster convergence and to reduce the vani…
Mutual information is widely applied to learn latent representations of observations, whilst its implication in classification neural networks remain to be better explained. We show that optimising the parameters of classification neural networks with softmax cross-entropy is equivalent to maximising the mutual informa…
The Yarowsky algorithm is a rule-based semi-supervised learning algorithm that has been successfully applied to some problems in computational linguistics. The algorithm was not mathematically well understood until (Abney 2004) which analyzed some specific variants of the algorithm, and also proposed some new algorithm…
DSL uses supervised learning to optimize portfolios, improving stability and performance.
problem Optimizing robust portfolios in financial markets.
method DSL reframes portfolio construction as a supervised learning problem, using cross-entropy loss and optimizing Sharpe or Sortino ratios. Deep Ensemble methods are employed to reduce variance.
result DSL outperforms traditional and machine learning methods, achieving higher median returns and more stable risk-adjusted performance.
Reinforcement learning (RL) is about sequential decision making and is traditionally opposed to supervised learning (SL) and unsupervised learning (USL). In RL, given the current state, the agent makes a decision that may influence the next state as opposed to SL (and USL) where, the next state remains the same, regard…
Cross-entropy loss linked to metric learning, outperforming complex pairwise losses.
problem Improving metric learning performance without complex optimization schemes.
method Theoretical analysis linking cross-entropy to pairwise losses, showing cross-entropy as an upper bound and equivalent to mutual information maximization.
result Minimizing cross-entropy is equivalent to maximizing mutual information, leading to state-of-the-art performance.
We present a method for training multi-label, massively multi-class image classification models, that is faster and more accurate than supervision via a sigmoid cross-entropy loss (logistic regression). Our method consists in embedding high-dimensional sparse labels onto a lower-dimensional dense sphere of unit-normed …
We propose a max-pooling based loss function for training Long Short-Term Memory (LSTM) networks for small-footprint keyword spotting (KWS), with low CPU, memory, and latency requirements. The max-pooling loss training can be further guided by initializing with a cross-entropy loss trained network. A posterior smoothin…
Synthesizing human's movements such as dancing is a flourishing research field which has several applications in computer graphics. Recent studies have demonstrated the advantages of deep neural networks (DNNs) for achieving remarkable performance in motion and music tasks with little effort for feature pre-processing.…
In deep neural network, the cross-entropy loss function is commonly used for classification. Minimizing cross-entropy is equivalent to maximizing likelihood under assumptions of uniform feature and class distributions. It belongs to generative training criteria which does not directly discriminate correct class from co…
The graph convolution network (GCN) is a widely-used facility to realize graph-based semi-supervised learning, which usually integrates node features and graph topologic information to build learning models. However, as for multi-label learning tasks, the supervision part of GCN simply minimizes the cross-entropy loss …
Convolution Neural Networks (CNN) have recently achieved state-of-the art performance on handwritten Chinese character recognition (HCCR). However, most of CNN models employ the SoftMax activation function and minimize cross entropy loss, which may cause loss of inter-class information. To cope with this problem, we pr…