arXiv research

This work examines uncertainty sampling in binary classification using equivalent loss.

New loss function equivalence reveals PER's uniform sampling can be improved.

Research characterizes learnability of multilabel ranking problems.

New method simplifies checking consistency of differentiable loss functions.

Improved online learning for hidden-convex losses achieves optimal regret.

Paper explores connections between loss functions and consistency in binary classification and regression.

AUC (area under ROC curve) is an important evaluation criterion, which has been popularly used in many learning tasks such as class-imbalance learning, cost-sensitive learning, learning to rank, etc. Many learning approaches try to optimize AUC, while owing to the non-convexity and discontinuousness of AUC, almost all …

We introduce a class of utility-based market makers that always accept orders at their risk-neutral prices. We derive necessary and sufficient conditions for such market makers to have bounded loss. We prove that hyperbolic absolute risk aversion utility market makers are equivalent to weighted pseudospherical scoring …

Optimal resource allocation in censored semi-bandits with unknown thresholds.

Learning a generative model is a key component of model-based reinforcement learning. Though learning a good model in the tabular setting is a simple task, learning a useful model in the approximate setting is challenging. In this context, an important question is the loss function used for model learning as varying th…

This work establishes the equivalence between neural networks and support vector machines.

PAC-Bayesian bounds for MLPs with cross entropy loss validated.

Unified framework approximates gradient descent's implicit bias in high dimensions.

We formalize and study the natural approach of designing convex surrogate loss functions via embeddings, for problems such as classification, ranking, or structured prediction. In this approach, one embeds each of the finitely many predictions (e.g.\ rankings) as a point in $\mathbb{R}^d$, assigns the original loss val…

The paper develops robust risk measures for uncertain loss positions.

Dynamic regret minimization is shown equivalent to static regret minimization for linear losses.

We present $α$-loss, $α\in [1,\infty]$, a tunable loss function for binary classification that bridges log-loss ($α=1$) and $0$-$1$ loss ($α= \infty$). We prove that $α$-loss has an equivalent margin-based form and is classification-calibrated, two desirable properties for a good surrogate loss function for the ideal y…

This paper introduces new risk measures for evaluating losses with varying time horizons.

In deep neural network, the cross-entropy loss function is commonly used for classification. Minimizing cross-entropy is equivalent to maximizing likelihood under assumptions of uniform feature and class distributions. It belongs to generative training criteria which does not directly discriminate correct class from co…

Investors face constraints in Heston's model; optimal allocation differs from naive capped strategy.

In this paper, we study censored Semi-Bandits, a novel variant of the semi-bandits problem. The learner is assumed to have a fixed amount of resources, which it allocates to the arms at each time step. The loss observed from an arm is random and depends on the amount of resources allocated to it. More specifically, the…

Study shows transductive learning is equivalent to PAC learning for most natural loss functions.

This paper examines how different loss functions affect neural network features and performance.

Gradient descent is a simple and widely used optimization method for machine learning. For homogeneous linear classifiers applied to separable data, gradient descent has been shown to converge to the maximal margin (or equivalently, the minimal norm) solution for various smooth loss functions. The previous theory does …

The paper studies risk-sensitive learning schemes and provides learning bounds for empirical OCE minimizers.

The paper introduces a new loss function to prevent overfitting in semi-supervised graph networks.

Differentiable structure learning addresses DAGs with multiple global minimizers.

In this work, we introduce the {\em average top-$k$} (\atk) loss as a new aggregate loss for supervised learning, which is the average over the $k$ largest individual losses over a training dataset. We show that the \atk loss is a natural generalization of the two widely used aggregate losses, namely the average loss a…

In this work we establish the equivalence of algorithmic regularization and explicit convex penalization for generic convex losses. We introduce a geometric condition for the optimization path of a convex function, and show that if such a condition is satisfied, the optimization path of an iterative algorithm on the un…

Biased mean regression estimates factors exceeding expected loss or radiation release severity.

Using established principles from Statistics and Information Theory, we show that invariance to nuisance factors in a deep neural network is equivalent to information minimality of the learned representation, and that stacking layers and injecting noise during training naturally bias the network towards learning invari…

We determine the optimal amount of life insurance for a household of two wage earners. We consider the simple case of exponential utility, thereby removing wealth as a factor in buying life insurance, while retaining the relationship among life insurance, income, and the probability of dying and thus losing that income…

Paper shows equivalence between two alignment methods and introduces a new algorithm.

The notion of developing statistical methods in machine learning which are robust to adversarial perturbations in the underlying data has been the subject of increasing interest in recent years. A common feature of this work is that the adversarial robustification often corresponds exactly to regularization methods whi…

Mitigates overoptimization in RLHF by reformulating SFT loss as a preference optimization loss.

The permutation symmetry of neurons in each layer of a deep neural network gives rise not only to multiple equivalent global minima of the loss function, but also to first-order saddle points located on the path between the global minima. In a network of $d-1$ hidden layers with $n_k$ neurons in layers $k = 1, \ldots, …

Study optimal investment decisions for diverse risk-tolerant agents.

Symmetrizes loss functions to improve neural network robustness against noisy labels.

Improved diffusion bridge sampling with rKL-LD loss.

Generative Adversarial Networks (GANs) were intuitively and attractively explained under the perspective of game theory, wherein two involving parties are a discriminator and a generator. In this game, the task of the discriminator is to discriminate the real and generated (i.e., fake) data, whilst the task of the gene…

A new learning method using hyperbolic geometry for class labels.

We study the asymptotic behavior of the difference between the values at risk VaR(L) and VaR(L+S) for heavy tailed random variables L and S for application in sensitivity analysis of quantitative operational risk management within the framework of the advanced measurement approach of Basel II (and III). Here L describe…

Submodularity is studied for convex risk measures, including Expected Shortfall.

We establish linear regret bounds for convex smooth losses using Fenchel-Young losses.

New loss functions based on f-divergences improve language model performance.

Paper improves PAC-Bayes bounds for various loss types.

We develop efficient algorithms to train $\ell_1$-regularized linear classifiers with large dimensionality $d$ of the feature space, number of classes $k$, and sample size $n$. Our focus is on a special class of losses that includes, in particular, the multiclass hinge and logistic losses. Our approach combines several…

New property shows VaR subadditivity for comonotonic loss variables.