arXiv research

ALIS uses probabilistic sampling to minimize true loss in active learning.

Proposes a new loss function for causal machine-learning.

In this paper, we propose a Dual Focal Loss (DFL) function, as a replacement for the standard cross entropy (CE) function to achieve a better treatment of the unbalanced classes in a dataset. Our DFL method is an improvement on the recently reported Focal Loss (FL) cross-entropy function, which proposes a scaling metho…

Paper analyzes proper losses and their performance in machine learning tasks.

Local update methods' performance depends on learning rates, affecting convergence rates and alignment with true loss.

Paper develops a generative model using Wasserstein-2 loss.

New method estimates grouping loss in neural networks to improve confidence scores.

Focal loss improves classification but not class-posterior probability estimation.

Random Matrix Theory explains loss surface Hessians in neural networks.

In many applications of classifier learning, training data suffers from label noise. Deep networks are learned using huge training data where the problem of noisy labels is particularly relevant. The current techniques proposed for learning deep networks under label noise focus on modifying the network architecture and…

Stochastic gradient descent on l1 loss converges to true parameter in online robust regression.

A new method detects concept drift without true labels.

Generative Adversarial Networks (GANs) were intuitively and attractively explained under the perspective of game theory, wherein two involving parties are a discriminator and a generator. In this game, the task of the discriminator is to discriminate the real and generated (i.e., fake) data, whilst the task of the gene…

UMAP's true loss function differs from what was previously thought, focusing on nearest neighbor graph similarities.

The minimization of loss functions is the heart and soul of Machine Learning. In this paper, we propose an off-the-shelf optimization approach that can minimize virtually any non-differentiable and non-decomposable loss function (e.g. Miss-classification Rate, AUC, F1, Jaccard Index, Mathew Correlation Coefficient, etc…

Generative Adversarial Networks improve robust statistics for various distributions.

Early stopping improves neural networks' performance on binary classification tasks.

This paper improves risk bounds and calibration for smart predict-then-optimize method.

New method proves asymptotic normality for matrix sensing problems.

Improved computational complexity in statistical models using second-order information.

Study connects covariance cleaning theory to information theory for heavy-tailed distributions.

New method ensures generated data statistics match real data distributions.

It is now known that an extended Gaussian process model equipped with rescaling can adapt to different smoothness levels of a function valued parameter in many nonparametric Bayesian analyses, offering a posterior convergence rate that is optimal (up to logarithmic factors) for the smoothness class the true function be…

Proposes squentropy loss for improved classification accuracy and model calibration.

MRCs minimize worst-case expected 0-1 loss and provide performance guarantees.

The paper analyzes the generalization of deep neural networks for metric and similarity learning.

Paper proposes a new loss function for PU learning without negative examples.

Two-layer ReLU networks often converge to simpler solutions, improving generalization.

The paper analyzes error bounds and KL properties for noisy matrix recovery problems.

Sign-RIP improves robust low-rank matrix recovery by preserving norms even with corrupted measurements.

This paper examines how different decoding algorithms for LLMs align with various goals.

We present a powerful new loss function and training scheme for learning binary hash codes with any differentiable model and similarity function. Our loss function improves over prior methods by using log likelihood loss on top of an accurate approximation for the probability that two inputs fall within a Hamming dista…

Consider a transverse knot which is the binding of an open book for the ambient contact manifold. In this paper, we show that the transverse invariants defined by Lisca, Ozsvath, Stipsicz, and Szabo (LOSS) are nonvanishing for such transverse knots. This is true regardless of whether or not the ambient contact structur…

This work focuses on dynamic regret of online convex optimization that compares the performance of online learning to a clairvoyant who knows the sequence of loss functions in advance and hence selects the minimizer of the loss function at each step. By assuming that the clairvoyant moves slowly (i.e., the minimizers c…

In this paper, we propose a novel asymmetric $ε$-insensitive pinball loss function for quantile estimation. There exists some pinball loss functions which attempt to incorporate the $ε$-insensitive zone approach in it but, they fail to extend the $ε$-insensitive approach for quantile estimation in true sense. The propo…

The paper explores how the depth of neural networks affects their ability to represent data accurately.

New optimization method improves AUC for binary classification and changepoint detection.

Proposes a new generalization bound for Bayesian deep nets without strict assumptions.

New method trains generative models without discriminators, improving stability and accuracy.

We prove that the empirical risk of most well-known loss functions factors into a linear term aggregating all labels with a term that is label free, and can further be expressed by sums of the loss. This holds true even for non-smooth, non-convex losses and in any RKHS. The first term is a (kernel) mean operator --the …

UREs lead to overfitting in complex models, especially in complementary label learning.

Neural networks are a powerful class of functions that can be trained with simple gradient descent to achieve state-of-the-art performance on a variety of applications. Despite their practical success, there is a paucity of results that provide theoretical guarantees on why they are so effective. Lying in the center of…

The paper explains spikes in training loss as catapults, improving feature learning and generalization.

In this paper we refine the process of computing calibration functions for a number of multiclass classification surrogate losses. Calibration functions are a powerful tool for easily converting bounds for the surrogate risk (which can be computed through well-known methods) into bounds for the true risk, the probabili…

New insights into convergence and accuracy trade-offs in federated and meta-learning.

This note is a response to [7] in which it is claimed that [13, Proposition 11] is false. We demonstrate here that this assertion in [7] is false, and is based on a misreading of the notion of set membership in [13, Proposition 11]. We maintain that [13, Proposition 11] is true. ([7] = arXiv:1809.00593, [13] = arXiv:15…

We propose a novel autoencoding model called Pairwise Augmented GANs. We train a generator and an encoder jointly and in an adversarial manner. The generator network learns to sample realistic objects. In turn, the encoder network at the same time is trained to map the true data distribution to the prior in latent spac…

Proposes SOVR loss to improve adversarial robustness by increasing logit margins.