arXiv research

We study proper losses for discrete generative models without knowing the target distribution.

Paper proposes a new loss function for conditional models using soft targets.

We establish linear regret bounds for convex smooth losses using Fenchel-Young losses.

New bounds show polyhedral surrogates are optimal for generalization.

Deep generative networks can simulate from a complex target distribution, by minimizing a loss with respect to samples from that distribution. However, often we do not have direct access to our target distribution - our data may be subject to sample selection bias, or may be from a different but related distribution. W…

The use of orthogonal projections on high-dimensional input and target data in learning frameworks is studied. First, we investigate the relations between two standard objectives in dimension reduction, preservation of variance and of pairwise relative distances. Investigations of their asymptotic correlation as well a…

Structured entropy improves classification performance on structured targets.

This paper consider penalized empirical loss minimization of convex loss functions with unknown non-linear target functions. Using the elastic net penalty we establish a finite sample oracle inequality which bounds the loss of our estimator from above with high probability. If the unknown target is linear this inequali…

This paper calculates worst-case target semi-variances for uncertain losses.

When optimizing against the mean loss over a distribution of predictions in the context of a regression task, then even if there is a distribution of targets the optimal prediction distribution is always a delta function at a single value. Methods of constructing generative models need to overcome this tendency. We con…

Paper develops upper-bounds for target general loss in multiple source DA and DG settings.

Improved sampling via learned diffusions using variational losses.

Improves score estimation for noised targets using known clean scores.

Paper explains learning property of logistic and softmax losses for balanced and imbalanced class data.

Understanding and evaluating the robustness of neural networks under adversarial settings is a subject of growing interest. Attacks proposed in the literature usually work with models trained to minimize cross-entropy loss and output softmax probabilities. In this work, we present interesting experimental results that …

We investigate the use of a non-parametric independence measure, the Hilbert-Schmidt Independence Criterion (HSIC), as a loss-function for learning robust regression and classification models. This loss-function encourages learning models where the distribution of the residuals between the label and the model predictio…

In few-shot learning, a machine learning system learns from a small set of labelled examples relating to a specific task, such that it can generalize to new examples of the same task. Given the limited availability of labelled examples in such tasks, we wish to make use of all the information we can. Usually a model le…

A novel minimax classifier tackles imbalanced datasets with few minority samples.

We study prediction and estimation problems using empirical risk minimization, relative to a general convex loss function. We obtain sharp error rates even when concentration is false or is very restricted, for example, in heavy-tailed scenarios. Our results show that the error rate depends on two parameters: one captu…

We study a stochastic game where one player tries to find a strategy such that the state process reaches a target of controlled-loss-type, no matter which action is chosen by the other player. We provide, in a general setup, a relaxed geometric dynamic programming principle for this problem and derive, for the case of …

TILT improves target domain performance by penalizing an auxiliary component on unlabeled target inputs.

We explore methods of producing adversarial examples on deep generative models such as the variational autoencoder (VAE) and the VAE-GAN. Deep learning architectures are known to be vulnerable to adversarial examples, but previous work has focused on the application of adversarial examples to classification tasks. Deep…

Fisher loss improves deep domain adaptation by learning discriminative within-class compact and between-class separable representations.

This work introduces a new method for coupling base and target densities in generative models.

Malware detection is a popular application of Machine Learning for Information Security (ML-Sec), in which an ML classifier is trained to predict whether a given file is malware or benignware. Parameters of this classifier are typically optimized such that outputs from the model over a set of input samples most closely…

Enhanced $H$-consistency bounds derived under relaxed conditions.

There is growing evidence that converting targets to soft targets in supervised learning can provide considerable gains in performance. Much of this work has considered classification, converting hard zero-one values to soft labels---such as by adding label noise, incorporating label ambiguity or using distillation. In…

Poisoning attack is identified as a severe security threat to machine learning algorithms. In many applications, for example, deep neural network (DNN) models collect public data as the inputs to perform re-training, where the input data can be poisoned. Although poisoning attack against support vector machines (SVM) h…

Enhances ENet's prediction accuracy while maintaining uncertainty estimation.

Proposes Deep LTMLE for estimating dynamic treatment effects in longitudinal studies.

Diffusion models accurately recover mixture weights from generated samples despite score function insensitivity.

This paper addresses tracking of a moving target in a multi-agent network. The target follows a linear dynamics corrupted by an adversarial noise, i.e., the noise is not generated from a statistical distribution. The location of the target at each time induces a global time-varying loss function, and the global loss is…

New framework embeds generalization in learning dynamics using large deviation theory.

A new method for domain generalization using unlabeled data.

Improved loss scaling for stochastic momentum algorithms in high dimensions.

Study on $H$-consistency bounds for machine learning surrogates.

Quantum models face barren plateaus, but specific losses can be trainable.

A new method selects training samples for fine-tuning using validation set inference.

Theoretical framework for target propagation shows differences from backpropagation.

New method recalibrates VaR for option books, reducing forecast errors.

We analyze the generalization and robustness of the batched weighted average algorithm for V-geometrically ergodic Markov data. This algorithm is a good alternative to the empirical risk minimization algorithm when the latter suffers from overfitting or when optimizing the empirical risk is hard. For the generalization…

Data augmentation (DA) has been widely utilized to improve generalization in training deep neural networks. Recently, human-designed data augmentation has been gradually replaced by automatically learned augmentation policy. Through finding the best policy in well-designed search space of data augmentation, AutoAugment…

In this work we study loss functions for learning and evaluating probability distributions over large discrete domains. Unlike classification or regression where a wide variety of loss functions are used, in the distribution learning and density estimation literature, very few losses outside the dominant $log\ loss$ ar…

Stable GFlowNets prevent loss spikes and mode collapse in training.

This work analyzes two methods for combining multiple binary labels in bipartite ranking.

Paper proposes AXE loss for non-autoregressive machine translation, improving performance.

End-to-end analysis of SGD for STL with adaptive sub-sampling.

EGFs use ergodicity to simplify generative flows for easier training and imitation learning.