arXiv research

Label smoothing improves model robustness against misspecification.

ModHiFi identifies critical components for model modification without gradients or loss function.

New loss function equivalence reveals PER's uniform sampling can be improved.

Paper connects sampling and labeling biases in large-output spaces.

Understanding and evaluating the robustness of neural networks under adversarial settings is a subject of growing interest. Attacks proposed in the literature usually work with models trained to minimize cross-entropy loss and output softmax probabilities. In this work, we present interesting experimental results that …

We present the Tamed Cross Entropy (TCE) loss function, a robust derivative of the standard Cross Entropy (CE) loss used in deep learning for classification tasks. However, unlike other robust losses, the TCE loss is designed to exhibit the same training properties than the CE loss in noiseless scenarios. Therefore, th…

Prb-GAN uses dropout and variational inference to improve GAN performance.

The one-bit quantization is implemented by one single comparator that operates at low power and a high rate. Hence one-bit compressive sensing (1bit-CS) becomes attractive in signal processing. When measurements are corrupted by noise during signal acquisition and transmission, 1bit-CS is usually modeled as minimizing …

Thompson Sampling remains differentially private with minimal modifications.

Overfitting in deep learning has been the focus of a number of recent works, yet its exact impact on the behavior of neural networks is not well understood. This study analyzes overfitting by examining how the distribution of logits alters in relation to how much the model overfits. Specifically, we find that when trai…

Training Generative Adversarial Networks (GANs) is notoriously challenging. We propose and study an architectural modification, self-modulation, which improves GAN performance across different data sets, architectures, losses, regularizers, and hyperparameter settings. Intuitively, self-modulation allows the intermedia…

In order to push the performance on realistic computer vision tasks, the number of classes in modern benchmark datasets has significantly increased in recent years. This increase in the number of classes comes along with increased ambiguity between the class labels, raising the question if top-1 error is the right perf…

SurvNAM explains survival model predictions using machine learning.

In this study, we consider classification problems based on neural networks in data-imbalanced environment. Learning from an imbalanced data set is one of the most important and practical problems in the field of machine learning. A weighted loss function based on cost-sensitive approach is a well-known effective metho…

New method estimates insurance risk dependencies.

A new adversarial attack method using structured search and contextual bandits.

We deal with the \textit{selective classification} problem (supervised-learning problem with a rejection option), where we want to achieve the best performance at a certain level of coverage of the data. We transform the original $m$-class classification problem to $(m+1)$-class where the $(m+1)$-th class represents th…

LQF linearizes deep models for better interpretability.

New framework for learning from imbalanced data with theoretical guarantees.

A hybrid loss framework improves time series forecasting by balancing global and component errors.

Gradient EM converges exponentially to optimal solution in agnostic mixtures.

Paper corrects GIRP algorithm to ensure isotonic models.

We provide novel theoretical results regarding local optima of regularized $M$-estimators, allowing for nonconvexity in both loss and penalty functions. Under restricted strong convexity on the loss and suitable regularity conditions on the penalty, we prove that \emph{any stationary point} of the composite objective f…

Substantial evidence indicates that major psychiatric disorders are associated with distributed neural dysconnectivity, leading to strong interest in using neuroimaging methods to accurately predict disorder status. In this work, we are specifically interested in a multivariate approach that uses features derived from …

Training activation quantized neural networks involves minimizing a piecewise constant function whose gradient vanishes almost everywhere, which is undesirable for the standard back-propagation or chain rule. An empirical way around this issue is to use a straight-through estimator (STE) (Bengio et al., 2013) in the ba…

In this study, classification problems based on feedforward neural networks in a data-imbalanced environment are considered. Learning from an imbalanced dataset is one of the most important practical problems in the field of machine learning. A weighted loss function (WLF) based on a cost-sensitive approach is a well-k…

Reviews recent findings on neural network landscapes.

Modified BFGS and LBFGS++ libraries boost performance for non-parallelizable functions.

Develops a comprehensive theory of corruption in supervised learning.

We propose a new algorithm to incorporate class conditional information into the critic of GANs via a multi-class generalization of the commonly used Hinge loss that is compatible with both supervised and semi-supervised settings. We study the compromise between training a state of the art generator and an accurate cla…

New framework to understand and exploit curvature in deep learning loss landscapes.

We propose a new stochastic optimization framework for empirical risk minimization problems such as those that arise in machine learning. The traditional approaches, such as (mini-batch) stochastic gradient descent (SGD), utilize an unbiased gradient estimator of the empirical average loss. In contrast, we develop a co…

We derive the fast convergence rates of a deep neural network (DNN) classifier with the rectified linear unit (ReLU) activation function learned using the hinge loss. We consider three cases for a true model: (1) a smooth decision boundary, (2) smooth conditional class probability, and (3) the margin condition (i.e., t…

New method wraps black-box classifiers to reduce bias.

Solves regression problems with CP by converting to classification.

Paper proposes a method to improve deep neural networks' confidence estimates.

Computational methods that predict differential gene expression from histone modification signals are highly desirable for understanding how histone modifications control the functional heterogeneity of cells through influencing differential gene regulation. Recent studies either failed to capture combinatorial effects…

The paper examines bounds for stop-loss payoffs using transformed random variables.

In classical Q-learning, the objective is to maximize the sum of discounted rewards through iteratively using the Bellman equation as an update, in an attempt to estimate the action value function of the optimal policy. Conventionally, the loss function is defined as the temporal difference between the action value and…

DenseHMM improves HMMs by learning dense representations that enable gradient-based optimization.

New method prevents class collapse in metric learning with margin-based losses.

New method learns multi-class from single-class data with confidences.

Random Fourier features is a widely used, simple, and effective technique for scaling up kernel methods. The existing theoretical analysis of the approach, however, remains focused on specific learning tasks and typically gives pessimistic bounds which are at odds with the empirical results. We tackle these problems an…

This work improves testing of machine learning model modifications using novel statistical methods.

MCD automates counterfactual design searches for multi-modal tasks.

Simple image augmentation boosts deep RL from pixels.

Study improves interpretability in generative models by disentangling latent variables in scientific datasets.

G-TRACER optimizes deep learning by promoting flat minima.