Label smoothing improves model robustness against misspecification.
problem Improving model robustness against model misspecification.
method Introducing modified label smoothing (MLSLR) that maintains consistent probability estimation while modifying the loss function.
result MLSLR exhibits higher robustness against model misspecification than conventional label smoothing.
ModHiFi identifies critical components for model modification without gradients or loss function.
problem Modifying open weight models without access to training data or loss function.
method Theoretical analysis of Lipschitz-continuous networks, Subset Fidelity metric, and ModHiFi algorithm.
result ModHiFi-P and ModHiFi-U achieve significant performance improvements in model pruning and unlearning.
New loss function equivalence reveals PER's uniform sampling can be improved.
problem Improving Prioritized Experience Replay (PER) for better learning efficiency.
method Transforming non-uniformly sampled data loss functions into uniformly sampled ones.
result Some environments can replace PER with a new loss function without performance loss.
Paper connects sampling and labeling biases in large-output spaces.
problem Efficient training in large-output spaces with label imbalance.
method Unified approach to address sampling and labeling biases.
result Different negative sampling schemes trade-off performance on dominant and rare labels.
Understanding and evaluating the robustness of neural networks under adversarial settings is a subject of growing interest. Attacks proposed in the literature usually work with models trained to minimize cross-entropy loss and output softmax probabilities. In this work, we present interesting experimental results that …
We present the Tamed Cross Entropy (TCE) loss function, a robust derivative of the standard Cross Entropy (CE) loss used in deep learning for classification tasks. However, unlike other robust losses, the TCE loss is designed to exhibit the same training properties than the CE loss in noiseless scenarios. Therefore, th…
Prb-GAN uses dropout and variational inference to improve GAN performance.
problem GANs struggle with mode loss and training instability.
method Introduces Prb-GANs with dropout and variational inference for parameter distribution.
result Improves GAN performance through dropout and variational inference.
The one-bit quantization is implemented by one single comparator that operates at low power and a high rate. Hence one-bit compressive sensing (1bit-CS) becomes attractive in signal processing. When measurements are corrupted by noise during signal acquisition and transmission, 1bit-CS is usually modeled as minimizing …
Thompson Sampling remains differentially private with minimal modifications.
problem Ensuring privacy in Thompson Sampling for multi-arm bandits.
method Demonstrated differential privacy of original Thompson Sampling, provided per-round guarantees, and introduced modifications for tighter privacy.
result Privacy guarantees can be tuned by modifying the algorithm, and these modifications impact expected regret.
Overfitting in deep learning has been the focus of a number of recent works, yet its exact impact on the behavior of neural networks is not well understood. This study analyzes overfitting by examining how the distribution of logits alters in relation to how much the model overfits. Specifically, we find that when trai…
Training Generative Adversarial Networks (GANs) is notoriously challenging. We propose and study an architectural modification, self-modulation, which improves GAN performance across different data sets, architectures, losses, regularizers, and hyperparameter settings. Intuitively, self-modulation allows the intermedia…
SurvNAM explains survival model predictions using machine learning.
problem Explaining predictions of black-box survival models.
method SurvNAM is a modified Neural Additive Model (NAM) trained with a specific loss function based on Cox model and GAM.
result SurvNAM allows local and global explanation of survival model predictions.
In order to push the performance on realistic computer vision tasks, the number of classes in modern benchmark datasets has significantly increased in recent years. This increase in the number of classes comes along with increased ambiguity between the class labels, raising the question if top-1 error is the right perf…
In this study, we consider classification problems based on neural networks in data-imbalanced environment. Learning from an imbalanced data set is one of the most important and practical problems in the field of machine learning. A weighted loss function based on cost-sensitive approach is a well-known effective metho…
New method estimates insurance risk dependencies.
problem Complex dependence between insurance risks.
method Modified continuous generalised method of moments (CGMM).
result Comparable estimators to Maximum Likelihood Estimation.
A new adversarial attack method using structured search and contextual bandits.
problem Black-box adversarial attacks on deep learning models.
method Structured search space and Bayesian optimization for contextual bandits.
result Achieves state-of-the-art success rates and query efficiencies.
We deal with the \textit{selective classification} problem (supervised-learning problem with a rejection option), where we want to achieve the best performance at a certain level of coverage of the data. We transform the original m-class classification problem to (m+1)-class where the (m+1)-th class represents th…
A new BN method corrects size mismatch in WLF for imbalanced data.
problem Learning from imbalanced datasets in neural networks.
method Proposes weighted batch normalization (WBN) to correct size mismatch between BN and WLF.
result WBN corrects size mismatch and improves classification performance in imbalanced datasets.
LQF linearizes deep models for better interpretability.
problem Lack of interpretability in deep neural networks.
method Simple modifications to architecture, loss function, and optimization.
result Comparable performance to non-linear fine-tuning, with interpretability.
New framework for learning from imbalanced data with theoretical guarantees.
problem Class imbalance in machine learning, especially in multi-class problems.
method Theoretical framework and new margin loss function for imbalanced classification.
result Proves strong H-consistency of the proposed margin loss function. A hybrid loss framework improves time series forecasting by balancing global and component errors.
problem Current time series methods may prioritize less significant sub-series, leading to forecasting bias.
method Proposes a hybrid loss framework combining global and component losses, dynamically adjusting weights.
result Improves time series forecasting performance by 0.5-2% on average.
Gradient EM converges exponentially to optimal solution in agnostic mixtures.
problem Fitting k parametric functions to given data points without a generative model. method Gradient EM algorithm for agnostic mixtures of arbitrary parametric functions.
result Gradient EM converges exponentially to population loss minimizers with high probability.
Paper corrects GIRP algorithm to ensure isotonic models.
problem GIRP algorithm fails to produce isotonic models.
method Modified GIRP algorithm with binary partitioning.
result Correct solution exists and can be found.
We provide novel theoretical results regarding local optima of regularized M-estimators, allowing for nonconvexity in both loss and penalty functions. Under restricted strong convexity on the loss and suitable regularity conditions on the penalty, we prove that \emph{any stationary point} of the composite objective f…
Substantial evidence indicates that major psychiatric disorders are associated with distributed neural dysconnectivity, leading to strong interest in using neuroimaging methods to accurately predict disorder status. In this work, we are specifically interested in a multivariate approach that uses features derived from …
Training activation quantized neural networks involves minimizing a piecewise constant function whose gradient vanishes almost everywhere, which is undesirable for the standard back-propagation or chain rule. An empirical way around this issue is to use a straight-through estimator (STE) (Bengio et al., 2013) in the ba…
Reviews recent findings on neural network landscapes.
problem Non-convexity of loss functions causing bad landscapes.
method Rigorous geometric analysis and empirical exploration.
result Wide neural nets may have sub-optimal local minima.
Modified BFGS and LBFGS++ libraries boost performance for non-parallelizable functions.
problem Improving performance of non-parallelizable functions using SIMD and AAD.
method Modifications to BFGS and LBFGS++ libraries, utilizing SIMD and Automatic Differentiation (AAD).
result Up to 3.8 times faster for European Swaption curve calibration and 1.4 times faster for LMM model calibration.
Improves GANs by incorporating class information with a multi-hinge loss.
problem Improving GANs to better respect class information.
method Proposes a multi-class generalization of the Hinge loss for GANs.
result Improves Inception Scores and Frechet Inception Distance on Imagenet.
Develops a comprehensive theory of corruption in supervised learning.
problem Widespread corruption in data collection affects supervised learning problems.
method Introduces a general theory of corruption using Markov kernels, distinguishing and comparing corruption types.
result Establishes a unified framework for corruption types and develops mitigation strategies.
New framework to understand and exploit curvature in deep learning loss landscapes.
problem Understanding and optimizing the loss landscape in deep learning models.
method New conceptual framework and techniques to estimate and exploit curvature of expected loss changes.
result Alice algorithm optimizes training by incorporating curvature terms and step bounds.
We propose a new stochastic optimization framework for empirical risk minimization problems such as those that arise in machine learning. The traditional approaches, such as (mini-batch) stochastic gradient descent (SGD), utilize an unbiased gradient estimator of the empirical average loss. In contrast, we develop a co…
We derive the fast convergence rates of a deep neural network (DNN) classifier with the rectified linear unit (ReLU) activation function learned using the hinge loss. We consider three cases for a true model: (1) a smooth decision boundary, (2) smooth conditional class probability, and (3) the margin condition (i.e., t…
New method wraps black-box classifiers to reduce bias.
problem Reduction of bias in black-box predictions.
method Post-processing with α-trees and boosting algorithms.
result Demonstrated effectiveness in reducing bias across multiple fairness metrics.
Solves regression problems with CP by converting to classification.
problem Challenges in CP for heteroscedastic, multimodal, or skewed regression outputs.
method Converts regression to classification, uses CP for classification to obtain CP sets for regression.
result Simple approach yields good results on practical problems.
Paper proposes a method to improve deep neural networks' confidence estimates.
problem Overconfident predictions limit practical use of deep neural networks in safety-critical applications.
method Proposes a novel loss function, Correctness Ranking Loss, to regularize class probabilities.
result The method produces well-ranked confidence estimates and is effective for out-of-distribution detection and active learning.
Computational methods that predict differential gene expression from histone modification signals are highly desirable for understanding how histone modifications control the functional heterogeneity of cells through influencing differential gene regulation. Recent studies either failed to capture combinatorial effects…
The paper examines bounds for stop-loss payoffs using transformed random variables.
problem Bounding stop-loss payoffs for a difference of two random variables.
method Analyzes crossing points of cdfs of original and transformed random variables.
result Unique pairwise crossing points for mortality-linked securities under symmetric copulas.
In classical Q-learning, the objective is to maximize the sum of discounted rewards through iteratively using the Bellman equation as an update, in an attempt to estimate the action value function of the optimal policy. Conventionally, the loss function is defined as the temporal difference between the action value and…
DenseHMM improves HMMs by learning dense representations that enable gradient-based optimization.
problem Learning dense representations for hidden states and observables in HMMs.
method DenseHMM uses kernelized transition probabilities and two optimization schemes.
result DenseHMM achieves superior performance and expressiveness compared to standard HMMs.
New method prevents class collapse in metric learning with margin-based losses.
problem Class collapse in metric learning due to diverse intra-class samples.
method Proposed a sampling method to select nearest same-class samples as positive elements in tuple.
result Demonstrated clear benefits on various fine-grained image retrieval datasets.
New method learns multi-class from single-class data with confidences.
problem Learning multi-class from single-class data without additional data.
method Empirical risk minimization framework for multi-class classification.
result Method achieves Bayes-consistency with noisy confidences.
Random Fourier features is a widely used, simple, and effective technique for scaling up kernel methods. The existing theoretical analysis of the approach, however, remains focused on specific learning tasks and typically gives pessimistic bounds which are at odds with the empirical results. We tackle these problems an…
MCD automates counterfactual design searches for multi-modal tasks.
problem Designing for multi-objective goals and complex constraints.
method Model-agnostic counterfactual search method for multi-modal design modifications.
result MCD streamlines and automates counterfactual search, recommending effective design modifications.
This work improves testing of machine learning model modifications using novel statistical methods.
problem Overfitting and conservative Bonferroni correction when testing multiple model modifications.
method Introduces alpha-recycling and SRGPs to control error rate and approve more beneficial modifications.
result Novel statistical methods approve a higher number of beneficial modifications than previous approaches.
Simple image augmentation boosts deep RL from pixels.
problem Training deep RL models from raw pixel inputs is challenging.
method Applying standard computer vision data augmentation to reinforcement learning.
result Our method significantly improves SAC's performance on DeepMind control suite.
Study improves interpretability in generative models by disentangling latent variables in scientific datasets.
problem Extracting generative factors from complex, high-dimensional datasets in unsupervised or semi-supervised settings.
method Introducing Aux-VAE, a novel architecture within the VAE framework, which disentangles latent variables by guiding them with auxiliary variables.
result Aux-VAE achieves disentanglement with minimal modifications to the standard VAE loss function, validated on multiple datasets.
G-TRACER optimizes deep learning by promoting flat minima.
problem Promoting generalization in deep learning architectures.
method Geometric TRACE Ratio regularization, curvature-regularized optimizers.
result Converges to a neighborhood of local minima of unregularized objective.