arXiv research

Logit regularization induces logit clustering, affecting classifier performance.

Improves adversarial robustness by constraining logits with a bounded function.

LAWN normalizes logits to improve deep network adaptability and generalization.

Recently, Kannan et al. [2018] proposed several logit regularization methods to improve the adversarial robustness of classifiers. We show that the computationally fast methods they propose - Clean Logit Pairing (CLP) and Logit Squeezing (LSQ) - just make the gradient-based optimization problem of crafting adversarial …

ELM improves neural model embeddings for long-tail learning.

New algorithm avoids spurious sharpness minimization for NLP models.

While great progress has been made at making neural networks effective across a wide range of visual tasks, most models are surprisingly vulnerable. This frailness takes the form of small, carefully chosen perturbations of their input, known as adversarial examples, which represent a security threat for learned vision …

Adversarial training is one of the strongest defenses against adversarial attacks, but it requires adversarial examples to be generated for every mini-batch during optimization. The expense of producing these examples during training often precludes adversarial training from use on complex image datasets. In this study…

As datasets capturing human choices grow in richness and scale -- particularly in online domains -- there is an increasing need for choice models that escape traditional choice-theoretic axioms such as regularity, stochastic transitivity, and Luce's choice axiom. In this work we introduce the Pairwise Choice Markov Cha…

Logit-GFN accelerates GFlowNets training by scaling logits based on temperature.

A new logit model derived from the Weibull manifold.

Proposes SOVR loss to improve adversarial robustness by increasing logit margins.

Generative classifiers have been shown promising to detect illegal inputs including adversarial examples and out-of-distribution samples. Supervised Deep Infomax~(SDIM) is a scalable end-to-end framework to learn generative classifiers. In this paper, we propose a modification of SDIM termed SDIM-\emph{logit}. Instead …

Despite having excellent performances for a wide variety of tasks, modern neural networks are unable to provide a reliable confidence value allowing to detect misclassifications. This limitation is at the heart of what is known as an adversarial example, where the network provides a wrong prediction associated with a s…

In this paper, we develop improved techniques for defending against adversarial examples at scale. First, we implement the state of the art version of adversarial training at unprecedented scale on ImageNet and investigate whether it remains effective in this setting - an important open scientific question (Athalye et …

Grokking occurs at numerical stability edge, requiring regularization to prevent.

Logit dynamics formula reveals self-regulation in softmax policy gradient methods.

Proposes a convex model for mixed logit to handle individual heterogeneity.

MANO normalizes logits to estimate test accuracy without labels.

We establish a theoretical link between adversarial training and operator norm regularization for deep neural networks. Specifically, we prove that $\ell_p$-norm constrained projected gradient ascent based adversarial training with an $\ell_q$-norm loss on the logits of clean and perturbed inputs is equivalent to data-…

New method uses low logit rank to simplify complex language models.

Logit models are usually applied when studying individual travel behavior, i.e., to predict travel mode choice and to gain behavioral insights on traveler preferences. Recently, some studies have applied machine learning to model travel mode choice and reported higher out-of-sample predictive accuracy than traditional …

Logit distance bounds representational similarity of models.

We consider neural network training, in applications in which there are many possible classes, but at test-time, the task is a binary classification task of determining whether the given example belongs to a specific class, where the class of interest can be different each time the classifier is applied. For instance, …

Label smoothing improves model robustness against misspecification.

New algorithm reduces switching costs in multinomial logit bandit problems.

We evaluate the robustness of Adversarial Logit Pairing, a recently proposed defense against adversarial examples. We find that a network trained with Adversarial Logit Pairing achieves 0.6% accuracy in the threat model in which the defense is considered. We provide a brief overview of the defense and the threat models…

New method distills cloud models into edge-friendly ones.

Paper tackles long-tailed labels in classification problems.

Deep learning classifiers are known to be vulnerable to adversarial examples. A recent paper presented at ICML 2019 proposed a statistical test detection method based on the observation that logits of noisy adversarial examples are biased toward the true class. The method is evaluated on CIFAR-10 dataset and is shown t…

The standard Gibbs sampler of Mixed Multinomial Logit (MMNL) models involves sampling from conditional densities of utility parameters using Metropolis-Hastings (MH) algorithm due to unavailability of conjugate prior for logit kernel. To address this non-conjugacy concern, we propose the application of Pólygamma data a…

The paper proposes an efficient method to scale Bayesian inference for mixed multinomial logit models to very large datasets.

Proposes a new model for context-dependent decision-making.

Logit correction improves model performance by correcting spurious correlations.

This article presents a proof of the existence of Bertrand-Nash equilibrium prices with multi-product firms and under the Logit model of demand that does not rely on restrictive assumptions on product characteristics, firm homogeneity or symmetry, product costs, or linearity of the utility function. The proof is based …

The paper models network formation using mixed logit models.

The paper calibrates uncertainty in dropout variational inference models.

In discrete choice modeling (DCM), model misspecifications may lead to limited predictability and biased parameter estimates. In this paper, we propose a new approach for estimating choice models in which we divide the systematic part of the utility specification into (i) a knowledge-driven part, and (ii) a data-driven…

Overfitting in deep learning has been the focus of a number of recent works, yet its exact impact on the behavior of neural networks is not well understood. This study analyzes overfitting by examining how the distribution of logits alters in relation to how much the model overfits. Specifically, we find that when trai…

Analyzes how inclusion/exclusion from STOXX Europe 600 Index affects company prices.

Auxiliary Tuning adapts pre-trained models for novel tasks efficiently.

A new network model combines features of DCBM, LSM, and β-model, using a cancellation trick for parameter estimation.

The paper uncovers symmetries in large language models through layer-peeled optimization.

SLED improves factuality in LLMs without external knowledge.

Dark Experience improves continual learning with a simple, strong baseline.

SC unifies ICL calibration methods and improves LLM performance.

Generating and eliminating adversarial examples has been an intriguing topic in the field of deep learning. While previous research verified that adversarial attacks are often fragile and can be defended via image-level processing, it remains unclear how high-level features are perturbed by such attacks. We investigate…

The paper proposes effective margin regularization to improve adversarial robustness in deep neural networks.