BoC probe assesses neural network confidence coherence, revealing architecture-specific uncertainty.
problem Poor calibration and OOD detection in neural networks.
method Bag-of-Coins (BoC) probe compares softmax confidence to pairwise dominance probabilities.
result BoC reveals clear ID/OOD separation for some architectures but not others.
Support vector machines (SVMs) naturally embody sparseness due to their use of hinge loss functions. However, SVMs can not directly estimate conditional class probabilities. In this paper we propose and study a family of coherence functions, which are convex and differentiable, as surrogates of the hinge function. The …
Variance-Calibrated Modulation (VCM) addresses the likelihood trap in LLMs by reshaping the probability distribution before truncation.
problem LLMs fall into the likelihood trap, leading to repetitive degeneration and vocabulary dullness.
method VCM reshapes the probability distribution before truncation through Contextual Searchlight and Adaptive Self-Debiasing.
result VCM mitigates the likelihood trap across open-ended generation, factual QA, and mathematical reasoning.
Unified kernel for prediction markets reduces belief variance forecast error.
problem Lack of standardized tools for quoting and hedging belief risk in prediction markets.
method Logit jump-diffusion model with risk-neutral drift, calibration pipeline, and coherent derivative layer.
result Model reduces forecast error compared to diffusion-only and probability-space baselines.
Logit regularization induces logit clustering, affecting classifier performance.
problem Understanding the mechanism of logit regularization in classification.
method Analysis of logit regularization in linear classification, proving logit clustering leads to Fisher's Linear Discriminant alignment.
result Logit regularization can halve critical sample complexity and induce robust generalization.
Improves adversarial robustness by constraining logits with a bounded function.
problem Improving adversarial robustness in deep learning models.
method Addition of a bounded function before softmax to constrain logits.
result Our method improves adversarial robustness without requiring adversarial training.
Logit-GFN accelerates GFlowNets training by scaling logits based on temperature.
problem Training temperature-conditional GFlowNets is numerically challenging.
method Logit-GFN uses a learned function of temperature to scale policy logits.
result Logit-GFN greatly accelerates GFlowNets training and improves generalization and mode discovery.
A new logit model derived from the Weibull manifold.
problem No potential function on the Weibull manifold.
method Extracted a logit model from the two-parameter Weibull model.
result Found a completely integrable Hamiltonian gradient system on the logit model.
Proposes SOVR loss to improve adversarial robustness by increasing logit margins.
problem Adversarial training's difficulty in robustness against sophisticated attacks.
method Introduces SOVR loss function that switches from cross-entropy to one-vs-the-rest loss for important samples.
result SOVR loss increases logit margins of important samples, improving robustness against Auto-Attack.
Generative classifiers have been shown promising to detect illegal inputs including adversarial examples and out-of-distribution samples. Supervised Deep Infomax~(SDIM) is a scalable end-to-end framework to learn generative classifiers. In this paper, we propose a modification of SDIM termed SDIM-\emph{logit}. Instead …
In this paper, we develop improved techniques for defending against adversarial examples at scale. First, we implement the state of the art version of adversarial training at unprecedented scale on ImageNet and investigate whether it remains effective in this setting - an important open scientific question (Athalye et …
Logit dynamics formula reveals self-regulation in softmax policy gradient methods.
problem Understanding the stability and convergence of softmax policy gradient methods.
method Deriving the exact formula for the L2 norm of the logit update vector.
result Logit update magnitudes are modulated by action probability and policy concentration.
Proposes a convex model for mixed logit to handle individual heterogeneity.
problem Non-convex optimization in mixed logit models for individual heterogeneity.
method Sparse and low-rank decomposition for convex formulation.
result Convex formulation avoids simulation-based approximation and unstable model interpretation.
Recently, Kannan et al. [2018] proposed several logit regularization methods to improve the adversarial robustness of classifiers. We show that the computationally fast methods they propose - Clean Logit Pairing (CLP) and Logit Squeezing (LSQ) - just make the gradient-based optimization problem of crafting adversarial …
MANO normalizes logits to estimate test accuracy without labels.
problem Estimating test accuracy of OOD samples without labels.
method Applies Lp norm to normalized logits. result Achieves state-of-the-art performance across various architectures.
New method uses low logit rank to simplify complex language models.
problem Understanding and learning from modern language models.
method Exploiting the low logit rank structure of language models for efficient learning.
result An efficient algorithm for learning low logit rank models from queries.
Logit models are usually applied when studying individual travel behavior, i.e., to predict travel mode choice and to gain behavioral insights on traveler preferences. Recently, some studies have applied machine learning to model travel mode choice and reported higher out-of-sample predictive accuracy than traditional …
Logit distance bounds representational similarity of models.
problem Approximating linear similarity when distributions are close.
method Defined a logit distance and proved its relationship to representational dissimilarity.
result Logit distance bounds representational similarity, providing nontrivial control in practice.
We consider neural network training, in applications in which there are many possible classes, but at test-time, the task is a binary classification task of determining whether the given example belongs to a specific class, where the class of interest can be different each time the classifier is applied. For instance, …
LAWN normalizes logits to improve deep network adaptability and generalization.
problem Large logits and weights lead to overfitting in deep networks.
method Logit Attenuating Weight Normalization (LAWN) constrains weight norms in the final sub-network.
result LAWN improves generalization and adaptability of deep networks.
New algorithm reduces switching costs in multinomial logit bandit problems.
problem Minimizing switching costs in multinomial logit bandit problems.
method Proposed AT-DUCB and FH-DUCB algorithms with low assortment switching costs.
result AT-DUCB and FH-DUCB algorithms achieve almost optimal minimax regret with low switching costs.
We evaluate the robustness of Adversarial Logit Pairing, a recently proposed defense against adversarial examples. We find that a network trained with Adversarial Logit Pairing achieves 0.6% accuracy in the threat model in which the defense is considered. We provide a brief overview of the defense and the threat models…
ELM improves neural model embeddings for long-tail learning.
problem Learning skewed label distributions in neural models.
method Enforces margins in logit space and regularizes embedding distribution.
result ELM reduces generalization gap and tightens tail class embeddings.
New method distills cloud models into edge-friendly ones.
problem Cloud-to-edge model compression with limited data exchange.
method Two-step workflow of deprivatization and distillation.
result Outperforms previous state-of-the-art approaches on various benchmarks.
Paper tackles long-tailed labels in classification problems.
problem Imbalanced or long-tailed label distribution in real-world classification problems.
method Logit adjustment applied post-hoc or during training to encourage a large relative margin between rare and dominant labels.
result Unified and generalised techniques for coping with long-tailed labels, improving generalisation and performance.
Deep learning classifiers are known to be vulnerable to adversarial examples. A recent paper presented at ICML 2019 proposed a statistical test detection method based on the observation that logits of noisy adversarial examples are biased toward the true class. The method is evaluated on CIFAR-10 dataset and is shown t…
The standard Gibbs sampler of Mixed Multinomial Logit (MMNL) models involves sampling from conditional densities of utility parameters using Metropolis-Hastings (MH) algorithm due to unavailability of conjugate prior for logit kernel. To address this non-conjugacy concern, we propose the application of Pólygamma data a…
The paper proposes an efficient method to scale Bayesian inference for mixed multinomial logit models to very large datasets.
problem Efficiency in Bayesian inference for mixed multinomial logit models on large datasets.
method Amortized Variational Inference with stochastic backpropagation, automatic differentiation, and GPU acceleration.
result The proposed method achieves significant computational speedups over traditional methods for large datasets.
Proposes a new model for context-dependent decision-making.
problem Constant preference parameters in decision models are too rigid.
method Introduces Context-aware Bayesian mixed multinomial logit model using neural networks.
result Models context-dependent intra-respondent heterogeneity effectively.
Logit correction improves model performance by correcting spurious correlations.
problem Spurious correlations lead to poor model performance during inference.
method Proposes logit correction (LC) loss to mitigate spurious correlations.
result LC loss outperforms state-of-the-art solutions by 5.5% absolute improvement.
This article presents a proof of the existence of Bertrand-Nash equilibrium prices with multi-product firms and under the Logit model of demand that does not rely on restrictive assumptions on product characteristics, firm homogeneity or symmetry, product costs, or linearity of the utility function. The proof is based …
The paper models network formation using mixed logit models.
problem Modeling network formation in various fields.
method Mixed logit models, specifically the repeated-choice (RC) model.
result The RC model outperforms the multinomial logit (MNL) model in estimating network formation.
The paper calibrates uncertainty in dropout variational inference models.
problem Miscalibration of model uncertainty in dropout variational inference.
method Logit scaling methods are extended to recalibrate model uncertainty.
result Logit scaling reduces miscalibration, improving reliability of predictions.
In discrete choice modeling (DCM), model misspecifications may lead to limited predictability and biased parameter estimates. In this paper, we propose a new approach for estimating choice models in which we divide the systematic part of the utility specification into (i) a knowledge-driven part, and (ii) a data-driven…
Analyzes how inclusion/exclusion from STOXX Europe 600 Index affects company prices.
problem Understanding price dynamics of companies in STOXX Europe 600 Index.
method Used logit models and neural networks to analyze data.
result Identified independent variables affecting price changes.
Auxiliary Tuning adapts pre-trained models for novel tasks efficiently.
problem Adapting pre-trained models for new tasks efficiently.
method Supplementing pre-trained model with an auxiliary model that shifts output distribution.
result Achieved similar results to training from scratch with fewer resources.
A new network model combines features of DCBM, LSM, and β-model, using a cancellation trick for parameter estimation.
problem Challenging parameter fitting in network models.
method Introducing a cancellation trick to resolve parameter fitting issues in the logit-DCBM.
result R-SCORE significantly improves community detection over existing methods.
As datasets capturing human choices grow in richness and scale -- particularly in online domains -- there is an increasing need for choice models that escape traditional choice-theoretic axioms such as regularity, stochastic transitivity, and Luce's choice axiom. In this work we introduce the Pairwise Choice Markov Cha…
SLED improves factuality in LLMs without external knowledge.
problem Unreliable or factually incorrect outputs from large language models.
method Contrasts final layer logits with early layers' logits, uses approximate gradient to refine outputs.
result Consistently improves factual accuracy over existing methods.
Generating and eliminating adversarial examples has been an intriguing topic in the field of deep learning. While previous research verified that adversarial attacks are often fragile and can be defended via image-level processing, it remains unclear how high-level features are perturbed by such attacks. We investigate…
Optimal design for multinomial logit models improves assortment selection efficiency.
problem Optimal experimental design for multinomial logit models with feedback.
method Two complementary approaches: MILP reformulation and lifted design.
result Achieves statistical efficiency and scalability for MNL bandits.
Graph neural networks improve residential location choice predictions.
problem Capturing spatial dependence in discrete choice models.
method Graph Neural Networks (GNN) for analyzing spatial alternatives.
result GNN-DCMs outperform classical models in residential location choice predictions.
Simple k-NN filtering improves model accuracy on noisy labels.
problem Training models with noisy labels reduces performance and is hard to identify.
method A simple k-nearest neighbor-based filtering approach on the logit layer. result Improves model accuracy compared to recent methods.
Defines coherent manifolds and their quantum applications.
problem Understanding quantum spaces and coherent states.
method Generalizes quantum field theory and Schrödinger equation solutions.
result Solves Schrödinger equation on coherent manifolds.
CoRe method improves time series forecasting coherency without strict constraints.
problem Noisy hierarchical time series data that doesn't perfectly adhere to aggregation constraints.
method Coherency Regularization using neural networks.
result Improved forecast accuracy and coherence, especially in noisy data scenarios.
In this paper we present a theoretical framework for studying coherent acceptability indices in a dynamic setup. We study dynamic coherent acceptability indices and dynamic coherent risk measures, and we establish a duality between them. We derive a representation theorem for dynamic coherent risk measures in terms of …
Logit-link models reveal socio-temporal effects on microfinance delinquency.
problem Understanding and quantifying socio-temporal factors affecting microfinance loan delinquency.
method Developed and evaluated discrete-time logit-link models with fixed-effects and frailty extensions.
result Simple random intercept structures capture latent heterogeneity in microfinance repayment behavior.
The paper tackles learning mixtures of two multinomial logits, showing identifiability and presenting an algorithm.
problem Learning an arbitrary mixture of two multinomial logits.
method Reduction to solving a system of univariate quartic equations, followed by an algorithm using polynomial and linear samples.
result Identifiability of the mixture models may only fail on an algebraic variety of negligible measure.