We propose a principle and loss functions for efficient single-class classification.
problem Efficient binary classification for specific classes in high-dimensional data.
method Define Single Logit Classification (SLC) task, propose Principle of Logit Separation, and design loss functions.
result Loss functions aligned with the Principle of Logit Separation improve SLC accuracy by 20%.
Logit regularization induces logit clustering, affecting classifier performance.
problem Understanding the mechanism of logit regularization in classification.
method Analysis of logit regularization in linear classification, proving logit clustering leads to Fisher's Linear Discriminant alignment.
result Logit regularization can halve critical sample complexity and induce robust generalization.
Generative classifier derived from any discriminative classifier rejects illegal inputs.
problem Detecting and rejecting illegal inputs like adversarial examples and out-of-distribution samples.
method SDIM-logit: learns generative classifier from logits of any discriminative classifier, imposing statistical constraints.
result SDIM-logit inherits performance of base classifier without loss and can reject illegal inputs.
Paper tackles long-tailed labels in classification problems.
problem Imbalanced or long-tailed label distribution in real-world classification problems.
method Logit adjustment applied post-hoc or during training to encourage a large relative margin between rare and dominant labels.
result Unified and generalised techniques for coping with long-tailed labels, improving generalisation and performance.
LAWN normalizes logits to improve deep network adaptability and generalization.
problem Large logits and weights lead to overfitting in deep networks.
method Logit Attenuating Weight Normalization (LAWN) constrains weight norms in the final sub-network.
result LAWN improves generalization and adaptability of deep networks.
A neural network-based two-sample test improves classification accuracy.
problem Differentiating between two sub-exponential densities.
method Difference of logit function from trained classification neural network.
result Network complexity scales with intrinsic dimensionality for low-dimensional manifolds.
New CNN approach reduces overconfidence in object classification predictions.
problem Overconfident predictions from deep models, especially SoftMax layer.
method Introduces CNN probabilistic approach using Logit layer for Bayesian inference.
result Proposed approach shows promising performance compared to SoftMax.
Logit-link models reveal socio-temporal effects on microfinance delinquency.
problem Understanding and quantifying socio-temporal factors affecting microfinance loan delinquency.
method Developed and evaluated discrete-time logit-link models with fixed-effects and frailty extensions.
result Simple random intercept structures capture latent heterogeneity in microfinance repayment behavior.
Improves adversarial robustness by constraining logits with a bounded function.
problem Improving adversarial robustness in deep learning models.
method Addition of a bounded function before softmax to constrain logits.
result Our method improves adversarial robustness without requiring adversarial training.
Optimal design for multinomial logit models improves assortment selection efficiency.
problem Optimal experimental design for multinomial logit models with feedback.
method Two complementary approaches: MILP reformulation and lifted design.
result Achieves statistical efficiency and scalability for MNL bandits.
Logit-GFN accelerates GFlowNets training by scaling logits based on temperature.
problem Training temperature-conditional GFlowNets is numerically challenging.
method Logit-GFN uses a learned function of temperature to scale policy logits.
result Logit-GFN greatly accelerates GFlowNets training and improves generalization and mode discovery.
Label smoothing improves model robustness against misspecification.
problem Improving model robustness against model misspecification.
method Introducing modified label smoothing (MLSLR) that maintains consistent probability estimation while modifying the loss function.
result MLSLR exhibits higher robustness against model misspecification than conventional label smoothing.
Logit-Coordinate models improve representation of mixed data types.
problem Representation of mixed continuous-categorical data in generative models.
method Logit-coordinate framework combining categorical and continuous variables.
result Logit-Coordinate models outperform one-hot encoding in simulations and real data.
A new logit model derived from the Weibull manifold.
problem No potential function on the Weibull manifold.
method Extracted a logit model from the two-parameter Weibull model.
result Found a completely integrable Hamiltonian gradient system on the logit model.
A new teacher-class network method compresses DNNs by distributing knowledge to multiple student networks.
problem Overwhelming size of Deep Neural Networks (DNNs).
method Single teacher with multiple student networks, transferring knowledge to each student.
result The combined knowledge of the class of students achieves better performance and reduces parameters.
Proposes SOVR loss to improve adversarial robustness by increasing logit margins.
problem Adversarial training's difficulty in robustness against sophisticated attacks.
method Introduces SOVR loss function that switches from cross-entropy to one-vs-the-rest loss for important samples.
result SOVR loss increases logit margins of important samples, improving robustness against Auto-Attack.
Improved defense against adversarial examples using logit pairing.
problem Defending against adversarial examples at scale.
method Adversarial training and logit pairing.
result Adversarial logit pairing achieves state of the art defense on ImageNet.
Logit pairing methods don't provide robustness against adversarial examples.
problem Improving adversarial robustness of classifiers.
method Logit regularization methods (CLP, LSQ, ALP) to craft adversarial examples.
result Adversarial Logit Pairing (ALP) may provide robustness, but the increase is small.
This paper introduces new loss functions for balanced multi-class classification.
problem Balancing class imbalance in multi-class classification.
method Introduces two new surrogate loss families: GLA and GCA.
result GCA losses offer stronger theoretical guarantees in imbalanced settings.
Linear filtration helps delete training data from models.
problem Deleting training data from models when individuals request it.
method Linear filtration as a computationally efficient sanitization method.
result Demonstrates benefits in an adversarial setting over naive deletion schemes.
New model shows weak teachers can help strong students learn even with imperfect labels.
problem Improving strong student's performance with weak teacher's imperfect pseudolabels.
method Stylized overparameterized spiked covariance model with Gaussian covariates, proving two phases of generalization.
result Provable successful and random guessing phases of strong student's generalization.
Adversarial Logit Pairing shows 0.6% accuracy in threat model.
problem Evaluating robustness of Adversarial Logit Pairing against adversarial examples.
method Evaluation of Adversarial Logit Pairing's robustness through accuracy testing.
result Adversarial Logit Pairing achieves 0.6% accuracy in threat model.
DMNL bandits optimize assortment choices balancing relevance and diversity.
problem Balancing relevance-driven choice with within-assortment diversity.
method Augments MNL choice probabilities with a submodular diversity function, proposing a white-box UCB-based algorithm.
result Achieves at least a (1−e+11)-approximate regret bound of $ ilde{O}\left(d \sqrt{T/K}
ight)$. This paper compares machine learning and logit models for predicting travel mode choice.
problem Comparing interpretability of machine learning and logit models for travel mode choice.
method Empirical evaluation of machine learning and logit models on stated preference survey data.
result Machine learning produces significantly higher predictive accuracy than logit models.
Logit dynamics formula reveals self-regulation in softmax policy gradient methods.
problem Understanding the stability and convergence of softmax policy gradient methods.
method Deriving the exact formula for the L2 norm of the logit update vector.
result Logit update magnitudes are modulated by action probability and policy concentration.
Proposes a convex model for mixed logit to handle individual heterogeneity.
problem Non-convex optimization in mixed logit models for individual heterogeneity.
method Sparse and low-rank decomposition for convex formulation.
result Convex formulation avoids simulation-based approximation and unstable model interpretation.
A method to reuse 98% of parameters for multi-task learning.
problem Improving efficiency in deep learning parameter usage.
method Learning model patches for each task, reusing pretrained network parameters.
result Significant improvement in transfer learning accuracy with fewer parameters.
New method tightens federated probe-logit distillation rates under varying bandwidths.
problem Estimating conditional distributions in federated learning with heterogeneous bandwidth constraints.
method Developed a new federated probe-logit distillation (FPLD) method with optimal allocation for varying bandwidths.
result Achieved matching lower and upper bounds for the minimax rate under heterogeneous bandwidths.
MANO normalizes logits to estimate test accuracy without labels.
problem Estimating test accuracy of OOD samples without labels.
method Applies Lp norm to normalized logits. result Achieves state-of-the-art performance across various architectures.
Model quantifies retail attractiveness based on floor space and agglomeration.
problem Lack of theoretical framework for modeling retail location choices at the unit level.
method Cross-Nested Logit model based on random utility theory.
result Superlinear scaling of attractiveness with floor space and agglomeration effect.
New method uses low logit rank to simplify complex language models.
problem Understanding and learning from modern language models.
method Exploiting the low logit rank structure of language models for efficient learning.
result An efficient algorithm for learning low logit rank models from queries.
Unified framework suppresses model bias in semi-supervised learning with decoupled sampling control.
problem Class imbalance in semi-supervised learning, especially with distributional mismatches.
method Unified framework SC-SSL with decoupled sampling control, explicit expansion capability, and adaptive sampling probabilities.
result Consistent and state-of-the-art performance across various benchmark datasets and distribution settings.
Logit distance bounds representational similarity of models.
problem Approximating linear similarity when distributions are close.
method Defined a logit distance and proved its relationship to representational dissimilarity.
result Logit distance bounds representational similarity, providing nontrivial control in practice.
New models improve choice prediction accuracy.
problem Model misspecifications in discrete choice models lead to limited predictability and biased estimates.
method Proposes a new approach to estimate choice models by dividing the systematic part into knowledge-driven and data-driven components, which learns a new representation from available variables.
result The new models (L-MNL and L-NL) outperform traditional models in predictive performance and parameter estimation.
The paper develops a statistical theory explaining overfitting in imbalanced classification.
problem Overfitting in high-dimensional imbalanced classification.
method Developed a statistical theory for support vector machines and logistic regression.
result Overfitting is more severe for the minority class due to truncation or skewing effects in high-dimensional data.
PLD distills knowledge using choice-theoretic Plackett-Luce model.
problem Model compression and knowledge transfer between large and small networks.
method PLD uses a weighted list-wise ranking loss based on the Plackett-Luce model.
result PLD achieves consistent gains across diverse architectures and distillation methods.
Proposes MCLLO for assessing and recalibrating multiclass probability predictions.
problem Limited multicategory recalibration methods for assessing and comparing model calibration.
method MCLLO recalibration method that assesses calibration without model access and is easy to interpret.
result MCLLO outperforms other methods in simulations and real-world case studies.
Study dynamic assortment and positioning of products with varying display effects.
problem Dynamic assortment and positioning of products with varying display effects.
method Design round-based learning algorithms for both multiplicative and general position effects models, and develop efficient subroutines for optimization.
result First regret-optimal characterization for both models, with matching upper and lower bounds.
A two-layer network corrects logits to defend adversarial attacks.
problem Detecting and defending adversarial attacks on deep learning models.
method A two-layer network trained on mixed logits to recover original predictions.
result Shows promising results in defense with high accuracy and interpretability.
New model improves website ranking by considering user choices as a whole.
problem Optimizing content ordering for user clicks in website design.
method Introduced multinomial logit (MNL) choice model to LTR framework, proposing UCB algorithms.
result Proved theoretical bounds on regret for UCB algorithms in both known and unknown position parameter settings.
New method bypasses statistical and classifier-based detection of adversarial examples.
problem Vulnerability of deep learning classifiers to adversarial examples.
method Classifier-based adaptation of statistical test method and Logit Mimicry Attack.
result Reduces detection performance to less than 2.2% TPR and 1.6% TPR for statistical test and classifier-based methods, respectively, even at 5% FPR.
New algorithm reduces switching costs in multinomial logit bandit problems.
problem Minimizing switching costs in multinomial logit bandit problems.
method Proposed AT-DUCB and FH-DUCB algorithms with low assortment switching costs.
result AT-DUCB and FH-DUCB algorithms achieve almost optimal minimax regret with low switching costs.
This paper provides a theoretical and computational justification of the long held claim that of the similarity of the probit and logit link functions often used in binary classification. Despite this widespread recognition of the strong similarities between these two link functions, very few (if any) researchers have …
ELM improves neural model embeddings for long-tail learning.
problem Learning skewed label distributions in neural models.
method Enforces margins in logit space and regularizes embedding distribution.
result ELM reduces generalization gap and tightens tail class embeddings.
New method distills cloud models into edge-friendly ones.
problem Cloud-to-edge model compression with limited data exchange.
method Two-step workflow of deprivatization and distillation.
result Outperforms previous state-of-the-art approaches on various benchmarks.
Detects misclassifications and adversarial examples using neural network logits.
problem Unable to detect misclassifications and adversarial examples in neural networks.
method Introspection using pretrained neural network logits.
result Simple 3-layer neural network trained on logits detects misclassifications competitively.
The paper proposes an efficient method to scale Bayesian inference for mixed multinomial logit models to very large datasets.
problem Efficiency in Bayesian inference for mixed multinomial logit models on large datasets.
method Amortized Variational Inference with stochastic backpropagation, automatic differentiation, and GPU acceleration.
result The proposed method achieves significant computational speedups over traditional methods for large datasets.
Proposes a new model for context-dependent decision-making.
problem Constant preference parameters in decision models are too rigid.
method Introduces Context-aware Bayesian mixed multinomial logit model using neural networks.
result Models context-dependent intra-respondent heterogeneity effectively.