MMA training maximizes margins for adversarial robustness.
problem Adversarial robustness of neural networks.
method Directly maximizes margins through adaptive adversarial training.
result MMA training improves adversarial robustness compared to fixed ε adversarial training.
A fast method for training linear classifiers maximizes margins.
problem Training linear classifiers with maximum margins.
method Momentum-based gradient method derived from convex dual with Nesterov acceleration.
result Exponentially faster convergence rate compared to standard methods.
New DAM method improves AUC scores in medical image classification.
problem Maximizing AUC in large-scale medical image classification.
method Proposes AUC margin loss for robust optimization, conducts extensive empirical studies.
result Improves performance on four medical image classification tasks, achieving 1st place on Stanford CheXpert.
The paper proposes effective margin regularization to improve adversarial robustness in deep neural networks.
problem Adversarial vulnerability of deep neural networks (DNNs).
method Regularization of effective weight norm during training to maximize effective margins.
result Effective margin regularization (EMR) boosts adversarial robustness in both standard and adversarial training.
New loss function improves convergence rate for neural networks.
problem Improving convergence speed of neural networks for classification tasks.
method Proposes a modified hinge loss function with gradients to critical points.
result Margin converges to max-margin at O(1/t) rate, faster than exponential losses. Gradient descent in neural networks maximizes margin.
problem Optimizing neural networks using gradient descent.
method Gradient descent or gradient flow on homogeneous neural networks.
result Normalized margin increases over time if training loss decreases below a threshold.
This work improves adversarial robustness by boosting model ensembles with margin maximization.
problem Single models are insufficient for defending against adversarial attacks.
method Margin-boosting approach to learn ensembles with maximum margin.
result Our algorithm outperforms existing ensembling techniques and large models trained end-to-end.
Paper corrects Max-Margin loss for multi-label tasks.
problem Max-Margin loss inconsistency in multi-label classification.
method Introduced Restricted-Max-Margin loss.
result Consistent loss for multi-label tasks under milder conditions.
Gradient penalty improves GAN performance by inducing a large-margin classifier.
problem Improving GAN performance and addressing vanishing gradients.
method A unifying framework of expected margin maximization, showing gradient penalties induce large-margin classifiers.
result Gradient penalties reduce vanishing gradients and produce better generated outputs.
Gradient descent converges to max-margin solution for hinge loss.
problem Applying gradient descent to the hinge loss for linear classifiers.
method Homotopic gradient descent applied to the hinge loss.
result Explicit convergence rates to max-margin solution for separable data.
A fast Bayesian optimization method using threshold-guided marginal likelihood maximization.
problem Efficiently optimizing models with Gaussian process regression.
method Guided marginal likelihood maximization with a pre-defined threshold to reduce model selection steps.
result Significantly reduces execution time without compromising optimization quality.
Study how regularization and optimization affect margin in deep models.
problem Understanding margin maximization in deep learning models.
method Analyze the limit of loss minimization with diverging norm constraints and margin paths.
result Discovers lexicographic max-margin solutions for homogeneous models and shows convergence under certain conditions.
Paper analyzes GMM for separable data with various parameter structures.
problem Classifying separable data with logistic models and their generalizations.
method Introduces and analyzes Generalized Margin Maximizer (GMM) for logistic models with specific parameter structures.
result GMM outperforms max-margin classifiers in various parameter settings and structures.
We exhibit a strong link between frequentist PAC-Bayesian risk bounds and the Bayesian marginal likelihood. That is, for the negative log-likelihood loss function, we show that the minimization of PAC-Bayesian generalization risk bounds maximizes the Bayesian marginal likelihood. This provides an alternative explanatio…
New insights into how linear classifiers and leaky ReLU networks can overfit without harming generalization.
problem Understanding conditions for benign overfitting in linear classifiers and leaky ReLU networks.
method Utilizing Karush--Kuhn--Tucker (KKT) conditions for margin maximization.
result Satisfaction of KKT conditions leads to benign overfitting in linear classifiers and leaky ReLU networks.
Advances robustness of metric learning by adversarial margin in input space.
problem Improving robustness of metric learning algorithms.
method Imposing adversarial margin in input space, minimizing perturbation loss.
result Enlarged adversarial margin improves generalization and robustness.
New framework for learning from imbalanced data with theoretical guarantees.
problem Class imbalance in machine learning, especially in multi-class problems.
method Theoretical framework and new margin loss function for imbalanced classification.
result Proves strong H-consistency of the proposed margin loss function. New insights into deep learning: reducing training data significantly improves performance.
problem Understanding and improving generalization in deep learning models.
method Analyzing the distribution of classification margins and dynamically reducing the training set.
result The area under the curve of the margin distribution is a good measure of generalization.
Gradient descent reveals the exact implicit bias via dual optimization for linearly separable data.
problem Characterizing the implicit bias of gradient descent on linearly separable data.
method Primal-dual analysis with smoothed margin for general losses, and exponential loss with specific step sizes.
result Proves faster convergence rates for implicit bias and margin maximization.
We study the implicit bias of gradient descent methods in solving a binary classification problem over a linearly separable dataset. The classifier is described by a nonlinear ReLU model and the objective function adopts the exponential loss function. We first characterize the landscape of the loss function and show th…
Regulator allocates buffers to prevent financial contagion in networks with common assets.
problem Containment of default contagion in financial networks with common asset exposures.
method Allocates nonnegative buffer vectors under linear budget constraints to maximize default or insolvency resilience margins or minimize worst-case systemic losses.
result Exact synthesis results for buffer allocation under ℓ∞ and ℓ1 uncertainty sets, showing significant gains over uniform and exposure-proportional allocations. The concept of refinement from probability elicitation is considered for proper scoring rules. Taking directions from the axioms of probability, refinement is further clarified using a Hilbert space interpretation and reformulated into the underlying data distribution setting where connections to maximal marginal diver…
New findings on the max margin problem in neural networks.
problem Understanding the max margin problem in neural networks.
method Analyzing gradient flow and max margin problem in linear and ReLU networks.
result The KKT point is not always an optimum of the max margin problem.
Unified binary and multiclass margin-based classification methods.
problem No consensus on multiclass loss functions analogous to binary margin loss.
method Showed multiclass loss functions can be expressed in relative margin form.
result Extended classification-calibration result to multiclass.
Maximizing margins leads to lossless compression of training data.
problem Generalization in supervised learning.
method Information-theoretic interpretation of margin maximization.
result Margin maximization is a form of lossless maximal compression.
Derives VC bounds for deep neural networks using radius-margin approach.
problem Understanding the capacity of deep neural networks.
method Applies radius-margin bounds from SVM to deep feed-forward architectures.
result Derives new VC bounds different from earlier ones.
Paper introduces a variational approach for PU learning with improved performance and stability.
problem Learning binary classifiers from only positive and unlabeled data.
method Variational principle for PU learning, quantitatively evaluating modeling error, efficient loss function, margin maximizing loss function.
result Improved discriminative performance and numerical stability of the variational PU learning method.
GD iterates for non-homogeneous deep nets increase margin and converge in direction.
problem Understanding implicit bias in non-homogeneous deep networks.
method Characterization of GD iterates' properties starting from small empirical risk.
result GD iterates converge in direction despite diverging norms, satisfying KKT conditions.
We analyze bias-variance of margin losses.
problem Understanding model overfitting/underfitting.
method Bias-variance decomposition for strictly convex margin losses.
result Expected risk decomposes into central model risk and data variation.
Improved SincNet for better speaker recognition.
problem Speaker recognition challenges and the need for better deep learning models.
method Proposes AM-SincNet, a SincNet-based model with an improved AM-Softmax layer.
result Improved speaker recognition performance, achieving a 40% Frame Error Rate reduction.
This paper proposes MMD-SVR to improve SVR's margin distribution for better generalization.
problem Improving SVR's generalization performance by maximizing the margin distribution of the whole dataset.
method Introducing MMD-SVR with coupled constraints to convert a non-convex optimization problem into a convex one.
result MMD-SVR significantly improves prediction accuracy and generalization compared to classic SVR.
Default-ERM shortcut learning persists even without additional information.
problem Default-ERM shortcut learning in perception tasks despite stable feature sufficiency.
method Studied linear perception task; developed margin control (MARG-CTRL) loss functions.
result Margin control mitigates shortcut learning on various tasks.
Paper introduces negative margin loss for better few-shot classification accuracy.
problem Improving few-shot classification accuracy with metric learning.
method Introduces negative margin loss and analyzes its impact on feature discriminability.
result Negative margin loss outperforms regular softmax loss on few-shot classification benchmarks.
Proposes SOVR loss to improve adversarial robustness by increasing logit margins.
problem Adversarial training's difficulty in robustness against sophisticated attacks.
method Introduces SOVR loss function that switches from cross-entropy to one-vs-the-rest loss for important samples.
result SOVR loss increases logit margins of important samples, improving robustness against Auto-Attack.
Deep networks converge in direction, with implications for predictions and margins.
problem Understanding convergence and alignment in deep learning networks.
method Developed a theory of unbounded nonsmooth Kurdyka-Łojasiewicz inequalities for functions definable in an o-minimal structure.
result Network weights, predictions, training errors, and margin distribution converge in direction and align with gradient flow.
Paper proposes angular loss for better face recognition and object classification.
problem Improving intra-class compactness and preventing overfitting in face recognition and object classification.
method Angular loss function to maximize angular gradient, reducing overfitting and requiring only one adjustable constant.
result Our method outperforms other methods in accuracy, discriminative information, and time-efficiency.
Gradient descent dynamics in deep networks leads to optimal margin solutions.
problem Controlling the complexity of deep networks for generalization.
method Gradient descent dynamics on normalized weights.
result Gradient descent dynamics converge to optimal margin solutions.
Paper establishes comparison theorems for large-margin learning.
problem Data piling issue in high-dimension and low-sample size SVM.
method Large-margin unified machines (LUM) loss functions.
result New comparison theorems for all LUM loss functions.
An explicit solution found for maximizing/minimizing agreement in a 2x2 table.
problem Maximizing or minimizing agreement between clusterings with given marginals.
method Formal framework for several agreement measures, explicit solution for 2x2 table.
result An explicit solution for the 2x2 case.
Paper proposes adaptive margin loss to improve few-shot learning.
problem Few-shot learning's difficulty in generalizing from a few examples.
method Develops class-relevant and task-relevant additive margin losses.
result Boosts performance of metric-based meta-learning approaches.
New method prevents class collapse in metric learning with margin-based losses.
problem Class collapse in metric learning due to diverse intra-class samples.
method Proposed a sampling method to select nearest same-class samples as positive elements in tuple.
result Demonstrated clear benefits on various fine-grained image retrieval datasets.
New margin-based regularization and selective sampling improve deep neural network performance.
problem Improving deep neural network performance on various classification tasks.
method Multi-margin regularization (MMR) and minimal margin score (MMS) for selective sampling.
result Improved results on multiple classification tasks across domains.
New margin-based learning guarantees improve generalization bounds.
problem Improving generalization bounds for machine learning models.
method Relative deviation margin bounds using empirical margin loss and Rademacher complexity.
result Distribution-dependent generalization bounds for unbounded loss functions.
Proposes a method to optimize neural network initialization using marginal likelihood maximization.
problem Optimizing hyperparameters for neural network initialization.
method Leverages the connection between neural networks and Gaussian processes to infer optimal hyperparameters.
result Marginal likelihood maximization provides near-optimal prediction performance on MNIST classification tasks.
An econometric or statistical model may undergo a marginal gain if we admit a new variable to the model, and a marginal loss if we remove an existing variable from the model. Assuming equality of opportunity among all candidate variables, we derive a valuation framework by the expected marginal gain and marginal loss i…
Paper explores connections between loss functions and consistency in binary classification and regression.
problem Consistency in binary classification and regression applications.
method Characterization of conformable loss functions and derivation of a new Huber-type loss function.
result Margin-based loss functions are equivalent to loss functions of squared standardized logistic regression residuals.
Consider a classification problem where we have both labeled and unlabeled data available. We show that for linear classifiers defined by convex margin-based surrogate losses that are decreasing, it is impossible to construct any semi-supervised approach that is able to guarantee an improvement over the supervised clas…
The paper introduces a new bias measure, infra-marginality, to quantify unfairness in group fairness.
problem The trade-off between group fairness and individual-level bias in decision-making.
method Proposes a new notion of η-infra-marginality, proves its independence from accuracy, and provides practical methods to measure and avoid it. result High accuracy does not lead to high infra-marginality, but maximizing group fairness often increases infra-marginality.