New findings show the large margins theory is insufficient for explaining ensemble methods.
problem Explaining the performance of ensemble methods, especially boosting.
method Illustrated by counterexamples that show how to improve margin distribution without improving test set performance.
result The large margins theory is not sufficient to explain the performance of ensemble methods.
Enhances ordinal embedding with less data by focusing on margin distribution.
problem Insufficient labeled data for ordinal embedding.
method Proposes Distributional Margin based Ordinal Embedding (DMOE) to improve generalization with less data.
result Demonstrates improved generalization performance with less labeled data.
This study analyzes adversarial training on linearly separable data and finds that gradient updates can achieve large margins in polynomial iterations.
problem Ensuring robustness in machine learning models trained on linearly separable data.
method Analysis of adversarial training with gradient updates on linearly separable data.
result Gradient updates in adversarial training can achieve large margins in polynomial iterations, whereas non-smooth methods require exponentially many iterations.
Framework for private, noise-tolerant, and efficient learning algorithms.
problem Private and efficient learning of large-margin halfspaces in noisy environments.
method Simple framework using differential privacy and noise tolerance conditions.
result Noise-tolerant and private PAC learners for large-margin halfspaces with sample complexity independent of dimension.
We present a new boosting algorithm, motivated by the large margins theory for boosting. We give experimental evidence that the new algorithm is significantly more robust against label noise than existing boosting algorithm.
Paper reinterprets marginal productivity theory using vectorial products, challenging traditional ethical interpretations.
problem Challenges traditional ethical interpretations of marginal productivity theory.
method Formulates marginal productivity theory using vectorial marginal products, contrasting with traditional scalar approach.
result Vectorial marginal products conflict with traditional distributive shares picture of property.
Paper establishes comparison theorems for large-margin learning.
problem Data piling issue in high-dimension and low-sample size SVM.
method Large-margin unified machines (LUM) loss functions.
result New comparison theorems for all LUM loss functions.
New private algorithms learn large-margin halfspaces efficiently.
problem Learning large-margin halfspaces with privacy constraints.
method Differentially private algorithms based on a new approach.
result Sample complexity depends only on the margin, not dimension.
New algorithm learns halfspaces with large margins efficiently.
problem Learning halfspaces with large margins in noisy data.
method Proper learning algorithm with tight complexity bounds.
result Nearly tight complexity characterization for α=1.01-approximate learning. Analyzes large-margin classifiers under high-dimensional data.
problem Selecting the best classifier among various margin-based methods.
method Investigates asymptotic performance of large-margin classifiers under two component mixture models.
result Analytical results closely match with Monte Carlo simulations.
Gradient penalty improves GAN performance by inducing a large-margin classifier.
problem Improving GAN performance and addressing vanishing gradients.
method A unifying framework of expected margin maximization, showing gradient penalties induce large-margin classifiers.
result Gradient penalties reduce vanishing gradients and produce better generated outputs.
The key issue of few-shot learning is learning to generalize. This paper proposes a large margin principle to improve the generalization capacity of metric based methods for few-shot learning. To realize it, we develop a unified framework to learn a more discriminative metric space by augmenting the classification loss…
New method accelerates large margin metric learning for nearest neighbor classification.
problem Efficiently learning metrics for nearest neighbor classification.
method Triplet mining and stratified sampling for large margin metric learning.
result Improved efficiency and scalability of optimization.
Study generalization of voting classifiers using margin-based bounds.
problem Understanding the generalization of ensemble classifiers like voting.
method Proved margin-based generalization bounds using PAC-Bayes theory and Dirichlet posteriors.
result Provided state-of-the-art guarantees on classification tasks.
This paper connects masked pre-training to Bayesian model selection.
problem Understanding the success of masked pre-training and its generalization.
method The paper shows masked pre-training corresponds to maximizing the marginal likelihood.
result Masked pre-training with a suitable scoring function maximizes the marginal likelihood.
Paper improves forest representation learning by optimizing margin distribution.
problem Improving generalization gap in forest representation learning.
method Reformulated as an additive model, optimizing margin distribution ratio λ.
result Substantially improved upper bound of generalization gap from O(√(ln m / m)) to O((ln m) / m).
We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L2 regularization: We introduce the margin-adapted dimension, which is a simple function of the second order statistics of the data distribution, and show distribution-specific upper and lower bounds on…
Theory for soft-margin classifiers on object manifolds.
problem Classifying object manifolds with variability.
method Mean-field theory of soft-margin classifiers applied to object manifolds.
result Prediction of classification errors and their dependence on regularization.
The paper proposes effective margin regularization to improve adversarial robustness in deep neural networks.
problem Adversarial vulnerability of deep neural networks (DNNs).
method Regularization of effective weight norm during training to maximize effective margins.
result Effective margin regularization (EMR) boosts adversarial robustness in both standard and adversarial training.
Study examines liquidation, leverage, and optimal margin requirements in Bitcoin futures markets.
problem Understanding and optimizing margin requirements in Bitcoin futures markets.
method Empirical analysis using generalized extreme value theory and BitMEX data.
result Margin requirements need to be significantly higher to reduce daily margin calls.
We present a formulation of deep learning that aims at producing a large margin classifier. The notion of margin, minimum distance to a decision boundary, has served as the foundation of several theoretically profound and empirically successful results for both classification and regression tasks. However, most large m…
Paper proposes a new classifier for hyperbolic spaces using horospherical boundaries.
problem Optimization of large margin classifiers in hyperbolic spaces.
method Horospherical decision boundaries for geodesically convex optimization.
result Geodesically convex optimization leads to globally optimal solutions.
We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L_2 regularization: We introduce the γ-adapted-dimension, which is a simple function of the spectrum of a distribution's covariance matrix, and show distribution-specific upper and lower bounds on the s…
Paper bridges theory and algorithm for domain adaptation.
problem Domain adaptation from theory to algorithm gap.
method Extended domain adaptation theories, introduced Margin Disparity Discrepancy, and transformed into adversarial learning algorithm.
result Empirical studies show state-of-the-art accuracies on domain adaptation tasks.
New lower bounds nearly match existing upper bounds for boosted classifiers.
problem Understanding the generalization performance of boosted classifiers.
method Margin-based lower bounds on boosted classifiers.
result Lower bounds nearly match the kth margin bound, settling the generalization performance of boosted classifiers. Game theory enhances preference learning, improving feature selection and interpretability.
problem Improving feature selection and interpretability in preference learning.
method Formulates preference learning as a two-player zero-sum game, proposing an algorithm to incrementally add features.
result Demonstrates the convergence of the algorithm and shows its effectiveness in feature selection and interpretability.
The paper develops a theory for speculative decoding acceptance criteria.
problem Speculative decoding's acceptance criteria and their rejection regions.
method Characterization of rejection regions as lower level sets of the target distribution, derivation of exact and margin-based certificates.
result Relaxed and tree-based acceptance criteria substantially enlarge the region of certified acceptance.
New method reduces variance in off-policy evaluation for RL.
problem Reducing variance in off-policy evaluation for RL with long horizons.
method Marginalized Importance Sampling (MIS) estimator.
result Achieves mean-squared error bound matching Cramer-Rao lower bound.
Paper provides first theoretical guarantees for hyperbolic space learning.
problem Learning a classifier in hyperbolic space for hierarchical data.
method Efficient algorithm for large-margin hyperplane learning in hyperbolic space.
result The low embedding dimension in hyperbolic space leads to superior classifier learning guarantees.
Causal invariance can improve finite-sample domain adaptation, but only when the target risk margins are large.
problem Finite-sample domain adaptation
method Linear regression with causal knowledge
result Adaptive aggregation can match best candidate predictor while avoiding negative transfer
Derives asymptotic generalization error for large-margin classifiers.
problem Understanding the generalization error of large-margin classifiers.
method Statistical physics replica method for deriving asymptotic expression.
result Establishes phase transition boundary for class separability.
We study two-layer belief networks of binary random variables in which the conditional probabilities Pr[childlparents] depend monotonically on weighted sums of the parents. In large networks where exact probabilistic inference is intractable, we show how to compute upper and lower bounds on many probabilities of intere…
Paper proposes LMM-PQS for cross-domain few-shot learning.
problem Cross-domain few-shot learning problem.
method Generates pseudo query images and fine-tunes feature extraction modules with a large margin mechanism.
result LMM-PQS outperforms baseline models in cross-domain few-shot learning.
This paper is devoted to the quantification and analysis of marginal risk contribution of a given single financial institution i to the risk of a financial system s. Our work expands on the CoVaR concept proposed by Adrian and Brunnermeier as a tool for the measurement of marginal systemic risk contribution. We first g…
Paper improves deep neural networks' generalization by focusing on margin distribution complexity.
problem Improving deep neural networks' generalization performance.
method Proves a generalization upper bound based on margin distribution statistics and optimizes a convex margin distribution loss function.
result Optimizing the ratio of margin standard deviation to expected margin enhances generalization performance.
Paper improves DP-ERM for binary linear classification with large-margin subsets.
problem Differentially private binary linear classification with large-margin subsets.
method Efficient (ε,δ)-DP algorithm with empirical zero-one risk bound. result Improved empirical zero-one risk bound for binary linear classification.
Paper proposes an approximate margin method for fast multi-class classification.
problem Challenges in multi-class classification with many classes.
method Uses ANN search structures and LSH for approximate margin estimation.
result Approximate margin method is highly competitive in time, memory, and performance.
Efficient algorithms improve learning of large-margin halfspaces.
problem Learning large-margin halfspaces efficiently and reproducibly.
method Design of efficient, dimension-independent, polynomial-time algorithms; SGD-based approach; DP-to-Replicability reduction.
result Improved sample complexity compared to previous algorithms, with optimal sample complexity for one algorithm.
New stability theory for Sinkhorn semigroups with explicit decay rates.
problem Stability and convergence of Sinkhorn iterations for various divergences.
method Operator-theoretic framework based on Lyapunov techniques.
result Explicit exponential decay rates for Sinkhorn iterates.
IMMIGRATE selects features with interaction terms using margin-based weights.
problem Unclear differentiation of feature interactions from marginal effects.
method Includes and trains weights for interaction terms, applies large margin principle, considers robustness and local/global information.
result Achieves state-of-the-art results on several tasks.
Maximum entropy distributions with discrete support in m dimensions arise in machine learning, statistics, information theory, and theoretical computer science. While structural and computational properties of max-entropy distributions have been extensively studied, basic questions such as: Do max-entropy distributio…
Efficiently computes robust option prices using multi-marginal martingale transport.
problem Computing robust option prices under martingale constraints.
method Extending state space, sequential martingale structure, entropic regularisation.
result Fast computation of optimal solutions for large problems.
Large GD stepsizes improve margins and speed up training for non-homogeneous networks.
problem Training efficiency and margin improvement in non-homogeneous two-layer networks.
method Investigation of two distinct phases in GD training, showing margin growth and empirical risk decrease.
result Large GD stepsizes lead to faster convergence and improved margins in non-homogeneous networks.
Gradient descent dynamics in deep networks leads to optimal margin solutions.
problem Controlling the complexity of deep networks for generalization.
method Gradient descent dynamics on normalized weights.
result Gradient descent dynamics converge to optimal margin solutions.
An econometric or statistical model may undergo a marginal gain if we admit a new variable to the model, and a marginal loss if we remove an existing variable from the model. Assuming equality of opportunity among all candidate variables, we derive a valuation framework by the expected marginal gain and marginal loss i…
Proposes MDR estimator for unbiased OPE with large action spaces.
problem Severe bias and variance tradeoffs in OPE with large action spaces.
method Marginalized Doubly Robust (MDR) estimator, reducing variance and bias.
result MDR estimator is unbiased under weaker assumptions than MIPS.
Paper analyzes U.S. broker call rate laws of motion and their implications.
problem Understanding the dynamics and pricing of margin loans in the U.S. market.
method Analysis of monthly observations, derivation of stochastic differential equations, application of arbitrage theory.
result Margin loan interest rate follows mean-reverting behavior, with total call loan volume constituting over 70% of leveraged portfolios.
New margin bound improves generalization for voting classifiers.
problem Improving generalization bounds for voting classifiers.
method Established a new margin-based generalization bound.
result Derives an optimal weak-to-strong learner with matching theoretical lower bound.