This paper studies convergence of risk aggregation methods using empirical margins and copulas.
problem Convergence of risk aggregation distributions in multivariate models.
method Empirical margins, Latin Hypercube Sampling, and convergence of sum distributions.
result Strong uniform consistency of estimated sum distribution function with convergence rate O(n−1/2). New margin-based learning guarantees improve generalization bounds.
problem Improving generalization bounds for machine learning models.
method Relative deviation margin bounds using empirical margin loss and Rademacher complexity.
result Distribution-dependent generalization bounds for unbounded loss functions.
Researchers test if larger margins lead to lower generalization error in ensemble methods.
problem Explaining why ensembles perform better than individual classifiers.
method Empirical testing of techniques to evaluate the relationship between margins and generalization error.
result Current research holds true: larger margins generally lead to lower generalization error.
We address the problem of learning the parameters in graphical models when inference is intractable. A common strategy in this case is to replace the partition function with its Bethe approximation. We show that there exists a regime of empirical marginals where such Bethe learning will fail. By failure we mean that th…
This study analyzes adversarial training on linearly separable data and finds that gradient updates can achieve large margins in polynomial iterations.
problem Ensuring robustness in machine learning models trained on linearly separable data.
method Analysis of adversarial training with gradient updates on linearly separable data.
result Gradient updates in adversarial training can achieve large margins in polynomial iterations, whereas non-smooth methods require exponentially many iterations.
This paper identifies and bounds ICE central moments using PO marginal central moments.
problem Identifying and characterizing treatment effect heterogeneity.
method Using only marginal central moments of potential outcomes, the paper identifies and bounds central moments of individual causal effects.
result Identification and bounding of central moments of ICE using marginal moments of POs.
Paper improves DP-ERM for binary linear classification with large-margin subsets.
problem Differentially private binary linear classification with large-margin subsets.
method Efficient (ε,δ)-DP algorithm with empirical zero-one risk bound. result Improved empirical zero-one risk bound for binary linear classification.
Data augmentation improves model robustness by enforcing a margin.
problem Understanding how data augmentation provably improves model robustness.
method Analyzed linear and nonlinear models, quantifying the margin introduced by data augmentation.
result Commonly used data augmentation techniques may only introduce significant margin after adding exponentially many points.
The paper introduces a new bias measure, infra-marginality, to quantify unfairness in group fairness.
problem The trade-off between group fairness and individual-level bias in decision-making.
method Proposes a new notion of η-infra-marginality, proves its independence from accuracy, and provides practical methods to measure and avoid it. result High accuracy does not lead to high infra-marginality, but maximizing group fairness often increases infra-marginality.
Large margin approach for deep neural networks.
problem Deep learning's lack of margin enforcement.
method Proposes a novel loss function to enforce margin across layers of deep networks.
result Improved performance on various datasets and tasks.
New method improves calibration of neural networks by targeting robust margins and local smoothness.
problem Poor calibration of neural networks, leading to unreliable confidence estimates.
method Intervene on training procedure by targeting robust margins and local smoothness.
result Improved out-of-sample calibration without sacrificing accuracy.
New AM regularization improves both accuracy and robustness.
problem Lack of robustness in deep neural networks.
method Average margin (AM) regularization for margin classifiers or deep neural networks.
result AM regularization can improve both accuracy and robustness to adversarial attacks.
Efficiently estimates marginal posteriors for complex simulations.
problem Bayesian inference in high-dimensional, intractable likelihood scenarios.
method Simulates and estimates low-dimensional marginal posteriors, using truncated indicators.
result Simulator efficiency and robustness testing of inference results.
In order to protect brokers from customer defaults in a volatile market, an active margin system is proposed for the transactions of margin lending in China. The probability of negative return under the condition that collaterals are liquidated in a falling market is used to measure the risk associated with margin loan…
This work improves adversarial robustness by boosting model ensembles with margin maximization.
problem Single models are insufficient for defending against adversarial attacks.
method Margin-boosting approach to learn ensembles with maximum margin.
result Our algorithm outperforms existing ensembling techniques and large models trained end-to-end.
New estimator reduces kernel mean estimation error.
problem Kernel mean estimation in reproducing kernel Hilbert spaces.
method Corrupt data with known distributions and estimate kernel mean under the corrupted distribution.
result The marginalized kernel mean estimator achieves lower estimation error.
New margin-based regularization and selective sampling improve deep neural network performance.
problem Improving deep neural network performance on various classification tasks.
method Multi-margin regularization (MMR) and minimal margin score (MMS) for selective sampling.
result Improved results on multiple classification tasks across domains.
Paper introduces negative margin loss for better few-shot classification accuracy.
problem Improving few-shot classification accuracy with metric learning.
method Introduces negative margin loss and analyzes its impact on feature discriminability.
result Negative margin loss outperforms regular softmax loss on few-shot classification benchmarks.
Develops bounds predicting deep learning generalization using optimal transport.
problem Discrepancy between theoretical error bounds and empirical observations in deep learning.
method Margin-based generalization bounds with optimal transport costs.
result Theoretical bounds robustly predict generalization error on large datasets.
An active margin system for margin loans is proposed for Chinese margin lending market, which uses cash and randomly selected stock as collateral. The conditional probability of negative return(CPNR) after a forced sale of securities from under-margined account in a falling market is used to measure the risk faced by t…
Study examines liquidation, leverage, and optimal margin requirements in Bitcoin futures markets.
problem Understanding and optimizing margin requirements in Bitcoin futures markets.
method Empirical analysis using generalized extreme value theory and BitMEX data.
result Margin requirements need to be significantly higher to reduce daily margin calls.
New algorithm optimizes variational objective for marginal inference.
problem Optimizing the tree-reweighted variational objective over the marginal polytope.
method Barrier Frank-Wolfe algorithm based on conditional gradient method, leveraging MAP solvers.
result More accurate results than TRW algorithms that optimize over local consistency relaxation.
Generative models often fail to preserve joint structure despite matching marginals.
problem Generative models fail to capture complex dependencies beyond univariate marginals.
method Introduced D_Sigma(P,Q) = ||Sigma_P - Sigma_Q||_F to measure covariance-level dependence fidelity.
result Covariance-level divergence can lead to structural instability in downstream inference.
MMA training maximizes margins for adversarial robustness.
problem Adversarial robustness of neural networks.
method Directly maximizes margins through adaptive adversarial training.
result MMA training improves adversarial robustness compared to fixed ε adversarial training.
Develops a new framework for perpetual futures on binary prediction markets.
problem Lack of effective risk management in perpetual futures on binary prediction markets.
method PIRAP framework with six components: index estimator, margin sizing, leverage, funding rule, halt protocol, and eligibility framework.
result Mixed results from empirical evaluation, with some pre-registered floors passing and others failing.
New loss function improves convergence rate for neural networks.
problem Improving convergence speed of neural networks for classification tasks.
method Proposes a modified hinge loss function with gradients to critical points.
result Margin converges to max-margin at O(1/t) rate, faster than exponential losses. Large GD stepsizes improve margins and speed up training for non-homogeneous networks.
problem Training efficiency and margin improvement in non-homogeneous two-layer networks.
method Investigation of two distinct phases in GD training, showing margin growth and empirical risk decrease.
result Large GD stepsizes lead to faster convergence and improved margins in non-homogeneous networks.
New algorithm optimizes margin distribution in binary classifiers.
problem Optimizing margin distribution in binary classifiers.
method Proposes an algorithm that searches the hypothesis space to ensure a pre-set margin level is a robust estimator of the margin location.
result Empirical tests show the method is effective and promising for classification.
We discuss the connection between information and copula theories by showing that a copula can be employed to decompose the information content of a multivariate distribution into marginal and dependence components, with the latter quantified by the mutual information. We define the information excess as a measure of d…
The marginal maximum a posteriori probability (MAP) estimation problem, which calculates the mode of the marginal posterior distribution of a subset of variables with the remaining variables marginalized, is an important inference problem in many models, such as those with hidden variables or uncertain parameters. Unfo…
Paper introduces a new margin bound for neural networks scaling with spectral complexity.
problem Improving generalization bounds for neural networks.
method Spectral complexity is defined as the product of the spectral norms of weight matrices, scaled by a correction factor.
result Empirical investigation shows correlation between bound, complexity, and excess risk for SGD-trained AlexNet.
Both in practice and in the academic literature, models for setting margin requirements in futures markets classically use daily closing price changes. However, as well documented by research on high-frequency data, financial markets have recently shown high intraday volatility, which could bring more risk than expecte…
Proposes a method to optimize neural network initialization using marginal likelihood maximization.
problem Optimizing hyperparameters for neural network initialization.
method Leverages the connection between neural networks and Gaussian processes to infer optimal hyperparameters.
result Marginal likelihood maximization provides near-optimal prediction performance on MNIST classification tasks.
MWGAN tackles multi-marginal matching problem with Wasserstein GAN.
problem Learning mappings to match a source domain to multiple target domains with cross-domain correlations.
method Develops a novel Multi-marginal Wasserstein GAN (MWGAN) with inner- and inter-domain constraints to minimize Wasserstein distance.
result Theoretical and empirical evaluations show MWGAN's effectiveness on balanced and imbalanced translation tasks.
This paper connects masked pre-training to Bayesian model selection.
problem Understanding the success of masked pre-training and its generalization.
method The paper shows masked pre-training corresponds to maximizing the marginal likelihood.
result Masked pre-training with a suitable scoring function maximizes the marginal likelihood.
New framework for learning from imbalanced data with theoretical guarantees.
problem Class imbalance in machine learning, especially in multi-class problems.
method Theoretical framework and new margin loss function for imbalanced classification.
result Proves strong H-consistency of the proposed margin loss function. This manuscript shows that AdaBoost and its immediate variants can produce approximate maximum margin classifiers simply by scaling step size choices with a fixed small constant. In this way, when the unscaled step size is an optimal choice, these results provide guarantees for Friedman's empirically successful "shrink…
Deep models maximize minimum margin for high accuracy but decrease average margin, leading to poor robustness.
problem Inadequate balance between accuracy and robustness in deep model training.
method Analyzed the training process of deep models and proposed a new regularizer to promote average margin.
result Demonstrated an intrinsic trade-off between accuracy and robustness, and proposed a regularizer to improve robustness.
This paper proposes MMD-SVR to improve SVR's margin distribution for better generalization.
problem Improving SVR's generalization performance by maximizing the margin distribution of the whole dataset.
method Introducing MMD-SVR with coupled constraints to convert a non-convex optimization problem into a convex one.
result MMD-SVR significantly improves prediction accuracy and generalization compared to classic SVR.
A new approach to cost-sensitive multiclass classification prioritizes certain classes over others.
problem Cost-sensitive multiclass classification where some classes are more important than others.
method Apportioned margin framework that shifts the decision boundary to prioritize certain classes.
result The method improves the error rate for important classes while reducing overall error.
New insights into deep learning: reducing training data significantly improves performance.
problem Understanding and improving generalization in deep learning models.
method Analyzing the distribution of classification margins and dynamically reducing the training set.
result The area under the curve of the margin distribution is a good measure of generalization.
Given a set of possible models (e.g., Bayesian network structures) and a data sample, in the unsupervised model selection problem the task is to choose the most accurate model with respect to the domain joint probability distribution. In contrast to this, in supervised model selection it is a priori known that the chos…
MSBM extends SB for multi-marginal trajectory inference.
problem Trajectory inference from multiple discrete snapshots.
method Multi-Marginal Schrödinger Bridge Matching (MSBM) using iterative Markovian fitting (IMF).
result MSBM effectively captures complex trajectories and respects intermediate distributions.
Given a set of possible models (e.g., Bayesian network structures) and a data sample, in the unsupervised model selection problem the task is to choose the most accurate model with respect to the domain joint probability distribution. In contrast to this, in supervised model selection it is a priori known that the chos…
The paper proposes effective margin regularization to improve adversarial robustness in deep neural networks.
problem Adversarial vulnerability of deep neural networks (DNNs).
method Regularization of effective weight norm during training to maximize effective margins.
result Effective margin regularization (EMR) boosts adversarial robustness in both standard and adversarial training.
Maximizing margins leads to lossless compression of training data.
problem Generalization in supervised learning.
method Information-theoretic interpretation of margin maximization.
result Margin maximization is a form of lossless maximal compression.
Paper introduces a new Bayesian score for discrete networks.
problem Learning the structure of discrete Bayesian networks.
method Empirical Bayes approach with MU+BDs scoring.
result MU+BDs score outperforms U+BDeu in structure learning and prediction.
New algorithm improves online multiclass classification with partial feedback.
problem Online multiclass classification with partial feedback.
method Inspired by complementary labels, a margin-based deterministic approach.
result Our method outperforms existing non-margin-based and stochastic methods.