arXiv research

Boosting and other ensemble methods combine a large number of weak classifiers through weighted voting to produce stronger predictive models. To explain the successful performance of boosting algorithms, Schapire et al. (1998) showed that AdaBoost is especially effective at increasing the margins of the training data. …

The paper improves SVM margin-based generalization bounds.

New margin bound improves generalization for voting classifiers.

New findings show margins are not sufficient for explaining gradient boosting performance.

For linear classifiers, the relationship between (normalized) output margin and generalization is captured in a clear and simple bound -- a large output margin implies good generalization. Unfortunately, for deep models, this relationship is less clear: existing analyses of the output margin give complicated bounds whi…

Improves sampling from complex hierarchical models using HMC and automatic marginalization.

New SVM margin bound improves generalization in machine learning.

Proposes MFSWB for marginal fairness in SWB, improving efficiency and performance.

Recent research has used margin theory to analyze the generalization performance for deep neural networks (DNNs). The existed results are almost based on the spectrally-normalized minimum margin. However, optimizing the minimum margin ignores a mass of information about the entire margin distribution, which is crucial …

Margin enlargement over training data has been an important strategy since perceptrons in machine learning for the purpose of boosting the robustness of classifiers toward a good generalization ability. Yet Breiman (1999) showed a dilemma that a uniform improvement on margin distribution does NOT necessarily reduces ge…

We propose the Margin Adaptation for Generative Adversarial Networks (MAGANs) algorithm, a novel training procedure for GANs to improve stability and performance by using an adaptive hinge loss function. We estimate the appropriate hinge loss margin with the expected energy of the target distribution, and derive princi…

Improved exploration in RL with latent state marginalization.

Paper proposes adaptive margin loss to improve few-shot learning.

The key issue of few-shot learning is learning to generalize. This paper proposes a large margin principle to improve the generalization capacity of metric based methods for few-shot learning. To realize it, we develop a unified framework to learn a more discriminative metric space by augmenting the classification loss…

Paper improves ISDA margin calculation using LSMC.

Consider a classification problem where we have both labeled and unlabeled data available. We show that for linear classifiers defined by convex margin-based surrogate losses that are decreasing, it is impossible to construct any semi-supervised approach that is able to guarantee an improvement over the supervised clas…

The paper proposes effective margin regularization to improve adversarial robustness in deep neural networks.

CMRM improves robustness in noisy label settings without requiring privileged knowledge.

Bayesian deep learning improves neural network accuracy and generalization.

We develop an HMC algorithm to easily marginalize random effects in LMMs.

Active learning can't improve over passive in certain settings.

Study improves curvature estimate for stable marginally outer trapped hypersurfaces with a free boundary.

MLS improves feature selection for imbalanced data.

Efficient algorithms improve learning of large-margin halfspaces.

Max-margin learning is a powerful approach to building classifiers and structured output predictors. Recent work on max-margin supervised topic models has successfully integrated it with Bayesian topic models to discover discriminative latent semantic structures and make accurate predictions for unseen testing data. Ho…

New margin-based learning guarantees improve generalization bounds.

New method improves calibration of neural networks by targeting robust margins and local smoothness.

Boosting algorithms produce a classifier by iteratively combining base hypotheses. It has been observed experimentally that the generalization error keeps improving even after achieving zero training error. One popular explanation attributes this to improvements in margins. A common goal in a long line of research, is …

Improved MUSE boosts performance and reduces error in Bayesian inference.

Quantization techniques have been applied in many challenging finance applications, including pricing claims with path dependence and early exercise features, stochastic optimal control, filtering problems and efficient calibration of large derivative books. Recursive Marginal Quantization of the Euler scheme has recen…

Adversarial robustness has become an important research topic given empirical demonstrations on the lack of robustness of deep neural networks. Unfortunately, recent theoretical results suggest that adversarial training induces a strict tradeoff between classification accuracy and adversarial robustness. In this paper,…

We present an improved algorithm for {\em quasi-properly} learning convex polyhedra in the realizable PAC setting from data with a margin. Our learning algorithm constructs a consistent polyhedron as an intersection of about $t \log t$ halfspaces with constant-size margins in time polynomial in $t$ (where $t$ is the nu…

We study adversarial robustness of neural networks from a margin maximization perspective, where margins are defined as the distances from inputs to a classifier's decision boundary. Our study shows that maximizing margins can be achieved by minimizing the adversarial loss on the decision boundary at the "shortest succ…

Improves hyperparameter learning in GP models with non-conjugate likelihoods.

This work improves adversarial robustness by boosting model ensembles with margin maximization.

One of the main open problems in the theory of multi-category margin classification is the form of the optimal dependency of a guaranteed risk on the number C of categories, the sample size m and the margin parameter gamma. From a practical point of view, the theoretical analysis of generalization performance contribut…

Data augmentation (DA) is commonly used during model training, as it significantly improves test error and model robustness. DA artificially expands the training set by applying random noise, rotations, crops, or even adversarial perturbations to the input data. Although DA is widely used, its capacity to provably impr…

A new sampler for FLMs improves token-level decoding controls.

In this paper, we reformulate the forest representation learning approach as an additive model which boosts the augmented feature instead of the prediction. We substantially improve the upper bound of generalization gap from $\mathcal{O}(\sqrt\frac{\ln m}{m})$ to $\mathcal{O}(\frac{\ln m}{m})$, while $λ$ - the margin r…

New margin-based regularization and selective sampling improve deep neural network performance.

Improved robustness of machine learning models with controlled Lipschitz constants.

Paper improves DP-ERM for binary linear classification with large-margin subsets.

New loss function improves convergence rate for neural networks.

Tall wheatgrass outperforms rye in energy and environmental metrics, marginally improving economic viability.

New method estimates marginal likelihood for deep learning models using training data alone.

Improved Gaussian process regression with tighter log marginal likelihood bounds.

MARVEL curbs memorization of noisy labels in deep nets.

Max-min margin Markov networks improve consistency in structured prediction.