arXiv research

New DP algorithms with margin guarantees for various hypothesis sets.

Paper proposes adaptive margin loss to improve few-shot learning.

In this paper, we reformulate the forest representation learning approach as an additive model which boosts the augmented feature instead of the prediction. We substantially improve the upper bound of generalization gap from $\mathcal{O}(\sqrt\frac{\ln m}{m})$ to $\mathcal{O}(\frac{\ln m}{m})$, while $λ$ - the margin r…

We present an improved algorithm for {\em quasi-properly} learning convex polyhedra in the realizable PAC setting from data with a margin. Our learning algorithm constructs a consistent polyhedron as an intersection of about $t \log t$ halfspaces with constant-size margins in time polynomial in $t$ (where $t$ is the nu…

MACQ method explains deep learning models by analyzing feature contributions across prediction levels.

Bayesian evidence helps compare models but can overfit.

We present a new active sampling method we call min-margin which trains multiple learners on bootstrap samples and then chooses the examples to label based on the candidates' minimum margin amongst the bootstrapped models. This extends standard margin sampling in a way that increases its diversity in a supervised manne…

New SVM margin bound improves generalization in machine learning.

Active learning can't improve over passive in certain settings.

Paper introduces negative margin loss for better few-shot classification accuracy.

A new approach for instance-optimal learning that bypasses impossibility results.

Max-margin learning is a powerful approach to building classifiers and structured output predictors. Recent work on max-margin supervised topic models has successfully integrated it with Bayesian topic models to discover discriminative latent semantic structures and make accurate predictions for unseen testing data. Ho…

Framework for private, noise-tolerant, and efficient learning algorithms.

The paper proposes effective margin regularization to improve adversarial robustness in deep neural networks.

We address the problem of learning the parameters in graphical models when inference is intractable. A common strategy in this case is to replace the partition function with its Bethe approximation. We show that there exists a regime of empirical marginals where such Bethe learning will fail. By failure we mean that th…

3MSBM learns smooth trajectories from multiple snapshots.

MARVEL curbs memorization of noisy labels in deep nets.

Bayesian approach learns invariances from data alone, but last layer approximation is not always sufficient.

Maximizing margins leads to lossless compression of training data.

We present a max-margin nonparametric latent feature model, which unites the ideas of max-margin learning and Bayesian nonparametrics to discover discriminative latent features for link prediction and automatically infer the unknown latent social dimension. By minimizing a hinge-loss using the linear expectation operat…

Algorithm learns dynamics from past observations.

Proposes a new distance metric for multi-marginal optimal transport.

An active margin system for margin loans is proposed for Chinese margin lending market, which uses cash and randomly selected stock as collateral. The conditional probability of negative return(CPNR) after a forced sale of securities from under-margined account in a falling market is used to measure the risk faced by t…

A framework estimates categorical distributions under constraints, ensuring generality and uniqueness.

The study establishes SQ lower bounds for learning halfspaces and ReLUs under Gaussian marginals.

New margin-based learning guarantees improve generalization bounds.

Max-min margin Markov networks improve consistency in structured prediction.

Given a matrix $A$, a linear feasibility problem (of which linear classification is a special case) aims to find a solution to a primal problem $w: A^Tw > \textbf{0}$ or a certificate for the dual problem which is a probability distribution $p: Ap = \textbf{0}$. Inspired by the continued importance of "large-margin cla…

Develops bounds predicting deep learning generalization using optimal transport.

Study shows exponential error reduction in multiclass classification without bias-variance trade-off.

Support vector regression (SVR) is one of the most popular machine learning algorithms aiming to generate the optimal regression curve through maximizing the minimal margin of selected training samples, i.e., support vectors. Recent researchers reveal that maximizing the margin distribution of whole training dataset ra…

This work proposes a new method to estimate joint probability from pairwise marginals, reducing sample complexity.

Boosting is one of the most successful ideas in machine learning. The most well-accepted explanations for the low generalization error of boosting algorithms such as AdaBoost stem from margin theory. The study of margins in the context of boosting algorithms was initiated by Schapire, Freund, Bartlett and Lee (1998) an…

We present a new replay-based method of continual classification learning that we term "conditional replay" which generates samples and labels together by sampling from a distribution conditioned on the class. We compare conditional replay to another replay-based continual learning paradigm (which we term "marginal rep…

This work improves adversarial robustness by boosting model ensembles with margin maximization.

New framework for learning from imbalanced data with theoretical guarantees.

Random projections have been applied in many machine learning algorithms. However, whether margin is preserved after random projection is non-trivial and not well studied. In this paper we analyse margin distortion after random projection, and give the conditions of margin preservation for binary classification problem…

Paper proposes LMM-PQS for cross-domain few-shot learning.

New method estimates marginal likelihood for deep learning models using training data alone.

Link prediction is a fundamental task in statistical network analysis. Recent advances have been made on learning flexible nonparametric Bayesian latent feature models for link prediction. In this paper, we present a max-margin learning method for such nonparametric latent feature relational models. Our approach attemp…

New method identifies drift and diffusivity from SDE marginals.

Batch normalization biases linear models towards uniform margins, improving performance in binary classification.

Margin enlargement over training data has been an important strategy since perceptrons in machine learning for the purpose of boosting the robustness of classifiers toward a good generalization ability. Yet Breiman (1999) showed a dilemma that a uniform improvement on margin distribution does NOT necessarily reduces ge…

This paper proposes a new method to improve domain adaptation by distinguishing between marginal and dependence structure differences.

In the absence of prior knowledge, ordinal embedding methods obtain new representation for items in a low-dimensional Euclidean space via a set of quadruple-wise comparisons. These ordinal comparisons often come from human annotators, and sufficient comparisons induce the success of classical approaches. However, colle…

Paper resolves conflicting Shapley value approaches by showing conditional is unsound and marginal is preferred.

Bayesian deep learning improves neural network accuracy and generalization.

Advances robustness of metric learning by adversarial margin in input space.