arXiv research

This study analyzes adversarial training on linearly separable data and finds that gradient updates can achieve large margins in polynomial iterations.

The support vector machine (SVM) is an important class of learning machines for function approach, pattern recognition, and time-serious prediction, etc. It maps samples into the feature space by so-called support vectors of selected samples, and then feature vectors are separated by maximum margin hyperplane. The pres…

Paper analyzes GMM for separable data with various parameter structures.

Mirror flow optimizes separable data problems, converging to a maximum margin classifier.

Adam's bias shifts from full-batch to max-margin of different norms for separable data.

Gradient descent converges to max-margin solution for hinge loss.

The paper proves a margin inequality for separating hyperplanes, useful for analyzing algorithmic bias.

Gradient descent reveals the exact implicit bias via dual optimization for linearly separable data.

Data augmentation improves model robustness by enforcing a margin.

Adam optimizes linear classifiers with separable data.

Gradient methods avoid overfitting on separable data.

The paper improves SVM margin-based generalization bounds.

Efficient algorithms for online multiclass linear classification with bandit feedback under linear separability conditions.

Gradient descent-based adversarial training converges to robust classifiers on linearly separable data.

Develops bounds predicting deep learning generalization using optimal transport.

Paper improves DP-ERM for binary linear classification with large-margin subsets.

Paper proves tight lower bounds for online multicalibration, separating it from marginal calibration.

One approach to monitoring a dynamic system relies on decomposition of the system into weakly interacting subsystems. An earlier paper introduced a notion of weak interaction called separability, and showed that it leads to exact propagation of marginals for prediction. This paper addresses two questions left open by t…

Tropical SVM tackles phylogenomics by classifying multi-locus data.

A new method for unsupervised domain adaptation using Gaussian processes.

Shallow neural nets classify objects perfectly if their distribution is linearly separable.

New indices for determining cluster compactness and separability.

New loss function improves convergence rate for neural networks.

Derives asymptotic generalization error for large-margin classifiers.

Study separates interventions on a causal Bayesian network using aggregate observations.

New findings on robust learning with well-separated data.

MACQ method explains deep learning models by analyzing feature contributions across prediction levels.

In this paper, we deal with the problem of marginalization over and conditioning on two disjoint subsets of the node set of chain graphs (CGs) with the LWF Markov property. For this purpose, we define the class of chain mixed graphs (CMGs) with three types of edges and, for this class, provide a separation criterion un…

GMC benchmark isolates retrieval in Transformers, revealing max-margin alignment.

Reliable measures of statistical dependence could be useful tools for learning independent features and performing tasks like source separation using Independent Component Analysis (ICA). Unfortunately, many of such measures, like the mutual information, are hard to estimate and optimize directly. We propose to learn i…

Stacked conformal prediction simplifies model validation.

DCC separates marginal estimation from dependence modeling for improved classification accuracy.

We provide a detailed study on the implicit bias of gradient descent when optimizing loss functions with strictly monotone tails, such as the logistic loss, over separable datasets. We look at two basic questions: (a) what are the conditions on the tail of the loss function under which gradient descent converges in the…

Posterior regularization enhances Bayesian hierarchical mixture clustering by improving node separation.

New insights into deep learning: reducing training data significantly improves performance.

A fast method for training linear classifiers maximizes margins.

Paper proposes adaptive margin loss to improve few-shot learning.

New research shows the maximum ℓ1-margin classifier doesn't adapt to sparse ground truths.

The paper analyzes the maximum margin algorithm's performance on noisy data.

Max-margin classifiers can overfit without harming performance in high dimensions.

We derive and analyze a new, efficient, pool-based active learning algorithm for halfspaces, called ALuMA. Most previous algorithms show exponential improvement in the label complexity assuming that the distribution over the instance space is close to uniform. This assumption rarely holds in practical applications. Ins…

We examine gradient descent on unregularized logistic regression problems, with homogeneous linear predictors on linearly separable datasets. We show the predictor converges to the direction of the max-margin (hard margin SVM) solution. The result also generalizes to other monotone decreasing loss functions with an inf…

We present a simple noise-robust margin-based active learning algorithm to find homogeneous (passing the origin) linear separators and analyze its error convergence when labels are corrupted by noise. We show that when the imposed noise satisfies the Tsybakov low noise condition (Mammen, Tsybakov, and others 1999; Tsyb…

Novel proof shows continuity of optimal transport feasible set mapping.

We study the connections between spectral clustering and the problems of maximum margin clustering, and estimation of the components of level sets of a density function. Specifically, we obtain bounds on the eigenvectors of graph Laplacian matrices in terms of the between cluster separation, and within cluster connecti…

We study the implicit bias of gradient descent methods in solving a binary classification problem over a linearly separable dataset. The classifier is described by a nonlinear ReLU model and the objective function adopts the exponential loss function. We first characterize the landscape of the loss function and show th…

GD with large, adaptive stepsizes achieves optimal risk in logistic regression.

Let X be a data matrix of rank ρ, whose rows represent n points in d-dimensional space. The linear support vector machine constructs a hyperplane separator that maximizes the 1-norm soft margin. We develop a new oblivious dimension reduction technique which is precomputed and can be applied to any input matrix X. We pr…