arXiv research

Shallow nonlinear networks can separate classes linearly with polynomially scaling width.

This paper investigates how data augmentation improves linear separation of manifold data.

Adversarial noises are linearly separable for random neural networks.

Randomly initialized neural networks can linearly separate arbitrary sets.

In this paper, we presented a novel semi-supervised one-class classification algorithm which assumes that class is linearly separable from other elements. We proved theoretically that class is linearly separable if and only if it is maximal by probability within the sets with the same mean. Furthermore, we presented an…

Gradient descent converges to perfect classification in neural nets for non-separable data.

This study analyzes adversarial training on linearly separable data and finds that gradient updates can achieve large margins in polynomial iterations.

Constructs classifiers for neural networks with specific data configurations.

We develop randomized (block) coordinate descent (CD) methods for linearly constrained convex optimization. Unlike most CD methods, we do not assume the constraints to be separable, but let them be coupled linearly. To our knowledge, ours is the first CD method that allows linear coupling constraints, without making th…

Adam optimizes linear classifiers with separable data.

Gradient descent reveals the exact implicit bias via dual optimization for linearly separable data.

Mirror flow optimizes separable data problems, converging to a maximum margin classifier.

The study quantifies how many objects can be linearly classified under all views.

Gradient descent-based adversarial training converges to robust classifiers on linearly separable data.

Improved mistake bound for group linear separable cases in online multiclass linear classification.

Neural networks use their hidden layers to transform input data into linearly separable data clusters, with a linear or a perceptron type output layer making the final projection on the line perpendicular to the discriminating hyperplane. For complex data with multimodal distributions this transformation is difficult t…

We show linear XOR classification is possible and propose equality separation for anomaly detection.

The paper connects decision tree interpretability and robustness through separation.

Proposes a new SVM model for binary classification with theoretical and practical advantages.

Graph convolution improves linear separability and generalizes to out-of-distribution data.

Gradient descent converges to max-margin solution for hinge loss.

This paper explains a mechanism called phase collapse that improves image classification accuracy.

We study the problem of efficient online multiclass linear classification with bandit feedback, where all examples belong to one of $K$ classes and lie in the $d$-dimensional Euclidean space. Previous works have left open the challenge of designing efficient algorithms with finite mistake bounds when the data is linear…

An important issue in neural network research is how to choose the number of nodes and layers such as to solve a classification problem. We provide new intuitions based on earlier results by An et al. (2015) by deriving an upper bound on the number of nodes in networks with two hidden layers such that linear separabili…

We provide new results concerning label efficient, polynomial time, passive and active learning of linear separators. We prove that active learning provides an exponential improvement over PAC (passive) learning of homogeneous linear separators under nearly log-concave distributions. Building on this, we provide a comp…

Recent years have witnessed the rapid development of block coordinate update (BCU) methods, which are particularly suitable for problems involving large-sized data and/or variables. In optimization, BCU first appears as the coordinate descent method that works well for smooth problems or those with separable nonsmooth …

Heuristic algorithm for portfolio optimization reduces solve times to milliseconds.

Neural networks with REctified Linear Unit (ReLU) activation functions (a.k.a. ReLU networks) have achieved great empirical success in various domains. Nonetheless, existing results for learning ReLU networks either pose assumptions on the underlying data distribution being e.g. Gaussian, or require the network size an…

Gradient descent, when applied to the task of logistic regression, outputs iterates which are biased to follow a unique ray defined by the data. The direction of this ray is the maximum margin predictor of a maximal linearly separable subset of the data; the gradient descent iterates converge to this ray in direction a…

DAMI uses interpretable regions to select informative samples for deep learning models.

The paper extends optimal transport for linear separability of sheared distributions in supervised learning.

Unhinged loss minimization fails to improve classifier accuracy for simple data.

Max-margin classifiers can overfit without harming performance in high dimensions.

Adam's bias shifts from full-batch to max-margin of different norms for separable data.

Large stepsize GD for logistic regression converges faster than expected.

We examine gradient descent on unregularized logistic regression problems, with homogeneous linear predictors on linearly separable datasets. We show the predictor converges to the direction of the max-margin (hard margin SVM) solution. The result also generalizes to other monotone decreasing loss functions with an inf…

Stochastic Gradient Descent (SGD) is a central tool in machine learning. We prove that SGD converges to zero loss, even with a fixed (non-vanishing) learning rate - in the special case of homogeneous linear classifiers with smooth monotone loss functions, optimized on linearly separable data. Previous works assumed eit…

Paper analyzes GMM for separable data with various parameter structures.

Deep networks achieve linear separability through progressive folding of data in higher dimensions.

This work simplifies SVM parameter selection using S&S ratio.

Study models deep learning training dynamics using locally elastic SDEs to reveal feature separability.

Neural networks favor simple features over complex ones, even when complex features are available.

BELIEF framework interprets GLMs using binary linear models.

Gradient descent converges with arbitrary stepsize for separable data under Fenchel-Young losses.

Kernel k-Means algorithm improves clustering of non-linear data.

Develops an algorithm to find the best subset of points for maximizing the coefficient of determination.

New loss function improves convergence rate for neural networks.

ICE algorithm solves exact 0-1 loss linear classification problem efficiently.