New method accelerates large margin metric learning for nearest neighbor classification.
problem Efficiently learning metrics for nearest neighbor classification.
method Triplet mining and stratified sampling for large margin metric learning.
result Improved efficiency and scalability of optimization.
We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L_2 regularization: We introduce the γ-adapted-dimension, which is a simple function of the spectrum of a distribution's covariance matrix, and show distribution-specific upper and lower bounds on the s…
Paper establishes comparison theorems for large-margin learning.
problem Data piling issue in high-dimension and low-sample size SVM.
method Large-margin unified machines (LUM) loss functions.
result New comparison theorems for all LUM loss functions.
Analyzes large-margin classifiers under high-dimensional data.
problem Selecting the best classifier among various margin-based methods.
method Investigates asymptotic performance of large-margin classifiers under two component mixture models.
result Analytical results closely match with Monte Carlo simulations.
Paper proposes a new classifier for hyperbolic spaces using horospherical boundaries.
problem Optimization of large margin classifiers in hyperbolic spaces.
method Horospherical decision boundaries for geodesically convex optimization.
result Geodesically convex optimization leads to globally optimal solutions.
Improves few-shot learning by adding a large margin to metric-based methods.
problem Few-shot learning's challenge of generalizing well with limited data.
method Unified framework with large margin distance loss function.
result Significant performance improvement with minimal computational overhead.
Proposes LM3FE for multi-modal feature extraction in image classification.
problem High-dimensional features and multi-modal data challenges.
method Large margin multi-modal multi-task feature extraction (LM3FE) framework.
result LM3FE outperforms single-task feature extraction and multi-modal feature extraction.
Large margin approach for deep neural networks.
problem Deep learning's lack of margin enforcement.
method Proposes a novel loss function to enforce margin across layers of deep networks.
result Improved performance on various datasets and tasks.
We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L2 regularization: We introduce the margin-adapted dimension, which is a simple function of the second order statistics of the data distribution, and show distribution-specific upper and lower bounds on…
Derives asymptotic generalization error for large-margin classifiers.
problem Understanding the generalization error of large-margin classifiers.
method Statistical physics replica method for deriving asymptotic expression.
result Establishes phase transition boundary for class separability.
pyLEMMINGS improves bioinformatics protein function prediction.
problem Lack of accurate instance-level protein annotations.
method Stochastic sub-gradient optimization for large-margin multiple instance classification and ranking.
result pyLEMMINGS achieves state-of-the-art performance in bioinformatics tasks.
Framework for private, noise-tolerant, and efficient learning algorithms.
problem Private and efficient learning of large-margin halfspaces in noisy environments.
method Simple framework using differential privacy and noise tolerance conditions.
result Noise-tolerant and private PAC learners for large-margin halfspaces with sample complexity independent of dimension.
Paper improves DP-ERM for binary linear classification with large-margin subsets.
problem Differentially private binary linear classification with large-margin subsets.
method Efficient (ε,δ)-DP algorithm with empirical zero-one risk bound. result Improved empirical zero-one risk bound for binary linear classification.
Proposes a gradient-based variable selection method for binary classification in RKHS.
problem Variable selection in high-dimensional data analysis.
method Gradient-based representation of large-margin classifier with group-lasso penalty.
result Selection consistency and risk bound of the estimated classifier.
Improves MKL for multi-class classification with better feature selection and representation.
problem Real-world multi-class classification problems with non-linear separations.
method Large-margin multiple kernel learning (LMMK) with sparsity term for discriminative feature selection.
result Competitive classification accuracy and sparse non-zero kernel weights.
Proposes an online metric learning method for multi-label classification.
problem Lack of consideration for label dependencies and theoretical analysis of loss functions in existing multi-label classification methods.
method Develops a novel online metric learning paradigm based on k-Nearest Neighbour (kNN) and large margin principle, adapted for online streaming data.
result The proposed OML algorithm outperforms state-of-the-art methods on benchmark multi-label datasets.
Binary classification is a common statistical learning problem in which a model is estimated on a set of covariates for some outcome indicating the membership of one of two classes. In the literature, there exists a distinction between hard and soft classification. In soft classification, the conditional class probabil…
As increasing amounts of sensitive personal information is aggregated into data repositories, it has become important to develop mechanisms for processing the data without revealing information about individual data instances. The differential privacy model provides a framework for the development and theoretical analy…
Unified approach to multiclass classification using Gabriel graphs.
problem Improving multiclass classification accuracy and efficiency.
method Integrates Gabriel graphs for binary and multiclass classification, proposing new activation functions and support edge neurons.
result Experimental results show superior performance compared to previous GG-based classifiers.
Support vector machines (SVMs) naturally embody sparseness due to their use of hinge loss functions. However, SVMs can not directly estimate conditional class probabilities. In this paper we propose and study a family of coherence functions, which are convex and differentiable, as surrogates of the hinge function. The …
A teacher can improve a learner's performance by selecting a smaller, more effective training subset.
problem Improving a learner's performance by selecting a smaller training subset.
method Sharp guarantees for two learners and a mixed-integer nonlinear programming-based algorithm for general learners.
result Empirical experiments show that the algorithm finds good super-teaching sets for regression and classification problems.
Improved acoustic scene classification with factorized CNN.
problem Acoustic scene classification in varying environments.
method Large-margin factorized CNN with triplet loss.
result Improved performance and better generalization on unseen data.
Stability is an important aspect of a classification procedure because unstable predictions can potentially reduce users' trust in a classification system and also harm the reproducibility of scientific conclusions. The major goal of our work is to introduce a novel concept of classification instability, i.e., decision…
Cross-entropy loss together with softmax is arguably one of the most common used supervision components in convolutional neural networks (CNNs). Despite its simplicity, popularity and excellent performance, the component does not explicitly encourage discriminative learning of features. In this paper, we propose a gene…
New private algorithms learn large-margin halfspaces efficiently.
problem Learning large-margin halfspaces with privacy constraints.
method Differentially private algorithms based on a new approach.
result Sample complexity depends only on the margin, not dimension.
DOC3 learns from contradictions to improve deep one class classification.
problem Deep one class classification problems.
method Formalizes learning from contradictions for one class large-margin loss, proposes DOC3 algorithm.
result DOC3 incurs lower generalization error compared to traditional inductive learning.
We consider a problem of risk estimation for large-margin multi-class classifiers. We propose a novel risk bound for the multi-class classification problem. The bound involves the marginal distribution of the classifier and the Rademacher complexity of the hypothesis class. We prove that our bound is tight in the numbe…
SqueezeFit reduces high-dimensional data to lower dimensions while preserving label distances.
problem Label-aware dimensionality reduction in high-dimensional spaces.
method Semidefinite programming relaxation of nearest neighbor classification.
result Provable recovery of a planted projection operator from labeled data.
This study analyzes adversarial training on linearly separable data and finds that gradient updates can achieve large margins in polynomial iterations.
problem Ensuring robustness in machine learning models trained on linearly separable data.
method Analysis of adversarial training with gradient updates on linearly separable data.
result Gradient updates in adversarial training can achieve large margins in polynomial iterations, whereas non-smooth methods require exponentially many iterations.
Learning to rank is a supervised learning problem where the output space is the space of rankings but the supervision space is the space of relevance scores. We make theoretical contributions to the learning to rank problem both in the online and batch settings. First, we propose a perceptron-like algorithm for learnin…
TAO outperforms other decision tree algorithms in accuracy.
problem Optimizing decision tree algorithms for better performance.
method Comparison of TAO with established algorithms on various datasets.
result TAO achieves higher accuracy in nearly all datasets.
Hyperbolic SVM improves classification in complex networks.
problem Accurately classifying points in hyperbolic space with hierarchical relationships.
method Introducing hyperbolic SVM, a hyperbolic formulation of SVM classifiers.
result Hyperbolic SVM outperforms Euclidean SVM in multi-class prediction tasks.
Paper provides first theoretical guarantees for hyperbolic space learning.
problem Learning a classifier in hyperbolic space for hierarchical data.
method Efficient algorithm for large-margin hyperplane learning in hyperbolic space.
result The low embedding dimension in hyperbolic space leads to superior classifier learning guarantees.
Tricks improve retail product image classification accuracy.
problem Retail Product Image Classification
method Various tricks including a new LCA layer, Instagram-pretrained Convnet, and Maximum Entropy loss.
result Increased accuracy of fine-tuned convnets by a large margin.
Despite the success of the popular kernelized support vector machines, they have two major limitations: they are restricted to Positive Semi-Definite (PSD) kernels, and their training complexity scales at least quadratically with the size of the data. Many natural measures of similarity between pairs of samples are not…
PGLMC tackles HDLSS problems with improved linear classifier.
problem Challenges in high-dimensional low-sample-size data sets.
method Population-guided large margin classifier (PGLMC) with comprehensive consideration of local structural information and training samples.
result PGLMC outperforms state-of-the-art methods in most cases.
New online learning algorithm combines PA and TER for binary classification.
problem Binary classification with non-separable data and data imbalance.
method Online Passive-Aggressive (PA) and Total-Error-Rate (TER) learning combined into PATER algorithm.
result PATER algorithms outperform existing online learning algorithms in efficiency and effectiveness.
New findings show the large margins theory is insufficient for explaining ensemble methods.
problem Explaining the performance of ensemble methods, especially boosting.
method Illustrated by counterexamples that show how to improve margin distribution without improving test set performance.
result The large margins theory is not sufficient to explain the performance of ensemble methods.
Gradient penalty improves GAN performance by inducing a large-margin classifier.
problem Improving GAN performance and addressing vanishing gradients.
method A unifying framework of expected margin maximization, showing gradient penalties induce large-margin classifiers.
result Gradient penalties reduce vanishing gradients and produce better generated outputs.
CoNDA improves domain classification for IPDAs by incorporating new and personalized domains.
problem Continuous learning for new and personalized domains in IPDAs.
method Neural network based approach for incremental learning.
result CoNDA achieves high accuracy and outperforms baselines.
Paper proposes LMM-PQS for cross-domain few-shot learning.
problem Cross-domain few-shot learning problem.
method Generates pseudo query images and fine-tunes feature extraction modules with a large margin mechanism.
result LMM-PQS outperforms baseline models in cross-domain few-shot learning.
Reweighting improves risk bounds in certain data regions.
problem Improving risk bounds in classification and heteroscedastic regression.
method Weighted empirical risk minimization with a data-dependent weight function.
result A weighted ERM estimator can achieve superior performance in specific sub-regions.
In this paper, we propose a general framework to learn a robust large-margin binary classifier when corrupt measurements, called anomalies, caused by sensor failure might be present in the training set. The goal is to minimize the generalization error of the classifier on non-corrupted measurements while controlling th…
In this paper, we propose a general framework to learn a robust large-margin binary classifier when corrupt measurements, called anomalies, caused by sensor failure might be present in the training set. The goal is to minimize the generalization error of the classifier on non-corrupted measurements while controlling th…
Determinantal point processes (DPPs) offer a powerful approach to modeling diversity in many applications where the goal is to select a diverse subset. We study the problem of learning the parameters (the kernel matrix) of a DPP from labeled training data. We make two contributions. First, we show how to reparameterize…
Efficient algorithms improve learning of large-margin halfspaces.
problem Learning large-margin halfspaces efficiently and reproducibly.
method Design of efficient, dimension-independent, polynomial-time algorithms; SGD-based approach; DP-to-Replicability reduction.
result Improved sample complexity compared to previous algorithms, with optimal sample complexity for one algorithm.
In many applications, input data are sampled functions taking their values in infinite dimensional spaces rather than standard vectors. This fact has complex consequences on data analysis algorithms that motivate modifications of them. In fact most of the traditional data analysis tools for regression, classification a…
Prototype networks on hyperspheres improve classification and regression.
problem Improving classification and regression performance.
method Using hyperspherical prototypes for classification and regression, optimizing prototypes through data-independent margin separation.
result Hyperspherical prototype networks outperform other methods in classification, regression, and their combination.