Improves few-shot learning by adding a large margin to metric-based methods.
problem Few-shot learning's challenge of generalizing well with limited data.
method Unified framework with large margin distance loss function.
result Significant performance improvement with minimal computational overhead.
Paper establishes comparison theorems for large-margin learning.
problem Data piling issue in high-dimension and low-sample size SVM.
method Large-margin unified machines (LUM) loss functions.
result New comparison theorems for all LUM loss functions.
New private algorithms learn large-margin halfspaces efficiently.
problem Learning large-margin halfspaces with privacy constraints.
method Differentially private algorithms based on a new approach.
result Sample complexity depends only on the margin, not dimension.
This study analyzes adversarial training on linearly separable data and finds that gradient updates can achieve large margins in polynomial iterations.
problem Ensuring robustness in machine learning models trained on linearly separable data.
method Analysis of adversarial training with gradient updates on linearly separable data.
result Gradient updates in adversarial training can achieve large margins in polynomial iterations, whereas non-smooth methods require exponentially many iterations.
New method accelerates large margin metric learning for nearest neighbor classification.
problem Efficiently learning metrics for nearest neighbor classification.
method Triplet mining and stratified sampling for large margin metric learning.
result Improved efficiency and scalability of optimization.
Framework for private, noise-tolerant, and efficient learning algorithms.
problem Private and efficient learning of large-margin halfspaces in noisy environments.
method Simple framework using differential privacy and noise tolerance conditions.
result Noise-tolerant and private PAC learners for large-margin halfspaces with sample complexity independent of dimension.
Paper proposes a new classifier for hyperbolic spaces using horospherical boundaries.
problem Optimization of large margin classifiers in hyperbolic spaces.
method Horospherical decision boundaries for geodesically convex optimization.
result Geodesically convex optimization leads to globally optimal solutions.
We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L_2 regularization: We introduce the γ-adapted-dimension, which is a simple function of the spectrum of a distribution's covariance matrix, and show distribution-specific upper and lower bounds on the s…
Paper provides first theoretical guarantees for hyperbolic space learning.
problem Learning a classifier in hyperbolic space for hierarchical data.
method Efficient algorithm for large-margin hyperplane learning in hyperbolic space.
result The low embedding dimension in hyperbolic space leads to superior classifier learning guarantees.
New findings show the large margins theory is insufficient for explaining ensemble methods.
problem Explaining the performance of ensemble methods, especially boosting.
method Illustrated by counterexamples that show how to improve margin distribution without improving test set performance.
result The large margins theory is not sufficient to explain the performance of ensemble methods.
We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L2 regularization: We introduce the margin-adapted dimension, which is a simple function of the second order statistics of the data distribution, and show distribution-specific upper and lower bounds on…
Gradient penalty improves GAN performance by inducing a large-margin classifier.
problem Improving GAN performance and addressing vanishing gradients.
method A unifying framework of expected margin maximization, showing gradient penalties induce large-margin classifiers.
result Gradient penalties reduce vanishing gradients and produce better generated outputs.
Analyzes large-margin classifiers under high-dimensional data.
problem Selecting the best classifier among various margin-based methods.
method Investigates asymptotic performance of large-margin classifiers under two component mixture models.
result Analytical results closely match with Monte Carlo simulations.
Paper proposes LMM-PQS for cross-domain few-shot learning.
problem Cross-domain few-shot learning problem.
method Generates pseudo query images and fine-tunes feature extraction modules with a large margin mechanism.
result LMM-PQS outperforms baseline models in cross-domain few-shot learning.
Derives asymptotic generalization error for large-margin classifiers.
problem Understanding the generalization error of large-margin classifiers.
method Statistical physics replica method for deriving asymptotic expression.
result Establishes phase transition boundary for class separability.
Large margin approach for deep neural networks.
problem Deep learning's lack of margin enforcement.
method Proposes a novel loss function to enforce margin across layers of deep networks.
result Improved performance on various datasets and tasks.
Determinantal point processes (DPPs) offer a powerful approach to modeling diversity in many applications where the goal is to select a diverse subset. We study the problem of learning the parameters (the kernel matrix) of a DPP from labeled training data. We make two contributions. First, we show how to reparameterize…
Efficient algorithms improve learning of large-margin halfspaces.
problem Learning large-margin halfspaces efficiently and reproducibly.
method Design of efficient, dimension-independent, polynomial-time algorithms; SGD-based approach; DP-to-Replicability reduction.
result Improved sample complexity compared to previous algorithms, with optimal sample complexity for one algorithm.
Proposes LM3FE for multi-modal feature extraction in image classification.
problem High-dimensional features and multi-modal data challenges.
method Large margin multi-modal multi-task feature extraction (LM3FE) framework.
result LM3FE outperforms single-task feature extraction and multi-modal feature extraction.
Proposes L-Softmax loss for CNNs to improve feature discriminativeness.
problem Lack of explicit feature discriminativeness in cross-entropy loss.
method Introduces L-Softmax loss that encourages intra-class compactness and inter-class separability.
result Deeply learned features with L-Softmax loss are more discriminative, boosting performance.
pyLEMMINGS improves bioinformatics protein function prediction.
problem Lack of accurate instance-level protein annotations.
method Stochastic sub-gradient optimization for large-margin multiple instance classification and ranking.
result pyLEMMINGS achieves state-of-the-art performance in bioinformatics tasks.
As increasing amounts of sensitive personal information is aggregated into data repositories, it has become important to develop mechanisms for processing the data without revealing information about individual data instances. The differential privacy model provides a framework for the development and theoretical analy…
Paper improves DP-ERM for binary linear classification with large-margin subsets.
problem Differentially private binary linear classification with large-margin subsets.
method Efficient (ε,δ)-DP algorithm with empirical zero-one risk bound. result Improved empirical zero-one risk bound for binary linear classification.
Proposes a gradient-based variable selection method for binary classification in RKHS.
problem Variable selection in high-dimensional data analysis.
method Gradient-based representation of large-margin classifier with group-lasso penalty.
result Selection consistency and risk bound of the estimated classifier.
New method selects stable and accurate classifiers.
problem Classification instability and reproducibility issues.
method Two-stage algorithm: select stable classifiers first, then choose the one with minimal DBI.
result Consistent selection of optimal classifiers with minimal GE and DBI.
A teacher can improve a learner's performance by selecting a smaller, more effective training subset.
problem Improving a learner's performance by selecting a smaller training subset.
method Sharp guarantees for two learners and a mixed-integer nonlinear programming-based algorithm for general learners.
result Empirical experiments show that the algorithm finds good super-teaching sets for regression and classification problems.
Learning to rank is a supervised learning problem where the output space is the space of rankings but the supervision space is the space of relevance scores. We make theoretical contributions to the learning to rank problem both in the online and batch settings. First, we propose a perceptron-like algorithm for learnin…
We present a new boosting algorithm, motivated by the large margins theory for boosting. We give experimental evidence that the new algorithm is significantly more robust against label noise than existing boosting algorithm.
Introduces gradient decay in Softmax for better generalization.
problem Improving generalization performance in neural networks.
method Gradient decay hyperparameter in Softmax for varying gradient rates based on probability.
result Gradient decay rate affects generalization performance and can be tuned for better optimization.
We introduce a method to learn a mixture of submodular "shells" in a large-margin setting. A submodular shell is an abstract submodular function that can be instantiated with a ground set and a set of parameters to produce a submodular function. A mixture of such shells can then also be so instantiated to produce a mor…
Improves MKL for multi-class classification with better feature selection and representation.
problem Real-world multi-class classification problems with non-linear separations.
method Large-margin multiple kernel learning (LMMK) with sparsity term for discriminative feature selection.
result Competitive classification accuracy and sparse non-zero kernel weights.
Support vector machines (SVMs) naturally embody sparseness due to their use of hinge loss functions. However, SVMs can not directly estimate conditional class probabilities. In this paper we propose and study a family of coherence functions, which are convex and differentiable, as surrogates of the hinge function. The …
Study develops large margin machine learning models for predicting host-pathogen protein interactions.
problem Identifying host-pathogen interactions to develop new drugs for infectious diseases.
method Large margin machine learning models, specifically SVM with weighted negative sampling and distance-based weight assignment.
result Proposed and validated a new method for predicting host-pathogen protein interactions.
In this paper, we consider unsupervised partitioning problems, such as clustering, image segmentation, video segmentation and other change-point detection problems. We focus on partitioning problems based explicitly or implicitly on the minimization of Euclidean distortions, which include mean-based change-point detect…
Proposes an online metric learning method for multi-label classification.
problem Lack of consideration for label dependencies and theoretical analysis of loss functions in existing multi-label classification methods.
method Develops a novel online metric learning paradigm based on k-Nearest Neighbour (kNN) and large margin principle, adapted for online streaming data.
result The proposed OML algorithm outperforms state-of-the-art methods on benchmark multi-label datasets.
Enhances motion data analysis using metric learning for DTW.
problem Improving classification accuracy in motion capture data analysis.
method Extends LMNN principle to DTW, treating component-wise dissimilarity values as features.
result Significantly enhances classification accuracy in motion capture data analysis.
Despite the success of the popular kernelized support vector machines, they have two major limitations: they are restricted to Positive Semi-Definite (PSD) kernels, and their training complexity scales at least quadratically with the size of the data. Many natural measures of similarity between pairs of samples are not…
StratLearner learns strategies to prevent misinformation in social networks.
problem Learning strategies to protect against misinformation in social networks without knowing the diffusion model.
method Structured prediction framework using random features and large margin method.
result Our method produces near-optimal protectors without diffusion model information and outperforms other methods.
AGM uses adversarial approach for robust prediction in structured prediction problems.
problem Structured prediction problems with complex relationships between variables.
method Adversarial Graphical Models (AGM) for distributionally robust prediction.
result AGM achieves Fisher consistency and flexibility in loss metrics.
Unified approach to multiclass classification using Gabriel graphs.
problem Improving multiclass classification accuracy and efficiency.
method Integrates Gabriel graphs for binary and multiclass classification, proposing new activation functions and support edge neurons.
result Experimental results show superior performance compared to previous GG-based classifiers.
Binary classification is a common statistical learning problem in which a model is estimated on a set of covariates for some outcome indicating the membership of one of two classes. In the literature, there exists a distinction between hard and soft classification. In soft classification, the conditional class probabil…
This paper improves video summarization using a new algorithm and dataset.
problem Efficiently summarizing videos for browsing and searching.
method Improves sequential determinantal point process (SeqDPP) with a large-margin algorithm and a new probabilistic distribution.
result Significantly improved video summarization model with better user input integration and diversity.
New algorithm learns halfspaces with large margins efficiently.
problem Learning halfspaces with large margins in noisy data.
method Proper learning algorithm with tight complexity bounds.
result Nearly tight complexity characterization for α=1.01-approximate learning. Improved acoustic scene classification with factorized CNN.
problem Acoustic scene classification in varying environments.
method Large-margin factorized CNN with triplet loss.
result Improved performance and better generalization on unseen data.
PGLMC tackles HDLSS problems with improved linear classifier.
problem Challenges in high-dimensional low-sample-size data sets.
method Population-guided large margin classifier (PGLMC) with comprehensive consideration of local structural information and training samples.
result PGLMC outperforms state-of-the-art methods in most cases.
We consider a problem of risk estimation for large-margin multi-class classifiers. We propose a novel risk bound for the multi-class classification problem. The bound involves the marginal distribution of the classifier and the Rademacher complexity of the hypothesis class. We prove that our bound is tight in the numbe…
New study shows non-interactive privacy model requires exponentially more data.
problem Privacy constraints limit data analysis efficiency.
method Developed new technique to prove lower bounds on data samples needed.
result Exponential lower bound on samples needed for learning tasks.
Enhances ordinal embedding with less data by focusing on margin distribution.
problem Insufficient labeled data for ordinal embedding.
method Proposes Distributional Margin based Ordinal Embedding (DMOE) to improve generalization with less data.
result Demonstrates improved generalization performance with less labeled data.