Classification is an important topic in statistics and machine learning with great potential in many real applications. In this paper, we investigate two popular large margin classification methods, Support Vector Machine (SVM) and Distance Weighted Discrimination (DWD), under two contexts: the high-dimensional, low-sa…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,738 papers · 148 categories
Trend · papers per month
48 results for “high-dimensional SVM”
SVM generalizes well even with many support vectors in high dimensions.
problem Generalization of SVM in high-dimensional spaces with many support vectors.
method Identified new deterministic equivalences and proved conditions for support vector proliferation.
result Broadened conditions for SVM generalization in high-dimensional settings and proved converse result.
Efficient Distributed Learning over Decentralized Networks with Convoluted Support Vector Machinestat.ML
Paper tackles efficient SVM classification over decentralized networks.
problem Efficiently classifying high-dimensional data over decentralized networks.
method Convolution-based smoothing technique for nonsmooth hinge loss function, combined with an efficient ADMM algorithm.
result Provable linear convergence of the ADMM algorithm and near-optimal statistical convergence of the sparse estimator.
SVM and linear regression models coincide in high dimensions.
problem Understanding the connection between SVM and linear regression in high-dimensional data.
method Analyzing feature models and proving lower bounds on dimensionality.
result A sharp phase transition in Gaussian feature models, with support vector proliferation occurring only in very high dimensions.
A new method for high-dimensional classification using Bernstein polynomials.
problem Computational difficulties in high-dimensional SVM hinge loss.
method Proposes Bernstein support vector machine (BernSVM) and two efficient algorithms.
result Achieves a prediction accuracy rate of with high probability.
This work proposes a model averaging method for SVM that avoids redundant covariates and achieves asymptotic optimality.
problem Redundant covariates impair SVM performance in high-dimensional settings.
method Frequentist model averaging procedure for SVM using cross-validation to select optimal weights.
result The proposed method achieves asymptotic optimality in SVM model averaging.
Support vector machine (SVM) is a particularly powerful and flexible supervised learning model that analyzes data for both classification and regression, whose usual algorithm complexity scales polynomially with the dimension of data space and the number of data points. To tackle the big data challenge, a quantum SVM a…
The support vector machine (SVM) is a powerful and widely used classification algorithm. This paper uses the Karush-Kuhn-Tucker conditions to provide rigorous mathematical proof for new insights into the behavior of SVM. These insights provide perhaps unexpected relationships between SVM and two other linear classifier…
This paper improves SVM prediction uncertainty quantification methods.
problem Lack of comprehensive UQ methods for SVM predictions.
method Developed SSVQR model for sparse PI estimation and feature selection algorithm.
result Proposed SSVQR model achieves sparse solutions and improves PI quality.
We consider classification tasks in the regime of scarce labeled training data in high dimensional feature space, where specific expert knowledge is also available. We propose a new hybrid optimization algorithm that solves the elastic-net support vector machine (SVM) through an alternating direction method of multipli…
When applying the support vector machine (SVM) to high-dimensional classification problems, we often impose a sparse structure in the SVM to eliminate the influences of the irrelevant predictors. The lasso and other variable selection techniques have been successfully used in the SVM to perform automatic variable selec…
The current study proposes a dimension reduction method, stepwise support vector machine (SVM), to reduce the dimensions of large p small n datasets. The proposed method is compared with other dimension reduction methods, namely, the Pearson product difference correlation coefficient (PCCs), recursive feature eliminati…
A sparse semismooth Newton based augmented Lagrangian method for large-scale support vector machinesmath.OC
Support vector machines (SVMs) are successful modeling and prediction tools with a variety of applications. Previous work has demonstrated the superiority of the SVMs in dealing with the high dimensional, low sample size problems. However, the numerical difficulties of the SVMs will become severe with the increase of t…
Paper shows SVMs can interpolate data in various settings.
problem Understanding SVM performance and generalization.
method Flexible analysis framework for proving SVM interpolation in diverse settings.
result Support vector machines can interpolate data in many cases not previously covered.
Novel defense algorithm improves SVMs against data poisoning attacks.
problem Vulnerability of SVMs to targeted training data manipulations like poisoning attacks.
method Developed a weighted SVM using K-LID to de-emphasize suspicious data samples.
result Significant reduction in classification error rates (10% on average) with the proposed defense.
This paper uses MIO to select features for kernel SVM classification.
problem Feature selection for kernel SVM classification.
method Mixed-integer optimization (MIO) for feature subset selection.
result The MIO approach can often outperform linear-SVM-based methods in prediction performance.
Sparse support vector machine (SVM) is a popular classification technique that can simultaneously learn a small set of the most interpretable features and identify the support vectors. It has achieved great successes in many real-world applications. However, for large-scale problems involving a huge number of samples a…
We propose a novel non-parametric adaptive anomaly detection algorithm for high dimensional data based on rank-SVM. Data points are first ranked based on scores derived from nearest neighbor graphs on n-point nominal data. We then train a rank-SVM using this ranked data. A test-point is declared as an anomaly at alpha-…
Improved SRHT for linear SVM classification with higher accuracy.
problem Inefficient random projection methods for high-dimensional data.
method Importance sampling and deterministic top- sampling for effective low-dimensional embedding.
result Higher classification accuracy on real-life datasets.
SVMs improve forest fire detection accuracy on challenging datasets.
problem Rapid and accurate detection of forest fires.
method Training SVMs on labeled fire image datasets, focusing on data preprocessing, feature extraction, and model training.
result SVMs enhance detection accuracy on complex datasets, revealing key parameters affecting performance.
A new method for causal inference in high-dimensional data using machine learning.
problem Causal inference in high-dimensional observational data.
method Support Points Sample Splitting (SPSS) for efficient double machine learning (DML) in causal inference.
result Deep learning with SPSS and hybrid methods outperform SVM with SPSS in computational efficiency and estimation quality.
RHPSVM improves SVM performance with robust loss function.
problem Outliers and resampling instability in SVM models.
method RHPSVM uses a rescaled Huberized pinball loss function.
result RHPSVM outperforms existing SVM models in noisy and small-sample scenarios.
Support vector machines (SVMs) rely on the inherent geometry of a data set to classify training data. Because of this, we believe SVMs are an excellent candidate to guide the development of an analytic feature selection algorithm, as opposed to the more commonly used heuristic methods. We propose a filter-based feature…
Support Vector Machines (SVMs) can solve structured multi-output learning problems such as multi-label classification, multiclass classification and vector regression. SVM training is expensive especially for large and high dimensional datasets. The bottleneck of the SVM training often lies in the kernel value computat…
In this article, a large dimensional performance analysis of kernel least squares support vector machines (LS-SVMs) is provided under the assumption of a two-class Gaussian mixture model for the input data. Building upon recent advances in random matrix theory, we show, when the dimension of data and their number $…
New research shows the maximum ℓ1-margin classifier doesn't adapt to sparse ground truths.
problem Understanding the limitations of the maximum ℓ1-margin classifier in high-dimensional settings.
method Analyzing convergence and prediction error rates of the maximum ℓ1-margin classifier.
result Proves tight upper and lower bounds for prediction error, showing benign overfitting.
Out-of-Distribution generalization of quantile regression with heavy tailed inputs: an SVM approachstat.ML
Novel SVM approach for extreme quantile regression with heavy tailed inputs.
problem Learning from extreme values in quantile regression.
method Support Vector Machine framework for handling high-dimensional and nonlinear settings.
result Established finite-sample learning guarantees under mild regularity assumptions.
One-class support vector machine (OC-SVM) for a long time has been one of the most effective anomaly detection methods and extensively adopted in both research as well as industrial applications. The biggest issue for OC-SVM is yet the capability to operate with large and high-dimensional datasets due to optimization c…
The linear Support Vector Machine (SVM) is a classic classification technique in machine learning. Motivated by applications in modern high dimensional statistics, we consider penalized SVM problems involving the minimization of a hinge-loss function with a convex sparsity-inducing regularizer such as: the L1-norm on t…
A nonlinear channel estimator using complex Least Square Support Vector Machines (LS-SVM) is proposed for pilot-aided OFDM system and applied to Long Term Evolution (LTE) downlink under high mobility conditions. The estimation algorithm makes use of the reference signals to estimate the total frequency response of the …
Event-based models (EBM) are a class of disease progression models that can be used to estimate temporal ordering of neuropathological changes from cross-sectional data. Current EBMs only handle scalar biomarkers, such as regional volumes, as inputs. However, regional aggregates are a crude summary of the underlying hi…
Proposed SMO algorithm for OC-SVM+ significantly outperforms non-sequential algorithms.
problem One-class SVM with privileged information
method Sequential Minimal Optimization (SMO) algorithm
result Finite-time convergence established
Support Vector Machines, SVMs, and the Large Margin Nearest Neighbor algorithm, LMNN, are two very popular learning algorithms with quite different learning biases. In this paper we bring them into a unified view and show that they have a much stronger relation than what is commonly thought. We analyze SVMs from a metr…
In many applications, input data are sampled functions taking their values in infinite dimensional spaces rather than standard vectors. This fact has complex consequences on data analysis algorithms that motivate modifications of them. In fact most of the traditional data analysis tools for regression, classification a…
Unified Pin-SVM improves accuracy over existing Pin-SVM model.
problem Difficulty in Pin-SVM model for .
method Unified Pin-SVM model that solves a QPP for .
result Significant improvement in accuracy over existing Pin-SVM model.
Localized SVMs maintain SVM's consistency properties for large datasets.
problem Inefficient computational requirements of global SVMs for large data sets.
method Localized SVMs apply different hyperparameters to different regions of the input space.
result Localized SVMs inherit - and risk consistency from global SVMs.
Efficient learning of minimax risk classifiers in high dimensions.
problem Efficient learning of classifiers in high-dimensional data.
method Iterative algorithm leveraging constraint generation methods for minimax risk classifiers.
result The algorithm provides efficient learning and feature selection in high-dimensional scenarios.
Robust support vector model based on bounded asymmetric elastic net loss for binary classificationstat.ML
BAEN-SVM improves SVM robustness to noisy data.
problem Noise and geometric irrationalities in SVM.
method Bounded asymmetric elastic net loss combined with SVM.
result BAEN-SVM is robust to noise and geometrically well-defined.
Support vector machines (SVMs) are invaluable tools for many practical applications in artificial intelligence, e.g., classification and event recognition. However, popular SVM solvers are not sufficiently efficient for applications with a great deal of samples as well as a large number of features. In this paper, thus…
Introduces Soft-SVM for binary classification bridging logistic and SVM.
problem Data separability issues in binary classification.
method Soft-SVM regression using convex relaxation of hinge loss with softness and class-separation parameters.
result Soft-SVM performs well in classification and prediction errors.
In this paper, we consider asymptotic properties of the support vector machine (SVM) in high-dimension, low-sample-size (HDLSS) settings. We show that the hard-margin linear SVM holds a consistency property in which misclassification rates tend to zero as the dimension goes to infinity under certain severe conditions. …
Voice activity detection (VAD), used as the front end of speech enhancement, speech and speaker recognition algorithms, determines the overall accuracy and efficiency of the algorithms. Therefore, a VAD with low complexity and high accuracy is highly desirable for speech processing applications. In this paper, we propo…
Prior knowledge can be used to improve predictive performance of learning algorithms or reduce the amount of data required for training. The same goal is pursued within the learning using privileged information paradigm which was recently introduced by Vapnik et al. and is aimed at utilizing additional information avai…
SMM preserves matrix data structure for SVM classification.
problem Preserving spatial correlations in matrix data for SVM.
method SMM uses spectral elastic net combining nuclear and Frobenius norms.
result SMM improves SVM performance on matrix data.
MKL--SVMstat.ML
Paper introduces MKL--SVM for SVM with loss.
problem Optimization of SVM with loss function.
method MKL framework combined with ADMM algorithm for solving the optimization problem.
result Performance of MKL--SVM comparable to SimpleMKL.
Weighted SVM (or fuzzy SVM) is the most widely used SVM variant owning its effectiveness to the use of instance weights. Proper selection of the instance weights can lead to increased generalization performance. In this work, we extend the span error bound theory to weighted SVM and we introduce effective hyperparamete…
Training of one-vs.-rest SVMs can be parallelized over the number of classes in a straight forward way. Given enough computational resources, one-vs.-rest SVMs can thus be trained on data involving a large number of classes. The same cannot be stated, however, for the so-called all-in-one SVMs, which require solving a …
We describe a novel binary classification technique called Banded SVM (B-SVM). In the standard C-SVM formulation of Cortes et al. (1995), the decision rule is encouraged to lie in the interval [1, \infty]. The new B-SVM objective function contains a penalty term that encourages the decision rule to lie in a user specif…