Tuning SVM and boosting models using optimization algorithms.
problem Tuning parameters for SVM and boosting models across various datasets.
method Used grid search to identify parameter ranges and optimization algorithms to select models.
result Optimization algorithms outperformed grid search in selecting well-performing models.
This study uses MILP for feature selection in SVM with budget constraint.
problem Feature selection in SVM with a budget constraint.
method Mixed Integer Linear Programming (MILP) formulation with a budget constraint.
result The proposed MILP formulation and procedures efficiently solve the feature selection problem.
Proposes cost-sensitive feature selection for SVMs.
problem Asymmetric misclassification costs in feature selection.
method Mathematical optimization-based approach for SVMs.
result Substantial reduction in feature count with desired error rates.
Paper proposes a new method to automatically select Gaussian kernel bandwidth for SVDD.
problem Selecting optimal Gaussian kernel bandwidth for SVDD is crucial but challenging.
method Automatic unsupervised method for selecting Gaussian kernel bandwidth.
result The selected bandwidth is competitive with existing methods and can be computed quickly.
Study evaluates SVM active learning for imbalanced datasets.
problem Imbalanced datasets in text classification and relation extraction.
method Combines SVM with closest-to-hyperplane and query-by-committee selection methods for imbalance.
result ClosestPA algorithm consistently outperforms other methods.
The support vector machine (SVM) is an important class of learning machines for function approach, pattern recognition, and time-serious prediction, etc. It maps samples into the feature space by so-called support vectors of selected samples, and then feature vectors are separated by maximum margin hyperplane. The pres…
Proposes an L1-regularized functional SVM for binary classification with functional covariates.
problem Binary classification with multivariate functional covariates.
method L1-regularized functional support vector machine (SVM) with an accompanying algorithm.
result The proposed classifier performs well in prediction and feature selection.
A mean field variational Bayes approach to support vector machines (SVMs) using the latent variable representation on Polson & Scott (2012) is presented. This representation allows circumvention of many of the shortcomings associated with classical SVMs including automatic penalty parameter selection, the ability to ha…
LIBTwinSVM offers a free library for efficient Twin Support Vector Machines.
problem Large-scale classification problems.
method Efficient implementation of Twin Support Vector Machines.
result Effectiveness demonstrated through benchmarks.
This work proposes a new algorithm for training a re-weighted L2 Support Vector Machine (SVM), inspired on the re-weighted Lasso algorithm of Candès et al. and on the equivalence between Lasso and SVM shown recently by Jaggi. In particular, the margin required for each training vector is set independently, defining a n…
This work improves classification performance by selecting critical samples for Nyström methods.
problem Lack of supervision in Nyström methods leads to poor classification performance.
method Selects Nyström support vectors via negative margin criterion to create better feature maps.
result Significantly improves classification performance compared to unsupervised methods.
Paper optimizes sparse feature selection for cancer detection using GSVP and SVM.
problem Sparse feature selection for cancer detection.
method Regularized GSVP with proximal gradient descent, feature selection via SVM.
result Near-perfect balanced accuracy with few selected features.
This work proposes a model averaging method for SVM that avoids redundant covariates and achieves asymptotic optimality.
problem Redundant covariates impair SVM performance in high-dimensional settings.
method Frequentist model averaging procedure for SVM using cross-validation to select optimal weights.
result The proposed method achieves asymptotic optimality in SVM model averaging.
New method selects kernel bandwidth for SVDD and OCSVM.
problem Selecting optimal Gaussian kernel bandwidth for SVDD and OCSVM.
method Exploits low-rank representation of kernel matrix to suggest bandwidth.
result Method performs well for both low-dimensional and high-dimensional data.
This paper proposes MMD-SVR to improve SVR's margin distribution for better generalization.
problem Improving SVR's generalization performance by maximizing the margin distribution of the whole dataset.
method Introducing MMD-SVR with coupled constraints to convert a non-convex optimization problem into a convex one.
result MMD-SVR significantly improves prediction accuracy and generalization compared to classic SVR.
Efficient ADMM algorithm solves nonconvex SVMs with various penalties.
problem Solving nonconvex penalized SVMs due to nondifferentiability, nonsmoothness, and nonconvexity.
method ADMM-based algorithm for a wide range of nonconvex penalties.
result The proposed algorithm outperforms other methods on benchmark datasets.
A new algorithm for faster model selection in twin multi-class SVM.
problem Challenges in effective solution of multi-classification and fast model selection in twin multi-class SVM.
method Sample data set partition strategy, Lagrangian multipliers, piecewise linear update, initialization algorithm, and event-based iteration.
result Comparable classification performance achieved without solving quadratic programming problems.
A novel feature selection method for SVM improves model accuracy and interpretability.
problem Feature selection in nonlinear SVM classification problems.
method Embedded min-max optimization problem, leveraging duality theory.
result Improves model accuracy and interpretability on benchmark data sets.
A fast method for selecting Gaussian kernel bandwidth in kernel-based classifiers.
problem High computational complexity in estimating Gaussian kernel bandwidth.
method Developed based on reproducing kernel Hilbert space operators.
result Proposed method outperforms state-of-the-art methods in computational time and performance.
Lasso-Zero improves support recovery in high-dimensional linear models.
problem Support recovery in high-dimensional linear models with sparse β 0 β^0 β 0 . method Lasso-Zero uses an 'overfit, then threshold' approach with noise dictionaries.
result Lasso-Zero outperforms competitors in support recovery and trade-off between true positives and false discoveries.
A method selects key genes from tumor transcriptomics data using kernel methods and improves classification performance.
problem Feature selection for tumor classification using gene expression data.
method Multiple Kernel Learning with latent regularization and non-linear dimensionality reduction.
result Improved tumor classification performance on unseen test samples.
Support vector machines (SVMs) rely on the inherent geometry of a data set to classify training data. Because of this, we believe SVMs are an excellent candidate to guide the development of an analytic feature selection algorithm, as opposed to the more commonly used heuristic methods. We propose a filter-based feature…
Automates optimization of support vector machine classification steps.
problem Optimal preprocessing and classification selection for complex data.
method Theoretical view on classical classifiers, preprocessing interpretation, integrated framework.
result Semiautomatic optimization of processing chain with numerous algorithms integration.
New method improves scalability of Gaussian processes for discrete data.
problem Scalability issue in Gaussian processes for discrete domains.
method Simulated annealing for selecting inducing points.
result Simulated annealing outperforms SVM and full GP on DNA sequence data.
RFSVM with random features achieves faster learning rates.
problem Improving the learning rate of SVM with random features.
method Support Vector Machine with N ≪ m N\ll m N ≪ m random features, optimized feature map, and reweighted feature selection. result RFSVM achieves faster learning rates than O ( 1 / m ) O(1/\sqrt{m}) O ( 1/ m ) under low noise assumptions. Study proposes a hybrid method for medium-term load forecasting.
problem Accurate medium-term load forecasting for power system operation and planning.
method Support Vector Regression (SVR) combined with Symbiotic Organism Search Optimization (SOSO) for parameter optimization and feature selection.
result The proposed method outperformed previous methods in the EUNITE competition dataset.
New tree-based SVM methods for multi-class classification.
problem Efficient multi-class classification for large datasets.
method Entropy and generalization error estimation for binary classifiers in tree nodes.
result Proposed methods outperform traditional techniques in speed and accuracy.
An incremental SVDD algorithm for online data using Gaussian kernel.
problem Efficiently handling online or large data for SVDD.
method Incremental learning algorithm using Gaussian kernel, focusing on existing support vectors and new data points.
result Significant gains in efficiency with almost no loss in outlier detection accuracy or objective function value.
New method ranks network features in real-time for adaptive IDS.
problem Static feature selection ignores dynamic network data.
method Incremental learning of SVM for real-time feature ranking.
result Adapts to changes in normal network behavior and new attack patterns.
Proposes novel wSVMs for sparse learning and accurate probability estimation.
problem Sparse features with redundant noise limit the performance of existing wSVMs.
method Develops ℓ 1 \ell^1 ℓ 1 -norm and elastic net regularized wSVMs for automatic variable selection and probability estimation. result Elastic net regularized wSVMs achieve superior performance in variable selection and probability estimation.
New SVM feature selection methods improve wafer testing accuracy.
problem Need for improved SVM feature selection methods.
method Entropy measurement and K-medoid clustering for feature selection.
result One-class SVM's enhance wafer testing accuracy.
Efficient learning of minimax risk classifiers in high dimensions.
problem Efficient learning of classifiers in high-dimensional data.
method Iterative algorithm leveraging constraint generation methods for minimax risk classifiers.
result The algorithm provides efficient learning and feature selection in high-dimensional scenarios.
Sparse PCA selects variables with FDR control for improved performance.
problem Sparse PCA selects irrelevant variables when maximizing explained variance.
method Proposes FDR-controlled selection using T-Rex selector.
result Significant performance improvement over traditional sparse PCA.
Support Vector Data Description (SVDD) is a popular outlier detection technique which constructs a flexible description of the input data. SVDD computation time is high for large training datasets which limits its use in big-data process-monitoring applications. We propose a new iterative sampling-based method for SVDD…
Random Machines improves SVM performance with free kernel choice.
problem Efficiency and accuracy in solving classification and regression problems.
method Bagged-weighted support vector model with free kernel choice.
result Improved accuracy and reduced computational time.
When applying the support vector machine (SVM) to high-dimensional classification problems, we often impose a sparse structure in the SVM to eliminate the influences of the irrelevant predictors. The lasso and other variable selection techniques have been successfully used in the SVM to perform automatic variable selec…
Enhances EEG signal classification using non-Gaussian neutral vectors.
problem Challenges in classifying EEG signals for brain-computer interfaces.
method Transformed mDWT coefficients into neutral vectors, applied feature selection.
result Feature selection improves classification accuracy.
We investigate the issue of model selection and the use of the nonconformity (strangeness) measure in batch learning. Using the nonconformity measure we propose a new training algorithm that helps avoid the need for Cross-Validation or Leave-One-Out model selection strategies. We provide a new generalisation error boun…
A new algorithm FS-PeSOA extracts features from large datasets.
problem Extracting useful knowledge from large datasets.
method Adaptive Penguin Search Optimization Algorithm (FS-PeSOA) based on penguin hunting strategy.
result FS-PeSOA finds global optimum solution for feature subset selection.
This work develops scalable model selection methods with fast update and selection.
problem Efficient model selection for large pools of candidate models.
method Isolated model embedding, which supports asymptotically fast update and selection.
result Standardized Embedder achieves competitive model selection performances.
EDSVM uses elite observations to guide SVM classification.
problem Classical SVMs lack ways to encode trusted models or preferences.
method EDSVM augments SVMs by guiding slack variables for elite observations.
result EDSVM models closely track reference SVMs while achieving competitive performance.
For a variety of regularized optimization problems in machine learning, algorithms computing the entire solution path have been developed recently. Most of these methods are quadratic programs that are parameterized by a single parameter, as for example the Support Vector Machine (SVM). Solution path algorithms do not …
A novel SVR parameter optimization method using GSA outperforms other meta-heuristics in stock market forecasting.
problem Optimizing SVR parameters for reliable regression performance on small sample sizes.
method Golden Sine Algorithm (GSA) for parameter tuning of SVR.
result The GSA-based SVR outperforms eleven other meta-heuristics in terms of accuracy and computing time.
This paper improves SVM prediction uncertainty quantification methods.
problem Lack of comprehensive UQ methods for SVM predictions.
method Developed SSVQR model for sparse PI estimation and feature selection algorithm.
result Proposed SSVQR model achieves sparse solutions and improves PI quality.
Kernelized Support Vector Machines (SVMs) are among the best performing supervised learning methods. But for optimal predictive performance, time-consuming parameter tuning is crucial, which impedes application. To tackle this problem, the classic model selection procedure based on grid-search and cross-validation was …
Adaptive kNN method closes the gap for unbounded support distributions.
problem Inability of standard kNN to achieve minimax optimal rate for unbounded support distributions.
method Adaptive kNN method with varying k for different samples.
result The adaptive kNN method matches the minimax lower bound.
Improved greedy 2-coordinate updates for optimization problems with constraints.
problem Minimizing smooth functions subject to constraints.
method Exploiting a connection to steepest descent in the 1-norm, we give faster convergence rates and efficient computation.
result Greedy selection converges faster than random selection and can be computed in O ( n log n ) O(n \log n) O ( n log n ) time. Incorporating feature selection into a classification or regression method often carries a number of advantages. In this paper we formalize feature selection specifically from a discriminative perspective of improving classification/regression accuracy. The feature selection method is developed as an extension to the r…