The least-squares support vector machine is a frequently used kernel method for non-linear regression and classification tasks. Here we discuss several approximation algorithms for the least-squares support vector machine classifier. The proposed methods are based on randomized block kernel matrices, and we show that t…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “least-squares support vector machines”
Support vector machines (SVMs) are an important tool in modern data analysis. Traditionally, support vector machines have been fitted via quadratic programming, either using purpose-built or off-the-shelf algorithms. We present an alternative approach to SVM fitting via the majorization--minimization (MM) paradigm. Alg…
Sparse LR-LSSVM improves kernel machine performance.
problem Improving kernel machine performance with controlled model size.
method Introduces LR-LSSVM with low rank kernels and a two-step optimization algorithm.
result Proposed algorithm's performance is comparable or superior to existing kernel machines.
A quantum-inspired classical algorithm speeds up LS-SVM classification.
problem Big data challenge in SVM classification.
method Improved indirect sampling technique for LS-SVM.
result Algorithm achieves logarithmic runtime for low rank data matrices.
SVM and linear regression models coincide in high dimensions.
problem Understanding the connection between SVM and linear regression in high-dimensional data.
method Analyzing feature models and proving lower bounds on dimensionality.
result A sharp phase transition in Gaussian feature models, with support vector proliferation occurring only in very high dimensions.
The paper analyzes bootstrap ensemble classifiers in high-dimensional settings.
problem Performance of bootstrap ensemble classifiers in high-dimensional data.
method Random Matrix Theory applied to LSSVM ensemble.
result Strategies to optimize performance of LSSVM ensemble.
In this article, a large dimensional performance analysis of kernel least squares support vector machines (LS-SVMs) is provided under the assumption of a two-class Gaussian mixture model for the input data. Building upon recent advances in random matrix theory, we show, when the dimension of data and their number $…
This paper compares TFC and LS-SVM for solving DEs, showing TFC is faster and more accurate.
problem Solving differential equations efficiently and accurately.
method Theory of Functional Connections (TFC) integrated with LS-SVM.
result TFC is faster and more accurate than LS-SVM for solving DEs.
Quantum LS-SVM simplifies matrix inversion for faster machine learning.
problem Speeding up machine learning algorithms for large datasets.
method Introduces a novel quantum algorithm using continuous variables to simplify matrix inversion in LS-SVM, and proposes a hybrid quantum-classical approach for sparse solutions.
result Quantum LS-SVM achieves exponential speed-up and can solve classically difficult tasks.
RFSVM with random features achieves faster learning rates.
problem Improving the learning rate of SVM with random features.
method Support Vector Machine with random features, optimized feature map, and reweighted feature selection.
result RFSVM achieves faster learning rates than under low noise assumptions.
Classification and regression tasks in overparameterized models show different generalization properties.
problem Comparing classification and regression in overparameterized models.
method Comparison of least-squares minimum-norm interpolation and hard-margin SVM using different loss functions.
result Interpolating solutions generalize well with 0-1 loss but not with square loss.
Improved robustness in kernel-based regression via novel loss function and IRLS.
problem Noise sensitivity in kernel-based regression methods.
method Proposed -loss function and iteratively reweighted least squares (IRLS) optimization.
result Improved noise robustness in kernel-based regression methods.
Paper develops ML-based PLA verifiers that operate like the likelihood test.
problem Designing secure PLA verifiers when no attack information is available.
method Developed neural network and OCLSSVM models trained as two-class classifiers on legitimate data.
result One-class models can operate as the likelihood test at convergence.
New memory models improve LSSVM's generalization and reduce time cost.
problem Improving LSSVM's generalization performance and reducing training time.
method Introducing MIMM and WIMM models that modify LSSVM's memory influence mechanism.
result MIMM and WIMM models outperform LSSVM in generalization and time efficiency.
Due to the extremely volatile nature of financial markets, it is commonly accepted that stock price prediction is a task full of challenge. However in order to make profits or understand the essence of equity market, numerous market participants or researchers try to forecast stock price using various statistical, econ…
Conditional expectiles are becoming an increasingly important tool in finance as well as in other areas of applications. We analyse a support vector machine type approach for estimating conditional expectiles and establish learning rates that are minimax optimal modulo a logarithmic factor if Gaussian RBF kernels are u…
Eigen-decomposition simplifies quadratic programming with equality constraints.
problem Optimizing solutions under linear equality constraints in quadratic programming.
method Eigenvalue decomposition of the quadratic term matrix to project optimal solutions.
result Established a linear mapping between EQP formulations with and without diagonalized .
We prove rates of convergence in the statistical sense for kernel-based least squares regression using a conjugate gradient algorithm, where regularization against overfitting is obtained by early stopping. This method is directly related to Kernel Partial Least Squares, a regression method that combines supervised dim…
This article proposes a performance analysis of kernel least squares support vector machines (LS-SVMs) based on a random matrix approach, in the regime where both the dimension of data and their number grow large at the same rate. Under a two-class Gaussian mixture model for the input data, we prove that the LS…
Application of fuzzy support vector machine in stock price forecast. Support vector machine is a new type of machine learning method proposed in 1990s. It can deal with classification and regression problems very successfully. Due to the excellent learning performance of support vector machine, the technology has becom…
We generalize a support vector machine to a support spinor machine by using the mathematical structure of wedge product over vector machine in order to extend field from vector field to spinor field. The separated hyperplane is extended to Kolmogorov space in time series data which allow us to extend a structure of sup…
We propose a novel algorithm for greedy forward feature selection for regularized least-squares (RLS) regression and classification, also known as the least-squares support vector machine or ridge regression. The algorithm, which we call greedy RLS, starts from the empty feature set, and on each iteration adds the feat…
Support vector machines have attracted much attention in theoretical and in applied statistics. Main topics of recent interest are consistency, learning rates and robustness. In this article, it is shown that support vector machines are qualitatively robust. Since support vector machines can be represented by a functio…
Expectile regression is a nice tool for investigating conditional distributions beyond the conditional mean. It is well-known that expectiles can be described with the help of the asymmetric least square loss function, and this link makes it possible to estimate expectiles in a non-parametric framework by a support vec…
Regularized kernel methods such as, e.g., support vector machines and least-squares support vector regression constitute an important class of standard learning algorithms in machine learning. Theoretical investigations concerning asymptotic properties have manly focused on rates of convergence during the last years bu…
Study on using random subspaces for ERM with various loss functions.
problem Improving learning accuracy with computational savings from random subspaces.
method Random subspaces of a hypothesis space, considering data-dependent subspaces.
result Unified analysis showing computational efficiency can be improved without performance loss.
Paper develops distributed inference for SVM binary classification.
problem Challenges in modern data size for existing statistical inference.
method Proposes MDL estimator for linear SVM, computationally efficient.
result MDL estimator achieves optimal statistical efficiency.
A new method for high-dimensional classification using Bernstein polynomials.
problem Computational difficulties in high-dimensional SVM hinge loss.
method Proposes Bernstein support vector machine (BernSVM) and two efficient algorithms.
result Achieves a prediction accuracy rate of with high probability.
We propose an algorithm for exploring the entire regularization path of asymmetric-cost linear support vector machines. Empirical evidence suggests the predictive power of support vector machines depends on the regularization parameters of the training algorithms. The algorithms exploring the entire regularization path…
We consider the question of learning in general topological vector spaces. By exploiting known (or parametrized) covariance structures, our Main Theorem demonstrates that any continuous linear map corresponds to a certain isomorphism of embedded Hilbert spaces. By inverting this isomorphism and extending continuously, …
Support Vector Machine (SVM) is an efficient classification approach, which finds a hyperplane to separate data from different classes. This hyperplane is determined by support vectors. In existing SVM formulations, the objective function uses L2 norm or L1 norm on slack variables. The number of support vectors is a me…
Statistical downscaling of global climate models (GCMs) allows researchers to study local climate change effects decades into the future. A wide range of statistical models have been applied to downscaling GCMs but recent advances in machine learning have not been explored. In this paper, we compare four fundamental st…
SVM generalizes well even with many support vectors in high dimensions.
problem Generalization of SVM in high-dimensional spaces with many support vectors.
method Identified new deterministic equivalences and proved conditions for support vector proliferation.
result Broadened conditions for SVM generalization in high-dimensional settings and proved converse result.
Derives a primal-dual MLSVD formulation for multilinear data.
problem Efficiently decompose multilinear data for signal analysis and deep learning.
method Kernelizable primal-dual formulation of MLSVD.
result Derives a new MLSVD formulation with computational advantages.
Distributed estimation of principal support vector machines for sufficient dimension reductionstat.ML
New algorithms reduce computational burden for principal support vector machines.
problem High computational cost of principal support vector machines for large datasets.
method Two distributed estimation algorithms for principal support vector machines.
result Statistical efficiency is maintained with distributed algorithms.
The paper proposes an algorithm to enumerate K best models with distinct support vectors for SVM.
problem Finding multiple models with distinct support vectors for non-standard machine learning applications.
method A K-best model enumeration algorithm for SVM that efficiently finds models with distinct support vectors in the dual SVM problem.
result The algorithm efficiently finds the next best model with small latency, useful for interactive examination of requirements.
Tuning-Free Structured Sparse Recovery of Multiple Measurement Vectors using Implicit Regularizationcs.LG
A tuning-free method recovers jointly sparse signals in MMV using implicit regularization.
problem Recovering jointly sparse signals in MMV with minimal tuning or prior knowledge.
method Reparameterizes MMV estimation matrix into decoupled factors and applies gradient descent to a least-squares objective.
result Gradient descent dynamics exhibit a momentum-like effect, converging towards an idealized row-sparse solution.
LIBTwinSVM offers a free library for efficient Twin Support Vector Machines.
problem Large-scale classification problems.
method Efficient implementation of Twin Support Vector Machines.
result Effectiveness demonstrated through benchmarks.
Paper proposes a method to improve interpretability in kernel learning models.
problem Improving interpretability in flexible machine learning models.
method Proposes a quantitative index for interpretability and a universal learning framework to balance interpretability and generalization performance.
result Demonstrates a method to achieve global optimal solution in balancing interpretability and generalization performance.
The support vector clustering algorithm is a well-known clustering algorithm based on support vector machines using Gaussian or polynomial kernels. The classical support vector clustering algorithm works well in general, but its performance degrades when applied on big data. In this paper, we have investigated the perf…
Paper connects GLM and LRM for better classification performance.
problem Improving classification performance using statistical inference.
method Derives a statistical test based on SVM and permutation analysis.
result MLE-based inference provides better parameter estimation.
Distributed SVM algorithm improves performance in real-world applications.
problem Improving SVM performance in distributed computing environments.
method Proposes HPSVM, a distributed SVM algorithm that minimizes inter-machine communications.
result HPSVM achieves similar or better results than state-of-the-art SVM techniques.
Quantum algorithm solves SOCP and SVM problems faster than classical methods.
problem Quantum algorithms for solving SOCP and SVM problems.
method Quantum interior-point method (IPM) for SOCP, scaling as O(n^k).
result Quantum algorithm exhibits polynomial speedup over classical methods.
Solving Support Vector Machines in Reproducing Kernel Banach Spaces with Positive Definite Functionsstat.ML
In this paper we solve support vector machines in reproducing kernel Banach spaces with reproducing kernels defined on nonsymmetric domains instead of the traditional methods in reproducing kernel Hilbert spaces. Using the orthogonality of semi-inner-products, we can obtain the explicit representations of the dual (nor…
Rgtsvm provides a fast and flexible support vector machine (SVM) implementation for the R language. The distinguishing feature of Rgtsvm is that support vector classification and support vector regression tasks are implemented on a graphical processing unit (GPU), allowing the libraries to scale to millions of examples…
Parallel computing has played an important role in speeding up convex optimization methods for big data analytics and large-scale machine learning (ML). However, the scalability of these optimization methods is inhibited by the cost of communicating and synchronizing processors in a parallel setting. Iterative ML metho…
General Vector Machinestat.ML
The support vector machine (SVM) is an important class of learning machines for function approach, pattern recognition, and time-serious prediction, etc. It maps samples into the feature space by so-called support vectors of selected samples, and then feature vectors are separated by maximum margin hyperplane. The pres…
A theorem for Hilbert space-valued functions simplifies machine learning models.
problem Learning functions in Hilbert space-valued spaces.
method Generalized Representer Theorem for Hilbert space-valued functions.
result Unified view of supervised and semi-supervised learning methods.