We prove the statistical consistency of kernel Partial Least Squares Regression applied to a bounded regression learning problem on a reproducing kernel Hilbert space. Partial Least Squares stands out of well-known classical approaches as e.g. Ridge Regression or Principal Components Regression, as it is not defined as…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “partial least squares regression”
The derivation of statistical properties for Partial Least Squares regression can be a challenging task. The reason is that the construction of latent components from the predictor variables also depends on the response variable. While this typically leads to good performance and interpretable models in practice, it ma…
This paper reviews SDR methods for multivariate response regression.
problem Handling sufficient dimension reduction for multivariate response regression.
method Characterizes SDR estimators as inverse or forward regression methods.
result Pooled marginal, projective resampling, distance-based, ordinary least squares, partial least squares, and semiparametric SDR estimators are discussed.
A new PMA method combines PCA and PLS for better classification.
problem Improving classification performance in data analysis.
method Combines PCA and PLS through multiple sub-PLS models and PCA on the joint coefficient matrix.
result The proposed PMA method achieves better classification performance and stability.
This paper presents regression models obtained from a process of blind prediction of peptide binding affinity from provided descriptors for several distinct datasets as part of the 2006 Comparative Evaluation of Prediction Algorithms (COEPRA) contest. This paper finds that kernel partial least squares, a nonlinear part…
Dual-sPLS improves feature selection and prediction in high-dimensional data.
problem Relating variables to a response in high-dimensional chemometric problems.
method Generalizes PLS1 algorithm with dual norm penalizations and a shrinking ratio parameter.
result Favorably compares to similar regression methods on simulated and real chemical data.
This study examines the relationship between PLS and OLS regression using eigenvalue distributions.
problem Analyzing the difference between PLS and OLS regression in terms of eigenvalue distributions.
method Examined the distance between PLS and OLS regression coefficients using the Mahalanobis distance and eigenvalue distributions of the regressor covariance matrix.
result Provided a bound on the distance between PLS and OLS regression coefficients that depends only on the eigenvalue distribution of the regressor covariance matrix.
We prove rates of convergence in the statistical sense for kernel-based least squares regression using a conjugate gradient algorithm, where regularization against overfitting is obtained by early stopping. This method is directly related to Kernel Partial Least Squares, a regression method that combines supervised dim…
PLS-Lasso integrates dimension reduction into regression for financial index tracking.
problem Dimension reduction and regression are traditionally treated separately in multivariate data analysis.
method PLS-Lasso integrates dimension reduction directly into the regression process, presenting two formulations: PLS-Lasso-v1 and PLS-Lasso-v2.
result PLS-Lasso-v1 and PLS-Lasso-v2 outperform Lasso in financial index tracking.
Bayesian optimization selects wavelengths for sugar content estimation in NIR spectroscopy.
problem Improving prediction accuracy and interpretability of spectral data for sugar content estimation.
method Formulated as a binary black-box optimization problem, proposed method uses Bayesian optimization with a sparse quadratic surrogate model and Thompson sampling.
result Improves prediction accuracy of partial least squares regression and yields more consistent wavelength regions.
We prove statistical rates of convergence for kernel-based least squares regression from i.i.d. data using a conjugate gradient algorithm, where regularization against overfitting is obtained by early stopping. This method is related to Kernel Partial Least Squares, a regression method that combines supervised dimensio…
Functional PLS improves prediction and inference for scalar responses from functional predictors.
problem Estimating scalar responses from functional predictors in an ill-posed inverse problem.
method Functional partial least squares (PLS) estimator with adaptive early stopping and new tests.
result PLS attains nearly minimax-optimal convergence rates and detects local alternatives.
Unified multi-view learning framework using OPLS with regularization and deep extensions.
problem Improving multi-view learning for classification and feature extraction.
method Orthonormalized Partial Least Squares (OPLS) with regularization and deep extensions.
result Unified multi-view learning framework with improved performance.
This paper reviews and compares supervised linear dimension-reduction techniques.
problem Lack of information in the response during unsupervised PCA reduces predictive performance.
method Review and comparison of supervised linear dimension-reduction techniques.
result PLS and LSPCA consistently outperform other techniques in simulations.
We compare the risk of ridge regression to a simple variant of ordinary least squares, in which one simply projects the data onto a finite dimensional subspace (as specified by a Principal Component Analysis) and then performs an ordinary (un-regularized) least squares regression in this subspace. This note shows that …
Improved Least-Squares Monte Carlo with finite-difference ansatz.
problem Improving accuracy and stability in option pricing.
method Constructing an ansatz using finite-difference solution for conditional expected continuation payoffs.
result Reduces mean squared error and final pricing error.
Reduced-rank method improves least-squares regression under output regularity.
problem Least-squares regression with infinite dimensional outputs.
method Reduced-rank method for solving least-squares problems with output regularity assumptions.
result Learning bounds and improved statistical performance compared to full-rank method.
The runtime for Kernel Partial Least Squares (KPLS) to compute the fit is quadratic in the number of examples. However, the necessity of obtaining sensitivity measures as degrees of freedom for model selection or confidence intervals for more detailed analysis requires cubic runtime, and thus constitutes a computationa…
Randomized matrix compression techniques, such as the Johnson-Lindenstrauss transform, have emerged as an effective and practical way for solving large-scale problems efficiently. With a focus on computational efficiency, however, forsaking solutions quality and accuracy becomes the trade-off. In this paper, we investi…
Matrix factorization is a popular approach to solving matrix estimation problems based on partial observations. Existing matrix factorization is based on least squares and aims to yield a low-rank matrix to interpret the conditional sample means given the observations. However, in many real applications with skewed and…
GMLS-Nets extend CNNs to unstructured data points.
problem Learning from irregularly spaced data points in science and engineering.
method Introducing GMLS for non-parametric estimation and parameterizing it for learning operators with unstructured stencils.
result GMLS-Nets provide a framework for functional regression and quantity prediction from unstructured data.
Correcting bias in least squares regression with volume-rescaled sampling.
problem Bias in linear least squares solutions without distributional assumptions.
method Volume-rescaled sampling to correct bias in i.i.d. samples.
result Combined sample becomes unbiased with rescaled volume additional sample.
Paper uses deep learning to solve PDEs without supervision.
problem Solving elliptic PDEs without labeled data.
method Uses deep neural networks and least-squares functionals.
result Demonstrates effectiveness on 1D second-order elliptic PDEs.
We find a convex model for traditional nonlinear regression under L2 loss.
problem Nonlinear regression under L2 loss with non-convex optimization.
method Showed a convex nonlinear regression model for least squares problem.
result Existence of a convex model simplifies training complex systems.
P3LS preserves privacy while integrating data across companies.
problem Privacy concerns in cross-organizational data exchange and integration.
method Privacy-preserving federated learning technique using SVD-based PLS and random masks.
result Improves prediction performance on process-related indicators.
From STLS to Projection-based Dictionary Selection in Sparse Regression for System Identificationstat.ML
This work improves SINDy-type algorithms for system identification using score-guided dictionary selection.
problem Improving accuracy and interpretability in dynamical system identification.
method Score-guided library selection to refine dictionary terms in sparse regression.
result Score-guided methods enhance SINDy's robustness in discovering governing equations.
Five simple soft sensor methodologies with two update conditions were compared on two experimentally-obtained datasets and one simulated dataset. The soft sensors investigated were moving window partial least squares regression (and a recursive variant), moving window random forest regression, the mean moving window of…
Study shows time-varying stock returns across economic states.
problem Equity premium predictability varies by economic state.
method State-switching predictive regression using yield curve slope.
result The Aligned Economic Index improves stock return prediction.
Improved robustness in kernel-based regression via novel loss function and IRLS.
problem Noise sensitivity in kernel-based regression methods.
method Proposed -loss function and iteratively reweighted least squares (IRLS) optimization.
result Improved noise robustness in kernel-based regression methods.
We consider the problem of nonparametric regression under shape constraints. The main examples include isotonic regression (with respect to any partial order), unimodal/convex regression, additive shape-restricted regression, and constrained single index model. We review some of the theoretical properties of the least …
Proposes a method for coarse graph alignment using sparse partial least squares.
problem Aligning graphs with community structures when there's no natural one-to-one mapping.
method Sparse partial least squares method incorporating observed graph structures and imposing sparsity.
result Demonstrates effectiveness in simulations.
Study improves least squares estimation for heavy-tailed errors.
problem Improving least squares estimation under heteroscedastic and heavy-tailed errors.
method Analyzes the rate of convergence of least squares estimator under bounded conditional variance and finitely many moments of errors.
result Upper bounds on rates of convergence of LSE for heavy-tailed errors are found.
New deep learning solver for high-dimensional derivative pricing.
problem High-dimensional derivatives pricing problems.
method Combines deep learning with least square regression for backward SDE solving.
result Accurate and efficient pricing of complex derivatives.
Improved robust regression for heavy-tailed and contaminated data.
problem Linear regression with heavy-tailed and adversarially contaminated covariates and responses.
method Applying a filtering algorithm to covariates and then using Huber regression, least trimmed squares, or least absolute deviation estimators on the remaining data.
result Near-optimal error rates achieved for the Huber regression estimator.
In this paper we propose a computationally efficient algorithm for on-line variable selection in multivariate regression problems involving high dimensional data streams. The algorithm recursively extracts all the latent factors of a partial least squares solution and selects the most important variables for each facto…
Gradient flow in least squares regression is at least 1.69 times riskier than ridge regression.
problem Comparing the risk of gradient descent iterates to ridge regression in least squares regression.
method Continuous-time view of gradient descent, proving risk bounds.
result Gradient flow's risk is at least 1.69 times that of ridge regression.
New algorithm estimates partially-observed linear systems with better rates than previous methods.
problem Estimating parameters of partially-observed linear systems with long-term dependencies and semi-parametric noise.
method Prefiltered least squares estimator with semi-parametric noise model.
result First algorithm provably estimates parameters of partially-observed linear systems with rates not dependent on dependency decay rate.
R2T hybrid model improves robust regression for asymmetric noise.
problem Least-squares regression fails with asymmetric structured noise.
method Transformer encoder, compression NN, fixed symbolic equation.
result Median regression MSE of 6e-6 to 3.5e-5 on synthetic data.
Sparse linear regression, which entails finding a sparse solution to an underdetermined system of linear equations, can formally be expressed as an -constrained least-squares problem. The Orthogonal Least-Squares (OLS) algorithm sequentially selects the features (i.e., columns of the coefficient matrix) to greedil…
edPLS adds Gaussian noise to PLS regression to protect data privacy.
problem Protecting sensitive data in PLS regression models.
method Integrates Gaussian noise into PLS algorithm based on global sensitivity.
result Effective at preserving privacy while maintaining competitive prediction accuracy.
Principal Component Analysis (PCA) is a very successful dimensionality reduction technique, widely used in predictive modeling. A key factor in its widespread use in this domain is the fact that the projection of a dataset onto its first principal components minimizes the sum of squared errors between the original …
The paper examines prediction and estimation risks of ridgeless least squares under general error assumptions.
problem Prediction and estimation risks of ridgeless least squares under realistic error structures.
method Analysis of prediction and estimation risks under general regression error assumptions, including clustered or serial dependence.
result The benefits of overparameterization extend to time series, panel, and grouped data.
ESNs trained with Tikhonov least squares approximate ergodic dynamical systems in L2(μ) norm.
problem Approximating ergodic dynamical systems using ESNs.
method Tikhonov least squares regression on ESNs trained on observations from an ergodic dynamical system.
result ESNs trained with Tikhonov least squares approximate the target function in the L2(μ) norm.
Local Control Regression: Improving the Least Squares Monte Carlo Method for Portfolio Optimizationq-fin.PM
The least squares Monte Carlo algorithm has become popular for solving portfolio optimization problems. A simple approach is to approximate the value functions on a discrete grid of portfolio weights, then use control regression to generalize the discrete estimates. However, the classical global control regression can …
Lecture notes on advanced linear regression methods.
problem Understanding the properties of linear regression estimators in high dimensions.
method Proposition-proof exploration of least squares, ridgeless, ridge, and lasso estimators.
result Detailed analysis of the existence, uniqueness, relations, computation, and non-asymptotic properties of these estimators.
Deep learning solves complex volatility equations.
problem Solving path-dependent PDEs in rough volatility.
method Interpreting PDE as BSDE, using neural network reservoir approach.
result Proved theoretical convergence for least-square regression.
Algorithm solves robust linear regression with block Lewis weights.
problem Group distributionally robust least squares problem.
method Algorithm based on geometric construction and block Lewis weights, using accelerated proximal methods.
result Improves over known methods for moderate accuracy regimes and matches state-of-the-art guarantees.
Efficiently estimates private least squares with linear error growth.
problem Private estimation of ordinary least squares with bounded residuals and leverage.
method Scaled noise added to a stable nonprivate estimator of the regression vector.
result Near-optimal accuracy guarantee with linear error growth in dimension.