CD converges linearly for MCP/SCAD penalized least squares.
problem Recovering sparse signals from data.
method Coordinate descent for MCP/SCAD penalized least squares.
result CD converges linearly to solutions of MCP/SCAD penalized least squares.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “least squares problem”
Iterative Hessian sketch: Fast and accurate solution approximation for constrained least-squaresmath.OC
We study randomized sketching methods for approximately solving least-squares problem with a general convex constraint. The quality of a least-squares approximation can be assessed in different ways: either in terms of the value of the quadratic objective function (cost approximation), or in terms of some distance meas…
A new algorithm solves nonnegative least squares faster with nonnegative data.
problem Nonnegative least squares problems with nonnegative data.
method Primal-dual perspective accelerated algorithm with adaptive restart.
result Oracle complexity independent of matrix constants, solvable to multiplicative error.
Reduced-rank method improves least-squares regression under output regularity.
problem Least-squares regression with infinite dimensional outputs.
method Reduced-rank method for solving least-squares problems with output regularity assumptions.
result Learning bounds and improved statistical performance compared to full-rank method.
Proposes a partitioned least squares model for feature grouping.
problem Modeling with feature groups to assess variable importance.
method Two methods: alternating least squares and exact reformulation.
result Exact method provides better results in less time.
New method speeds up solving L0-regularized least-squares problems.
problem Solving L0-regularized least-squares problems efficiently.
method Safe peeling for Branch-and-Bound algorithm.
result Significant gains in solving time and node exploration.
Consider Least Squares Monte Carlo (LSM) algorithm, which is proposed by Longstaff and Schwartz (2001) for pricing American style securities. This algorithm is based on the projection of the value of continuation onto a certain set of basis functions via the least squares problem. We analyze the stability of the algori…
A fast sketching algorithm solves regularized least squares problems efficiently.
problem Solving large-scale optimization problems with convex or nonconvex regularization.
method Sketching for Regularized Optimization (SRO) algorithm that generates a sketch of the original data matrix and solves the sketched problem.
result General theoretical results for the approximation error between the original and sketched problems, including minimax rates for sparse signal estimation.
Randomized matrix compression techniques, such as the Johnson-Lindenstrauss transform, have emerged as an effective and practical way for solving large-scale problems efficiently. With a focus on computational efficiency, however, forsaking solutions quality and accuracy becomes the trade-off. In this paper, we investi…
We propose a new forward-backward stochastic differential equation solver for high-dimensional derivatives pricing problems by combining deep learning solver with least square regression technique widely used in the least square Monte Carlo method for the valuation of American options. Our numerical experiments demonst…
Effective Dimension Adaptive Sketching Methods for Faster Regularized Least-Squares Optimizationcs.LG
New algorithm reduces sketching dimension to effective problem size.
problem Solving L2-regularized least-squares problems efficiently.
method Randomized algorithm using Gaussian and SRHT embeddings.
result Preserves convergence guarantees with reduced embedding dimension.
We find a convex model for traditional nonlinear regression under L2 loss.
problem Nonlinear regression under L2 loss with non-convex optimization.
method Showed a convex nonlinear regression model for least squares problem.
result Existence of a convex model simplifies training complex systems.
The least-squares support vector machine is a frequently used kernel method for non-linear regression and classification tasks. Here we discuss several approximation algorithms for the least-squares support vector machine classifier. The proposed methods are based on randomized block kernel matrices, and we show that t…
Sparse linear regression, which entails finding a sparse solution to an underdetermined system of linear equations, can formally be expressed as an -constrained least-squares problem. The Orthogonal Least-Squares (OLS) algorithm sequentially selects the features (i.e., columns of the coefficient matrix) to greedil…
We introduce a novel semi-supervised version of the least squares classifier. This implicitly constrained least squares (ICLS) classifier minimizes the squared loss on the labeled data among the set of parameters implied by all possible labelings of the unlabeled data. Unlike other discriminative semi-supervised method…
Ordinary least squares (OLS) is the default method for fitting linear models, but is not applicable for problems with dimensionality larger than the sample size. For these problems, we advocate the use of a generalized version of OLS motivated by ridge regression, and propose two novel three-step algorithms involving l…
Paper proposes a pre-conditioning technique to speed up gradient-descent convergence in distributed linear least-squares problems.
problem Expediting convergence of gradient-descent method for ill-conditioned distributed linear least-squares problems.
method Iterative pre-conditioning technique to improve convergence rate of gradient-descent method.
result Pre-conditioned gradient-descent achieves superlinear convergence for unique solutions and improved linear convergence otherwise.
Optimal multiscale learning of linear operators
problem Statistical and computational limits of learning bounded linear operators between Sobolev spaces
method Reformulate as an infinite-dimensional matrix regression problem with heterogeneous multiscale structure
result Establish minimax rates and construct a finite-resolution blockwise least-squares estimator attaining these rates
Study of regularized least squares in RKKS with indefinite kernels.
problem Asymptotic properties of regularized least squares with indefinite kernels in RKKS.
method Introducing a bounded hyper-sphere constraint, theoretical demonstration of globally optimal solution, modified error decomposition techniques, matrix perturbation theory.
result Derivation of learning rates in RKKS, same as RKHS under certain conditions.
We introduce the implicitly constrained least squares (ICLS) classifier, a novel semi-supervised version of the least squares classifier. This classifier minimizes the squared loss on the labeled data among the set of parameters implied by all possible labelings of the unlabeled data. Unlike other discriminative semi-s…
A new screening rule 'dynamic Sasvi' improves sparse optimization speed.
problem Sparse optimization problem identification.
method Flexible framework based on Fenchel-Rockafellar duality for norm-regularized least squares.
result Dynamic Sasvi can eliminate more features and increase solver speed.
This work proves convergence of adaptive resampling for random Fourier features.
problem Sampling Fourier frequencies well for high-dimensional data.
method Data adaptive resampling of Fourier frequencies, asymptotically optimal.
result Proves convergence of adaptive resampling method for regression and classification problems.
We prove the statistical consistency of kernel Partial Least Squares Regression applied to a bounded regression learning problem on a reproducing kernel Hilbert space. Partial Least Squares stands out of well-known classical approaches as e.g. Ridge Regression or Principal Components Regression, as it is not defined as…
New algorithm reduces bias and variance in weighted least-squares solutions.
problem Inconsistent linear least-squares problems with rapidly decaying singular values.
method Regularized block Kaczmarz (ReBlocK) algorithm.
result ReBlocK outperforms RBK and minibatch SGD for inconsistent problems.
The paper tackles robust reinforcement learning with performance guarantees.
problem Finding a robust policy for RMDP with state space uncertainties.
method Proposes RLSPI algorithm for learning optimal robust policy with performance bounds.
result Demonstrates the performance of RLSPI on standard benchmark problems.
Least squares estimator fails to achieve optimal risk in bounded distributions, but non-linear predictors can.
problem Optimal risk in bounded distributions for constrained least squares.
method Comparison of least squares and non-linear predictors.
result Non-linear predictors can achieve optimal risk in bounded distributions.
Algorithm solves robust linear regression with block Lewis weights.
problem Group distributionally robust least squares problem.
method Algorithm based on geometric construction and block Lewis weights, using accelerated proximal methods.
result Improves over known methods for moderate accuracy regimes and matches state-of-the-art guarantees.
The paper explores how overfitting can lead to better predictions in high-dimensional data.
problem Understanding the behavior of linear models in high-dimensional settings with more predictors than observations.
method Analysis of ordinary least squares, penalized least squares, and spectral shrinkage estimates.
result The phenomenon of double descent, where model performance can improve with increasing model complexity.
Lower Bounds and a Near-Optimal Shrinkage Estimator for Least Squares using Random Projectionsmath.OC
Improved estimator for least squares using random projections achieves smaller error.
problem Improving the accuracy of least squares solutions for large-scale problems.
method James-Stein estimator applied to Gaussian sketching of least squares problems.
result Upper and lower bounds match when SNR is small and data matrix is well-conditioned.
Efficient method for high-dimensional American option pricing and hedging.
problem High-dimensional American option pricing and hedging.
method Gradient-enhanced sparse Hermite polynomial expansions combined with least squares Monte Carlo.
result Outperforms state-of-the-art methods in high dimensions with comparable computational cost.
Sequential Least-Squares Estimators with Fast Randomized Sketching for Linear Statistical Modelsstat.ML
A new method for estimating large-scale linear models with improved precision.
problem Estimating large-scale linear statistical models efficiently.
method Sequential Least-Squares Estimators with Fast Randomized Sketching (SLSE-FRS), integrating Sketch-and-Solve and Iterative-Sketching methods.
result SLSE-FRS produces high-precision estimators, outperforming state-of-the-art methods.
Matrix factorization is a popular approach to solving matrix estimation problems based on partial observations. Existing matrix factorization is based on least squares and aims to yield a low-rank matrix to interpret the conditional sample means given the observations. However, in many real applications with skewed and…
Illustrates interleaved learning with Kalman Filter for linear least squares.
problem Improving machine learning algorithms through interleaved learning.
method Simple statistical and optimization framework based on Kalman Filter.
result Demonstrates the effectiveness of interleaved learning.
Extends phase retrieval methods to handle sensing vector errors.
problem Phase retrieval with errors in sensing vectors.
method Total Least Squares (TLS) framework applied to gradient descent.
result Gradient descent can efficiently solve TLS phase retrieval.
Proposes a method for coarse graph alignment using sparse partial least squares.
problem Aligning graphs with community structures when there's no natural one-to-one mapping.
method Sparse partial least squares method incorporating observed graph structures and imposing sparsity.
result Demonstrates effectiveness in simulations.
This book introduces linear models and their theories rigorously.
problem Understanding linear models and their theories.
method Explains linear models from three perspectives, introduces maximum likelihood estimation, and proves least squares is the best unbiased linear model.
result Least squares is the best unbiased linear model in terms of mean squared error.
Cross validation residuals are well known for the ordinary least squares model. Here leave-M-out cross validation is extended to generalised least squares. The relationship between cross validation residuals and Cook's distance is demonstrated, in terms of an approximation to the difference in the generalised residual …
Study identifies and validates a method for system identification of Markov jump linear systems.
problem System identification for autonomous Markov jump linear systems with complete state observations.
method Proposes switched least squares method for identification and derives rates of convergence.
result Data-independent rate of convergence is , showing strong consistency.
Reinforced optimal controlmath.OC
Improved reinforcement method for optimal control problems.
problem Optimal control problems with limited computational cost.
method Reinforced least squares Monte Carlo method for stochastic control problems.
result Significant improvement in method's efficiency and accuracy.
Support vector machines (SVMs) are an important tool in modern data analysis. Traditionally, support vector machines have been fitted via quadratic programming, either using purpose-built or off-the-shelf algorithms. We present an alternative approach to SVM fitting via the majorization--minimization (MM) paradigm. Alg…
We compare the risk of ridge regression to a simple variant of ordinary least squares, in which one simply projects the data onto a finite dimensional subspace (as specified by a Principal Component Analysis) and then performs an ordinary (un-regularized) least squares regression in this subspace. This note shows that …
Bayesian system ID improves robustness to sparse, noisy data.
problem Robust system identification with sparse, noisy data.
method Probabilistic formulation of system identification using Bayesian posterior.
result The log posterior is more robust and less affected by multiple minima.
Optimal hashing embeddings reduce linear least squares solving time.
problem Efficiently solving large-scale linear least squares problems.
method Optimal hashing sketching matrices for linear least squares.
result Ski-LLS outperforms state-of-the-art solvers on various problem types.
This study examines the relationship between PLS and OLS regression using eigenvalue distributions.
problem Analyzing the difference between PLS and OLS regression in terms of eigenvalue distributions.
method Examined the distance between PLS and OLS regression coefficients using the Mahalanobis distance and eigenvalue distributions of the regressor covariance matrix.
result Provided a bound on the distance between PLS and OLS regression coefficients that depends only on the eigenvalue distribution of the regressor covariance matrix.
This paper optimizes sampling for least-squares approximation.
problem Optimizing sampling for least-squares approximation in arbitrary linear spaces.
method Introducing the Christoffel function to construct near-optimal random sampling strategies.
result The number of samples scales log-linearly in the dimension of the approximation space.
Improved SGD for non-strongly-convex regression with faster convergence.
problem Non-strongly-convex least squares regression problems.
method Modified accelerated gradient descent.
result Achieves optimal prediction error rates of and forgets initial conditions faster to .
Estimates LATE using combined datasets, overcoming data limitations.
problem Estimating LATE when compliance is incomplete and data is split.
method Combines separately observed datasets to estimate LATE using direct and weighted least squares methods.
result Proposes a stable and practical estimator for LATE.
We describe stochastic Newton and stochastic quasi-Newton approaches to efficiently solve large linear least-squares problems where the very large data sets present a significant computational burden (e.g., the size may exceed computer memory or data are collected in real-time). In our proposed framework, stochasticity…