arXiv research

We study randomized sketching methods for approximately solving least-squares problem with a general convex constraint. The quality of a least-squares approximation can be assessed in different ways: either in terms of the value of the quadratic objective function (cost approximation), or in terms of some distance meas…

ESNs trained with Tikhonov least squares approximate ergodic dynamical systems in L2(μ) norm.

This paper optimizes sampling for least-squares approximation.

New method for option pricing using Monte Carlo and least squares.

Cross validation residuals extended to GLS models.

The paper proposes a least squares method for binary compressive sampling with low intrinsic dimension signals.

Total least squares regression sped up with input sparsity time.

The kernel least mean squares (KLMS) algorithm is a computationally efficient nonlinear adaptive filtering method that "kernelizes" the celebrated (linear) least mean squares algorithm. We demonstrate that the least mean squares algorithm is closely related to the Kalman filtering, and thus, the KLMS can be interpreted…

Paper uses deep learning to solve PDEs without supervision.

New methods improve accuracy and scalability for large datasets in multi-class classification.

A fast sketching algorithm solves regularized least squares problems efficiently.

Improved regression analysis using Padé approximants with new residuals and regularization.

Sparse linear regression, which entails finding a sparse solution to an underdetermined system of linear equations, can formally be expressed as an $l_0$-constrained least-squares problem. The Orthogonal Least-Squares (OLS) algorithm sequentially selects the features (i.e., columns of the coefficient matrix) to greedil…

Study of regularized least squares in RKKS with indefinite kernels.

In this paper, we propose a low-rank approximation method based on discrete least-squares for the approximation of a multivariate function from random, noisy-free observations. Sparsity inducing regularization techniques are used within classical algorithms for low-rank approximation in order to exploit the possible sp…

Improved RL algorithm reduces regret in large state spaces.

Improved kernel ridge regression using conjugate gradients.

Develops a distributed least squares approximation method for regression problems.

Subsampling methods have been recently proposed to speed up least squares estimation in large scale settings. However, these algorithms are typically not robust to outliers or corruptions in the observed covariates. The concept of influence that was developed for regression diagnostics can be used to detect such corrup…

Improved function approximation for noisy data.

Stochastic Newton and quasi-Newton methods solve large linear least-squares problems efficiently.

We prove the statistical consistency of kernel Partial Least Squares Regression applied to a bounded regression learning problem on a reproducing kernel Hilbert space. Partial Least Squares stands out of well-known classical approaches as e.g. Ridge Regression or Principal Components Regression, as it is not defined as…

Improved SGD for non-strongly-convex regression with faster convergence.

Randomized matrix compression techniques, such as the Johnson-Lindenstrauss transform, have emerged as an effective and practical way for solving large-scale problems efficiently. With a focus on computational efficiency, however, forsaking solutions quality and accuracy becomes the trade-off. In this paper, we investi…

Local control regression improves portfolio optimization accuracy.

New ATD methods improve TD learning efficiency and stability.

Paper introduces $\ell$-DER for regression tasks using morphological operators and convex-concave procedure.

Unified theory and debiasing framework for random oblique projections in high dimensions.

The paper establishes prediction bounds for trend filtering with higher order total variation penalties.

Improved reinforcement method for optimal control problems.

The runtime for Kernel Partial Least Squares (KPLS) to compute the fit is quadratic in the number of examples. However, the necessity of obtaining sensitivity measures as degrees of freedom for model selection or confidence intervals for more detailed analysis requires cubic runtime, and thus constitutes a computationa…

New method for optimistic planning in MDPs using regularization.

Improved robustness in kernel-based regression via novel loss function and IRLS.

New algorithms reduce contextual bandits' regret without knowing reward noise variances.

Algorithm approximates functions on manifolds using noisy data.

The paper tackles robust reinforcement learning with performance guarantees.

We propose a stochastic approximation (SA) based method with randomization of samples for policy evaluation using the least squares temporal difference (LSTD) algorithm. Our proposed scheme is equivalent to running regular temporal difference learning with linear function approximation, albeit with samples picked unifo…

We study distributed learning with the least squares regularization scheme in a reproducing kernel Hilbert space (RKHS). By a divide-and-conquer approach, the algorithm partitions a data set into disjoint data subsets, applies the least squares regularization scheme to each data subset to produce an output function, an…

This book introduces linear models and their theories rigorously.

This work proves convergence of adaptive resampling for random Fourier features.

Efficient method for high-dimensional American option pricing and hedging.

Improved estimator for least squares using random projections achieves smaller error.

Study on RNNs' ability to approximate past-dependent Hölder functions and their application to regression.

A new method speeds up ALS for recommender systems by subsampling key elements.

Paper shows how to fit regression models on encrypted data.

CD converges linearly for MCP/SCAD penalized least squares.

This work refutes the conventional wisdom and shows acceleration can be made robust for least squares regression.

Bayesian method detects change points and clusters in piece-wise constant signals.