A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Unified theory and debiasing framework for random oblique projections in high dimensions.
Subsampling methods have been recently proposed to speed up least squares estimation in large scale settings. However, these algorithms are typically not robust to outliers or corruptions in the observed covariates. The concept of influence that was developed for regression diagnostics can be used to detect such corrup…
A new method speeds up ALS for recommender systems by subsampling key elements.
Unified methodology for statistical inference in least squares and PCA via randomized sketching.
For massive data, the family of subsampling algorithms is popular to downsize the data volume and reduce computational burden. Existing studies focus on approximating the ordinary least squares estimate in linear regression, where statistical leverage scores are often used to define subsampling probabilities. In this p…
Ensemble methods that average over a collection of independent predictors that are each limited to a subsampling of both the examples and features of the training data command a significant presence in machine learning, such as the ever-popular random forest, yet the nature of the subsampling effect, particularly of th…
For optimization on large-scale data, exactly calculating its solution may be computationally difficulty because of the large size of the data. In this paper we consider subsampled optimization for fast approximating the exact solution. In this approach, one gets a surrogate dataset by sampling from the full data, and …
Early stopping is a well known approach to reduce the time complexity for performing training and model selection of large scale learning machines. On the other hand, memory/space (rather than time) complexity is the main constraint in many applications, and randomized subsampling techniques have been proposed to tackl…
We study Nyström type subsampling approaches to large scale kernel methods, and prove learning bounds in the statistical learning setting, where random sampling and high probability estimates are considered. In particular, we prove that these approaches can achieve optimal learning bounds, provided the subsampling leve…
Large sample size brings the computation bottleneck for modern data analysis. Subsampling is one of efficient strategies to handle this problem. In previous studies, researchers make more fo- cus on subsampling with replacement (SSR) than on subsampling without replacement (SSWR). In this paper we investigate a kind of…
A significant hurdle for analyzing large sample data is the lack of effective statistical computing and inference methods. An emerging powerful approach for analyzing large sample data is subsampling, by which one takes a random subsample from the original full sample and uses it as a surrogate for subsequent computati…
In this paper, we study the Nystr{ö}m type subsampling for large scale kernel methods to reduce the computational complexities of big data. We discuss the multi-penalty regularization scheme based on Nystr{ö}m type subsampling which is motivated from well-studied manifold regularization schemes. We develop a theoretica…
Theory and method for reducing prediction variance in noisy feature-subsampled ridge ensembles.
New algorithm reduces sketching dimension to effective problem size.
New insights into how randomization affects greedy model selection.
Iterative Hessian sketch (IHS) is an effective sketching method for modeling large-scale data. It was originally proposed by Pilanci and Wainwright (2016; JMLR) based on randomized sketching matrices. However, it is computationally intensive due to the iterative sketch process. In this paper, we analyze the IHS algorit…
Optimal hashing embeddings reduce linear least squares solving time.
We consider a least squares regression problem where the data has been generated from a linear model, and we are interested to learn the unknown regression parameters. We consider "sketch-and-solve" methods that randomly project the data first, and do regression after. Previous works have analyzed the statistical and c…
The paper analyzes the risk of bagging regularized M-estimators under proportional asymptotics.
We propose and study kernel conjugate gradient methods (KCGM) with random projections for least-squares regression over a separable Hilbert space. Considering two types of random projections generated by randomized sketches and Nyström subsampling, we prove optimal statistical results with respect to variants of norms …
Study ridge ensembles in proportional feature-to-sample size regime, proving risk equivalence and GCV consistency.
Efficiently removes specific data subsets without retraining for GDPR compliance.
The paper tackles extrapolation in extreme regions of regression problems.
Enhances random forest performance with exogenous randomness.
CD converges linearly for MCP/SCAD penalized least squares.
Develops an empirical likelihood framework for random forests and ensembles.
Illustrates interleaved learning with Kalman Filter for linear least squares.
Jackknife variance estimation validated for generalized U-statistics.
We study randomized sketching methods for approximately solving least-squares problem with a general convex constraint. The quality of a least-squares approximation can be assessed in different ways: either in terms of the value of the quadratic objective function (cost approximation), or in terms of some distance meas…
Cross validation residuals are well known for the ordinary least squares model. Here leave-M-out cross validation is extended to generalised least squares. The relationship between cross validation residuals and Cook's distance is demonstrated, in terms of an approximation to the difference in the generalised residual …
The paper analyzes the statistical cost of tuning kernel hyperparameters in robust regression.
We compare the risk of ridge regression to a simple variant of ordinary least squares, in which one simply projects the data onto a finite dimensional subspace (as specified by a Principal Component Analysis) and then performs an ordinary (un-regularized) least squares regression in this subspace. This note shows that …
A new method for streaming PCA provides confidence intervals for eigenvector entries.
The paper improves Kaczmarz algorithm with momentum for linear least squares.
New algorithm improves online binary classification with constant time complexity.
Reduced-rank method improves least-squares regression under output regularity.
Develops data subsampling techniques for Poisson regression models.
In this paper we tackle the problem of estimating the power-law tail exponent of income distributions by using the Hill's estimator. A subsample semi-parametric bootstrap procedure minimising the mean squared error is used to choose the power-law cutoff value optimally. This technique is applied to personal income data…
The paper analyzes bagging in overparameterized learning, deriving risk properties and optimal subsample sizes.
Proposes a partitioned least squares model for feature grouping.
ESNs trained with Tikhonov least squares approximate ergodic dynamical systems in L2(μ) norm.
The kernel least mean squares (KLMS) algorithm is a computationally efficient nonlinear adaptive filtering method that "kernelizes" the celebrated (linear) least mean squares algorithm. We demonstrate that the least mean squares algorithm is closely related to the Kalman filtering, and thus, the KLMS can be interpreted…
A new algorithm solves nonnegative least squares faster with nonnegative data.
The paper identifies saddlepoints in unsupervised auto-encoding neural nets.
The paper proposes a least squares method for binary compressive sampling with low intrinsic dimension signals.
This paper studies an unsupervised deep learning-based numerical approach for solving partial differential equations (PDEs). The approach makes use of the deep neural network to approximate solutions of PDEs through the compositional construction and employs least-squares functionals as loss functions to determine para…
This paper explains CART random forests using stochastic control theory.
We propose a new forward-backward stochastic differential equation solver for high-dimensional derivatives pricing problems by combining deep learning solver with least square regression technique widely used in the least square Monte Carlo method for the valuation of American options. Our numerical experiments demonst…