Estimates Heston model parameters from continuous data.
problem Estimating parameters of the Heston model from continuous data.
method Least squares estimation for subcritical Heston model.
result Strong consistency and asymptotic normality of estimators proved.
The paper analyzes the risk of a least squares estimator under a spike covariance model.
problem Risk analysis of the least squares estimator under a spike covariance model.
method Assumes spike covariance matrices, studies risk as d / n i g h t a r r o w ∞ d/n
ightarrow \infty d / ni g h t a r r o w ∞ . result Risk of the minimum norm least squares estimator vanishes compared to the null estimator.
New method corrects least-squares temporal difference for better lambda-return estimation.
problem Improving lambda-return estimation in reinforcement learning.
method Uncorrected least-squares temporal difference with a correction method.
result Enhanced accuracy in temporal difference learning.
Study improves least squares estimation for heavy-tailed errors.
problem Improving least squares estimation under heteroscedastic and heavy-tailed errors.
method Analyzes the rate of convergence of least squares estimator under bounded conditional variance and finitely many moments of errors.
result Upper bounds on rates of convergence of LSE for heavy-tailed errors are found.
A new method for estimating large-scale linear models with improved precision.
problem Estimating large-scale linear statistical models efficiently.
method Sequential Least-Squares Estimators with Fast Randomized Sketching (SLSE-FRS), integrating Sketch-and-Solve and Iterative-Sketching methods.
result SLSE-FRS produces high-precision estimators, outperforming state-of-the-art methods.
Efficiently estimates private least squares with linear error growth.
problem Private estimation of ordinary least squares with bounded residuals and leverage.
method Scaled noise added to a stable nonprivate estimator of the regression vector.
result Near-optimal accuracy guarantee with linear error growth in dimension.
We study asymptotic properties of some (essentially conditional least squares) parameter estimators for the subcritical Heston model based on discrete time observations derived from conditional least squares estimators of some modified parameters.
The ratio of two probability densities can be used for solving various machine learning tasks such as covariate shift adaptation (importance sampling), outlier detection (likelihood-ratio test), and feature selection (mutual information). Recently, several methods of directly estimating the density ratio have been deve…
Improved estimator for least squares using random projections achieves smaller error.
problem Improving the accuracy of least squares solutions for large-scale problems.
method James-Stein estimator applied to Gaussian sketching of least squares problems.
result Upper and lower bounds match when SNR is small and data matrix is well-conditioned.
We prove the statistical consistency of kernel Partial Least Squares Regression applied to a bounded regression learning problem on a reproducing kernel Hilbert space. Partial Least Squares stands out of well-known classical approaches as e.g. Ridge Regression or Principal Components Regression, as it is not defined as…
Least squares estimator fails to achieve optimal risk in bounded distributions, but non-linear predictors can.
problem Optimal risk in bounded distributions for constrained least squares.
method Comparison of least squares and non-linear predictors.
result Non-linear predictors can achieve optimal risk O ( d / n ) O(d/n) O ( d / n ) in bounded distributions. New algorithms estimate Jacobian matrices for large-scale machine learning.
problem Efficiently computing search directions for large nonlinear least squares.
method Exploit low-rank structure in Hessian to estimate Jacobian matrices.
result Two algorithms perform well compared to state-of-the-art methods.
The paper establishes prediction bounds for trend filtering with higher order total variation penalties.
problem Estimating signals with jumps of varying orders using total variation regularization.
method Combining oracle inequalities and interpolating vectors to bound effective sparsity.
result The ℓ 1 \ell_1 ℓ 1 -penalty on ( k − 1 ) e x t t h (k-1)^{ ext{th}} ( k − 1 ) e x t t h order differences allows adaptive estimation for k ∈ { 1 , 2 , 3 , 4 } k \in \{1,2,3,4\} k ∈ { 1 , 2 , 3 , 4 } . Synthesizes robust estimators for domain adaptation.
problem Improving prediction accuracy in target domain with limited data.
method Synthesizes a family of robust least squares estimators using convex optimization.
result Robust strategies can outperform non-robust interpolations.
The paper examines prediction and estimation risks of ridgeless least squares under general error assumptions.
problem Prediction and estimation risks of ridgeless least squares under realistic error structures.
method Analysis of prediction and estimation risks under general regression error assumptions, including clustered or serial dependence.
result The benefits of overparameterization extend to time series, panel, and grouped data.
Efficiently estimates rotations with corrupted data.
problem Rotation synchronization under high corruption and noise.
method Message passing algorithm with reweighted least squares.
result Superior performance over state-of-the-art methods.
Least Squares EM converges globally for log-concave mixtures.
problem Location estimation in mixtures of two log-concave densities.
method Least Squares EM algorithm applied to log-concave mixtures.
result Least Squares EM converges globally to the true location parameter.
Least squares estimation works well for symmetric positive semidefinite matrices without regularization.
problem Estimation of symmetric positive semidefinite matrices without regularization.
method Simple least squares estimation with extsf{spd} constraint.
result Constrained least squares estimation performs as well as regularization-based approaches.
The paper explores how overfitting can lead to better predictions in high-dimensional data.
problem Understanding the behavior of linear models in high-dimensional settings with more predictors than observations.
method Analysis of ordinary least squares, penalized least squares, and spectral shrinkage estimates.
result The phenomenon of double descent, where model performance can improve with increasing model complexity.
Least Squares Estimators are suboptimal for 5D convex functions.
problem Suboptimality of Least Squares Estimators in estimating multidimensional convex functions.
method Analysis of natural subclasses of convex functions in random and fixed design settings.
result Risk of LSE is n − 2 / d n^{-2/d} n − 2/ d while minimax risk is n − 4 / ( d + 4 ) n^{-4/(d+4)} n − 4/ ( d + 4 ) for d ≥ 5 d \geq 5 d ≥ 5 . Paper criticizes mFLMS algorithm for power signal estimation.
problem Criticism of mFLMS algorithm for power signal parameter estimation.
method Recently proposed mFLMS algorithm for power signal parameter estimation.
result Simulation results show mFLMS does not outperform LMS.
Optimal multiscale learning of linear operators
problem Statistical and computational limits of learning bounded linear operators between Sobolev spaces
method Reformulate as an infinite-dimensional matrix regression problem with heterogeneous multiscale structure
result Establish minimax rates and construct a finite-resolution blockwise least-squares estimator attaining these rates
New algorithm estimates partially-observed linear systems with better rates than previous methods.
problem Estimating parameters of partially-observed linear systems with long-term dependencies and semi-parametric noise.
method Prefiltered least squares estimator with semi-parametric noise model.
result First algorithm provably estimates parameters of partially-observed linear systems with rates not dependent on dependency decay rate.
Improved robust regression for heavy-tailed and contaminated data.
problem Linear regression with heavy-tailed and adversarially contaminated covariates and responses.
method Applying a filtering algorithm to covariates and then using Huber regression, least trimmed squares, or least absolute deviation estimators on the remaining data.
result Near-optimal error rates achieved for the Huber regression estimator.
The OLS estimator optimally identifies stable linear systems with a finite number of samples.
problem Identifying stable linear systems with a finite number of samples.
method Finite-time analysis of the Ordinary Least Squares (OLS) estimator for stable linear systems.
result The OLS estimator achieves optimal sample complexity for stable systems, matching existing lower bounds up to universal factors.
Estimates LATE using combined datasets, overcoming data limitations.
problem Estimating LATE when compliance is incomplete and data is split.
method Combines separately observed datasets to estimate LATE using direct and weighted least squares methods.
result Proposes a stable and practical estimator for LATE.
New kriging method improves mean estimation and uncertainty.
problem Improving mean estimation and uncertainty in kriging.
method Proposes a rational kriging method using generalized least squares and Gaussian process.
result Generalized least squares estimate is more well-behaved than ordinary kriging.
Paper proposes robust LAD estimators for 2D sinusoidal model, proving consistency and normality.
problem Estimation of parameters in 2D sinusoidal models with outliers or heavy-tailed noise.
method Least absolute deviation (LAD) estimators for robust parameter estimation.
result Strong consistency and asymptotic normality of LAD estimators for 2D sinusoidal model parameters.
This paper reviews SDR methods for multivariate response regression.
problem Handling sufficient dimension reduction for multivariate response regression.
method Characterizes SDR estimators as inverse or forward regression methods.
result Pooled marginal, projective resampling, distance-based, ordinary least squares, partial least squares, and semiparametric SDR estimators are discussed.
New methods estimate probabilities from pairwise comparisons, adapting to difficulty.
problem Estimating probabilities from pairwise comparisons with varying difficulty.
method Adaptive estimators using an adaptivity index based on indifference sets.
result CRL estimator has adaptivity index upper bounded by n \sqrt{n} n up to logarithmic factors. Investigates least squares estimation in deterministic MoE models.
problem Complexity and difficulty in analyzing MoE models.
method Least squares estimation under deterministic MoE models.
result Rates of estimating strongly identifiable experts are faster than polynomial experts.
Combines BTEM and T-PLS for accurate spectral recovery and calibration.
problem Calibrating pure spectra of minority components in mixtures without prior knowledge.
method Band target entropy minimization (BTEM) and target partial least squares (T-PLS).
result Estimated amounts from BTEM-T-PLS similar to MCR-ALS on simple mixtures, superior on complex ones.
A recently introduced Importance Sampling strategy based on a least squares optimization is applied to the Monte Carlo simulation of Libor Market Models. Such Least Squares Importance Sampling (LSIS) allows the automatic optimization of the sampling distribution within a trial class by means of a quick presimulation al…
This book introduces linear models and their theories rigorously.
problem Understanding linear models and their theories.
method Explains linear models from three perspectives, introduces maximum likelihood estimation, and proves least squares is the best unbiased linear model.
result Least squares is the best unbiased linear model in terms of mean squared error.
Distributed learning with least squares regularization achieves good performance without eigenfunction assumptions.
problem Efficiently learning from large datasets distributed across multiple machines.
method Divide-and-conquer approach, least squares regularization, RKHS, error bounds in expectation.
result The global estimator is a good approximation to the full data estimator, with sharp error bounds.
Bayesian method improves online NARMAX model identification.
problem Online identification of nonlinear systems with small sample sizes and low noise.
method Variational Bayesian inference using message passing algorithm for polynomial NARMAX models.
result Variational Bayesian estimator outperforms recursive and offline least-squares methods.
The study explores nonparametric regression with shape constraints using least squares estimation.
problem Nonparametric regression under shape constraints.
method Least squares estimation (LSE) with focus on isotonic, unimodal, convex, and additive shape-restricted regression.
result Adaptive nature of the LSE and its risk behavior, with pointwise limiting distribution theory for isotonic regression.
The paper proposes a least squares method for binary compressive sampling with low intrinsic dimension signals.
problem Recovering signals from binary measurements with noise and sign flips.
method Least squares decoder for signals with low generative intrinsic dimension.
result The least squares decoder achieves a sharp estimation error of O ( k log ( L n ) m ) O(\sqrt{\frac{k\log (Ln)}{m}}) O ( m k l o g ( L n ) ) under certain conditions. Develops a distributed least squares approximation method for regression problems.
problem Solving large-scale regression problems on distributed systems.
method Approximates local objective functions using a local quadratic form and combines estimators by weighted average.
result Statistically efficient combined estimator with one round of communication.
Functional PLS improves prediction and inference for scalar responses from functional predictors.
problem Estimating scalar responses from functional predictors in an ill-posed inverse problem.
method Functional partial least squares (PLS) estimator with adaptive early stopping and new tests.
result PLS attains nearly minimax-optimal convergence rates and detects local alternatives.
New method for unbiased regression reduces excess risk.
problem Least squares regression with optimal solution and Hessian matrix.
method Averaged stochastic gradient descent with time-average estimator.
result Unbiased estimator with O(1/k) expected excess risk.
Develops asymptotic analysis for RandNLA sampling estimators in least-squares problems.
problem Lack of distributional information for RandNLA estimators in statistical inference.
method Asymptotic analysis of sampling estimators for least-squares problems in two settings.
result Sampling estimators are asymptotically normally distributed under mild conditions.
Proposes a new regression method using L p L_p L p -norms for non-Gaussian noise.
problem Non-Gaussian noise in residuals affects the performance of local least squares regression.
method Introduces local polynomial L p L_p L p -norm regression, replacing weighted least squares with weighted L p L_p L p -norm estimation. result Demonstrates superior performance over local least squares in one-dimensional data and higher dimensions.
Many problems in financial engineering involve the estimation of unknown conditional expectations across a time interval. Often Least Squares Monte Carlo techniques are used for the estimation. One method that can be combined with Least Squares Monte Carlo is the "Regress-Later" method. Unlike conventional methods wher…
Estimates time-varying parameters from two OLS estimates.
problem Time-varying linear regression with hidden dynamics.
method Combines two OLS estimates for stable linear dynamics.
result Finite sample guarantee on estimation error.
Bayesian optimization selects wavelengths for sugar content estimation in NIR spectroscopy.
problem Improving prediction accuracy and interpretability of spectral data for sugar content estimation.
method Formulated as a binary black-box optimization problem, proposed method uses Bayesian optimization with a sparse quadratic surrogate model and Thompson sampling.
result Improves prediction accuracy of partial least squares regression and yields more consistent wavelength regions.
Local control regression improves portfolio optimization accuracy.
problem Expensive and inaccurate global control regression for portfolio optimization.
method Introduced local control regression combined with adaptive grids.
result Choosing a coarse grid for local regression produces accurate results.
Subsampling methods have been recently proposed to speed up least squares estimation in large scale settings. However, these algorithms are typically not robust to outliers or corruptions in the observed covariates. The concept of influence that was developed for regression diagnostics can be used to detect such corrup…