Generalized cross validation (GCV) is one of the most important approaches used to estimate parameters in the context of inverse problems and regularization techniques. A notable example is the determination of the smoothness parameter in splines. When the data are generated by a state space model, like in the spline c…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
18 results for “GCV”
Corrects GCV for inconsistent risk estimation in finite ensembles of penalized estimators.
problem Inconsistent risk estimation of GCV for finite ensembles of penalized estimators.
method Identifies a correction involving an additional scalar correction based on degrees of freedom adjusted training errors from each ensemble component.
result CGCV maintains computational advantages of GCV and is model-free uniformly consistent for ridge regression.
Study ridge ensembles in proportional feature-to-sample size regime, proving risk equivalence and GCV consistency.
problem Characterizing and optimizing ridge ensembles in proportional feature-to-sample size regimes.
method Proportional asymptotics analysis, GCV for tuning, proving risk equivalence.
result Risk of optimal full ridgeless ensemble matches optimal ridge predictor's risk.
We improve prediction risk estimation for large datasets using sketching and ridge regression.
problem Estimating prediction risks for large datasets efficiently and accurately.
method Random matrix theory, generalized cross validation, sketched ridge regression ensembles, and ensemble trick.
result Consistent risk estimation and prediction intervals for large-scale datasets.
New insights into ridge regression with correlated data, improving risk prediction.
problem Understanding and predicting risk in ridge regression with correlated samples.
method Random matrix theory and free probability for asymptotic analysis; modified GCV estimator (CorrGCV) for unbiased prediction.
result GCV estimator fails for out-of-sample risk with correlated data; CorrGCV provides an unbiased estimator.
Deep learning method improves myelin water fraction estimation.
problem Estimating myelin water fraction in the brain using magnetic resonance relaxometry.
method Combines input layer regularization with automated regularization hyperparameter tuning.
result Proposed method outperforms classical methods and multi-layer perceptrons on in vivo brain data.
Tuning parameter selection is of critical importance for kernel ridge regression. To this date, data driven tuning method for divide-and-conquer kernel ridge regression (d-KRR) has been lacking in the literature, which limits the applicability of d-KRR for large data sets. In this paper, by modifying the Generalized Cr…
Analysis of cross-validation for early-stopped gradient descent in high-dimensional regression.
problem Inconsistency of GCV for early-stopped GD in high-dimensional least squares regression.
method Theoretical analysis of GCV and LOOCV applied to early-stopped GD in high-dimensional least squares regression.
result LOOCV converges uniformly to the prediction risk of early-stopped GD, while GCV is generically inconsistent.
The paper proves a non-asymptotic test error approximation for KRR.
problem Understanding the test error of Kernel Ridge Regression.
method Established a non-asymptotic deterministic approximation for test error of KRR.
result The test error of KRR can be approximated by a closed-form estimate derived from the spectrum of the kernel operator.
Random matrix theory predicts neural representations generalize well.
problem Understanding why neural representations generalize well in practice.
method Applied random matrix theory to kernel regression and neural networks.
result GCV estimator accurately predicts generalization risk in overparameterized settings.
New method for cross-validation in high-dimensional data with dependent or heavy-tailed covariates.
problem Inconsistent cross-validation in high-dimensional settings with dependent or heavy-tailed covariates.
method ROTI-GCV framework for cross-validation under proportional asymptotics regime.
result Demonstrated accuracy of ROTI-GCV in synthetic and semi-synthetic settings.
Hybrid regularization avoids double descent in random feature models.
problem Avoiding the double descent phenomenon in random feature models.
method Combines early stopping and weight decay, using GCV for hyperparameter selection.
result Hybrid method successfully avoids double descent and achieves comparable generalization.
New findings on kernel regression in the quadratic regime, improving understanding of machine learning models.
problem Understanding kernel ridge regression in the quadratic asymptotic regime.
method Extended study of kernel regression to the quadratic regime, establishing approximation bounds and spectral distributions.
result Broad class of inner-product kernels exhibit behavior similar to a quadratic kernel, with precise asymptotic training and test errors characterized.
Optimal self-distillation improves generative models' velocity risk and mode recovery.
problem Improving generative models' velocity risk and mode recovery.
method Proved optimal self-distillation for rectified flow via linear probing, derived mixing coefficient, and provided validation tuning.
result Optimal self-distillation improves velocity risk and mode recovery.
Resolution and Scale Independent Function Matching Using a String Energy Penalized Spline Priorstat.ML
The extension of the classical Bayesian penalized spline method to inference on vector-valued functions is considered, with an emphasis on characterizing the suitability of the method for general application.We show that the standard quadratic penalty is exactly analogous to the energy of a stretched string, with the p…
New techniques improve the accuracy of identifying nonlinear systems from noisy data.
problem Identifying nonlinear dynamical systems from noisy state measurements.
method Comparative study of local and global smoothing techniques to denoise state measurements and improve sparse regression methods.
result Global smoothing methods outperform local methods in improving the accuracy of governing equation recovery.
A Doubly Regularized Linear Discriminant Analysis Classifier with Automatic Parameter Selectioncs.LG
Proposes R2LDA for improved LDA classifier performance.
problem Poor performance of LDA classifiers in small to comparable data sizes.
method Doubly regularized LDA with automatic parameter selection.
result Consistent and effective performance in noisy test data.
Solve for the Hyperparameter, Skip the Search: Kolmogorov-Optimal Scaling Laws for Spline Regressioncs.LG
KOLMOGOROV-OPTIMAL RESOLUTION ESTIMATION (KORE) solves spline regression without exhaustive search
problem Hyperparameter tuning in spline regression
method Solving for optimal resolution analytically
result KORE matches exhaustive cross-validation and outperforms tuned models