Optimizes Lasso hyperparameters using leave-one-out CV.
problem Finding optimal hyperparameters for Lasso regression.
method Develops an algorithm to compute exact or approximate leave-one-out CV.
result Algorithm finds optimal hyperparameters for Lasso.
Optimizes hyperparameter tuning for models using approximate leave-one-out cross-validation.
problem Finding optimal hyperparameters for regularized models using approximate leave-one-out cross-validation.
method Derive efficient formulas for gradient and hessian of approximate leave-one-out cross-validation, apply second-order optimization.
result Demonstrates the effectiveness of the approach on real-world data sets.
ALO-CV approximates leave-one-out error in proportional regime.
problem Estimating generalization error in high-dimensional settings.
method Developed new analysis for ALO-CV, showed consistency under strong convexity.
result ALO-CV approximates leave-one-out error up to negligible error.
The paper improves ALO for ℓ1-regularized models.
problem Estimating out-of-sample error for ℓ1-regularized models. method Developed a novel theory for ℓ1-regularized problems, bounding ALO error. result For ℓ1-regularized problems, ALO error goes to zero as p goes to infinity. The paper proves LOO CV is reliable under estimator stability.
problem Ensuring the reliability of leave-one-out cross validation.
method Using concentration inequalities based on logarithmic Sobolev inequality.
result LOO CV is a valid procedure under estimator stability.
A fast method for LOOCV in k-NN regression reduces computation time.
problem Efficient computation of LOOCV for k-NN regression.
method Identical LOOCV estimate to (k+1)-NN MSE on training data.
result LOOCV computation can be done with (k+1)-NN regression once.
Improved LOO cross-validation for function approximation.
problem Estimating the Integrated Squared Error (ISE) for function approximation.
method Weighted Leave-One-Out cross-validation based on Gaussian Process.
result Significantly more precise ISE estimation compared to unweighted LOO.
Paper accelerates conformal prediction by using approximate leave-one-out estimators.
problem Limited computational cost for conformal prediction.
method Incorporates approximate leave-one-out estimators to accelerate conformal prediction.
result ALO-based methods achieve comparable coverage and efficiency to exact methods but with significantly reduced runtime.
Model inference, such as model comparison, model checking, and model selection, is an important part of model development. Leave-one-out cross-validation (LOO) is a general approach for assessing the generalizability of a model, but unfortunately, LOO does not scale well to large datasets. We propose a combination of u…
We show how to adjust the coefficient of determination (R2) when used for measuring predictive accuracy via leave-one-out cross-validation.
Study improves understanding of non-differentiable penalties in high-dimensional settings.
problem Theoretical understanding of non-differentiable penalties like generalized LASSO and nuclear norm in high-dimensional settings.
method Proportional high-dimensional regime analysis with finite sample upper bounds on expected squared error.
result LO provides accurate estimation of out-of-sample risk in high-dimensional settings.
Algorithm identifies and corrects noisy labels using Gaussian process regression.
problem Detecting and correcting real-valued noisy labels from mixed data.
method Gaussian process regression with heteroscedastic noise model and leave-one-out cross-validation.
result The method can pinpoint corrupted sample points and improve regression models.
A new method improves robustness and efficiency of Bayesian LOO-CV.
problem Computational expense and unreliability of classical LOO-CV in high-dimensional Bayesian models.
method Proposes a mixture estimator to compute Bayesian LOO-CV criteria with finite asymptotic variance.
result Improved robustness and efficiency in high-dimensional problems.
In this article, we derive concentration inequalities for the cross-validation estimate of the generalization error for stable predictors in the context of risk assessment. The notion of stability has been first introduced by \cite{DEWA79} and extended by \cite{KEA95}, \cite{BE01} and \cite{KUNIY02} to characterize cla…
New method optimizes hyperparameters for non-smooth problems efficiently.
problem Efficiently tuning hyperparameters for non-smooth cost functions.
method Combines hyperparameter search with proximal gradient updates.
result Method converges to local optimum of LOO validation error.
New methods improve anomaly detection with reduced false positives.
problem Effective anomaly detection with controlled error rates.
method Leave-one-out-, bootstrap-, and cross-conformal anomaly detection methods.
result Improved anomaly detection with reduced false positives.
The paper analyzes LOCV for high-dimensional risk estimation, proving error bounds.
problem Estimating out-of-sample prediction error in high-dimensional settings.
method Theoretical analysis of leave-one-out cross validation (LOCV) in penalized regression.
result Finite sample upper bounds on LOCV error, showing it converges to zero as n,p → ∞.
The future predictive performance of a Bayesian model can be estimated using Bayesian cross-validation. In this article, we consider Gaussian latent variable models where the integration over the latent values is approximated using the Laplace method or expectation propagation (EP). We study the properties of several B…
Enhances polynomial chaos models with uncertainty intervals.
problem Uncertainty quantification in surrogate models.
method Jackknife-based conformal prediction integrated into polynomial chaos expansions.
result Produces accurate predictive intervals for low-accuracy models.
Proposes DeGLIF to denoise graph data for label noise robustness.
problem Label noise in graph data makes node classification challenging.
method Uses leave-one-out influence function to denoise graph data.
result DeGLIF improves accuracy in node classification on noisy datasets.
RandALO speeds up risk estimation for large datasets.
problem Estimating out-of-sample risk for large, high-dimensional models.
method RandALO: a randomized approximate leave-one-out estimator.
result RandALO is a computationally efficient risk estimator in high dimensions.
The study evaluates different parameter selection methods for Gaussian process interpolation.
problem Choosing optimal parameters for Gaussian process interpolation.
method Empirical study using scoring rules and leave-one-out selection criteria.
result The choice of model family is often more important than the selection criterion.
We speed up Gaussian process cross-validation calculations and improve model diagnostics.
problem Efficiently calculating cross-validation residuals and their covariances in Gaussian processes.
method Generalized fast Gaussian process leave-one-out formulae to multiple-fold cross-validation, highlighting covariance structures.
result Correcting for residual covariances in cross-validation improves back to Maximum Likelihood Estimation.
The paper improves confidence intervals for test error using cross-validation.
problem Improving confidence intervals for test error in machine learning.
method Develops central limit theorems and consistent estimators for cross-validation.
result Provides asymptotically-exact confidence intervals and hypothesis tests.
New cross-validation methods for Gaussian process regression with efficient gradient computation.
problem Estimating parameters of Gaussian process covariance functions.
method Derive new cross-validation criteria and efficient adjoint computation of gradients.
result Efficient method for evaluating cross-validation criteria and their gradients.
Risk estimation is at the core of many learning systems. The importance of this problem has motivated researchers to propose different schemes, such as cross validation, generalized cross validation, and Bootstrap. The theoretical properties of such estimates have been extensively studied in the low-dimensional setting…
Bayesian model averaging, model selection and its approximations such as BIC are generally statistically consistent, but sometimes achieve slower rates og convergence than other methods such as AIC and leave-one-out cross-validation. On the other hand, these other methods can br inconsistent. We identify the "catch-up …
Weighted SVM (or fuzzy SVM) is the most widely used SVM variant owning its effectiveness to the use of instance weights. Proper selection of the instance weights can lead to increased generalization performance. In this work, we extend the span error bound theory to weighted SVM and we introduce effective hyperparamete…
In this article, we derive concentration inequalities for the cross-validation estimate of the generalization error for subagged estimators, both for classification and regressor. General loss functions and class of predictors with both finite and infinite VC-dimension are considered. We slightly generalize the formali…
Backward Conformal Prediction offers flexible control over prediction set sizes while ensuring coverage guarantees.
problem Providing reliable prediction sets with controlled sizes in applications like medical diagnosis.
method Defines a rule that constrains prediction set sizes based on observed data, adapting coverage levels.
result Maintains computable coverage guarantees while ensuring interpretable, well-controlled prediction set sizes.
Cross-validation (CV) is a technique for evaluating the ability of statistical models/learning systems based on a given data set. Despite its wide applicability, the rather heavy computational cost can prevent its use as the system size grows. To resolve this difficulty in the case of Bayesian linear regression, we dev…
This paper introduces Kernel-based Information Criterion (KIC) for model selection in regression analysis. The novel kernel-based complexity measure in KIC efficiently computes the interdependency between parameters of the model using a variable-wise variance and yields selection of better, more robust regressors. Expe…
We propose a novel algorithm for greedy forward feature selection for regularized least-squares (RLS) regression and classification, also known as the least-squares support vector machine or ridge regression. The algorithm, which we call greedy RLS, starts from the empty feature set, and on each iteration adds the feat…
UDM reparameterization improves language model generation.
problem Mismatch between UDM training objective and denoising posterior.
method Leave-one-out denoising and absorbing state reformulation.
result Improved UDM generation through leave-one-out parameterization.
Decoding, ie prediction from brain images or signals, calls for empirical evaluation of its predictive power. Such evaluation is achieved via cross-validation, a method also used to tune decoders' hyper-parameters. This paper is a review on cross-validation procedures for decoding in neuroimaging. It includes a didacti…
Paper analyzes robust matrix completion with efficient nonconvex method and leave-one-out analysis.
problem Robust matrix completion with sparse noise.
method Alternates between projected gradient step for low-rank and thresholding step for sparse noise.
result Achieves linear convergence for general thresholding functions.
In this article, we derive concentration inequalities for the cross-validation estimate of the generalization error for empirical risk minimizers. In the general setting, we prove sanity-check bounds in the spirit of \cite{KR99} \textquotedblleft\textit{bounds showing that the worst-case error of this estimate is not m…
FastMuyGPs speeds up GP predictions for large datasets.
problem High cost of Gaussian process predictions for large data.
method Combines cross-validation, batching, nearest neighbors sparsification, and precomputation.
result Superior accuracy and competitive runtime compared to other methods.
Adaptive coverage policies improve conformal prediction accuracy.
problem Fixed coverage levels in traditional conformal prediction lead to uninformative predictions.
method Optimizes adaptive coverage policy using a neural network trained on leave-one-out calibration.
result Adaptive coverage policies produce more informative and flexible prediction sets.
Framework designs antiviral drugs using deep learning and RL.
problem Designing effective antiviral drugs for SARS-CoV-2.
method Deep learning framework with conditional molecular generator and RL.
result Framework generates more antiviral ligands than a VAE baseline.
New method improves online nonparametric estimators with minimal extra computation.
problem Model selection and hyperparameter tuning for online nonparametric estimators.
method Weighted rolling validation procedure for online cross-validation.
result Improves base estimators to achieve better heuristic performance and adaptive convergence rate.
Bayesian EM method improves ridge regression tuning without LOOCV's limitations.
problem Tuning ridge regression hyper-parameter λ efficiently and accurately. method Bayesian formulation of ridge regression with expectation maximization (EM) procedure.
result Guaranteed unique optimal solution for large n without candidate λ specification. The present paper provides a new generic strategy leading to non-asymptotic theoretical guarantees on the Leave-one-Out procedure applied to a broad class of learning algorithms. This strategy relies on two main ingredients: the new notion of Lq stability, and the strong use of moment inequalities. Lq stability e…
Analysis of cross-validation for early-stopped gradient descent in high-dimensional regression.
problem Inconsistency of GCV for early-stopped GD in high-dimensional least squares regression.
method Theoretical analysis of GCV and LOOCV applied to early-stopped GD in high-dimensional least squares regression.
result LOOCV converges uniformly to the prediction risk of early-stopped GD, while GCV is generically inconsistent.
LOOCV is often useful for analyzing small, structured experimental designs.
problem The effectiveness of cross-validation in analyzing designed experiments.
method Empirical study comparing LOOCV and other model selection methods.
result LOOCV is often useful in the analysis of small, structured experimental designs.
Consider the following class of learning schemes: β^:=argminβ∑j=1nℓ(xj⊤β;yj)+λR(β),(1) where xi∈Rp and yi∈R denote the ith feature and response variable …
LOO-StabCP speeds up CP for multiple predictions.
problem Balancing computational efficiency and prediction accuracy in CP.
method Leave-One-Out Stable Conformal Prediction (LOO-StabCP) using algorithmic stability.
result LOO-StabCP is faster and more accurate than RO-StabCP.
We investigate the issue of model selection and the use of the nonconformity (strangeness) measure in batch learning. Using the nonconformity measure we propose a new training algorithm that helps avoid the need for Cross-Validation or Leave-One-Out model selection strategies. We provide a new generalisation error boun…