ALO-CV approximates leave-one-out error in proportional regime.
problem Estimating generalization error in high-dimensional settings.
method Developed new analysis for ALO-CV, showed consistency under strong convexity.
result ALO-CV approximates leave-one-out error up to negligible error.
Paper accelerates conformal prediction by using approximate leave-one-out estimators.
problem Limited computational cost for conformal prediction.
method Incorporates approximate leave-one-out estimators to accelerate conformal prediction.
result ALO-based methods achieve comparable coverage and efficiency to exact methods but with significantly reduced runtime.
The paper proves LOO CV is reliable under estimator stability.
problem Ensuring the reliability of leave-one-out cross validation.
method Using concentration inequalities based on logarithmic Sobolev inequality.
result LOO CV is a valid procedure under estimator stability.
Optimizes hyperparameter tuning for models using approximate leave-one-out cross-validation.
problem Finding optimal hyperparameters for regularized models using approximate leave-one-out cross-validation.
method Derive efficient formulas for gradient and hessian of approximate leave-one-out cross-validation, apply second-order optimization.
result Demonstrates the effectiveness of the approach on real-world data sets.
The paper improves ALO for ℓ1-regularized models.
problem Estimating out-of-sample error for ℓ1-regularized models. method Developed a novel theory for ℓ1-regularized problems, bounding ALO error. result For ℓ1-regularized problems, ALO error goes to zero as p goes to infinity. Proposes DeGLIF to denoise graph data for label noise robustness.
problem Label noise in graph data makes node classification challenging.
method Uses leave-one-out influence function to denoise graph data.
result DeGLIF improves accuracy in node classification on noisy datasets.
Improved LOO cross-validation for function approximation.
problem Estimating the Integrated Squared Error (ISE) for function approximation.
method Weighted Leave-One-Out cross-validation based on Gaussian Process.
result Significantly more precise ISE estimation compared to unweighted LOO.
VarGrad reduces variance in ELBO gradient estimation for variational inference.
problem Improving the variance of gradient estimators in variational inference.
method VarGrad uses a new log-variance loss to estimate the ELBO gradient, achieving lower variance than the score function method.
result VarGrad offers a lower variance gradient estimator compared to other methods.
Optimizes Lasso hyperparameters using leave-one-out CV.
problem Finding optimal hyperparameters for Lasso regression.
method Develops an algorithm to compute exact or approximate leave-one-out CV.
result Algorithm finds optimal hyperparameters for Lasso.
Algorithm identifies and corrects noisy labels using Gaussian process regression.
problem Detecting and correcting real-valued noisy labels from mixed data.
method Gaussian process regression with heteroscedastic noise model and leave-one-out cross-validation.
result The method can pinpoint corrupted sample points and improve regression models.
A new method improves robustness and efficiency of Bayesian LOO-CV.
problem Computational expense and unreliability of classical LOO-CV in high-dimensional Bayesian models.
method Proposes a mixture estimator to compute Bayesian LOO-CV criteria with finite asymptotic variance.
result Improved robustness and efficiency in high-dimensional problems.
Study improves understanding of non-differentiable penalties in high-dimensional settings.
problem Theoretical understanding of non-differentiable penalties like generalized LASSO and nuclear norm in high-dimensional settings.
method Proportional high-dimensional regime analysis with finite sample upper bounds on expected squared error.
result LO provides accurate estimation of out-of-sample risk in high-dimensional settings.
Study shows influence functions are poor for neural networks but useful for identifying influential examples.
problem Influence functions misalign with leave-one-out retraining in neural networks.
method Decomposed the discrepancy into five terms and studied their contributions across different architectures and datasets.
result Influence functions are a good approximation to the proximal Bregman response function (PBRF), useful for identifying influential examples.
Paper proposes diagnostics for error and variance estimation in randomized matrix computations.
problem Safe use of randomized matrix algorithms in applications.
method Leave-one-out error estimator and jackknife resampling method.
result Provides rapid diagnostics to assess quality of randomized matrix computations.
RandALO speeds up risk estimation for large datasets.
problem Estimating out-of-sample risk for large, high-dimensional models.
method RandALO: a randomized approximate leave-one-out estimator.
result RandALO is a computationally efficient risk estimator in high dimensions.
The paper analyzes LOCV for high-dimensional risk estimation, proving error bounds.
problem Estimating out-of-sample prediction error in high-dimensional settings.
method Theoretical analysis of leave-one-out cross validation (LOCV) in penalized regression.
result Finite sample upper bounds on LOCV error, showing it converges to zero as n,p → ∞.
A fast method for LOOCV in k-NN regression reduces computation time.
problem Efficient computation of LOOCV for k-NN regression.
method Identical LOOCV estimate to (k+1)-NN MSE on training data.
result LOOCV computation can be done with (k+1)-NN regression once.
A new method reduces variance in training discrete latent variable models.
problem High variance in stochastic gradient estimators for discrete latent variable models.
method Double control variates for score function estimators using Taylor expansions.
result Our method can have lower variance compared to other estimators.
The least squares Monte Carlo (LSM) algorithm proposed by Longstaff and Schwartz (2001) is widely used for pricing Bermudan options. The LSM estimator contains undesirable look-ahead bias, and the conventional technique of avoiding it requires additional simulation paths. We present the leave-one-out LSM (LOOLSM) algor…
Study examines influence diagnostics in high-dimensional M-estimation.
problem Understanding influence diagnostics in high-dimensional settings.
method Characterized the distribution of leave-one-out influences in high-dimensional Gaussian M-estimation.
result The distribution of influences converges to a limiting measure in high-dimensional settings.
UDM reparameterization improves language model generation.
problem Mismatch between UDM training objective and denoising posterior.
method Leave-one-out denoising and absorbing state reformulation.
result Improved UDM generation through leave-one-out parameterization.
Risk estimation is at the core of many learning systems. The importance of this problem has motivated researchers to propose different schemes, such as cross validation, generalized cross validation, and Bootstrap. The theoretical properties of such estimates have been extensively studied in the low-dimensional setting…
Paper optimizes estimation of quadratic functionals in nonparametric IV models.
problem Optimal estimation of a nonlinear functional in ill-posed inverse regression.
method Adaptive, minimax estimation using leave-one-out, sieve NPIV estimator with data-driven sieve dimension selection.
result Adaptive estimator achieves minimax optimal rate in various ill-posed cases.
A new KDE model prevents singular solutions and accelerates optimization for probabilistic modeling.
problem Adapting to varying densities in data regions for probabilistic modeling.
method Adaptive KDE model with individual bandwidths, LOO-MLL criterion, and modified EM algorithm.
result The proposed models prevent singular solutions and have promising performance.
Paper analyzes robust matrix completion with efficient nonconvex method and leave-one-out analysis.
problem Robust matrix completion with sparse noise.
method Alternates between projected gradient step for low-rank and thresholding step for sparse noise.
result Achieves linear convergence for general thresholding functions.
We propose and analyze estimators for statistical functionals of one or more distributions under nonparametric assumptions. Our estimators are based on the theory of influence functions, which appear in the semiparametric statistics literature. We show that estimators based either on data-splitting or a leave-one-out t…
The present paper provides a new generic strategy leading to non-asymptotic theoretical guarantees on the Leave-one-Out procedure applied to a broad class of learning algorithms. This strategy relies on two main ingredients: the new notion of Lq stability, and the strong use of moment inequalities. Lq stability e…
The future predictive performance of a Bayesian model can be estimated using Bayesian cross-validation. In this article, we consider Gaussian latent variable models where the integration over the latent values is approximated using the Laplace method or expectation propagation (EP). We study the properties of several B…
Enhances polynomial chaos models with uncertainty intervals.
problem Uncertainty quantification in surrogate models.
method Jackknife-based conformal prediction integrated into polynomial chaos expansions.
result Produces accurate predictive intervals for low-accuracy models.
LOO-StabCP speeds up CP for multiple predictions.
problem Balancing computational efficiency and prediction accuracy in CP.
method Leave-One-Out Stable Conformal Prediction (LOO-StabCP) using algorithmic stability.
result LOO-StabCP is faster and more accurate than RO-StabCP.
Paper introduces a method to explain deep learning models and identify good generalization.
problem Limited interpretability of neural networks hinders progress and real-world applications.
method Polytope interpolation method for local explainability and generalization assessment.
result Developed a method to identify deep learning models with good generalization properties.
Weighted SVM (or fuzzy SVM) is the most widely used SVM variant owning its effectiveness to the use of instance weights. Proper selection of the instance weights can lead to increased generalization performance. In this work, we extend the span error bound theory to weighted SVM and we introduce effective hyperparamete…
In this article, we derive concentration inequalities for the cross-validation estimate of the generalization error for stable predictors in the context of risk assessment. The notion of stability has been first introduced by \cite{DEWA79} and extended by \cite{KEA95}, \cite{BE01} and \cite{KUNIY02} to characterize cla…
Model inference, such as model comparison, model checking, and model selection, is an important part of model development. Leave-one-out cross-validation (LOO) is a general approach for assessing the generalizability of a model, but unfortunately, LOO does not scale well to large datasets. We propose a combination of u…
The paper introduces the Banzhaf value for robust data valuation in machine learning, addressing stochastic model performance.
problem Inconsistent data value rankings due to model performance noise.
method Introduces the Banzhaf value and Maximum Sample Reuse (MSR) principle for efficient estimation.
result The Banzhaf value outperforms other semivalues in robust data valuation.
Efficient PI for neural networks without distributional assumptions.
problem No distributional assumptions for efficient predictive inference.
method Differential privacy and linear approximation for leave-one-out models.
result Rigorous coverage guarantees with reduced computation.
Backward Conformal Prediction offers flexible control over prediction set sizes while ensuring coverage guarantees.
problem Providing reliable prediction sets with controlled sizes in applications like medical diagnosis.
method Defines a rule that constrains prediction set sizes based on observed data, adapting coverage levels.
result Maintains computable coverage guarantees while ensuring interpretable, well-controlled prediction set sizes.
Paper tackles robust matrix completion with heavy-tailed noise.
problem Estimating a low-rank matrix from noisy incomplete data.
method Adaptive Huber loss for robustness, nonconvex algorithm with spectral initialization.
result Achieves minimax-optimal statistical estimation error under bounded second moment condition.
New cross-validation methods for Gaussian process regression with efficient gradient computation.
problem Estimating parameters of Gaussian process covariance functions.
method Derive new cross-validation criteria and efficient adjoint computation of gradients.
result Efficient method for evaluating cross-validation criteria and their gradients.
New methods improve anomaly detection with reduced false positives.
problem Effective anomaly detection with controlled error rates.
method Leave-one-out-, bootstrap-, and cross-conformal anomaly detection methods.
result Improved anomaly detection with reduced false positives.
Consider the following class of learning schemes: β^:=argminβ∑j=1nℓ(xj⊤β;yj)+λR(β),(1) where xi∈Rp and yi∈R denote the ith feature and response variable …
Improves gradient estimation for discrete distributions with variance reduction techniques.
problem Excessive variance in gradient estimation for discrete distributions.
method Stein operators for discrete distributions and control variates.
result Substantially lower variance in gradient estimation.
Gradient descent with small initialization solves matrix completion without regularization.
problem Symmetric matrix completion from observed entries.
method Vanilla gradient descent with small initialization.
result GD converges to the ground truth matrix without regularization in over-parameterized scenario.
Deep learning methods have predominantly been applied to large artificial neural networks. Despite their state-of-the-art performance, these large networks typically do not generalize well to datasets with limited sample sizes. In this paper, we take a different approach by learning multiple layers of kernels. We combi…
We show how to adjust the coefficient of determination (R2) when used for measuring predictive accuracy via leave-one-out cross-validation.
In this article, we derive concentration inequalities for the cross-validation estimate of the generalization error for subagged estimators, both for classification and regressor. General loss functions and class of predictors with both finite and infinite VC-dimension are considered. We slightly generalize the formali…
CARMS improves gradient estimation for categorical variables.
problem Accurately backpropagating gradients through categorical variables.
method CARMS combines REINFORCE with antithetic sampling to create unbiased gradient estimators.
result CARMS outperforms competing methods on various tasks.
The study evaluates different parameter selection methods for Gaussian process interpolation.
problem Choosing optimal parameters for Gaussian process interpolation.
method Empirical study using scoring rules and leave-one-out selection criteria.
result The choice of model family is often more important than the selection criterion.