ALO-CV approximates leave-one-out error in proportional regime.
problem Estimating generalization error in high-dimensional settings.
method Developed new analysis for ALO-CV, showed consistency under strong convexity.
result ALO-CV approximates leave-one-out error up to negligible error.
A new method improves robustness and efficiency of Bayesian LOO-CV.
problem Computational expense and unreliability of classical LOO-CV in high-dimensional Bayesian models.
method Proposes a mixture estimator to compute Bayesian LOO-CV criteria with finite asymptotic variance.
result Improved robustness and efficiency in high-dimensional problems.
Bayesian LOO-CV method speeds up model evaluation for big data.
problem Inefficient LOO-CV for large datasets.
method Combining approximate inference and PPS sampling.
result Good properties for large data demonstrated.
RandALO speeds up risk estimation for large datasets.
problem Estimating out-of-sample risk for large, high-dimensional models.
method RandALO: a randomized approximate leave-one-out estimator.
result RandALO is a computationally efficient risk estimator in high dimensions.
Optimizes Lasso hyperparameters using leave-one-out CV.
problem Finding optimal hyperparameters for Lasso regression.
method Develops an algorithm to compute exact or approximate leave-one-out CV.
result Algorithm finds optimal hyperparameters for Lasso.
Cross-validation (CV) is a technique for evaluating the ability of statistical models/learning systems based on a given data set. Despite its wide applicability, the rather heavy computational cost can prevent its use as the system size grows. To resolve this difficulty in the case of Bayesian linear regression, we dev…
The paper proves LOO CV is reliable under estimator stability.
problem Ensuring the reliability of leave-one-out cross validation.
method Using concentration inequalities based on logarithmic Sobolev inequality.
result LOO CV is a valid procedure under estimator stability.
LOOCV is often useful for analyzing small, structured experimental designs.
problem The effectiveness of cross-validation in analyzing designed experiments.
method Empirical study comparing LOOCV and other model selection methods.
result LOOCV is often useful in the analysis of small, structured experimental designs.
This paper formalizes and compares different CV methods for estimating classifier performance.
problem Variations of cross-validation methods for estimating classifier performance are not well understood.
method Mathematical formalization and analysis of different CV methods, proving their properties and suggesting a smooth estimator.
result The repeated K-fold CV is the only smooth estimator, but it estimates both conditional and mean performance accurately. A method to improve surrogate model accuracy using multiple fidelity models.
problem Efficiently combining models of varying accuracy and computational cost.
method Multifidelity Gaussian process models and leave-one-out cross-validation.
result Reduced LOO-CV error at the highest fidelity through adaptive learning.
A new method for approximating CV and bootstrap with higher-order infinitesimal jackknife.
problem Efficiently approximating cross-validation and bootstrap methods for machine learning.
method Higher-order infinitesimal jackknife (HOIJ) using Taylor series approximations and automatic differentiation.
result HOIJ provides higher-order accuracy and can be computed efficiently even in high dimensions.
This work improves confidence intervals for Cox model test error using nested CV.
problem Insufficient understanding of confidence intervals for cross-validation in Cox model.
method Generalized nested cross-validation to Cox proportional hazards model.
result Improved coverage of confidence intervals for Cox model test error.
Upper bounds for CV errors apply to lasso and other models.
problem Bounding CV errors for lasso and similar models.
method Rademacher complexity and Orlicz-Ψν norm. result Upper bounds are tight and stable for lasso.
In this paper we consider the problem of Gaussian process classifier (GPC) model selection with different Leave-One-Out (LOO) Cross Validation (CV) based optimization criteria and provide a practical algorithm using LOO predictive distributions with such criteria to select hyperparameters. Apart from the standard avera…
CV outperforms mean-variance for stock returns, minimizing risk and maximizing growth.
problem Traditional risk assessment methods underperform in stock market analysis.
method Derived new CV equation and used it to analyze stock performance.
result Stocks with low but positive CV grow exponentially, outperforming high-risk stocks.
We prove that for k≥5 there does not exist a continuous map ∂CV(Fk)→PCurr(Fk) that is either Out(Fk)-equivariant or Out(Fk)-anti-equivariant. Here ∂CV(Fk) is the "length-function" boundary of Culler-Vogtmann's Outer space CV(Fk), and PCurr(Fk) is the space of pr…
Cross-validation (CV) is often used to select the regularization parameter in high dimensional problems. However, when applied to the sparse modeling method Lasso, CV leads to models that are unstable in high-dimensions, and consequently not suited for reliable interpretation. In this paper, we propose a model-free cri…
Trajectory-wise CVs reduce variance in policy gradient methods.
problem High variance in estimating policy gradient estimates.
method Proposes trajectory-wise control variates to reduce variance without bias.
result Trajectory-wise CVs are optimal for variance reduction under reasonable assumptions.
New method approximates CV for model assessment and selection.
problem Efficient model assessment and selection with large number of folds.
method Approximates expensive refitting with a single Newton step warm-started from full training set optimizer.
result Uniform non-asymptotic, deterministic model assessment guarantees for approximate CV.
The paper analyzes cross-validation for correlated data and introduces a bias-corrected estimator.
problem Cross-validation with squared error loss assumes independent and identically distributed (i.i.d.) data, which is often violated in correlated data.
method The paper presents a criterion for standard CV suitability and introduces a bias-corrected estimator (CVc) for correlated data. result The bias-corrected estimator (CVc) yields an unbiased estimate of prediction error in settings where standard CV is invalid. New method approximates CV efficiently for large-scale problems.
problem High computational cost of standard CV in large-scale problems.
method Iterative first-order algorithm to approximate CV solution.
result Extends CV approximation guarantees to non-converged solutions.
CV inference can be invalid for relatively unstable model comparisons.
problem The validity of cross-validation for model comparison is questioned when models are relatively unstable.
method The study proves that simple, individually stable models can generate relatively unstable comparisons, invalidating CV inference.
result The Lasso and soft-thresholding generate relatively unstable comparisons, invalidating CV inferences.
A method to approximate cross-validation for manifold regularization in semi-supervised learning.
problem Efficiency of cross-validation in selecting hyper-parameters for manifold regularization.
method Using Bouligand influence function (BIF) for Taylor expansion to approximate cross-validation.
result The proposed method significantly improves efficiency with minimal time cost.
Geometric study of Outer Space using envelopes and geodesics.
problem Understanding the geometry and structure of Outer Space CVn. method Study of envelopes in the asymmetric Lipschitz metric of CVn. result For almost all pairs of points in CVn, their envelopes have dimension 3n−4. New CVs preserve transition rates in molecular dynamics.
problem Designing CVs that accurately capture rare events in high-dimensional systems.
method Integrating manifold learning and group-invariant featurization to construct neural network-based CVs that satisfy orthogonality conditions.
result Achieved a CV for butane that reproduces the anti-gauche transition rate with less than ten percent relative error.
New method learns collective variables using autoencoders for molecular simulations.
problem Learning low-dimensional slow degrees of freedom (collective variables) for molecular simulations.
method Iterative method involving CV learning with autoencoders and reweighting scheme.
result Achieves convergence of learned collective variables.
Optimizes hyperparameter tuning for models using approximate leave-one-out cross-validation.
problem Finding optimal hyperparameters for regularized models using approximate leave-one-out cross-validation.
method Derive efficient formulas for gradient and hessian of approximate leave-one-out cross-validation, apply second-order optimization.
result Demonstrates the effectiveness of the approach on real-world data sets.
Stochastic GD converges linearly for CV@R learning under certain conditions.
problem Optimizing CV@R in statistical learning with non-convex loss functions.
method Stochastic Gradient Descent with Polyak-Łojasiewicz condition.
result Stochastic GD achieves linear convergence for CV@R learning.
Careful tuning of a regularization parameter is indispensable in many machine learning tasks because it has a significant impact on generalization performances. Nevertheless, current practice of regularization parameter tuning is more of an art than a science, e.g., it is hard to tell how many grid-points would be need…
Bayesian models discover CVs for complex systems, enhancing sampling methods.
problem Limitations in modeling complex systems in biochemistry and materials science.
method Formulated CV discovery as a Bayesian inference problem, using deep learning and variational inference.
result Discovered CVs improve predictive ability for alanine dipeptide and ALA-15 peptides.
Study finds unsupervised imputation before cross-validation can reduce computational costs without significantly degrading model performance.
problem High computational costs in pipeline modeling algorithms with imputation steps.
method Empirical assessment of unsupervised imputation before vs during cross-validation.
result Reduced variance of imputation before cross-validation leads to lower overall root mean squared error.
Meta-CVs leverage task similarity to reduce variance with limited data.
problem Reducing variance in Monte Carlo estimators with few samples.
method Meta-learning control variates for related tasks.
result Meta-CVs lead to significant variance reduction in settings with limited data.
For the free group FN of finite rank N≥2 we construct a canonical Bonahon-type continuous and Out(FN)-invariant \emph{geometric intersection form} \[ <, >: \bar{cv}(F_N)\times Curr(F_N)\to \mathbb R_{\ge 0}. \] Here cvˉ(FN) is the closure of unprojectivized Culler-Vogtmann's Outer space cv(FN)…
New CV method reduces bias in spatial prediction models.
problem Bias in standard cross-validation due to uneven sampling.
method Target-Weighted Cross-Validation (TWCV) framework.
result Weighted CV approaches reduce bias in prediction error.
A method for efficient CV estimates in Bayesian hierarchical models.
problem Computational infeasibility of cross-validation in Bayesian hierarchical regression models.
method Conditioning on variance-covariance parameters to transform CV into an optimization problem.
result Equivalent or improved predictive estimates compared to full cross-validation.
Macromolecular and biomolecular folding landscapes typically contain high free energy barriers that impede efficient sampling of configurational space by standard molecular dynamics simulation. Biased sampling can artificially drive the simulation along pre-specified collective variables (CVs), but success depends crit…
Cross-validation (CV) is one of the main tools for performance estimation and parameter tuning in machine learning. The general recipe for computing CV estimate is to run a learning algorithm separately for each CV fold, a computationally expensive process. In this paper, we propose a new approach to reduce the computa…
Ridge regression CV loss may have multiple local optima.
problem Can we globally optimize cross-validation loss in ridge regression?
method Analyzing quasiconvexity of CV loss in ridge regression.
result CV loss may fail to be quasiconvex and have multiple local optima.
The paper analyzes the risk of CV-tuned regularized estimators and connects it to SURE.
problem Understanding the risk of CV-tuned regularized estimators.
method Derives asymptotic risk function of CV-tuned estimators and connects it to SURE.
result The risk function provides a more detailed picture of predictive performance than uniform bounds.
Study the boundary of a space related to Outer space.
problem Analyze the structure of the boundary of a specific space.
method Equivariant deformation retract, proper and cocompact action, homotopy equivalence.
result The boundary is homotopy equivalent to the subcomplex of vertices at infinity.
GPMI method interpolates uncertain atrial conduction velocity on non-Euclidean manifolds.
problem Uncertainty in atrial conduction velocity calculations.
method Gaussian Process Manifold Interpolation (GPMI) on human atrial manifolds.
result GPMI accounts for atrial topology and calculates CV uncertainty.
We define a new compactification of outer space CVN (the \emph{Pacman compactification}) which is an absolute retract, for which the boundary is a Z-set. The classical compactification CVN made of very small FN-actions on R-trees, however, fails to be locally 4-connected as soon as $N…
We consider the class of risk measures associated with optimized certainty equivalents. This class includes several popular examples, such as CV@R and monotone mean-variance. Numerical schemes are developed for the computation of these risk measures using Fourier transform methods. This leads, in particular, to a very …
Paper accelerates conformal prediction by using approximate leave-one-out estimators.
problem Limited computational cost for conformal prediction.
method Incorporates approximate leave-one-out estimators to accelerate conformal prediction.
result ALO-based methods achieve comparable coverage and efficiency to exact methods but with significantly reduced runtime.
A \emph{geodesic current} on a free group F is an F-invariant measure on the set ∂2F of pairs of distinct points of ∂F. The space of geodesic currents on F is a natural companion of Culler-Vogtmann's Outer space cv(F) and studying them together yields new information about both spaces as we…
UDM reparameterization improves language model generation.
problem Mismatch between UDM training objective and denoising posterior.
method Leave-one-out denoising and absorbing state reformulation.
result Improved UDM generation through leave-one-out parameterization.
Improved traffic flow prediction model using Kalman filter noise reduction.
problem Low accuracy in predicting traffic flow parameters due to limited connected vehicle data.
method Combined LSTM with Kalman filter-based RTS noise reduction.
result Reduced prediction errors by 50-70% for speed and space headway.
Selection of appropriate collective variables for enhancing sampling of molecular simulations remains an unsolved problem in computational biophysics. In particular, picking initial collective variables (CVs) is particularly challenging in higher dimensions. Which atomic coordinates or transforms there of from a list o…