ALO-CV approximates leave-one-out error in proportional regime.
problem Estimating generalization error in high-dimensional settings.
method Developed new analysis for ALO-CV, showed consistency under strong convexity.
result ALO-CV approximates leave-one-out error up to negligible error.
A new method improves robustness and efficiency of Bayesian LOO-CV.
problem Computational expense and unreliability of classical LOO-CV in high-dimensional Bayesian models.
method Proposes a mixture estimator to compute Bayesian LOO-CV criteria with finite asymptotic variance.
result Improved robustness and efficiency in high-dimensional problems.
Bayesian LOO-CV method speeds up model evaluation for big data.
problem Inefficient LOO-CV for large datasets.
method Combining approximate inference and PPS sampling.
result Good properties for large data demonstrated.
RandALO speeds up risk estimation for large datasets.
problem Estimating out-of-sample risk for large, high-dimensional models.
method RandALO: a randomized approximate leave-one-out estimator.
result RandALO is a computationally efficient risk estimator in high dimensions.
Optimizes Lasso hyperparameters using leave-one-out CV.
problem Finding optimal hyperparameters for Lasso regression.
method Develops an algorithm to compute exact or approximate leave-one-out CV.
result Algorithm finds optimal hyperparameters for Lasso.
Develops an approximate CV formula for Bayesian linear regression.
problem Heavy computational cost of cross-validation in Bayesian linear regression.
method Develops an approximate formula for leave-one-out CV error without actual CV.
result Formula provides an efficient way to evaluate Bayesian linear regression models.
The paper proves LOO CV is reliable under estimator stability.
problem Ensuring the reliability of leave-one-out cross validation.
method Using concentration inequalities based on logarithmic Sobolev inequality.
result LOO CV is a valid procedure under estimator stability.
LOOCV is often useful for analyzing small, structured experimental designs.
problem The effectiveness of cross-validation in analyzing designed experiments.
method Empirical study comparing LOOCV and other model selection methods.
result LOOCV is often useful in the analysis of small, structured experimental designs.
This paper formalizes and compares different CV methods for estimating classifier performance.
problem Variations of cross-validation methods for estimating classifier performance are not well understood.
method Mathematical formalization and analysis of different CV methods, proving their properties and suggesting a smooth estimator.
result The repeated K-fold CV is the only smooth estimator, but it estimates both conditional and mean performance accurately. A new framework for quickly updating classifiers with minimal re-computation.
problem Efficiently updating classifiers with incremental data changes.
method Proposes a novel sensitivity analysis framework that provides bounds on the updated classifier without full re-optimization.
result The proposed framework can quickly provide tight bounds on the updated classifier with minimal computational cost.
A method to improve surrogate model accuracy using multiple fidelity models.
problem Efficiently combining models of varying accuracy and computational cost.
method Multifidelity Gaussian process models and leave-one-out cross-validation.
result Reduced LOO-CV error at the highest fidelity through adaptive learning.
A new method for approximating CV and bootstrap with higher-order infinitesimal jackknife.
problem Efficiently approximating cross-validation and bootstrap methods for machine learning.
method Higher-order infinitesimal jackknife (HOIJ) using Taylor series approximations and automatic differentiation.
result HOIJ provides higher-order accuracy and can be computed efficiently even in high dimensions.
This work improves confidence intervals for Cox model test error using nested CV.
problem Insufficient understanding of confidence intervals for cross-validation in Cox model.
method Generalized nested cross-validation to Cox proportional hazards model.
result Improved coverage of confidence intervals for Cox model test error.
Upper bounds for CV errors apply to lasso and other models.
problem Bounding CV errors for lasso and similar models.
method Rademacher complexity and Orlicz-Ψν norm. result Upper bounds are tight and stable for lasso.
In this paper we consider the problem of Gaussian process classifier (GPC) model selection with different Leave-One-Out (LOO) Cross Validation (CV) based optimization criteria and provide a practical algorithm using LOO predictive distributions with such criteria to select hyperparameters. Apart from the standard avera…
CV outperforms mean-variance for stock returns, minimizing risk and maximizing growth.
problem Traditional risk assessment methods underperform in stock market analysis.
method Derived new CV equation and used it to analyze stock performance.
result Stocks with low but positive CV grow exponentially, outperforming high-risk stocks.
We prove that for k≥5 there does not exist a continuous map ∂CV(Fk)→PCurr(Fk) that is either Out(Fk)-equivariant or Out(Fk)-anti-equivariant. Here ∂CV(Fk) is the "length-function" boundary of Culler-Vogtmann's Outer space CV(Fk), and PCurr(Fk) is the space of pr…
Cross-validation (CV) is often used to select the regularization parameter in high dimensional problems. However, when applied to the sparse modeling method Lasso, CV leads to models that are unstable in high-dimensions, and consequently not suited for reliable interpretation. In this paper, we propose a model-free cri…
Trajectory-wise CVs reduce variance in policy gradient methods.
problem High variance in estimating policy gradient estimates.
method Proposes trajectory-wise control variates to reduce variance without bias.
result Trajectory-wise CVs are optimal for variance reduction under reasonable assumptions.
New method approximates CV for model assessment and selection.
problem Efficient model assessment and selection with large number of folds.
method Approximates expensive refitting with a single Newton step warm-started from full training set optimizer.
result Uniform non-asymptotic, deterministic model assessment guarantees for approximate CV.
The paper analyzes cross-validation for correlated data and introduces a bias-corrected estimator.
problem Cross-validation with squared error loss assumes independent and identically distributed (i.i.d.) data, which is often violated in correlated data.
method The paper presents a criterion for standard CV suitability and introduces a bias-corrected estimator (CVc) for correlated data. result The bias-corrected estimator (CVc) yields an unbiased estimate of prediction error in settings where standard CV is invalid. New method approximates CV efficiently for large-scale problems.
problem High computational cost of standard CV in large-scale problems.
method Iterative first-order algorithm to approximate CV solution.
result Extends CV approximation guarantees to non-converged solutions.
CV inference can be invalid for relatively unstable model comparisons.
problem The validity of cross-validation for model comparison is questioned when models are relatively unstable.
method The study proves that simple, individually stable models can generate relatively unstable comparisons, invalidating CV inference.
result The Lasso and soft-thresholding generate relatively unstable comparisons, invalidating CV inferences.
A method to approximate cross-validation for manifold regularization in semi-supervised learning.
problem Efficiency of cross-validation in selecting hyper-parameters for manifold regularization.
method Using Bouligand influence function (BIF) for Taylor expansion to approximate cross-validation.
result The proposed method significantly improves efficiency with minimal time cost.
Geometric study of Outer Space using envelopes and geodesics.
problem Understanding the geometry and structure of Outer Space CVn. method Study of envelopes in the asymmetric Lipschitz metric of CVn. result For almost all pairs of points in CVn, their envelopes have dimension 3n−4. New CVs preserve transition rates in molecular dynamics.
problem Designing CVs that accurately capture rare events in high-dimensional systems.
method Integrating manifold learning and group-invariant featurization to construct neural network-based CVs that satisfy orthogonality conditions.
result Achieved a CV for butane that reproduces the anti-gauche transition rate with less than ten percent relative error.
Abstract reformulates DT theory using Frobenius and CV structures, proving convergence.
problem Formal families of Frobenius type and CV structures in DT theory.
method Abstract study of Frobenius and CV structures, proving convergence.
result Proves convergence relevant to DT theory.
New method learns collective variables using autoencoders for molecular simulations.
problem Learning low-dimensional slow degrees of freedom (collective variables) for molecular simulations.
method Iterative method involving CV learning with autoencoders and reweighting scheme.
result Achieves convergence of learned collective variables.
Stochastic GD converges linearly for CV@R learning under certain conditions.
problem Optimizing CV@R in statistical learning with non-convex loss functions.
method Stochastic Gradient Descent with Polyak-Łojasiewicz condition.
result Stochastic GD achieves linear convergence for CV@R learning.
Optimizes hyperparameter tuning for models using approximate leave-one-out cross-validation.
problem Finding optimal hyperparameters for regularized models using approximate leave-one-out cross-validation.
method Derive efficient formulas for gradient and hessian of approximate leave-one-out cross-validation, apply second-order optimization.
result Demonstrates the effectiveness of the approach on real-world data sets.
Careful tuning of a regularization parameter is indispensable in many machine learning tasks because it has a significant impact on generalization performances. Nevertheless, current practice of regularization parameter tuning is more of an art than a science, e.g., it is hard to tell how many grid-points would be need…
Bayesian models discover CVs for complex systems, enhancing sampling methods.
problem Limitations in modeling complex systems in biochemistry and materials science.
method Formulated CV discovery as a Bayesian inference problem, using deep learning and variational inference.
result Discovered CVs improve predictive ability for alanine dipeptide and ALA-15 peptides.
Study finds unsupervised imputation before cross-validation can reduce computational costs without significantly degrading model performance.
problem High computational costs in pipeline modeling algorithms with imputation steps.
method Empirical assessment of unsupervised imputation before vs during cross-validation.
result Reduced variance of imputation before cross-validation leads to lower overall root mean squared error.
Meta-CVs leverage task similarity to reduce variance with limited data.
problem Reducing variance in Monte Carlo estimators with few samples.
method Meta-learning control variates for related tasks.
result Meta-CVs lead to significant variance reduction in settings with limited data.
New stability bounds improve Leave-One-Out performance guarantees.
problem Improving theoretical guarantees for Leave-One-Out procedure.
method Introducing Lq stability and using moment inequalities. result New PAC exponential generalisation bounds for Leave-One-Out.
A new method reduces CV computation time for big data.
problem Efficiently estimating model performance and tuning parameters.
method A general approach for reducing CV computation time in incremental learning.
result Running time scales logarithmically with the number of CV folds.
New CV method reduces bias in spatial prediction models.
problem Bias in standard cross-validation due to uneven sampling.
method Target-Weighted Cross-Validation (TWCV) framework.
result Weighted CV approaches reduce bias in prediction error.
For the free group FN of finite rank N≥2 we construct a canonical Bonahon-type continuous and Out(FN)-invariant \emph{geometric intersection form} \[ <, >: \bar{cv}(F_N)\times Curr(F_N)\to \mathbb R_{\ge 0}. \] Here cvˉ(FN) is the closure of unprojectivized Culler-Vogtmann's Outer space cv(FN)…
A method for efficient CV estimates in Bayesian hierarchical models.
problem Computational infeasibility of cross-validation in Bayesian hierarchical regression models.
method Conditioning on variance-covariance parameters to transform CV into an optimization problem.
result Equivalent or improved predictive estimates compared to full cross-validation.
Ridge regression CV loss may have multiple local optima.
problem Can we globally optimize cross-validation loss in ridge regression?
method Analyzing quasiconvexity of CV loss in ridge regression.
result CV loss may fail to be quasiconvex and have multiple local optima.
The paper analyzes the risk of CV-tuned regularized estimators and connects it to SURE.
problem Understanding the risk of CV-tuned regularized estimators.
method Derives asymptotic risk function of CV-tuned estimators and connects it to SURE.
result The risk function provides a more detailed picture of predictive performance than uniform bounds.
Solves the initial CV problem for molecular simulations using machine learning.
problem Selecting appropriate collective variables for enhancing sampling in molecular simulations.
method Data-driven approach inspired by supervised machine learning (SML).
result Various SML algorithms can be used as initial collective variables (SML_cv) for accelerated sampling.
GPMI method interpolates uncertain atrial conduction velocity on non-Euclidean manifolds.
problem Uncertainty in atrial conduction velocity calculations.
method Gaussian Process Manifold Interpolation (GPMI) on human atrial manifolds.
result GPMI accounts for atrial topology and calculates CV uncertainty.
Study the boundary of a space related to Outer space.
problem Analyze the structure of the boundary of a specific space.
method Equivariant deformation retract, proper and cocompact action, homotopy equivalence.
result The boundary is homotopy equivalent to the subcomplex of vertices at infinity.
We consider the class of risk measures associated with optimized certainty equivalents. This class includes several popular examples, such as CV@R and monotone mean-variance. Numerical schemes are developed for the computation of these risk measures using Fourier transform methods. This leads, in particular, to a very …
Autoencoders discover and accelerate molecular dynamics simulations.
problem Efficient sampling of macromolecular folding landscapes with high free energy barriers.
method Employing auto-associative artificial neural networks to learn nonlinear collective variables (CVs) that are explicit and differentiable functions of atomic coordinates.
result Substantial speedups in exploration of configurational space and discovery of data-driven CVs.
Paper derives new formula for CV and WAIC to find optimal hyperparameters.
problem Finding optimal prior in Bayesian estimation.
method Derives new formula clarifying the relation among CV, WAIC, and generalization loss.
result Optimal hyperparameters minimize average generalization loss asymptotically.
A \emph{geodesic current} on a free group F is an F-invariant measure on the set ∂2F of pairs of distinct points of ∂F. The space of geodesic currents on F is a natural companion of Culler-Vogtmann's Outer space cv(F) and studying them together yields new information about both spaces as we…