New method stops experiments early for harm in diverse groups.
problem Early stopping of experiments for harmful treatment effects in diverse populations.
method Causal machine learning approach (CLASH) for early stopping.
result CLASH effectively stops experiments early for harmful treatment effects in diverse groups.
Early stopping improves logistic regression's calibration and consistency in high dimensions.
problem Improving the statistical performance of gradient descent in overparameterized logistic regression.
method Investigates the effects of early stopping on gradient descent in logistic regression.
result Early-stopped gradient descent is well-calibrated and statistically consistent, while asymptotic gradient descent is not.
This paper improves neural network predictions with early stopping using conformal calibration.
problem Lack of precise statistical guarantees for neural networks trained with early stopping.
method Conformalized early stopping that combines early stopping with conformal calibration.
result Models provide both accuracy and precise inferences without additional data splits.
The study reveals optimal early stopping behaviors in deep learning models.
problem Understanding optimal early stopping in deep learning models.
method Theoretical analysis of linear models and experimental validation.
result Two distinct behaviors of optimal early stopping time depending on model dimension relative to dataset features.
Gradient descent with early stopping achieves optimal sparse recovery.
problem Sparse regression with gradient descent and early stopping.
method Gradient descent on depth-N networks with early stopping.
result Implicit sparse regularization occurs with early stopping for general depth N.
This work bounds the run-time of nonconvex optimization with early stopping.
problem Bounding the expected run-time of nonconvex optimization with early stopping.
method Derives conditions for well-defined early stopping based on validation function norms and bounds the expected number of iterations and gradient evaluations.
result Guarantees the validity of early stopping and provides bounds on the expected run-time for various optimization algorithms.
Early stopping methods reduce unnecessary reasoning steps in LLMs by monitoring uncertainty signals.
problem LLMs sometimes generate unnecessary reasoning steps, especially under uncertainty.
method Statistically principled early stopping methods that monitor uncertainty signals during generation.
result Uncertainty-aware early stopping improves efficiency and reliability in LLM reasoning, especially in math reasoning.
The paper analyzes early stopping in linear regression and shows it's equivalent to ridge regularization.
problem Understanding the effect of early stopping on linear regression models.
method Characterization of gradient descent dynamics and analysis of excess risk.
result Early stopped solution is equivalent to minimum norm solution for a generalized ridge regularized problem.
The paper studies early stopping methods in linear contextual bandits.
problem Minimizing in-experiment regret and conducting robust post-experiment inferences in contextual bandits.
method The study proposes early stopping rules based on the Opportunity Cost and Threshold Method, using variances of estimators to quantify upper regret bounds.
result The proposed method provides a systematic approach to minimize in-experiment regret and conduct robust post-experiment inferences.
Early stopping improves sample quality in latent diffusion models.
problem Latent diffusion models degrade sample quality with conventional early stopping.
method Analyzed the interaction between latent dimension and stopping time under Gaussian framework.
result Lower-dimensional representations benefit from earlier termination, higher-dimensional spaces require later stopping.
The strategy of early stopping is a regularization technique based on choosing a stopping time for an iterative algorithm. Focusing on non-parametric regression in a reproducing kernel Hilbert space, we analyze the early stopping strategy for a form of gradient-descent applied to the least-squares loss function. We pro…
Early stopping method saves up to 75% computation time in policy search tasks.
problem Lengthy evaluation times in optimization problems, especially in robotics.
method A generalized early stopping criterion that only uses objective value at each time step.
result The method saves up to 75% computation time compared to no stopping.
Early stopping of iterative algorithms is an algorithmic regularization method to avoid over-fitting in estimation and classification. In this paper, we show that early stopping can also be applied to obtain the minimax optimal testing in a general non-parametric setup. Specifically, a Wald-type test statistic is obtai…
Early stopping improves neural networks' performance on binary classification tasks.
problem Improving shallow ReLU networks' performance on binary classification tasks.
method Gradient descent with early stopping on binary classification data.
result Gradient descent with early stopping achieves population risk arbitrarily close to optimal.
Early stopping is a well known approach to reduce the time complexity for performing training and model selection of large scale learning machines. On the other hand, memory/space (rather than time) complexity is the main constraint in many applications, and randomized subsampling techniques have been proposed to tackl…
Early stopping in meta-learning improved by analyzing neural activation patterns.
problem Early stopping in few-shot learning is challenging due to distributional shifts between meta-validation and meta-test sets.
method Activation-Based Early-stopping (ABE) analyzes hidden layer activations from unlabelled support examples to detect when target generalization diverges from source data.
result Simple activation statistics can effectively estimate target generalization, improving few-shot transfer learning across various algorithms and datasets.
Paper studies early-stopped mirror descent for noisy sparse phase retrieval.
problem Recovering a sparse signal from noisy quadratic measurements.
method Early-stopped mirror descent with hyperbolic entropy mirror map.
result Achieves nearly minimax-optimal rate of convergence for k-sparse signals. ScoreStop uses gradient tests to stop gradient boosting early.
problem Overfitting in gradient boosted decision trees.
method ScoreStop uses a functional score test based on gradients to stop boosting.
result ScoreStop is competitive with loss-based early stopping methods.
Paper proposes a method for early stopping in regression using reproducing kernels.
problem Early stopping for iterative learning algorithms in nonparametric regression.
method Data-driven rule based on minimum discrepancy principle, validated by fixed-point analysis of localized Rademacher complexities.
result The proposed rule is minimax-optimal and performs comparably to cross-validation.
Analysis of cross-validation for early-stopped gradient descent in high-dimensional regression.
problem Inconsistency of GCV for early-stopped GD in high-dimensional least squares regression.
method Theoretical analysis of GCV and LOOCV applied to early-stopped GD in high-dimensional least squares regression.
result LOOCV converges uniformly to the prediction risk of early-stopped GD, while GCV is generically inconsistent.
Sharp risk bounds for early-stopping in Gaussian linear regression are derived.
problem Minimizing in-sample mean squared error in high-dimensional Gaussian linear regression.
method Early-stopped mirror descent (ESMD) with local Gaussian width bounds.
result Sharp risk bounds extend to early-stopped mirror descent for least squares estimator (LSE).
Noisy labels are very common in real-world training data, which lead to poor generalization on test data because of overfitting to the noisy labels. In this paper, we claim that such overfitting can be avoided by "early stopping" training a deep neural network before the noisy labels are severely memorized. Then, we re…
In this paper, we propose an adaptive stopping rule for kernel-based gradient descent (KGD) algorithms. We introduce the empirical effective dimension to quantify the increments of iterations in KGD and derive an implementable early stopping strategy. We analyze the performance of the adaptive stopping rule in the fram…
SAM minimizes loss sharpness, improving adversarial transferability.
problem Improving adversarial transferability of deep neural networks.
method Evaluating surrogate models trained with seven minimizers, focusing on loss sharpness and flat neighborhoods.
result SAM minimizes loss sharpness, leading to better adversarial transferability.
Study robustness of early-stopping GD for linear regression attacks.
problem Robustness of gradient-descent methods to adversarial attacks.
method Early-stopping strategies, gradient-descent, Mahalanobis attacks, feature-dependent learning rates, data transformations.
result Early-stopped GD is optimally robust to Euclidean-norm attacks but sub-optimal for Mahalanobis attacks.
Improves early stopping in deep networks by adjusting stepsizes.
problem Epoch-wise double descent in deep networks.
method Analytical and empirical study of bias-variance tradeoffs in different network layers.
result Eliminating epoch-wise double descent through adjusting stepsizes of different layers improves early stopping performance.
Early stopping improves generalization in overparameterized diffusion models.
problem Understanding and optimizing generalization in overparameterized diffusion models.
method Revisiting diffusion models, showing generalization occurs before memorization, and developing a phase diagram.
result Generalization time scales with dataset size, supporting early-stopping criteria.
Early stopping is a widely used technique to prevent poor generalization performance when training an over-expressive model by means of gradient-based optimization. To find a good point to halt the optimizer, a common practice is to split the dataset into a training and a smaller validation set to obtain an ongoing est…
Early-stopped aggregation improves computational efficiency in adaptive statistical inference.
problem Computational inefficiency in model selection and aggregation for adaptive inference.
method Early-stopped aggregation (ESA) framework: compute only a few simpler models using early-stopping criteria.
result ESA achieves optimal adaptive contraction rates in variational Bayes and frequentist settings.
A new method for early stopping in neural networks without validation sets.
problem Determining when to stop training neural networks to avoid overfitting.
method Channel-wise DeepNNK (CW-DeepNNK) using non-negative kernel regression and polytope interpolation.
result The proposed early stopping criterion based on CW-DeepNNK performs better than standard validation-based methods.
MGD with early stopping tends to ridge regularization in least squares regression.
problem Characterizing the implicit regularization of MGD with early stopping.
method Continuous-time view of MGD (momentum gradient flow) and comparison with explicit ridge regularization.
result Under optimal tuning, the risk of MGF is no more than 1.54 times that of ridge.
Gradient boosting improved with lassoed trees achieves faster convergence.
problem Improving gradient boosting convergence in large nonparametric spaces.
method Lassoed gradient boosted trees with early stopping.
result Achieves faster than n−1/4 L2 convergence rate. Paper introduces a method to control early classification accuracy gaps.
problem Maintaining accuracy in early classification without full input processing.
method Statistical framework for a calibrated stopping rule.
result Reduces up to 94% of timesteps while controlling accuracy gaps.
New method estimates variable importance for large models efficiently.
problem Estimating variable importance for large, opaque models is computationally challenging.
method Combining early stopping and warm-start techniques for scalable variable importance estimation.
result The method provides theoretical guarantees and demonstrates improved accuracy and computational efficiency.
Study mirrors descent's early stopping for linear and kernel models, improving risk guarantees.
problem Understanding the statistical performance of early-stopped mirror descent algorithms.
method Characterized convexity of squared loss, identified link between offset Rademacher complexities and mirror descent convergence.
result Excess risk guarantees for mirror descent iterates traced by the path, expressed in terms of offset complexities.
The research proposes a stopping rule for reinforcement learning algorithms based on instance-dependent confidence.
problem Dramatic variation in convergence rates of reinforcement learning algorithms due to problem structure.
method Develops instance-dependent confidence regions and a data-dependent stopping rule for MDP policy evaluation and optimal value estimation.
result Proposes a stopping rule that adapts to the instance-specific difficulty of the problem, allowing for early termination.
We pursue an early stopping technique that helps Gaussian Restricted Boltzmann Machines (GRBMs) to gain good natural image representations in terms of overcompleteness and data fitting. GRBMs are widely considered as an unsuitable model for natural images because they gain non-overcomplete representations which include…
Distillation is a method to transfer knowledge from one model to another and often achieves higher accuracy with the same capacity. In this paper, we aim to provide a theoretical understanding on what mainly helps with the distillation. Our answer is "early stopping". Assuming that the teacher network is overparameteri…
In this work, we introduce a recently developed early classification mechanism to satellite-based agricultural monitoring. It augments existing classification models by an additional stopping probability based on the previously seen information. This mechanism is end-to-end trainable and derives its stopping decision s…
When a feed-forward neural network (FNN) is trained for source ranging in an ocean waveguide, it is difficult evaluating the range accuracy of the FNN on unlabeled test data. A fitting-based early stopping (FEAST) method is introduced to evaluate the range error of the FNN on test data where the distance of source is u…
Early stopping of iterative algorithms is a widely-used form of regularization in statistics, commonly used in conjunction with boosting and related gradient-type algorithms. Although consistency results have been established in some settings, such estimators are less well-understood than their analogues based on penal…
GD-trained shallow ReLU nets learn Lipschitz functions with noise.
problem Learning Lipschitz functions with additive noise in overparameterized neural networks.
method Gradient Descent (GD) with early stopping, focusing on the Neural Tangent Kernel (NTK).
result Early-stopped GD achieves minimax optimal rates for learning Lipschitz functions.
GD with early stopping trains shallow neural nets for nonparametric regression robustly.
problem Learning Lipschitz regression functions with noisy labels.
method Overparameterized shallow neural networks trained by GD with early stopping.
result Optimal rates of convergence for nonparametric regression.
A new tree-based model for varying coefficients using CGBM.
problem Modeling varying coefficients with high dimensionality and complex interactions.
method Tree-based varying coefficient model with CGBM for varying coefficients, dimension-wise early stopping, and feature importance scores.
result The model produces comparable out-of-sample loss to neural networks, demonstrating effectiveness.
Random matrix theory explains transient signal detectability in early-stopped gradient flow.
problem Transient signal detectability in early-stopped gradient flow.
method Random matrix theory applied to gradient flow in a linear teacher-student setting.
result Transient Baik-Ben Arous-Péché (BBP) transition in learning dynamics due to anisotropy and noise.
Adversarial training overfits, harming robustness; early stopping fixes this.
problem Adversarial robustness in deep learning overfits to training data.
method Empirical study of adversarially trained deep networks.
result Overfitting to training data harms robust performance in adversarial training.
Nowadays many financial derivatives, such as American or Bermudan options, are of early exercise type. Often the pricing of early exercise options gives rise to high-dimensional optimal stopping problems, since the dimension corresponds to the number of underlying assets. High-dimensional optimal stopping problems are,…
The paper speeds up hyperparameter optimisation in Gaussian processes.
problem Scaling hyperparameter optimisation to large datasets.
method Improvements to linear system solvers (pathwise gradient, warm starting, early stopping).
result Speed-ups of up to 72x and residual norm decreases of up to 7x.