Out-of-distribution (OOD) detection approaches usually present special requirements (e.g., hyperparameter validation, collection of outlier data) and produce side effects (e.g., classification accuracy drop, slower energy-inefficient inferences). We argue that these issues are a consequence of the SoftMax loss anisotro…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “validation loss”
Paper establishes a formula linking model performance to insurance loss ratio.
problem Improving model performance does not always lead to proportional improvements in loss ratio.
method Derives a closed-form formula connecting Pearson correlation to expected loss ratio.
result Model improvements have diminishing marginal returns in reducing loss ratio.
Paper improves PAC-Bayes bounds for various loss types.
problem Improving PAC-Bayes bounds for different types of losses.
method Introducing new high-probability PAC-Bayes bounds for bounded and general tail behaviors losses, and extending to anytime-valid bounds.
result New fast-rate and mixed-rate bounds for losses with bounded ranges, and parameter-free bounds for losses with general tail behaviors.
ScoreStop uses gradient tests to stop gradient boosting early.
problem Overfitting in gradient boosted decision trees.
method ScoreStop uses a functional score test based on gradients to stop boosting.
result ScoreStop is competitive with loss-based early stopping methods.
Ridge regression CV loss may have multiple local optima.
problem Can we globally optimize cross-validation loss in ridge regression?
method Analyzing quasiconvexity of CV loss in ridge regression.
result CV loss may fail to be quasiconvex and have multiple local optima.
This paper explores how train-validation splits help in NAS to prevent overfitting.
problem NAS overfits with train-validation splits and needs better generalization guarantees.
method Established refined properties of validation loss and risk for NAS.
result NAS with train-validation splits can select the most generalizable model.
New criterion assesses cluster separability for validation.
problem Validating cluster analysis results and determining the number of clusters.
method Distinguishability criterion, combined loss function-based framework.
result Validated cluster configurations and determined the number of clusters.
A new method selects training samples for fine-tuning using validation set inference.
problem Selecting training examples for fine-tuning with limited target data.
method Invert train-validation roles; select samples affecting most predictions.
result Our method achieves lower test log-loss than state-of-the-art approaches.
New method estimates grouping loss in neural networks to improve confidence scores.
problem Improving confidence scores in neural networks to reflect true posterior probabilities.
method Proposed an estimator to approximate the grouping loss.
result Modern neural networks exhibit grouping loss, especially in distribution shifts.
A new framework for time series forecasting that adapts to varying patterns.
problem Forecasting multivariate time series with predictive heterogeneity.
method Validation-driven clustering framework that applies specialization based on out-of-sample predictive performance.
result Improves robustness to heavy-tailed errors and local anomalies.
MD tree diagnoses model failures using loss landscape metrics.
problem Diagnose model failures without knowing training configuration.
method MD tree based on loss landscape metrics.
result MD tree achieves 87.7% accuracy in dataset transfer tasks, outperforming validation-based approaches.
Improved GP models for fast training and good performance.
problem Training scalable Gaussian process models efficiently.
method Cross-validation and nearest neighbor truncation for scalable GP training.
result Our method offers fast training and excellent predictive performance.
PAC-Bayesian bounds for MLPs with cross entropy loss validated.
problem Generalization bounds for MLPs with cross entropy loss.
method Introduced probabilistic explanations and proved PAC-Bayesian bounds using ELBO.
result MLPs with cross entropy loss inherently guarantee PAC-Bayesian generalization bounds.
Meta-learning performance depends on train-validation split type.
problem Understanding the importance of train-validation split in meta-learning.
method Theoretical and experimental study comparing train-val and train-train methods.
result Train-train method can achieve strictly better excess loss in realizable cases.
We propose a novel approach for loss reserving based on deep neural networks. The approach allows for joint modeling of paid losses and claims outstanding, and incorporation of heterogeneous inputs. We validate the models on loss reserving data across lines of business, and show that they improve on the predictive accu…
AGMMNs improve learning of copula models by adaptively selecting kernels.
problem Learning dependence structures in copula models.
method Adaptive bandwidth selection for MMD in GMMNs, increasing kernels based on validation loss.
result AGMMNs significantly improve training performance over GMMNs and parametric models.
K-fold cross-validation (CV) with squared error loss is widely used for evaluating predictive models, especially when strong distributional assumptions cannot be taken. However, CV with squared error loss is not free from distributional assumptions, in particular in cases involving non-i.i.d. data. This paper analyzes …
Proposes a test to ensure predictive algorithms predict intended outcomes better than unintended ones.
problem Unintended model behavior leading to prediction of unintended outcomes.
method Falsification framework using nonparametric hypothesis testing to compare prediction losses across outcomes.
result Establishes discriminant validity with respect to gender but not race in an admissions setting.
SmoothDARTS stabilizes DARTS-based architecture search by smoothing loss landscapes.
problem DARTS-based NAS methods suffer from instability, leading to deteriorating architectures.
method SmoothDARTS (SDARTS) uses perturbation-based regularization to smooth the loss landscape.
result SmoothDARTS improves the generalizability and performance of DARTS-based methods.
Proposes a neural network loss function for better uncertainty estimation.
problem Challenges in estimating predictive uncertainty of neural networks.
method Bayesian Validation Metric (BVM) framework with ensemble learning.
result Competitive and robust uncertainty estimation on in-distribution and out-of-distribution data.
Unified approach to structured prediction combining entropy regularization and neuro-symbolic logic.
problem Structured prediction challenges due to large output spaces and insufficient labeled data.
method Neuro-symbolic entropy regularization loss that restricts entropy regularization to valid structures.
result Models predict more accurately and are more likely to be valid.
Improved scalable machine learning under heavy-tailed data.
problem Machine learning scalability under heavy-tailed data without strong convexity.
method Simple robust validation sub-routine to boost confidence in gradient-based sub-processes.
result Substantial improvement in dimension dependence without strong convexity.
This paper aims to provide a better understanding of a symmetric loss. First, we emphasize that using a symmetric loss is advantageous in the balanced error rate (BER) minimization and area under the receiver operating characteristic curve (AUC) maximization from corrupted labels. Second, we prove general theoretical p…
We consider the problem of learning a loss function which, when minimized over a training dataset, yields a model that approximately minimizes a validation error metric. Though learning an optimal loss function is NP-hard, we present an anytime algorithm that is asymptotically optimal in the worst case, and is provably…
Robust Variable Selection for High-dimensional Regression with Missing Data and Measurement Errorsstat.ME
Robust variable selection for high-dimensional data with missing and measurement errors.
problem Missing data and measurement errors confound data distribution.
method Exponential loss function with inverse probability weighting and additive error models.
result The Atan punishment method improves robust variable selection.
Cross-validation pitfalls in change-point regression are addressed with new approaches.
problem Cross-validation's prediction error-based criterion may lead to under- or over-estimation of change-points.
method Proposes two approaches: absolute error loss and modified holdout sets.
result Consistent estimation of the number of change-points under certain conditions.
In contrast to the standard classification paradigm where the true class is given to each training pattern, complementary-label learning only uses training patterns each equipped with a complementary label, which only specifies one of the classes that the pattern does not belong to. The goal of this paper is to derive …
A new concordance loss improves model performance and reliability in survival prediction.
problem Inconsistent evaluation of deep survival models using likelihood losses.
method Proposed a value-monotone concordance loss (SCL) to improve reliability and optimization.
result SCL achieves comparable discrimination and is the best or within one standard deviation of the best C-index across multiple datasets.
New function class characterizes loss landscape of deep neural networks without over-parametrization.
problem Complex loss landscape of deep neural networks without over-parametrization.
method Proposed a novel class of functions to characterize loss landscape without over-parametrization.
result Gradient-based optimizers possess theoretical guarantees of convergence under the new function class assumption.
In this article, we derive concentration inequalities for the cross-validation estimate of the generalization error for subagged estimators, both for classification and regressor. General loss functions and class of predictors with both finite and infinite VC-dimension are considered. We slightly generalize the formali…
Study binary choice with asymmetric loss, offering simple solutions.
problem Binary choice with asymmetric loss in data-rich environments.
method Loss-based reweighting of logistic regression or machine learning techniques.
result Valid decisions on binary outcomes with general loss functions.
Validates policies using past observational data with guarantees about out-of-sample performance.
problem Evaluating decision policies using past data observed under a different policy.
method Sample-splitting method to draw inferences about the entire loss distribution with finite-sample coverage guarantees.
result Valid inferences about out-of-sample loss with finite-sample coverage guarantees, accounting for model misspecifications.
Optimizes exp-concave losses with a new risk bound.
problem Optimizing exp-concave losses with stochastic convex optimization.
method Empirical Risk Minimization with a unified geometric assumption and local norms.
result Provides an excess risk bound.
Deep neural networks (DNNs) have great expressive power, which can even memorize samples with wrong labels. It is vitally important to reiterate robustness and generalization in DNNs against label corruption. To this end, this paper studies the 0-1 loss, which has a monotonic relationship with an empirical adversary (r…
In this paper, we introduce a new concept of stability for cross-validation, called the -stability, and use it as a new perspective to build the general theory for cross-validation. The -stability mathematically connects the generalization ability and the stability of…
Targeted Cross-Validationstat.ML
Proposes TCV for selecting models in regions of interest.
problem Selecting models for specific regions of predictor variables.
method Targeted Cross-Validation (TCV) using weighted L2 loss.
result TCV is consistent in selecting the best performing candidate under weighted L2 loss.
Loss to followup is a significant issue in healthcare and has serious consequences for a study's validity and cost. Methods available at present for recovering loss to followup information are restricted by their expressive capabilities and struggle to model highly non-linear relations and complex interactions. In this…
Novel loss functions improve decision tree learning from noisy data.
problem Training decision trees with noisy labels.
method Introducing distribution losses and a new negative exponential loss.
result The negative exponential loss leads to efficient and robust decision tree learning.
Model predicts internal fraud in retail banking is cyclical and influenced by corruption.
problem Predicting and mitigating internal fraud losses in retail banking.
method Developed a dynamic model considering internal factors and macroeconomic indicators.
result Internal fraud losses are pro-cyclical and positively affected by corruption perceptions.
New IF method improves accuracy in deep neural networks with noisy data.
problem Inaccurate influence estimates in deep neural networks, especially with noisy data.
method Established a connection between influence estimation error, validation set risk, and sharpness, introducing a novel estimation form for flat validation minima.
result Our novel Influence Function approach provides more accurate influence estimates, validated across various tasks.
LALR adapts learning rate for faster convergence in regression and neural nets.
problem Finding optimal learning rates for faster convergence in regression and neural networks.
method Lipschitz continuity theory applied to Mean Absolute Error and Quantile loss functions.
result Adaptive learning rate policy enables up to 20x faster convergence.
Transformer models outperform LSTM in financial forecasting with MADL loss.
problem Optimizing loss functions for Transformer models in financial forecasting.
method Empirical experiments with MADL loss function on equity and cryptocurrency assets.
result Transformer models significantly outperform LSTM models in financial forecasting.
Paper introduces a new robust loss function for RL.
problem Heuristic selection of threshold parameters in quantile Huber loss.
method Derived from Wasserstein distance, captures noise in quantile values.
result Enhances robustness against outliers and enables parameter adjustment.
Proves and tests methods for learning time-series with breaks.
problem Learning time-series with structural breaks.
method Complete proofs and experimental validation of a regularized loss function.
result Experimental results support the validity of the techniques.
We propose a stepsize adaptation scheme for stochastic gradient descent. It operates directly with the loss function and rescales the gradient in order to make fixed predicted progress on the loss. We demonstrate its capabilities by conclusively improving the performance of Adam and Momentum optimizers. The enhanced op…
The paper analyzes the risk of CV-tuned regularized estimators and connects it to SURE.
problem Understanding the risk of CV-tuned regularized estimators.
method Derives asymptotic risk function of CV-tuned estimators and connects it to SURE.
result The risk function provides a more detailed picture of predictive performance than uniform bounds.
Study non-monotonic loss functions in CRC, achieving valid risk control with large calibration samples.
problem Non-monotonic loss functions in CRC, violating existing theory's monotonicity assumption.
method Finite grid selection, calibration sample size analysis, Lipschitz continuity, monotonicity, distribution shift.
result Valid CRC achieved with large calibration samples, optimal excess risk rate of .
In this article, we derive concentration inequalities for the cross-validation estimate of the generalization error for empirical risk minimizers. In the general setting, we prove sanity-check bounds in the spirit of \cite{KR99} \textquotedblleft\textit{bounds showing that the worst-case error of this estimate is not m…