New CV method reduces bias in spatial prediction models.
problem Bias in standard cross-validation due to uneven sampling.
method Target-Weighted Cross-Validation (TWCV) framework.
result Weighted CV approaches reduce bias in prediction error.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Validation weighting”
Cross-validation under sample selection bias can, in principle, be done by importance-weighting the empirical risk. However, the importance-weighted risk estimator produces sub-optimal hyperparameter estimates in problem settings where large weights arise with high probability. We study its sampling variance as a funct…
The strength of association between a pair of data vectors is represented by a nonnegative real number, called matching weight. For dimensionality reduction, we consider a linear transformation of data vectors, and define a matching error as the weighted sum of squared distances between transformed vectors with respect…
Proposes PFWCP for multi-agent tasks with privacy and validity guarantees.
problem Challenges in uncertainty quantification for multi-agent settings.
method Personalized federated weighted conformal prediction (PFWCP) combining local density ratio weighting and weighted quantile aggregation.
result Asymptotically valid coverage guarantees for each agent in heterogeneous settings.
Improves conformal prediction by combining multiple score functions and optimizing weights.
problem Limitations of single-score conformal predictors in multi-class classification.
method Combines multiple score functions and optimizes weights to minimize prediction set size.
result Consistently outperforms single-score conformal predictors while maintaining valid coverage.
OUI tool detects optimal Weight Decay for DNNs without validation data.
problem Optimal Weight Decay hyperparameter selection for DNNs.
method Overfitting-Underfitting Indicator (OUI) tool.
result OUI correlates with improved generalization and validation scores.
Method measures weight similarity in neural networks using normalization and statistical inference.
problem Quantifying weight similarity in non-convex neural networks.
method Chain normalization rule and hypothesis-training-testing statistical inference.
result Weights of identical neural networks converge to similar local solutions.
This paper identifies a problem with the usual procedure for L2-regularization parameter estimation in a domain adaptation setting. In such a setting, there are differences between the distributions generating the training data (source domain) and the test data (target domain). The usual cross-validation procedure requ…
Adapts conformal prediction for missing data, ensuring valid coverage.
problem Uncertainty quantification with missing covariates.
method Proposes a reweighted conformal prediction procedure for handling missing values.
result Guaranteed Marginal Coverage and Mask-Conditional Validity for general missing data mechanisms.
Improved LOO cross-validation for function approximation.
problem Estimating the Integrated Squared Error (ISE) for function approximation.
method Weighted Leave-One-Out cross-validation based on Gaussian Process.
result Significantly more precise ISE estimation compared to unweighted LOO.
Improves test set performance and reduces out-of-sample disappointment for unstable models.
problem Ensuring strong test set performance via cross-validation for unstable models.
method Nested k-fold cross-validation with hyperparameter selection based on a weighted sum of cross-validation metric and model stability measure.
result Improves out-of-sample MSE for sparse ridge regression and CART by 4% and 2% respectively, compared to k-fold cross-validation.
We improve prediction set coverage by assigning weights to individual sets.
problem Aggregating multiple prediction sets weakens overall coverage guarantee.
method Propose a framework for weighted aggregation of prediction sets.
result Achieve tighter coverage bounds that interpolate between and guarantees.
Adapts example weights to optimize black-box metrics.
problem Optimizing metrics defined by black-box functions.
method Adaptive example weighting and iterative post-shifting.
result Improves classification performance compared to baselines.
CDST improves ensemble prediction by adjusting model weights based on covariates.
problem Improving ensemble prediction accuracy in complex scenarios.
method Covariate-dependent stacking (CDST) with flexible model weights estimated via cross-validation.
result CDST consistently outperforms conventional model averaging methods in complex datasets.
Study shows how feature weighting affects neural network regularization.
problem Understanding how feature weighting influences neural network regularization.
method Derived equivalence paths connecting different weighting matrices and ridge regularization levels.
result Ridge estimators trained on weighted features are asymptotically equivalent when evaluated against test vectors.
New covariance estimator for financial portfolios.
problem Estimating large financial covariances in non-stationary environments.
method Exponentially weighted averages and cross-validation for nonlinearly shrinking sample eigenvalues.
result Our estimator performs well in large dimensions compared to existing estimators.
Cross-Validated Causal Inference: a Modern Method to Combine Experimental and Observational Dataecon.EM
New method combines experimental and observational data for causal inference.
problem Combining internal validity of experiments and larger sample sizes of observations.
method Empirical risk minimization (ERM) framework with cross-validation.
result Efficacy and reliability demonstrated on real and synthetic data.
We introduce a new normalization technique that exhibits the fast convergence properties of batch normalization using a transformation of layer weights instead of layer outputs. The proposed technique keeps the contribution of positive and negative weights to the layer output balanced. We validate our method on a set o…
Develops a prediction method based on sampling design.
problem Creating accurate individual predictions.
method Design-based approach using expected cross-validation results.
result Valid inference of unobserved prediction errors defined with respect to sampling design.
Online Estimation with Rolling Validation: Adaptive Nonparametric Estimation with Streaming Datamath.ST
New method improves online nonparametric estimators with minimal extra computation.
problem Model selection and hyperparameter tuning for online nonparametric estimators.
method Weighted rolling validation procedure for online cross-validation.
result Improves base estimators to achieve better heuristic performance and adaptive convergence rate.
When the weights in a particle filter are not available analytically, standard resampling methods cannot be employed. To circumvent this problem state-of-the-art algorithms replace the true weights with non-negative unbiased estimates. This algorithm is still valid but at the cost of higher variance of the resulting fi…
A new validation scheme improves Federated Learning performance.
problem Learning a joint model from distributed, heterogeneous data.
method DVW scheme that uses distributed validation sets.
result DVW outperforms FedAvg in heterogeneous environments.
PPL improves on Takacs-Fiksel estimation for Gibbs models.
problem Improving point process estimation methods.
method PPL uses cross-validation and a specific loss function to estimate parameters.
result PPL with specific loss functions and hyperparameters outperforms Takacs-Fiksel estimation in mean square error.
We present a novel regularization approach to train neural networks that enjoys better generalization and test error than standard stochastic gradient descent. Our approach is based on the principles of cross-validation, where a validation set is used to limit the model overfitting. We formulate such principles as a bi…
The paper validates Stokes' theorem for differential subcomplexes in positively graded Lie groups.
problem Validating Stokes' theorem for differential subcomplexes in positively graded Lie groups.
method Introducing geometric conditions and spectral complexes to recover Stokes' theorem on locally smooth intrinsic graphs.
result Stokes' theorem holds for Rumin complex and new spectral complexes on Carnot groups.
The study combines social interaction data into a single network, identifying stable groups of chimpanzees.
problem Identifying stable groups of chimpanzees based on social interactions over time.
method Network representation, weighted proximity weights, principled loss function, statistical tests.
result The approach detects stable groups of chimpanzees that stay related for a significant length of time.
Develop conformal prediction for dyadic regression under complex missingness.
problem Conformal prediction for dyadic regression under complex missingness mechanisms.
method Developing general technical tools and conformal prediction procedures for dyadic regression under complex missingness.
result Establishing asymptotic validity of weighted conformal prediction under a nonparametric graphon model for missingness mechanism.
Develops anytime-valid stopping rules for SGD based on observed trajectory.
problem Stopping stochastic gradient descent (SGD) based on observed trajectory.
method Develops anytime-valid confidence sequences for stochastic gradient methods.
result Statistically valid, time-uniform stopping rules for SGD across convex and nonconvex settings.
Real-world machine learning applications often have complex test metrics, and may have training and test data that are not identically distributed. Motivated by known connections between complex test metrics and cost-weighted learning, we propose addressing these issues by using a weighted loss function with a standard…
A new method for anomaly detection adapts to local non-stationarity in low-data regimes.
problem Adapting conformal anomaly detection to handle distribution shifts in real-world data.
method Proposes a continuous inference relaxation using continuous weighted kernel density estimation to decouple local adaptation from tail resolution.
result Restores detection capabilities and statistical power in low-data regimes while maintaining valid error control.
Targeted Cross-Validationstat.ML
Proposes TCV for selecting models in regions of interest.
problem Selecting models for specific regions of predictor variables.
method Targeted Cross-Validation (TCV) using weighted L2 loss.
result TCV is consistent in selecting the best performing candidate under weighted L2 loss.
Importance-weighting is a popular and well-researched technique for dealing with sample selection bias and covariate shift. It has desirable characteristics such as unbiasedness, consistency and low computational complexity. However, weighting can have a detrimental effect on an estimator as well. In this work, we empi…
Weighted SVM (or fuzzy SVM) is the most widely used SVM variant owning its effectiveness to the use of instance weights. Proper selection of the instance weights can lead to increased generalization performance. In this work, we extend the span error bound theory to weighted SVM and we introduce effective hyperparamete…
SaML guides ML models to avoid survey biases.
problem ML models trained on survey data often ignore survey design metadata.
method Nine-step guideline for incorporating survey design metadata in ML lifecycle.
result SaML provides valid population inference from survey data.
Paper uses GNN and conformal prediction for accurate edge weight prediction.
problem Predicting edge weights on graphs for various applications.
method Graph Neural Network (GNN) with conformal prediction and error reweighting.
result Our method provides better coverage and efficiency than baselines.
A Randomized Permutation Whole-Model Test Heuristic for Self-Validated Ensemble Models (SVEM)stat.ME
A new test validates ensemble models against the null hypothesis.
problem Validating ensemble models against the null hypothesis of a constant response.
method Randomized permutation test on SVEM model predictions.
result The test maintains Type I error rate even with more parameters than observations.
PPI uses predictions and weighting to infer from partially labeled data.
problem Valid inference with partially labeled data.
method Combines model-based predictions with bias correction from labeled data, using Horvitz-Thompson and Hájek corrections.
result IPW-adjusted PPI with estimated propensities performs similarly to known-probability case.
Extends double linear policy with time-varying weights and proves robust positive expectation.
problem Ensuring robustness in policy optimization with time-varying parameters.
method Employed a novel elementary symmetric polynomials characterization approach to prove robust positive expectation (RPE). Derived explicit expressions for expected cumulative gain-loss and variance.
result Proved the robust positive expectation property holds for the extended double linear policy.
The study assesses external validity by evaluating worst-case treatment effects across subpopulations.
problem Underrepresentation of marginalized groups and limited study populations.
method Develops a semiparametrically efficient estimator for worst-case treatment effects (WTE) and uses cross-fitting to guard against brittle findings.
result The proposed framework guards against invalid findings due to unanticipated population shifts.
Extend CPS to non-exchangeable settings with observation-specific permutation weights
problem Calibrated predictive bands under distributional shifts
method Encoding distributional shifts through observation-specific permutation weights
result Shift-aware predictive systems remain valid
The Theory Behind Overfitting, Cross Validation, Regularization, Bagging, and Boosting: Tutorialstat.ML
In this tutorial paper, we first define mean squared error, variance, covariance, and bias of both random variables and classification/predictor models. Then, we formulate the true and generalization errors of the model for both training and validation/test instances where we make use of the Stein's Unbiased Risk Estim…
DW-KNN: A Transparent Local Classifier Integrating Distance Consistency and Neighbor Reliabilitycs.LG
DW-KNN improves KNN by integrating distance and neighbor reliability for better prediction accuracy.
problem Standard KNN assumes all neighbors are equally reliable, leading to unreliable predictions in heterogeneous feature spaces.
method DW-KNN integrates exponential distance with neighbor validity, providing instance-level interpretability and reducing hyperparameter sensitivity.
result DW-KNN achieves 0.8988 average accuracy, ranks 2nd among six methods, and has the lowest cross-validation variance.
Paper explores weighted averaging schemes for SGD, achieving asymptotic normality and optimality.
problem Improving convergence of SGD in various settings.
method Develops a general weighted averaging scheme for SGD and establishes asymptotic normality.
result Establishes asymptotic normality and optimality of weighted averaged SGD solutions.
We propose an algorithm for inexpensive gradient-based hyperparameter optimization that combines the implicit function theorem (IFT) with efficient inverse Hessian approximations. We present results about the relationship between the IFT and differentiating through optimization, motivating our algorithm. We use the pro…
Unified framework for optimizing portfolios with distributions over weights, returns, and parameters.
problem Traditional portfolio optimization treats expected returns, covariances, and allocations as fixed. Modern practice replaces at least one with a distribution.
method Unified framework using Gamma_theta(dw,dr) coupling to organize Bayesian, robust, chance-constrained, stochastic-allocation, and distributional reinforcement-learning methods.
result Synthetic and structural contributions, including a portfolio specialization of Wasserstein-CVaR duality and a static no-randomization theorem.
Robust Variable Selection for High-dimensional Regression with Missing Data and Measurement Errorsstat.ME
Robust variable selection for high-dimensional data with missing and measurement errors.
problem Missing data and measurement errors confound data distribution.
method Exponential loss function with inverse probability weighting and additive error models.
result The Atan punishment method improves robust variable selection.
Deep neural networks have been shown to be very powerful modeling tools for many supervised learning tasks involving complex input patterns. However, they can also easily overfit to training set biases and label noises. In addition to various regularizers, example reweighting algorithms are popular solutions to these p…
In this work, we utilize T1-weighted MR images and StackNet to predict fluid intelligence in adolescents. Our framework includes feature extraction, feature normalization, feature denoising, feature selection, training a StackNet, and predicting fluid intelligence. The extracted feature is the distribution of different…