arXiv research

Develops abstention procedure for nonparametric regression via variance testing.

Introduces TPV to analyze model robustness without labels.

PPAT uses predictions to improve risk estimation in active testing.

MFVI can overestimate predictive variance compared to the exact posterior

New method uses machine learning to improve statistical inference.

A simple method treats heteroscedastic variance variatively, improving model calibration and sample quality.

Adversarial training leads to large generalization gap, decomposed into bias and variance.

Unified approach combines prediction-powered inference and variance reduction for semi-supervised optimization.

New method uses interval-based metric to validate prediction uncertainty in machine learning.

This paper first answers the question "why do the two most powerful techniques Dropout and Batch Normalization (BN) often lead to a worse performance when they are combined together?" in both theoretical and statistical aspects. Theoretically, we find that Dropout would shift the variance of a specific neural unit when…

Researchers have constantly asked whether stock returns can be predicted by some macroeconomic data. However, it is known that macroeconomic data may exhibit nonstationarity and/or heavy tails, which complicates existing testing procedures for predictability. In this paper we propose novel empirical likelihood methods …

BYOV combines SSL and Bayesian methods for uncertainty estimation.

Machine learning reduces variance in online experiment results.

China's stock market is the largest emerging market all over the world. It is widely accepted that the Chinese stock market is far from efficiency and it possesses possible linear and nonlinear dependence. We study the predictability of returns in the Chinese stock market by employing the wild bootstrap automatic varia…

New framework tests mean-variance spanning in high dimensions.

Proposes a modified Morgan-Pitman test for evaluating variances in machine learning models.

New bin-wise scaling methods improve prediction uncertainty calibration for machine learning.

Scaling laws in linear regression explain model performance improvements with size and data.

The paper develops a method to evaluate recommendation systems without A/B testing.

USNRT uses tree-structured learning to improve uncertainty quantification of variance networks.

Paper analyzes GP derivatives for error propagation in geoscience.

New method for estimating out-of-sample R² from gene expression data.

Ordinary stochastic neural networks mostly rely on the expected values of their weights to make predictions, whereas the induced noise is mostly used to capture the uncertainty, prevent overfitting and slightly boost the performance through test-time averaging. In this paper, we introduce variance layers, a different k…

Covariate shift relaxes the widely-employed independent and identically distributed (IID) assumption by allowing different training and testing input distributions. Unfortunately, common methods for addressing covariate shift by trying to remove the bias between training and testing distributions using importance weigh…

In this paper, prediction for linear systems with missing information is investigated. New methods are introduced to improve the Mean Squared Error (MSE) on the test set in comparison to state-of-the-art methods, through appropriate tuning of Bias-Variance trade-off. First, the use of proposed Soft Weighted Prediction …

Gradient-flow optimization is reinterpreted as a statistical inference problem.

Paper improves volatility estimation using a Queue-Reactive model.

A new method improves treatment effect inferences in RCTs by adjusting for covariates and heteroskedasticity.

The bias-variance tradeoff tells us that as model complexity increases, bias falls and variances increases, leading to a U-shaped test error curve. However, recent empirical results with over-parameterized neural networks are marked by a striking absence of the classic U-shaped test error curve: test error keeps decrea…

This work uses ANOVA to understand how different factors contribute to test error in machine learning models.

We quantify predictive uncertainty using the posterior predictive variance.

Reduces quantifier variance with accuracy optimization of base classifier.

Language models allocate information storage, not collapsing into uniform representations.

A new permutation method improves two-sample testing power.

A common statistical problem in econometrics is to estimate the impact of a treatment on a treated unit given a control sample with untreated outcomes. Here we develop a generative learning approach to this problem, learning the probability distribution of the data, which can be used for downstream tasks such as post-t…

The infinitesimal jackknife (IJ) has recently been applied to the random forest to estimate its prediction variance. These theorems were verified under a traditional random forest framework which uses classification and regression trees (CART) and bootstrap resampling. However, random forests using conditional inferenc…

The paper proposes a new method for comparing logistic regression models across different populations.

Improves predictive algorithm performance with domain adaptation.

We present local ensembles, a method for detecting underspecification -- when many possible predictors are consistent with the training data and model class -- at test time in a pre-trained model. Our method uses local second-order information to approximate the variance of predictions across an ensemble of models from…

Project predicts stock prices for robust portfolio design in Indian sectors.

In this paper we formulate a regression problem to predict realized volatility by using option price data and enhance VIX-styled volatility indices' predictability and liquidity. We test algorithms including regularized regression and machine learning methods such as Feedforward Neural Networks (FNN) on S&P 500 Index a…

Paper explains how tree ensembles improve predictions by smoothing and regulating smoothness.

The accuracy of machine learning systems is a widely studied research topic. Established techniques such as cross-validation predict the accuracy on unseen data of the classifier produced by applying a given learning method to a given training data set. However, they do not predict whether incurring the cost of obtaini…

The relationship between the size and the variance of firm growth rates is known to follow an approximate power-law behavior $σ(S) \sim S^{-β(S)}$ where $S$ is the firm size and $β(S)\approx 0.2$ is an exponent weakly dependent on $S$. Here we show how a model of proportional growth which treats firms as classes compos…

This work develops formal statistical inference procedures for machine learning ensemble methods. Ensemble methods based on bootstrapping, such as bagging and random forests, have improved the predictive accuracy of individual trees, but fail to provide a framework in which distributional results can be easily determin…

Null-Calibrated Conformal Selection via Target-Membership Scores

This paper improves uncertainty quantification in ELM models.

Analyzes bias-variance in overparameterized linear models using random features.