arXiv research

Study shows exponential error reduction in multiclass classification without bias-variance trade-off.

This paper addresses error bounds and posterior variance for Gaussian process regression.

This paper optimizes portfolio selection by penalizing tracking error, improving Sharpe ratio.

The lasso has been studied extensively as a tool for estimating the coefficient vector in the high-dimensional linear model; however, considerably less is known about estimating the error variance in this context. In this paper, we propose the natural lasso estimator for the error variance, which maximizes a penalized …

This work uses ANOVA to understand how different factors contribute to test error in machine learning models.

Optimal estimator derived for partially observable LTI systems.

Stochastic gradient descent updates parameters with summation gradient computed from a random data batch. This summation will lead to unbalanced training process if the data we obtained is unbalanced. To address this issue, this paper takes the error variance and error mean both into consideration. The adaptively adjus…

Bayesian method recovers causal structure in SEMs with equal error variances.

In this paper, we prove that some Gaussian structural equation models with dependent errors having equal variances are identifiable from their corresponding Gaussian distributions. Specifically, we prove identifiability for the Gaussian structural equation models that can be represented as Andersson-Madigan-Perlman cha…

Paper proposes diagnostics for error and variance estimation in randomized matrix computations.

Optimizes embedding accuracy for data variance and error.

Proposes reducing random error in stochastic optimization by variance regularization.

The bias-variance tradeoff tells us that as model complexity increases, bias falls and variances increases, leading to a U-shaped test error curve. However, recent empirical results with over-parameterized neural networks are marked by a striking absence of the classic U-shaped test error curve: test error keeps decrea…

New insights into bias and variance in over-parameterized models.

Paper presents a method to reduce prediction variance of DNNs for unknown systems.

In this work, we consider the identifiability assumption of Gaussian linear structural equation models (SEMs) in which each variable is determined by a linear function of its parents plus normally distributed error. It has been shown that linear Gaussian structural equation models are fully identifiable if all error va…

Scaling laws in linear regression explain model performance improvements with size and data.

Temporal difference (TD) learning is a popular algorithm for policy evaluation in reinforcement learning, but the vanilla TD can substantially suffer from the inherent optimization variance. A variance reduced TD (VRTD) algorithm was proposed by Korda and La (2015), which applies the variance reduction technique direct…

Algorithm estimates common mean from Gaussian variables with unknown variances.

Generalizes bias-variance decomposition for Bregman divergences.

The paper examines prediction and estimation risks of ridgeless least squares under general error assumptions.

We revisit resampling procedures for error estimation in binary classification in terms of U-statistics. In particular, we exploit the fact that the error rate estimator involving all learning-testing splits is a U-statistic. Thus, it has minimal variance among all unbiased estimators and is asymptotically normally dis…

Analyzes error sources in global feature effect estimation methods.

The paper develops estimators for variance in graph structures using fused lasso.

The paper analyzes sparse high-dimensional linear regression with random design and unknown error variance, providing adaptiveness and concentration rates.

Proposes a new framework to optimize portfolios with reduced estimation errors.

MEVA aggregates model predictions to improve accuracy without needing model details.

Bias - variance decomposition of the expected error defined for regression and classification problems is an important tool to study and compare different algorithms, to find the best areas for their application. Here the decomposition is introduced for the survival analysis problem. In our experiments, we study bias -…

Paper improves Gumbel-Softmax estimator variance reduction.

Paper unifies bias and variance models for classification.

New method optimizes PCA for better prediction and variance.

This study investigates how Decision-Focused Learning improves stock return predictions for better portfolio optimization.

In this paper, we obtain generic bounds on the variances of estimation and prediction errors in time series analysis via an information-theoretic approach. It is seen in general that the error bounds are determined by the conditional entropy of the data point to be estimated or predicted given the side information or p…

VA-OPE improves OPE by incorporating variance information, achieving tighter error bounds.

Optimizes MCMC chains with neural control variates.

Data-driven optimization improves mean-variance portfolios by penalizing norms.

LoCoV reduces portfolio optimization errors from sample covariance matrices.

Robust and reliable covariance estimates play a decisive role in financial and many other applications. An important class of estimators is based on Factor models. Here, we show by extensive Monte Carlo simulations that covariance matrices derived from the statistical Factor Analysis model exhibit a systematic error, w…

Two derivations of PCA for distributional data.

Deep networks generalize well even when they fit training data perfectly, thanks to overparametrization.

In a financial market model, we consider the variance-optimal semi-static hedging of a given contingent claim, a generalization of the classic variance-optimal hedging. To obtain a tractable formula for the expected squared hedging error and the optimal hedging strategy, we use a Fourier approach in a general multidime…

Paper proposes methods to reduce bias and variance in recommender systems.

The paper estimates common mean of entangled Gaussians with bounded variances.

We study confidence intervals based on hard-thresholding, soft-thresholding, and adaptive soft-thresholding in a linear regression model where the number of regressors $k$ may depend on and diverge with sample size $n$. In addition to the case of known error variance, we define and study versions of the estimators when…

New method improves reliability of selecting individuals based on predicted treatment effects.

The study reveals a transition in neural network performance from infinite-width to variance-limited behavior as dataset size increases.

In this paper, we argue that, once the costs of maintaining the hedging portfolio are properly taken into account, semi-static portfolios should more properly be thought of as separate classes of derivatives, with non-trivial, model-dependent payoff structures. We derive new integral representations for payoffs of exot…

Paper analyzes GP derivatives for error propagation in geoscience.