Reduces quantifier variance with accuracy optimization of base classifier.
problem Minimizing quantifier variance under prior probability shift.
method Optimizes the Brier score of a base classifier for training data.
result Optimizing Brier score on training data reduces quantifier variance on test data.
We quantify predictive uncertainty using the posterior predictive variance.
problem Quantifying uncertainty in predictive models.
method Using the law of total variance, we generate expansions for the posterior predictive variance.
result Identify the main contributors to prediction intervals and quantify term-wise uncertainty.
We analyze bias-variance of margin losses.
problem Understanding model overfitting/underfitting.
method Bias-variance decomposition for strictly convex margin losses.
result Expected risk decomposes into central model risk and data variation.
Continuous Sweep improves binary quantifier performance.
problem Estimating class prevalence in datasets.
method Parametric binary quantifier inspired by Median Sweep, using parametric class distributions and mean of Adjusted Count estimates.
result Continuous Sweep outperforms other quantifiers in simulations and empirical data analysis.
New method quantifies uncertainty in reinforcement learning models.
problem Quantifying uncertainty over expected cumulative rewards in reinforcement learning.
method Proposes a new uncertainty Bellman equation to more accurately estimate value function variance.
result Our method converges to the true posterior variance over values and improves sample-efficiency.
Data balancing reduces variance in machine learning models.
problem Reduction of variance in machine learning models.
method Non-asymptotic statistical bound and eigenvalue decay of Markov operators.
result Data balancing across modalities and sources reduces variance.
USNRT uses tree-structured learning to improve uncertainty quantification of variance networks.
problem Improving uncertainty quantification of variance networks.
method Tree-structured local neural network model that partitions feature space into regions for training region-specific neural networks to predict mean and variance.
result USNRT shows superior performance in estimating uncertainty with variances on UCI datasets compared to recent methods.
Paper tackles unknown variances in best-arm identification.
problem Identifying the best arm with unknown variances in Gaussian distributions.
method Two approaches: empirical variance plugging or adapting transportation costs.
result The impact of unknown variances is small on sample complexity.
New method quantifies uncertainty at class level for better decision-making.
problem Improving cost-sensitive decision-making in classification tasks.
method Label-wise decomposition of uncertainty measures based on non-categorical metrics.
result Proposed measures adhere to desirable properties and improve uncertainty quantification.
Proposes deep quantile regression for uncertainty estimation in lesion detection.
problem Uncertainty quantification in lesion detection for critical applications.
method Quantile regression for aleatoric uncertainty, Variational AutoEncoder (VAE) with QR-VAE, binary quantile regression (BQR).
result Effective quantification of uncertainty in lesion detection and segmentation.
To improve the efficiency of Monte Carlo estimation, practitioners are turning to biased Markov chain Monte Carlo procedures that trade off asymptotic exactness for computational speed. The reasoning is sound: a reduction in variance due to more rapid sampling can outweigh the bias introduced. However, the inexactness …
This study optimizes model averaging for personalized collaborative learning.
problem Differences in data or objectives between nodes in federated learning.
method Weighted averaging between local and global models for scalar mean estimation.
result There is always some positive model averaging that reduces expected squared error.
BSG learns dynamic network spillovers and uncertainty quantification.
problem Identifying indirect spillovers and systemic risk in dynamic networks.
method Bayesian Spillover Graphs using FEVD and Bayesian time series models.
result Significant performance gains over baselines in identifying source and sink nodes.
A new method reduces data valuation variance for more trustworthy data trading.
problem Data valuation and trustworthy data trading in algorithmic prediction.
method Variance reduced Shapley value estimation using stratified sampling.
result VRDS method reduces estimation variance and improves data marketplace development.
Paper proposes a method to estimate variance reduction in DNN training using importance sampling.
problem Challenges in assessing variance reduction during DNN training using importance sampling.
method Proposes a method for estimating variance reduction using minibatches sampled under importance sampling.
result Demonstrates consistent reduction in variance, improved training efficiency, and enhanced model accuracy.
Paper introduces variance-based measures for second-order uncertainty quantification in classification problems.
problem Uncertainty in machine learning predictions and decision-making.
method Second-order uncertainty quantification using variance-based measures.
result Variance-based measures effectively quantify uncertainty on a class-based level and are competitive with entropy-based measures.
Paper introduces new risk measures for Kelly criterion.
problem Aggressive Kelly criterion investment strategy.
method Unified approach to risk assessment in Kelly criterion.
result Two new measures for quantifying risk.
A new statistical concept, lepto-variance, is defined for stock returns using Regression Trees.
problem Understanding the underlying structure of stock returns using statistical methods.
method Defining lepto-variance as the variance that cannot be removed by any regression tree of a specific depth and analyzing stock returns with 1- and 2-bit Regression Trees.
result Lepto-variance quantifies the resolving power of Regression Trees for stock returns, decomposing total variance into lepto-variance and macro-variance.
A new tradeoff between regularization and sharpness improves model performance in overparameterized settings.
problem Improving model performance in overparameterized settings with minimum-norm interpolators.
method Proposes a regularization-sharpness tradeoff for overparameterized linear regression with an ℓ^p penalty.
result Empirical validation shows the tradeoff terms can distinguish performant linear interpolators.
Boundary effects inflate variance in Gaussian processes, leading to acquisition bias.
problem Boundary-induced acquisition bias in Gaussian processes.
method Traced root cause to geometric mechanism of kernel truncation at domain boundaries.
result Boundary effects create distortion that worsens with dimensionality, affecting acquisition behavior.
Optimizes option portfolios for skewed-t returns using VaR and variance measures.
problem Optimizing portfolios for skewed-t returns with heavy tails and skewness.
method Uses variance and VaR measures, departing from normal returns, and provides explicit portfolio weights.
result Optimal portfolio weights differ significantly from variance optimal weights due to skewness.
Regularization helps resolve ambiguity in mean-variance models, improving predictive uncertainty quantification.
problem Signal-to-noise ambiguity in overparameterized mean-variance models.
method Statistical field theory framework to explain phase transition.
result Regularization reduces variability and improves predictive uncertainty quantification.
Unified theory explains diversity in ensemble learning.
problem Explaining diversity in ensemble learning across various scenarios.
method Developed a framework revealing diversity as a hidden dimension in bias-variance decomposition.
result Proved exact bias-variance-diversity decompositions for multiple losses in regression and classification.
Policy gradient methods have enjoyed great success in deep reinforcement learning but suffer from high variance of gradient estimates. The high variance problem is particularly exasperated in problems with long horizons or high-dimensional action spaces. To mitigate this issue, we derive a bias-free action-dependent ba…
Study shows Stochastic Mirror Descent optimizes convex problems with infinite noise variance.
problem Optimizing convex problems with infinite noise variance.
method Stochastic Mirror Descent algorithm with uniformly convex mirror maps.
result Demonstrates convergence rate quantified in terms of iterations, dimensionality, and geometric parameters.
We investigate the joint dynamics of spot and implied volatility from an empirical perspective. We focus on the equity market with the SPX Index our underlying of choice. Using only observable quantities, we extract the instantaneous variance curves implied by the market and study their daily variations jointly with sp…
New algorithms reduce regret in online MDPs by adapting to data and variance.
problem Adapting to both adversarial and stochastic environments in online MDPs.
method Develops algorithms based on global optimization and policy optimization, using optimistic follow-the-regularized-leader with log-barrier regularization.
result Achieves refined data-dependent and variance-dependent regret bounds.
Variance reduction methods such as SVRG and SpiderBoost use a mixture of large and small batch gradients to reduce the variance of stochastic gradients. Compared to SGD, these methods require at least double the number of operations per update to model parameters. To reduce the computational cost of these methods, we i…
The paper explores the trade-off between bias and variance in high-dimensional models.
problem Understanding the unavoidable trade-off between bias and variance in high-dimensional statistical models.
method Proposes a general strategy to obtain lower bounds on the variance of estimators with a specified bias, and applies it to various statistical models.
result Shows the extent to which the bias-variance trade-off is unavoidable and quantifies the performance loss for methods that do not balance it.
Variational inference is increasingly being addressed with stochastic optimization. In this setting, the gradient's variance plays a crucial role in the optimization procedure, since high variance gradients lead to poor convergence. A popular approach used to reduce gradient's variance involves the use of control varia…
Proposes SDE framework for uncertainty quantification in graph neural networks.
problem Lack of uncertainty quantification in graph neural networks.
method Introduces Latent Graph Neural Stochastic Differential Equations (LGNSDE) with Bayesian prior-posterior mechanism and Brownian motion.
result LGNSDEs provide theoretically sensible guarantees for uncertainty estimates and are robust to perturbations.
New unbiased variance estimator for random forests using Hoeffding decomposition.
problem Uncertainty quantification in random forests with large kernel sizes and small sample sizes.
method Proposes a new Hoeffding decomposition view for variance estimation, establishing unbiased estimators and ratio consistency.
result Establishes the ratio consistency of the proposed variance estimator, justifying confidence interval coverage rates.
AB-SAGA optimizes distributed optimization over directed graphs using variance reduction and stochastic weights.
problem Optimizing distributed stochastic optimization over directed graphs with stochastic weights.
method AB-SAGA combines variance reduction and network-level gradient tracking, using both row and column stochastic weights.
result AB-SAGA converges linearly to the global optimal with a constant step-size and achieves a linear speed-up over centralized methods.
The notion of disentangled autoencoders was proposed as an extension to the variational autoencoder by introducing a disentanglement parameter β, controlling the learning pressure put on the possible underlying latent representations. For certain values of β this kind of autoencoders is capable of encoding independ…
Proposes deep graph persistence to address neural persistence issues in deep learning.
problem Variance of weights and lack of spatial structure in deep neural networks impact neural persistence.
method Extends neural persistence to the whole network, considering interactions between layers.
result Deep graph persistence alleviates variance-related issues and captures persistent paths through the network.
Proposes a simpler method for quantifying uncertainty in time-series with volatility clustering.
problem Uncertainty quantification for time-series with volatility clustering.
method Proposes a Scale Mixture Distribution to quantify return forecast uncertainty in neural networks.
result The proposed method provides a favorable complexity-accuracy trade-off and separates model parameters into subnetworks.
PEMC uses ML to enhance Monte Carlo simulations, reducing variance and runtime.
problem Computational inefficiency in Monte Carlo simulations for complex tasks.
method Prediction-Enhanced Monte Carlo (PEMC) framework that uses ML surrogates as predictors.
result PEMC provides unbiased evaluations with reduced variance and runtime compared to standard Monte Carlo.
We report on time-varying network connectedness within three banking systems: North America, the EU, and ASEAN. The original method by Diebold and Yilmaz is improved by using exponentially weighted daily returns and ridge regularization on vector autoregression (VAR) and forecast error variance decomposition (FEVD). We…
Unified framework for clustering with auxiliary data.
problem Clustering with datasets reflecting similar but different latent structures.
method Adaptive Transfer Clustering (ATC) algorithm that optimizes bias-variance decomposition.
result ATC proves optimal under Gaussian mixture model and shows transfer benefits.
Introduces TSI, a variance-based measure for persistence barcodes.
problem Capturing structural variability in persistence barcodes.
method Variance-based scalar measure, TSI, and complementary TSigI.
result TSI captures structural variability complementary to entropy.
Bayesian framework improves variance component estimation in MET data.
problem Inaccurate estimation of variance components in MET data.
method Proposes a Bayesian updating framework using historical data.
result Stabilizes variance component estimation and quantifies uncertainty.
New method isolates epistemic uncertainty in diffusion models, improving plausibility scores.
problem Uncertainty quantification in diffusion models, especially epistemic uncertainty.
method Fisher information based approach using FLARE (Fisher-Laplace Randomized Estimator).
result FLARE improves uncertainty estimation in synthetic time-series generation tasks.
Paper proposes adaptive parameter selection for KGD algorithms.
problem Improving parameter selection for kernel-based gradient descent.
method Integrates bias-variance analysis with splitting method, introduces empirical effective dimension.
result Adaptive parameter selection strategy achieves optimal generalization error bound.
We study linear models under heavy-tailed priors from a probabilistic viewpoint. Instead of computing a single sparse most probable (MAP) solution as in standard deterministic approaches, the focus in the Bayesian compressed sensing framework shifts towards capturing the full posterior distribution on the latent variab…
New estimator reduces bias and variance in tensor and matrix denoising.
problem Optimal bias-variance tradeoff in matrix and tensor estimation.
method One-step variant of higher-order SVD (HOSVD) estimator.
result Achieves optimal bias-variance tradeoff in both matrix and tensor settings.
Study quantifies how LLMs capture higher-order statistical structure using cumulant expansion.
problem Understanding how LLMs internalize statistical structure during next-token prediction.
method Cumulant-expansion framework treating softmax entropy as perturbation around center distribution.
result Cumulants reveal distinct signatures for mathematical vs. general text prompts, quantifying feature-learning dynamics.
When randomized ensembles such as bagging or random forests are used for binary classification, the prediction error of the ensemble tends to decrease and stabilize as the number of classifiers increases. However, the precise relationship between prediction error and ensemble size is unknown in practice. In the standar…
Unified framework for output analysis using Monte Carlo sampling.
problem Accurately assess the quality of estimated values in predictive models.
method Unified output analysis framework through Monte Carlo sampling, leveraging fast iterative bootstrap sampling and higher-order influence functions.
result Clear advantage in building more robust confidence intervals with higher coverage probability.