Before training a neural net, a classic rule of thumb is to randomly initialize the weights so the variance of activations is preserved across layers. This is traditionally interpreted using the total variance due to randomness in both weights \emph{and} samples. Alternatively, one can interpret the rule of thumb as pr…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “total variance”
Improves diffusion models by controlling total variance and signal-to-noise-ratio.
problem Long sampling time in diffusion models.
method Total-Variance/Signal-to-Noise-Ratio (TV/SNR) disentangled framework.
result Improves generation performance by controlling TV and SNR independently.
Large batch sizes reduce gradient variance in DP-SGD, improving privacy.
problem Understanding why large batch sizes work in DP-SGD.
method Decomposed total gradient variance into subsampling and noise-induced variances, proving batch size independence in the limit.
result Large batch sizes reduce effective total gradient variance, improving privacy in DP-SGD.
New Monte Carlo method outperforms existing strategy for estimating Sobol' indices.
problem Estimating first-and total-orders Sobol' indices accurately.
method Comparing two Monte Carlo estimators for Sobol' indices.
result New method outperforms current approach in accuracy.
The paper develops estimators for variance in graph structures using fused lasso.
problem Variance estimation in graph-structured problems.
method Developed linear time estimator for homoscedastic case and total variation regularization estimator for heteroscedastic case.
result Minimax rates and consistency for variance estimation in various graph structures.
Two approaches integrate qualitative views into portfolio optimization, showing aggregation methods outperform robust optimization.
problem Incorporating qualitative views into portfolio optimization models.
method Robust optimization and order aggregation methods.
result Aggregation methods outperform robust optimization in portfolio performance analysis.
The paper analyzes the bias-variance tradeoff for Bregman divergences.
problem Understanding the bias-variance tradeoff for Bregman divergences.
method Analyzes the bias-variance tradeoff through operations in dual space.
result Derives several results including a generalized law of total variance and ensembling operations.
We quantify predictive uncertainty using the posterior predictive variance.
problem Quantifying uncertainty in predictive models.
method Using the law of total variance, we generate expansions for the posterior predictive variance.
result Identify the main contributors to prediction intervals and quantify term-wise uncertainty.
Contextual bandits study how reward variance affects regret bounds.
problem Investigating how small reward variance impacts regret bounds in contextual bandits.
method Analyzing two types of adversaries and function approximation complexities.
result Regret bounds are influenced by the eluder dimension and reward variance.
We develop a dynamic version of the SSVI parameterisation for the total implied variance, ensuring that European vanilla option prices are martingales, hence preventing the occurrence of arbitrage, both static and dynamic. Insisting on the constraint that the total implied variance needs to be null at the maturity of t…
New algorithm reduces MDP regret by accounting for state suboptimality gaps and variance.
problem Reducing regret in episodic MDPs with varying state suboptimality gaps.
method Introduced MVP algorithm with variance-aware gap-dependent regret bound.
result Achieved a variance-aware gap-dependent regret bound for MDPs.
We consider multi-level composite optimization problems where each mapping in the composition is the expectation over a family of random smooth mappings or the sum of some finite number of smooth mappings. We present a normalized proximal approximate gradient (NPAG) method where the approximate gradients are obtained v…
The paper introduces a method to decompose variance in twin networks for better treatment effect estimation.
problem Accurate treatment effect estimation requires reliable uncertainty measures to locate model failures.
method Layer-wise variance decomposition using Monte Carlo Dropout in twin networks.
result The encoder component dominates under distributional shift, providing a practical diagnostic for data collection.
Existing feature selection methods fail to properly account for interactions between features when evaluating feature subsets. In this paper, we attempt to remedy this issue by using orthogonal variance decomposition to evaluate features. The orthogonality of the decomposition allows us to directly calculate the total …
Improved regret bounds for contextual bandits considering variance sequences.
problem Establishing lower bounds for contextual bandits with variance-dependent regret.
method Developed variance-dependent lower bounds for linear contextual bandits under two settings: fixed and adaptive variance sequences.
result Lower bounds match upper bounds of SAVE algorithm up to logarithmic factors.
Measures three types of noise in LLM evaluations.
problem Separating signal from noise in LLM experiments.
method Defined and measured three types of noise: prediction, data, and total noise. Proposed the all-pairs paired method for statistical power.
result Total noise level is characteristic and predictable across all model pairs.
Flexible model captures commodity skews with maturity effects.
problem Capturing market skew in commodity futures with maturity effects.
method Non-parametric extension with leverage functions, calibrated using Monte Carlo simulation.
result Model accurately captures market smile and implied variance accumulation.
U-statistics improve gradient estimation in importance-weighted variational inference.
problem High variance in gradient estimation for importance-weighted variational inference.
method Use U-statistics to average base gradient estimators on overlapping batches of size m, achieving lower variance.
result U-statistic variance reduction leads to modest to significant improvements in inference performance.
VCAE improves autoencoder quality on MNIST and CelebA.
problem Overfitting and poor generative/reconstruction quality in autoencoders.
method Proposes variance-constrained autoencoder (VCAE) to enforce variance constraint on latent distribution.
result VCAE outperforms Wasserstein Autoencoder and Variational Autoencoder in quality.
Unified analysis of stochastic gradient methods for convex and smooth optimization.
problem Minimizing composite convex and smooth functions.
method Unified convergence analysis of various stochastic gradient methods.
result Unified convergence rates for a variety of methods including proximal SGD, variance reduced methods, quantization, and coordinate descent.
New theory shows how multi-head attention reduces variance and decorrelates outputs.
problem Understanding and optimizing multi-head attention in neural networks.
method Developed a statistical theory linking multi-head attention to ensemble Nadaraya-Watson estimators.
result MHA variance reduction depends on head decorrelation, not just head count.
A federated learning algorithm tackles linear bandits with adversarial actions, achieving optimal regret bounds.
problem Federated linear bandits with finite adversarial action sets.
method FedSupLinUCB algorithm, extending SupLinUCB and OFUL principles.
result Achieves a total regret of , matching minimax lower bound and being order-optimal.
We present an alternating augmented Lagrangian method for convex optimization problems where the cost function is the sum of two terms, one that is separable in the variable blocks, and a second that is separable in the difference between consecutive variable blocks. Examples of such problems include Fused Lasso estima…
No arbitrage SVIq-fin.MF
Characterizes no Butterfly arbitrage in SVI model parameters.
problem No Butterfly arbitrage in SVI implied total variance formula.
method Characterization using intermediary condition from Fukasawa (2012) and rescaling of SVI parameters.
result Simple range conditions on SVI parameters ensure no Butterfly arbitrage.
A new statistical concept, lepto-variance, is defined for stock returns using Regression Trees.
problem Understanding the underlying structure of stock returns using statistical methods.
method Defining lepto-variance as the variance that cannot be removed by any regression tree of a specific depth and analyzing stock returns with 1- and 2-bit Regression Trees.
result Lepto-variance quantifies the resolving power of Regression Trees for stock returns, decomposing total variance into lepto-variance and macro-variance.
ML helps select variables for minimum-variance portfolios, reducing risk and improving performance.
problem Optimizing minimum-variance portfolios with relevant predictors.
method Parameterized minimum-variance portfolio weights using a large pool of firm-level characteristics and their transformations.
result ML-selected predictors lead to lower risk and better performance in minimum-variance portfolios.
We propose a simple and general variant of the standard reparameterized gradient estimator for the variational evidence lower bound. Specifically, we remove a part of the total derivative with respect to the variational parameters that corresponds to the score function. Removing this term produces an unbiased gradient …
Improved variance reduction for Riemannian non-convex optimization with adaptive batch size.
problem Optimizing non-convex functions on Riemannian manifolds.
method Batch size adaptation in R-SVRG, R-SRG, and R-SPIDER.
result Achieves lower total complexities for various non-convex functions.
Principal component analysis (PCA) is a widely used technique for data analysis and dimension reduction with numerous applications in science and engineering. However, the standard PCA suffers from the fact that the principal components (PCs) are usually linear combinations of all the original variables, and it is thus…
Study bounds variance modulation function for K-spider distributions.
problem Bounding variance modulation function for K-spider distributions.
method Used folded moments and total probabilities of spider legs.
result Gave an interval for the variance modulation function.
An Axiomatic Assessment of Entropy- and Variance-based Uncertainty Quantification in Regressioncs.LG
This study introduces axioms to assess regression uncertainty measures.
problem Limited formal justification and evaluations of uncertainty measures in regression settings.
method Introduces axioms and analyzes entropy- and variance-based measures in a predictive exponential family context.
result Provides a principled foundation for reliable uncertainty assessment in regression.
We revisit resampling procedures for error estimation in binary classification in terms of U-statistics. In particular, we exploit the fact that the error rate estimator involving all learning-testing splits is a U-statistic. Thus, it has minimal variance among all unbiased estimators and is asymptotically normally dis…
Ensembles improve classifier performance by reducing bias, not variance.
problem Improving classifier performance through ensemble methods.
method Extended bias-variance decomposition for classification tasks, introducing dual reparameterization.
result Ensembling reduces bias in classifiers, contrary to the traditional view.
AEA dynamically aggregates ensemble targets for actor-critic learning.
problem Static ensemble aggregation methods struggle with overestimation bias and variance.
method Adaptive Ensemble Aggregation (AEA) dynamically constructs ensemble-based targets.
result AEA converges to optimal variance reduction and maximal Fisher information.
CV outperforms mean-variance for stock returns, minimizing risk and maximizing growth.
problem Traditional risk assessment methods underperform in stock market analysis.
method Derived new CV equation and used it to analyze stock performance.
result Stocks with low but positive CV grow exponentially, outperforming high-risk stocks.
Estimates Markov chain variance efficiently without storing samples.
problem Estimating the asymptotic variance of Markov chain functions.
method Linear stochastic approximation of Poisson equation solution.
result Optimal MSE convergence rate with finite sample guarantees.
We analyze total, asymmetric and frequency connectedness between oil and forex markets using high-frequency, intra-day data over the period 2007 -- 2017. By employing variance decompositions and their spectral representation in combination with realized semivariances to account for asymmetric and frequency connectednes…
Jumping VaR: Order Statistics Volatility Estimator for Jumps Classification and Market Risk Modelingq-fin.RM
This paper proposes a new integrated variance estimator based on order statistics within the framework of jump-diffusion models. Its ability to disentangle the integrated variance from the total process quadratic variation is confirmed by both simulated and empirical tests. For practical purposes, we introduce an itera…
This paper addresses the problem of segmenting a time-series with respect to changes in the mean value or in the variance. The first case is when the time data is modeled as a sequence of independent and normal distributed random variables with unknown, possibly changing, mean value but fixed variance. The main assumpt…
Adaptive linear bandit algorithm with best-of-three-worlds regret bounds.
problem Adaptive to adversarial and stochastic environments with varying sub-optimality gaps and corruption.
method Combines SCRiBLe algorithm with scaled-up sampling and optimistic online learning.
result Achieves best-of-three-worlds regret bounds of for adversarial and for stochastic environments.
Deep learning models can have low bias and variance, contrary to classical theory.
problem Understanding the performance of deep learning models at high complexity.
method Developed a fine-grained bias-variance decomposition for random feature kernel regression, analyzing the effects of sampling, initialization, and labels.
result The variance terms exhibit non-monotonic behavior and can diverge at the interpolation boundary, even in the absence of label noise.
This paper investigates the use of multiple directions of stratification as a variance reduction technique for Monte Carlo simulations of path-dependent options driven by Gaussian vectors. The precision of the method depends on the choice of the directions of stratification and the allocation rule within each strata. S…
USNRT uses tree-structured learning to improve uncertainty quantification of variance networks.
problem Improving uncertainty quantification of variance networks.
method Tree-structured local neural network model that partitions feature space into regions for training region-specific neural networks to predict mean and variance.
result USNRT shows superior performance in estimating uncertainty with variances on UCI datasets compared to recent methods.
New method allocates capital based on tail central moments for financial risk assessment.
problem Inability of CTE-based capital allocation to reflect tail behavior of losses.
method Developed TCM-based capital allocation for normal mean-variance mixture distributions.
result TCM-based method captures tail risk contributions not detected by CTE.
New algorithm reduces regret for linear bandits with unknown noise variance.
problem Finding optimal actions in linear bandits with varying noise variance.
method Adaptive algorithm with Freedman-type concentration inequality and multi-layer structure.
result Achieves regret for linear bandits.
New method quantifies uncertainty at class level for better decision-making.
problem Improving cost-sensitive decision-making in classification tasks.
method Label-wise decomposition of uncertainty measures based on non-categorical metrics.
result Proposed measures adhere to desirable properties and improve uncertainty quantification.
Language models allocate information storage, not collapsing into uniform representations.
problem Incomplete neural collapse in language model representations.
method Analyzing variance and information sharing across 14 models, proving an information floor.
result Within-class variance is allocated information storage, not collapsed into uniform representations.
Risk-Sensitive Deep RL: Variance-Constrained Actor-Critic Provably Finds Globally Optimal Policycs.LG
This work tackles risk-sensitive deep RL by optimizing policies with variance constraints.
problem Risk and aleatoric uncertainty in deep reinforcement learning.
method Lagrangian and Fenchel dualities to transform the problem into an unconstrained saddle-point policy optimization problem, and an actor-critic algorithm to iteratively update policy, Lagrange multiplier, and Fenchel dual variable.
result The proposed actor-critic algorithm finds a globally optimal policy at a sublinear rate.