Before training a neural net, a classic rule of thumb is to randomly initialize the weights so the variance of activations is preserved across layers. This is traditionally interpreted using the total variance due to randomness in both weights \emph{and} samples. Alternatively, one can interpret the rule of thumb as pr…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “sample variance”
This paper tackles variance issues in GNN training by proposing a method to reduce both embedding and gradient variances.
problem High variance in estimating stochastic gradients in GNN training, especially in large graphs.
method The paper proposes a decoupled variance reduction strategy that employs approximate gradient information to adaptively sample nodes with minimal variance.
result The proposed method achieves faster convergence and better generalization compared to existing sampling methods.
Paper proposes a method to estimate variance reduction in DNN training using importance sampling.
problem Challenges in assessing variance reduction during DNN training using importance sampling.
method Proposes a method for estimating variance reduction using minibatches sampled under importance sampling.
result Demonstrates consistent reduction in variance, improved training efficiency, and enhanced model accuracy.
Improved TD learning reduces variance and bias errors.
problem Inefficient optimization variance in TD learning.
method Proposed a mathematically solid analysis of VRTD, showing linear convergence rate and reduced variance and bias errors.
result VRTD converges to a fixed-point solution with reduced variance and bias errors compared to vanilla TD.
Unified method for MMD variance estimation improves accuracy and computational efficiency.
problem Variance estimation for MMD in nonparametric testing.
method Unified finite-sample characterization of MMD variance through U-statistic and Hoeffding decomposition; exact acceleration method for univariate case.
result Unified estimators improve accuracy and computational efficiency for MMD variance.
A new statistical concept, lepto-variance, is defined for stock returns using Regression Trees.
problem Understanding the underlying structure of stock returns using statistical methods.
method Defining lepto-variance as the variance that cannot be removed by any regression tree of a specific depth and analyzing stock returns with 1- and 2-bit Regression Trees.
result Lepto-variance quantifies the resolving power of Regression Trees for stock returns, decomposing total variance into lepto-variance and macro-variance.
Study shows gradient variance increases during deep learning training, contrary to common belief.
problem Understanding and minimizing gradient variance in deep learning models.
method Gradient Clustering method using stratified sampling to minimize gradient variance.
result Gradient variance increases during training, and smaller learning rates coincide with higher variance.
Develops Thompson Sampling algorithms for mean-variance bandits.
problem Risk in online decision making systems.
method Thompson Sampling algorithms for mean-variance MAB with comprehensive regret analyses.
result Achieves best known regret bounds for mean-variance MABs and information-theoretic bounds in some regimes.
A new method reduces data valuation variance for more trustworthy data trading.
problem Data valuation and trustworthy data trading in algorithmic prediction.
method Variance reduced Shapley value estimation using stratified sampling.
result VRDS method reduces estimation variance and improves data marketplace development.
Understanding the Curse of Horizon in Off-Policy Evaluation via Conditional Importance Samplingcs.LG
Off-policy policy estimators that use importance sampling (IS) can suffer from high variance in long-horizon domains, and there has been particular excitement over new IS methods that leverage the structure of Markov decision processes. We analyze the variance of the most popular approaches through the viewpoint of con…
New algorithm for training GNNs with learned weights.
problem Optimal sampling for GNNs with learned weights is intractable.
method Formulated as an adversary bandit problem, optimizing exploration and exploitation.
result Asymptotically approaches optimal variance within a factor of 3.
The paper optimizes RV estimation by efficient sampling in time-changed diffusion models.
problem Improving realized variance (RV) estimation in time-changed diffusion models.
method Theoretical analysis and simulations of hitting time and realized business time sampling schemes.
result Realized business time sampling is empirically most efficient for high noise levels.
We give improved constants for data dependent and variance sensitive confidence bounds, called empirical Bernstein bounds, and extend these inequalities to hold uniformly over classes of functionswhose growth function is polynomial in the sample size n. The bounds lead us to consider sample variance penalization, a nov…
Variance Reduction and Low Sample Complexity in Stochastic Optimization via Proximal Point Methodmath.OC
New method reduces variance in stochastic optimization with high confidence.
problem Achieving high-probability guarantees in stochastic optimization with weaker noise assumptions.
method Stochastic proximal point method combining proximal subproblem solver and probability booster.
result Demonstrates convergence with low sample complexity under bounded variance assumptions.
OSIRIS reduces variance in off-policy evaluation by omitting irrelevant states.
problem High variance in importance sampling-based OPE estimators.
method OSIRIS reduces variance by omitting likelihood ratios associated with states irrelevant to return.
result OSIRIS is unbiased and has lower variance than ordinary importance sampling.
Memory-efficient algorithm reduces variance in off-policy RL.
problem High variance in off-policy policy optimization.
method Memory-efficient, stochastically variance-reduced algorithm using off-policy samples.
result Empirically validated effectiveness of the proposed algorithm.
A new method interpolates between sampling and variational inference using stochastic mixtures.
problem Combining the strengths of sampling and variational inference methods.
method Develops a framework using stochastic mixtures of simple component distributions to interpolate between sampling and variational inference.
result Improves on both sampling and variational inference methods by reducing bias and variance.
Paper analyzes high-dimensional portfolio risks and finds empirical out-of-sample relative loss is more reliable.
problem Analyzing risks in high-dimensional portfolios using empirical variance.
method Derives asymptotic behavior of out-of-sample variance and relative loss in high-dimensional settings.
result Empirical out-of-sample relative loss is more reliable than variance in high-dimensional portfolios.
Efficiently simulates SABR model with novel sampling methods.
problem Sampling integrated variance and terminal forward price in SABR model.
method Moment-matched shifted lognormal approximation for integrated variance, CEV approximation for terminal forward price.
result Enhanced simulation scheme is highly efficient, accurate, and reliable.
VRCQ algorithm reduces variance in Q-learning for MDPs, achieving optimal sample complexity.
problem Estimating the optimal Q-function in MDPs with synchronous sampling.
method VRCQ combines direct variance reduction and Cascade Q-learning.
result VRCQ is minimax optimal and instance optimal for single-action problems.
VA-LUCB identifies best arm with variance constraint, achieving optimal sample complexity.
problem Identifying the best arm with variance constraint under fixed confidence.
method Parameter-free algorithm VA-LUCB, analyzing sample complexity and proving lower bounds.
result Optimal sample complexity up to a logarithmic factor in , demonstrated by experiments.
This paper balances bias and variance in adaptive importance sampling using mirror descent.
problem Large variance in adaptive importance sampling weights.
method Regularization strategy with power raised importance weights connected to mirror descent.
result The regularization parameter balances bias and variance.
A large portfolio of independent returns is optimized under the variance risk measure with a ban on short positions. The no-short selling constraint acts as an asymmetric regularizer, setting some of the portfolio weights to zero and keeping the out of sample estimator for the variance bounded, avoiding the di…
Introduce a variance-weighted batch distribution for diverse sampling in diffusion models.
problem Independent sampling in diffusion models.
method Introduce a variance-weighted batch distribution.
result Sampler with a transparent probabilistic target.
Improved sampling efficiency for inverse problems using variance-reduced diffusion methods.
problem Efficiently estimating noisy scores in inverse problems.
method Developed a nonparametric self-normalized importance sampling estimator and a state-dependent blending rule.
result Improved sample quality for fixed simulation budgets in synthetic targets and PDE-governed inverse problems.
Optimizes survey design for private mean estimation with reduced variance.
problem Minimizing variance in private mean estimation with privacy constraints.
method Formulates optimal survey design as an optimization problem, determining optimal subsampling sizes to minimize variance.
result Identifies the first privacy-aware stratified sampling scheme that minimizes variance under different privacy mechanisms.
A new measure -variance captures local distributional shape.
problem Summarizing distributional shape with local information.
method Random bipartite matchings and stochastic approximation.
result Easily approximated -variance measures capture local distributional properties.
AES improves policy gradient performance by adaptively selecting experience.
problem High variance in gradient estimators from past trajectories.
method AES learns an adaptive sampling distribution to minimise gradient variance.
result AES leads to significantly improved performance in continuous control tasks.
New method reduces model bias and variance by adjusting training sample weights based on label uncertainty.
problem Tradeoff between model bias and variance in classification models.
method Estimate label uncertainty, adjust training sample weights, and fine-tune decision boundary.
result Improves model performance and reduces variance in physical activity recognition.
Improved statistical inference for adaptive Thompson Sampling.
problem Statistical inference challenges in Thompson Sampling.
method Inflating posterior variance in Thompson Sampling.
result Asymptotically normal estimates of arm means with logarithmic regret increase.
Novel estimator reduces diffusion model variance.
problem High variance in score function estimation for diffusion models.
method Uses nearest neighbour samples to estimate the score function.
result Significant decrease in variance, leading to improved model performance.
PS-IG improves feature attribution by reducing noise and variance.
problem Improving feature attribution in machine learning models.
method Path-sampled integrated gradients (PS-IG) computes expected value over sampled baselines.
result PS-IG reduces attribution variance by a factor of 1/3 under uniform sampling.
Stochastic particle-optimization sampling (SPOS) is a recently-developed scalable Bayesian sampling framework that unifies stochastic gradient MCMC (SG-MCMC) and Stein variational gradient descent (SVGD) algorithms based on Wasserstein gradient flows. With a rigorous non-asymptotic convergence theory developed recently…
New method reduces variance in estimating PL model expectations.
problem High variance in Monte Carlo estimates of PL model expectations.
method Combining Gumbel top-k trick with quasi-Monte Carlo sampling.
result More sample-efficient estimators of PL model expectations.
Paper tackles unknown variances in best-arm identification.
problem Identifying the best arm with unknown variances in Gaussian distributions.
method Two approaches: empirical variance plugging or adapting transportation costs.
result The impact of unknown variances is small on sample complexity.
Estimates Markov chain variance efficiently without storing samples.
problem Estimating the asymptotic variance of Markov chain functions.
method Linear stochastic approximation of Poisson equation solution.
result Optimal MSE convergence rate with finite sample guarantees.
New strategies for identifying the best arm in bandits with decreasing variances.
problem Best arm identification in bandits with time-varying variances.
method Two policies: initial wait followed by continuous sampling, and periodic sampling with weighted average.
result Analytical guarantees and simulations show improved performance over existing methods.
Paper proposes SCott optimizer to reduce forecasting model training variance.
problem Large variance in gradient estimation for forecasting models.
method Stratified sampling and control variate to reduce gradient variance.
result SCott optimizer converges faster on time series forecasting problems.
Adaptive importance sampling for stochastic optimization is a promising approach that offers improved convergence through variance reduction. In this work, we propose a new framework for variance reduction that enables the use of mixtures over predefined sampling distributions, which can naturally encode prior knowledg…
Paper finds efficient OPE estimator for multiple logging policies with minimum variance.
problem Finding optimal importance sampling weights for multiple logging policies with varying variances.
method Established efficiency bound under stratified sampling and proposed an estimator achieving this bound.
result Proposed estimator achieves minimum variance for any instance.
FGTSVA improves Thompson Sampling for contextual bandits with optimal variance-aware regret.
problem Optimizing regret bounds for Thompson Sampling in contextual bandits.
method Developed FGTSVA, a variance-aware Thompson Sampling algorithm for contextual bandits with a new decoupling coefficient.
result Achieved optimal regret bound of .
Deep learning models can have low bias and variance, contrary to classical theory.
problem Understanding the performance of deep learning models at high complexity.
method Developed a fine-grained bias-variance decomposition for random feature kernel regression, analyzing the effects of sampling, initialization, and labels.
result The variance terms exhibit non-monotonic behavior and can diverge at the interpolation boundary, even in the absence of label noise.
Variational Variance: Simple, Reliable, Calibrated Heteroscedastic Noise Variance Parameterizationcs.LG
A simple method treats heteroscedastic variance variatively, improving model calibration and sample quality.
problem Brittle optimization impacts model likelihoods for mean and variance estimation.
method Proposes a variational approach to heteroscedastic variance, improving predictive mean and variance calibration.
result The proposed method significantly improves parameter calibration and sample quality for regression and VAEs.
A new method reduces variance in PG methods for RL, improving efficiency and convergence.
problem Improving sample efficiency and convergence of policy gradient methods in reinforcement learning.
method Proposes a gradient truncation mechanism and designs TSIVR-PG method to maximize rewards and utility.
result Shows sample complexity of TSIVR-PG to find ε-stationary policy and global ε-optimal policy.
Contrastive Divergence (CD) and Persistent Contrastive Divergence (PCD) are popular methods for training the weights of Restricted Boltzmann Machines. However, both methods use an approximate method for sampling from the model distribution. As a side effect, these approximations yield significantly different biases and…
A new permutation method improves two-sample testing power.
problem Two-sample testing with improved power and validity.
method Structured block-restricted cross-swaps.
result Block-restricted permutations achieve higher power than full permutations.
Paper introduces VDE, a variance-reduced determinant estimator.
problem Estimating determinants with low variance and efficiency.
method Combines variational inference and spherical normalizing flows.
result VDE achieves zero variance in ideal cases, requiring only one sample.
Many stochastic optimization algorithms work by estimating the gradient of the cost function on the fly by sampling datapoints uniformly at random from a training set. However, the estimator might have a large variance, which inadvertently slows down the convergence rate of the algorithms. One way to reduce this varian…