Before training a neural net, a classic rule of thumb is to randomly initialize the weights so the variance of activations is preserved across layers. This is traditionally interpreted using the total variance due to randomness in both weights \emph{and} samples. Alternatively, one can interpret the rule of thumb as pr…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “variance preservation”
A new aggregation strategy improves GNN performance and learning dynamics.
problem Improving expressivity and learning dynamics of GNNs.
method Proposes a variance-preserving aggregation function (VPA) for GNNs.
result VPA leads to increased predictive performance and improved learning dynamics.
Investigates projections onto explicit subspaces and their variance effects.
problem Understanding the variance preservation in explicit subspace projections.
method Investigates projections onto explicit subspaces of varying dimensionality and analyzes the variance effects.
result Developed new bounds for Euclidean distances and inner products.
Improves diffusion models by controlling total variance and signal-to-noise-ratio.
problem Long sampling time in diffusion models.
method Total-Variance/Signal-to-Noise-Ratio (TV/SNR) disentangled framework.
result Improves generation performance by controlling TV and SNR independently.
Variational Variance: Simple, Reliable, Calibrated Heteroscedastic Noise Variance Parameterizationcs.LG
A simple method treats heteroscedastic variance variatively, improving model calibration and sample quality.
problem Brittle optimization impacts model likelihoods for mean and variance estimation.
method Proposes a variational approach to heteroscedastic variance, improving predictive mean and variance calibration.
result The proposed method significantly improves parameter calibration and sample quality for regression and VAEs.
Beyond De Prado and Cotton: Hierarchical and Iterative Methods for General Mean-Variance Portfoliosq-fin.PM
New methods incorporate alpha signals into portfolio construction, improving performance.
problem Signal-blindness in existing portfolio construction methods.
method Introduces three methods: HRP-, HRP-, and CRISP.
result CRISP at intermediate consistently outperforms other methods.
TTRP method preserves distances in high-dimensional data with reduced storage and speed.
problem Preserving distances in high-dimensional datasets efficiently and accurately.
method Tensor train random projection (TTRP) using TT-ranks of one.
result TTRP is an expected isometric projection with bounded variance.
The article prices exchange options using variance gamma-like models.
problem Pricing exchange options under specific stochastic processes.
method Derives formulas for variance gamma and variance gamma++ processes, constructs multidimensional versions, calibrates parameters with real data.
result Closed formulas and numerical methods for evaluating exchange options.
Enhances machine learning models by preserving data structure, addressing statistical distortions.
problem Statistical distortions in synthetic data generated by Mixup.
method Proposes a generalized mixup method with a flexible weighting scheme to preserve data structure.
result Preserves statistical properties of original data while maintaining model performance.
SignSVRG improves SignSGD by reducing variance, achieving similar convergence rates.
problem Minimizing finite sums of convex and Lipschitz functions.
method Incorporates variance reduction techniques into SignSGD.
result Achieves convergence rates of for expected norm of the gradient and for smooth convex functions.
Dealing with high variance is a significant challenge in model-free reinforcement learning (RL). Existing methods are unreliable, exhibiting high variance in performance from run to run using different initializations/seeds. Focusing on problems arising in continuous control, we propose a functional regularization appr…
Flow matching improves synthetic data generation for tabular data.
problem Generating synthetic tabular data while preserving privacy.
method Flow matching (FM) and Variational FM for tabular data synthesis.
result FM outperforms diffusion baselines in tabular data synthesis.
Variance reduction (VR) methods boost the performance of stochastic gradient descent (SGD) by enabling the use of larger, constant stepsizes and preserving linear convergence rates. However, current variance reduced SGD methods require either high memory usage or an exact gradient computation (using the entire dataset)…
In this paper, we propose a novel reinforcement- learning algorithm consisting in a stochastic variance-reduced version of policy gradient for solving Markov Decision Processes (MDPs). Stochastic variance-reduced gradient (SVRG) methods have proven to be very successful in supervised learning. However, their adaptation…
PS-IG improves feature attribution by reducing noise and variance.
problem Improving feature attribution in machine learning models.
method Path-sampled integrated gradients (PS-IG) computes expected value over sampled baselines.
result PS-IG reduces attribution variance by a factor of 1/3 under uniform sampling.
SkMM selects data for finetuning by balancing bias and variance.
problem Balancing bias and variance in high-dimensional finetuning.
method Gradient sketching for bias reduction and moment matching for variance reduction.
result Gradient sketching selects samples efficiently and accurately.
A novel k-NN method estimates conditional mean and variance efficiently.
problem Joint estimation of conditional mean and variance.
method Integrates k-NN with automated variance selection.
result Achieves fast convergence rates and improved precision.
FPG uses fractional calculus for efficient reinforcement learning with long-term memory.
problem High variance and inefficient sampling in standard policy gradient methods for long-term temporal modeling.
method Fractional Policy Gradients (FPG) incorporating Caputo fractional derivatives for power-law temporal correlations.
result Achieves asymptotic variance reduction of order O(t^(-alpha)) and sample efficiency gains.
New initialization schemes preserve fractional moments of weights in deep networks, improving training and test performance.
problem Heavy-tailed distribution of stochastic gradients in DNNs during training.
method Developed initialization schemes that preserve any given fractional moment of order s < 2 over layers for various activations.
result The network output admits a heavy-tailed distribution with finite moments, improving training and test performance.
Adaptive collaboration for online personalized distributed learning with heterogeneous clientsstat.ML
A new method for online personalized learning reduces gradient variance by dynamically selecting peers.
problem Online personalized decentralized learning with statistically heterogeneous clients.
method Gradient-based collaboration criterion allowing clients to dynamically select peers with similar gradients.
result The method acts as a variance reduction method, achieving optimal performance in certain conditions.
Simplified analysis of diffusion models using discrete random variables.
problem Theoretical analysis of diffusion models is complex and requires rigorous proofs.
method Simplified framework for analyzing Euler--Maruyama discretization of VP-SDEs using Grönwall's inequality.
result Standard Gaussian noise can be replaced by discrete random variables without sacrificing convergence guarantee.
We show that stochastic recovery always leads to counter-intuitive behaviors in the risk measures of a CDO tranche - namely, continuity on default and positive credit spread risk cannot be ensured simultaneously. We then propose a simple recovery variance regularization method to control the magnitude of negative credi…
Multi-agent reinforcement learning (MARL) has recently received considerable attention due to its applicability to a wide range of real-world applications. However, achieving efficient communication among agents has always been an overarching problem in MARL. In this work, we propose Variance Based Control (VBC), a sim…
A new method resolves non-identifiability in reward modeling using anchor labels.
problem Non-identifiability in reward modeling from pairwise preferences alone.
method Anchor-guided Variance-aware Reward Modeling (AVRM) framework.
result AVRM resolves non-identifiability and improves reward modeling performance.
Efficiently simulates SABR model with novel sampling methods.
problem Sampling integrated variance and terminal forward price in SABR model.
method Moment-matched shifted lognormal approximation for integrated variance, CEV approximation for terminal forward price.
result Enhanced simulation scheme is highly efficient, accurate, and reliable.
Two new algorithms reduce feature space while preserving non-linear relationships.
problem High-dimensional data and overfitting issues.
method Bias-variance analysis for non-linear transformations and generalized linear models.
result Competitive performance on regression and classification tasks.
Proposes rounding method for precise treatment effect estimation under budget constraints.
problem Resource-constrained experimental design for precise treatment effect estimation.
method Dependent randomized rounding procedure to convert assignment probabilities into binary treatment decisions.
result Improved estimator precision through variance reduction and efficient inference.
Paper bridges statistical inference for DP-SGD, a privacy-preserving machine learning method.
problem Asymptotic statistical inference for Differentially Private Stochastic Gradient Descent (DP-SGD).
method Established asymptotic properties of SGD under randomized subsampling, extended to DP-SGD, proposed methods for constructing valid confidence intervals.
result Valid confidence intervals for DP-SGD output achieve nominal coverage rates while maintaining privacy.
Bagging reduces variance in LID estimation by preserving local distribution of NN distances.
problem High estimation variance from limited data in small neighborhoods.
method Subbagging to preserve local distribution of NN distances, combined with ensemble size.
result Bagging significantly reduces variance and MSE in LID estimation.
New method uses machine learning to improve statistical inference.
problem Performing inference on conditional functionals with scarce labeled data.
method Combines localization with prediction-based variance reduction.
result Valid and sharp confidence intervals for conditional functionals.
Two new estimators reduce costs and improve accuracy for EHR outcome prediction.
problem Sparse estimate distributions, high computational cost, and high sampling variance in EHR outcome prediction.
method Proposed SCOPE and REACH estimators that leverage next-token probability distributions.
result SCOPE and REACH match Monte Carlo accuracy with token reductions of 2.5-3.4 times and variance guarantees.
Bi-Lipschitz flows approximate a wide range of distributions.
problem Characterizing the expressivity of bi-Lipschitz normalizing flows.
method Linking score regularity to transport map bi-Lipschitzness via probability flow ODE.
result Gaussian pullbacks induced by bi-Lipschitz variance-preserving transport maps are -dense among all probability densities.
Deep neural networks suffer from over-fitting and catastrophic forgetting when trained with small data. One natural remedy for this problem is data augmentation, which has been recently shown to be effective. However, previous works either assume that intra-class variances can always be generalized to new classes, or e…
Proposes -PCA to learn identifiable linear transformations without whitening.
problem Cannot identify axes with equal variances in PCA.
method Unified model for linear and nonlinear PCA, introducing a missing piece to eliminate rotational indeterminacy.
result Eliminates subspace rotational indeterminacy in PCA.
PEMC uses ML to enhance Monte Carlo simulations, reducing variance and runtime.
problem Computational inefficiency in Monte Carlo simulations for complex tasks.
method Prediction-Enhanced Monte Carlo (PEMC) framework that uses ML surrogates as predictors.
result PEMC provides unbiased evaluations with reduced variance and runtime compared to standard Monte Carlo.
Proposes ENVAR for causal discovery in structural VAR models with equal noise variance.
problem Challenges in causal discovery from multivariate time series with contemporaneous effects.
method Introduces observational equivalence and the observational alignment discrepancy for structural VAR models with equal noise variance.
result Shows that multiple structural VAR parameterizations can induce the same stationary observed process law.
FlowSDR learns a low-dimensional projection preserving the response's conditional distribution.
problem Learning a low-dimensional projection that captures the response's conditional distribution.
method FlowSDR uses conditional log-likelihood maximization with monotone rational-quadratic spline flows to learn the projection and conditional density.
result FlowSDR outperforms existing SDR methods in various simulation settings and a face-age prediction task.
Unified analysis for deterministic samplers in diffusion models.
problem Challenges in analyzing deterministic samplers for diffusion models.
method Unified convergence analysis framework.
result Achieved polynomial iteration complexity for DDIM-type samplers.
New method reduces bias and variance in OPE for large action spaces.
problem High bias and variance in OPE for large, combinatorial action spaces.
method Factored action spaces and decomposed importance sampling.
result Decomposed IS estimators have less variance than non-decomposed versions.
A Scalable Gradient-Based Optimization Framework for Sparse Minimum-Variance Portfolio Selectionstat.ML
A scalable gradient-based framework for sparse portfolio selection.
problem Sparse minimum-variance portfolio selection with cardinality constraint.
method Gradient-based optimization with Boolean relaxation and tunable parameter.
result Matches commercial solvers in most instances, differing by a few assets with negligible error in portfolio variance.
In this paper we want to exploit further the semi-discrete method appeared in Halidias and Stamatiou (2015). We are interested in the numerical solution of mean reverting CEV processes that appear in financial mathematics models and are described as non negative solutions of certain stochastic differential equations wi…
Spectral Gradient Descent Mitigates Anisotropy-Driven Misalignment: A Case Study in Phase Retrievalstat.ML
SpecGD mitigates misalignment in phase retrieval models with anisotropic inputs.
problem Misalignment during gradient descent in phase retrieval models with anisotropic inputs.
method Spectral gradient descent modifies gradient updates to preserve directional information and remove spike amplification.
result SpecGD removes spike amplification, leading to stable alignment and accelerated noise contraction.
New estimator improves off-policy evaluation for large action spaces.
problem Conventional importance-weighting approaches suffer from excessive variance in off-policy evaluation for large discrete action spaces.
method Proposes OffCEM estimator based on conjunct effect model (CEM), applying importance weighting only to action clusters and using model-based reward estimation for residual effects.
result Proposed estimator is unbiased under local correctness condition, providing substantial improvements in OPE especially with many actions.
Bayesian investor learns unknown asset drift, trades mean-variance optimal portfolio, but policy is robust to observation model distortion.
problem Bayesian portfolio selection with observation model distortion
method Robust Bayesian portfolio selection
result Robust policy and its price are closed form, with price of robustness half the variance of the non-robust investor's loss.
It has often been stated that, within the class of continuous stochastic volatility models calibrated to vanillas, the price of a VIX future is maximized by the Dupire local volatility model. In this article we prove that this statement is incorrect: we build a continuous stochastic volatility model in which a VIX futu…
Class Normalization improves zero-shot learning models.
problem Improving zero-shot learning models in a continual setting.
method Class Normalization (CN) technique to address irregular loss surfaces and improve training efficiency.
result CN significantly outperforms state-of-the-art models on 4 standard ZSL datasets.
New projection techniques reduce the frequency of projections in solving LCPs.
problem Solving linearly constrained problems efficiently with reduced projection frequency.
method Delayed projection technique to call a projection less frequently.
result Theoretical and practical improvements in convergence rates and efficiency.
A new method for efficient portfolio optimization using graph structures.
problem Optimizing portfolio weights while reducing computational complexity.
method Hierarchical graph structures and Schur complement method.
result Optimal portfolio weights can be computed efficiently by inverting small submatrices.