arXiv research

We analyze the variance of Fisher information estimators in deep learning models.

Two Fisher information matrix estimators are analyzed for neural networks, focusing on their variances and trade-offs.

Language models allocate information storage, not collapsing into uniform representations.

A new measure $k$-variance captures local distributional shape.

We consider the mean-variance hedging problem under partial Information. The underlying asset price process follows a continuous semimartingale and strategies have to be constructed when only part of the information in the market is available. We show that the initial mean variance hedging problem is equivalent to a ne…

Method estimates group structure in panel data using variance information.

Improved SVRG method using BB techniques for faster convergence.

This paper tackles variance issues in GNN training by proposing a method to reduce both embedding and gradient variances.

Faster convergence of kernel mean embeddings using variance information.

New Riemannian optimization improves variance estimation in mixed models.

Investors with asymmetric information play a game to optimize their portfolios.

Study tight offline learning bounds for linear MDPs using variance information.

We present a set of log-price integrated variance estimators, equal to the sum of open-high-low-close bridge estimators of spot variances within $n$ subsequent time-step intervals. The main characteristics of some of the introduced estimators is to take into account the information on the occurrence times of the high a…

Variational approaches based on neural networks are showing promise for estimating mutual information (MI) between high dimensional variables. However, they can be difficult to use in practice due to poorly understood bias/variance tradeoffs. We theoretically show that, under some conditions, estimators such as MINE ex…

In this paper, we study the mean-variance portfolio selection problem under partial information with drift uncertainty. First we show that the market model is complete even in this case while the information is not complete and the drift is uncertain. Then, the optimal strategy based on partial information is derived, …

Study finds adding more information to robust option pricing does not improve bounds.

Deep learning model estimates mutual information with low bias and variance.

VA-OPE improves OPE by incorporating variance information, achieving tighter error bounds.

The paper analyzes how investors' wealth can decline collectively under partial information.

Bayesian framework improves variance component estimation in MET data.

New pricing model uses variance-gamma process for financial assets.

Data balancing reduces variance in machine learning models.

The paper explores the trade-off between bias and variance in high-dimensional models.

Improved robustness in optimization methods using second-order information.

Improved mean estimation for symmetric distributions with finite-sample guarantees.

This work introduces a bias-variance decomposition for proper scores, improving uncertainty estimation in predictive models.

New method improves convergence and reduces variance in noisy optimization problems.

New DR-IC estimator reduces bias and variance in OPE.

Active learning aims to train a classifier as fast as possible with as few labels as possible. The core element in virtually any active learning strategy is the criterion that measures the usefulness of the unlabeled data based on which new points to be labeled are picked. We propose a novel approach which we refer to …

Policy optimization on high-dimensional continuous control tasks exhibits its difficulty caused by the large variance of the policy gradient estimators. We present the action subspace dependent gradient (ASDG) estimator which incorporates the Rao-Blackwell theorem (RB) and Control Variates (CV) into a unified framework…

Humans are able to accelerate their learning by selecting training materials that are the most informative and at the appropriate level of difficulty. We propose a framework for distributing deep learning in which one set of workers search for the most informative examples in parallel while a single worker updates the …

A novel clustering method using a subset of data for better performance.

The posterior variance of Gaussian processes is a valuable measure of the learning error which is exploited in various applications such as safe reinforcement learning and control design. However, suitable analysis of the posterior variance which captures its behavior for finite and infinite number of training data is …

New approach combines PCA and t-sne for better data analysis.

Study shows Stochastic Mirror Descent optimizes convex problems with infinite noise variance.

Estimating and optimizing Mutual Information (MI) is core to many problems in machine learning; however, bounding MI in high dimensions is challenging. To establish tractable and scalable objectives, recent work has turned to variational bounds parameterized by neural networks, but the relationships and tradeoffs betwe…

Choosing appropriate step sizes is critical for reducing the computational cost of training large-scale neural network models. Mini-batch sub-sampling (MBSS) is often employed for computational tractability. However, MBSS introduces a sampling error, that can manifest as a bias or variance in a line search. This is bec…

We prove that the evidence lower bound (ELBO) employed by variational auto-encoders (VAEs) admits non-trivial solutions having constant posterior variances under certain mild conditions, removing the need to learn variances in the encoder. The proof follows from an unexpected journey through an array of topics: the clo…

We analyze Gibbs-based transfer learning algorithms using information theory.

Proposes reducing random error in stochastic optimization by variance regularization.

We present and prove properties of a new offline policy evaluator for an exploration learning setting which is superior to previous evaluators. In particular, it simultaneously and correctly incorporates techniques from importance weighting, doubly robust evaluation, and nonstationary policy evaluation approaches. In a…

This paper investigates optimal portfolio strategies in a financial market where the drift of the stock returns is driven by an unobserved Gaussian mean reverting process. Information on this process is obtained from observing stock returns and expert opinions. The latter provide at discrete time points an unbiased est…

TRSVR combines SVRG with trust-region for faster optimization.

This paper addresses dueling bandits with contextual information, improving regret bounds by accounting for variance.

The multi-armed bandit (MAB) problem is a classical learning task that exemplifies the exploration-exploitation tradeoff. However, standard formulations do not take into account {\em risk}. In online decision making systems, risk is a primary concern. In this regard, the mean-variance risk measure is one of the most co…

The paper bounds generalization error for iterative learning with bounded updates.

Optimal B-robust estimate is constructed for multidimensional parameter in drift coefficient of diffusion type process with small noise. Optimal mean-variance robust (optimal V -robust) trading strategy is find to hedge in mean-variance sense the contingent claim in incomplete financial market with arbitrary informatio…

Adaptive OMD reduces variance in learning optimal strategies for imperfect information games.