The paper tackles mean-variance analysis in Bayesian optimization under uncertainty.
problem Optimizing decisions in uncertain environments considering trade-offs between average and variance of risk.
method Developed bounds for mean and variance risk measures in Gaussian Process models and proposed AL algorithms for multi-task, multi-objective, and constrained optimization scenarios.
result Proposed AL algorithms effectively address the mean-variance trade-off in uncertain optimization scenarios.
Proposes counterfactual explainability for causal attribution, extending variance analysis methods.
problem Lack of mechanistic understanding in existing tools for explaining complex models.
method Extends global sensitivity analysis methods to causal explanations using directed acyclic graphs.
result Developed methods to estimate counterfactual explainability and applied to income inequality analysis.
Significant improvements in regret analysis for adaptive online learning problems.
problem Exploiting low variance in online learning problems without known variances.
method Novel peeling-based regret analysis leveraging elliptical potential `count` lemma.
result Significant improvements in regret bounds for linear bandits and linear mixture MDPs.
New methods solve graph sparsity optimization problems faster.
problem Complex graph sparsity optimization problems in disease outbreak monitoring and social network analysis.
method Stochastic variance-reduced gradient-based methods GraphSVRG-IHT and GraphSCSG-IHT.
result Our methods achieve linear convergence speed.
BS-VAE separates decoder variance and beta to improve VAE performance.
problem Blurriness in VAE outputs and difficulty in analyzing model performance.
method Explicitly separates beta and decoder variance in Beta-Sigma VAE.
result Superior performance in natural image synthesis and controllable parameters.
The paper identifies the minimum mean-variance spanning set and its importance in asset evaluation.
problem Estimating the minimum subset of assets that span the efficient frontier.
method Established identification conditions and developed a novel procedure for MSS estimation and inference.
result The MSS estimator accurately covers the true MSS and converges to it at any desired confidence level.
Optimizes variance reduction in Heston model using large and moderate deviations.
problem Improving variance reduction in stochastic volatility models.
method Large and moderate deviations theory applied to Heston model.
result Derives closed-form solutions for optimal change of measure.
We use the 2014 market history of two high-returning biotechnology exchange-traded funds to illustrate how ex post mean-variance analysis should not be done. Unfortunately, the way it should not be done is the way it generally is done -- to our knowledge.
The posterior variance of Gaussian processes is a valuable measure of the learning error which is exploited in various applications such as safe reinforcement learning and control design. However, suitable analysis of the posterior variance which captures its behavior for finite and infinite number of training data is …
Enhances sensitivity analysis for correlated inputs.
problem Estimating sensitivity indices in models with correlated inputs.
method Proposes an extension of Sobol' estimator using a linear correlation model.
result Improves accuracy in variance-based sensitivity analysis.
We revisit the stochastic variance-reduced policy gradient (SVRPG) method proposed by Papini et al. (2018) for reinforcement learning. We provide an improved convergence analysis of SVRPG and show that it can find an ε-approximate stationary point of the performance function within O(1/ε5/3) trajectories. This s…
In this paper, we propose a random projection approach to estimate variance in kernel ridge regression. Our approach leads to a consistent estimator of the true variance, while being computationally more efficient. Our variance estimator is optimal for a large family of kernels, including cubic splines and Gaussian ker…
Simplified analysis of diffusion models using discrete random variables.
problem Theoretical analysis of diffusion models is complex and requires rigorous proofs.
method Simplified framework for analyzing Euler--Maruyama discretization of VP-SDEs using Grönwall's inequality.
result Standard Gaussian noise can be replaced by discrete random variables without sacrificing convergence guarantee.
Optimal feature transfer identified through bias-variance analysis.
problem Optimizing feature transfer in transfer learning.
method Simple linear model with fine-grained bias-variance decomposition.
result Optimal pretrained feature transform is naturally sparse.
A new statistical concept, lepto-variance, is defined for stock returns using Regression Trees.
problem Understanding the underlying structure of stock returns using statistical methods.
method Defining lepto-variance as the variance that cannot be removed by any regression tree of a specific depth and analyzing stock returns with 1- and 2-bit Regression Trees.
result Lepto-variance quantifies the resolving power of Regression Trees for stock returns, decomposing total variance into lepto-variance and macro-variance.
Asymptotic analysis of short-maturity options on realized variance in local-stochastic volatility models.
problem Analyzing the behavior of short-maturity options on realized variance in local-stochastic volatility models.
method Large deviations theory and variational problems to solve rate functions for different cases.
result Explicit solutions for the rate function in the uncorrelated case and upper/lower bounds and expansions for the correlated case.
Bayesian nonparametrics are a class of probabilistic models in which the model size is inferred from data. A recently developed methodology in this field is small-variance asymptotic analysis, a mathematical technique for deriving learning algorithms that capture much of the flexibility of Bayesian nonparametric infere…
Probabilistic principal component analysis (PPCA) seeks a low dimensional representation of a data set in the presence of independent spherical Gaussian noise, Sigma = (sigma^2)*I. The maximum likelihood solution for the model is an eigenvalue problem on the sample covariance matrix. In this paper we consider the situa…
Unified analysis of stochastic gradient methods for convex and smooth optimization.
problem Minimizing composite convex and smooth functions.
method Unified convergence analysis of various stochastic gradient methods.
result Unified convergence rates for a variety of methods including proximal SGD, variance reduced methods, quantization, and coordinate descent.
VAEs analyzed using harmonic analysis, showing how variance controls frequency content and robustness.
problem Understanding and optimizing VAEs for robustness and frequency control.
method Viewing VAE latent space as Gaussian space, deriving results on variance and frequency content, and demonstrating soft Lipschitz constraints.
result Increasing encoder variance reduces high frequency content and improves adversarial robustness.
Bias - variance decomposition of the expected error defined for regression and classification problems is an important tool to study and compare different algorithms, to find the best areas for their application. Here the decomposition is introduced for the survival analysis problem. In our experiments, we study bias -…
Improved GP bandit algorithms for noiseless, varying noise, and RKHS norms.
problem Minimizing regret in Gaussian process bandits with unknown reward functions.
method New upper bound on maximum posterior variance, refined MVR and PE algorithms.
result Optimal regret bounds for noiseless, varying noise, and RKHS norms.
In this paper we propose a novel variance reduction approach for additive functionals of Markov chains based on minimization of an estimate for the asymptotic variance of these functionals over suitable classes of control variates. A distinctive feature of the proposed approach is its ability to significantly reduce th…
A new method detects anomalies in multivariate streams without unit dependence.
problem Detect anomalies in multivariate streams without unit dependence.
method Proposes SigMahaKNN combining variance norm and path signature.
result SigMahaKNN detects anomalies better than existing methods.
The paper analyzes the variance of different shuffling methods in stochastic gradient descent.
problem Understanding the variance of different shuffling methods in stochastic gradient descent.
method Power spectral density analysis to study the noise sequences of stochastic gradients.
result The stationary variances of iterates decrease in the order of SGD, SGD-RR, and SGD-SO.
Temporal difference (TD) learning is a popular algorithm for policy evaluation in reinforcement learning, but the vanilla TD can substantially suffer from the inherent optimization variance. A variance reduced TD (VRTD) algorithm was proposed by Korda and La (2015), which applies the variance reduction technique direct…
Techniques for reducing the variance of gradient estimates used in stochastic programming algorithms for convex finite-sum problems have received a great deal of attention in recent years. By leveraging dissipativity theory from control, we provide a new perspective on two important variance-reduction algorithms: SVRG …
New approach combines PCA and t-sne for better data analysis.
problem Multiscale complexity in high-dimensional data.
method Multiscale joint characterization using PCA and t-sne.
result Joint characterization detects signals not seen by PCA or t-sne alone.
Two derivations of PCA for distributional data.
problem PCA for datasets of distributions.
method Two derivations: variance maximization and reconstruction error minimization.
result Closed-form solution for distributional PCA.
A new measure k-variance captures local distributional shape.
problem Summarizing distributional shape with local information.
method Random bipartite matchings and stochastic approximation.
result Easily approximated k-variance measures capture local distributional properties. As application demands for zeroth-order (gradient-free) optimization accelerate, the need for variance reduced and faster converging approaches is also intensifying. This paper addresses these challenges by presenting: a) a comprehensive theoretical analysis of variance reduced zeroth-order (ZO) optimization, b) a nove…
New method optimizes PCA for better prediction and variance.
problem Improve PCA for better prediction and variance.
method Jointly optimize prediction error and variance explained.
result Our method outperforms existing approaches in both prediction and variance.
Improved variance reduction for Riemannian non-convex optimization with adaptive batch size.
problem Optimizing non-convex functions on Riemannian manifolds.
method Batch size adaptation in R-SVRG, R-SRG, and R-SPIDER.
result Achieves lower total complexities for various non-convex functions.
UCB-V algorithm improves on UCB for MAB problems with variance estimates.
problem Optimizing arm selection in MAB problems with variance information.
method Asymptotic and high probability analysis of UCB-V algorithm.
result UCB-V can exhibit instability in arm-pulling rates but achieves refined regret bounds.
Proposes a novel MTL approach based on bias-variance analysis.
problem Improving multi-task learning performance through shared knowledge.
method Two-phase iterative aggregation of targets and features using bias-variance analysis.
result Validation on synthetic and real-world datasets demonstrates the effectiveness of the proposed method.
This paper explores the non-convex composition optimization in the form including inner and outer finite-sum functions with a large number of component functions. This problem arises in some important applications such as nonlinear embedding and reinforcement learning. Although existing approaches such as stochastic gr…
Simplified analysis of SGD for linear regression with weight averaging.
problem Understanding SGD optimization in linear regression models.
method Simplified analysis using linear algebra tools, bypassing complex operator manipulations.
result Recovery of bias and variance bounds for SGD in linear regression.
Unified framework for output analysis using Monte Carlo sampling.
problem Accurately assess the quality of estimated values in predictive models.
method Unified output analysis framework through Monte Carlo sampling, leveraging fast iterative bootstrap sampling and higher-order influence functions.
result Clear advantage in building more robust confidence intervals with higher coverage probability.
We provide a simple explicit estimator for discretely observed Barndorff-Nielsen and Shephard models, prove rigorously consistency and asymptotic normality based on the single assumption that all moments of the stationary distribution of the variance process are finite, and give explicit expressions for the asymptotic …
New method reduces variance in stochastic optimization with high confidence.
problem Achieving high-probability guarantees in stochastic optimization with weaker noise assumptions.
method Stochastic proximal point method combining proximal subproblem solver and probability booster.
result Demonstrates convergence with low sample complexity under bounded variance assumptions.
We consider the task of classification in the high dimensional setting where the number of features of the given data is significantly greater than the number of observations. To accomplish this task, we propose a heuristic, called sparse zero-variance discriminant analysis (SZVD), for simultaneously performing linear …
A new method for online personalized learning reduces gradient variance by dynamically selecting peers.
problem Online personalized decentralized learning with statistically heterogeneous clients.
method Gradient-based collaboration criterion allowing clients to dynamically select peers with similar gradients.
result The method acts as a variance reduction method, achieving optimal performance in certain conditions.
Stochastic Gradient Descent has been widely studied with classification accuracy as a performance measure. However, these stochastic algorithms cannot be directly used when non-decomposable pairwise performance measures are used such as Area under the ROC curve (AUC) which is a common performance metric when the classe…
The paper analyzes how re-weighting helps in reducing variance in high-dimensional kernel methods under covariate shifts.
problem The challenge of high-dimensional kernel methods under covariate shifts and the role of re-weighting.
method Derives asymptotic expansion of high-dimensional kernels under covariate shifts, analyzes bias-variance decomposition, and characterizes the regularized kernel.
result Re-weighting helps in decreasing variance and can be seen as a data-dependent regularization.
This paper addresses error bounds and posterior variance for Gaussian process regression.
problem Deriving performance guarantees for Gaussian process regression without prior knowledge.
method Lipschitz continuity and analysis of posterior variance function.
result Uniform error bounds for Gaussian process regression are derived.
The portfolio optimization problem in which the variances of the return rates of assets are not identical is analyzed in this paper using the methodology of statistical mechanical informatics, specifically, replica analysis. We define two characteristic quantities of an optimal portfolio, namely, minimal investment ris…
This work uses ANOVA to understand how different factors contribute to test error in machine learning models.
problem Understanding why overparametrized models generalize well despite potentially fitting noise.
method Analysis of variance (ANOVA) to decompose test error into components of variance.
result The interaction between training samples and initialization can dominate variance, and there are phase transitions in variance behavior.
New Monte Carlo method outperforms existing strategy for estimating Sobol' indices.
problem Estimating first-and total-orders Sobol' indices accurately.
method Comparing two Monte Carlo estimators for Sobol' indices.
result New method outperforms current approach in accuracy.