New method for unbiased regression reduces excess risk.
problem Least squares regression with optimal solution and Hessian matrix.
method Averaged stochastic gradient descent with time-average estimator.
result Unbiased estimator with O(1/k) expected excess risk.
This paper presents a novel scaling method for unbiased risk estimation.
problem Challenges in risk assessment due to limited data, non-stationarity, and heavy tails.
method Develops a statistical framework for efficient risk scaling, extending beyond the square-root-of-time rule.
result Ensures robust and conservative risk estimation, applicable to small sample settings.
The estimation of risk measures recently gained a lot of attention, partly because of the backtesting issues of expected shortfall related to elicitability. In this work we shed a new and fundamental light on optimal estimation procedures of risk measures in terms of bias. We show that once the parameters of a model ne…
Paper proposes an unbiased risk estimator for PLLAC, handling unseen classes.
problem Handling unseen classes in PLLAC where some classes are not present in the training set.
method Proposes an unbiased risk estimator that estimates the distribution of augmented classes by differentiating known classes from unlabeled data.
result The estimator provides theoretical guarantees and converges to true risk minimizer as data increases.
Develops new methods to estimate treatment effects in survival data with competing risks.
problem Estimating treatment effects in survival data with competing risks.
method Censoring Unbiased Transformations (CUTs) for survival outcomes with and without competing risks.
result Consistent estimates of heterogeneous cumulative incidence effects and total effects using HTE learners.
From only positive (P) and unlabeled (U) data, a binary classifier could be trained with PU learning, in which the state of the art is unbiased PU learning. However, if its model is very flexible, empirical risks on training data will go negative, and we will suffer from serious overfitting. In this paper, we propose a…
This work improves texture segmentation by automatically tuning hyperparameters for Total-Variation.
problem The challenge is to automatically select hyperparameters for Total-Variation texture segmentation.
method The approach involves extending Stein's unbiased gradient estimator to handle correlated Gaussian noise, leading to an automatic tuning method.
result The method provides an automatic way to select hyperparameters for Total-Variation texture segmentation.
UREs lead to overfitting in complex models, especially in complementary label learning.
problem Overfitting in weakly supervised learning with complementary labels.
method Proposed a surrogate complementary loss (SCL) framework to reduce gradient variance.
result SCL mitigates overfitting and improves URE-based methods.
Randomized trials, also known as A/B tests, are used to select between two policies: a control and a treatment. Given a corresponding set of features, we can ideally learn an optimized policy P that maps the A/B test data features to action space and optimizes reward. However, although A/B testing provides an unbiased …
Pairwise similarities and dissimilarities between data points might be easier to obtain than fully labeled data in real-world classification problems, e.g., in privacy-aware situations. To handle such pairwise information, an empirical risk minimization approach has been proposed, giving an unbiased estimator of the cl…
New method learns from noisy data without knowing noise level.
problem Learning from noisy data without knowing noise level.
method Uses Stein's Unbiased Risk Estimate (SURE) without noise level knowledge.
result Outperforms other self-supervised methods on imaging problems.
Paper tackles weakly supervised learning from similarity-confidence data.
problem Learning binary classifier from unlabeled data pairs with confidence of similarity.
method Proposes an unbiased estimator of classification risk from Sconf data and risk correction scheme.
result Demonstrates effectiveness of proposed methods through experiments.
In this analytical study we derive the optimal unbiased value estimator (MVU) and compare its statistical risk to three well known value estimators: Temporal Difference learning (TD), Monte Carlo estimation (MC) and Least-Squares Temporal Difference Learning (LSTD). We demonstrate that LSTD is equivalent to the MVU if …
Using integration by parts on Gaussian space we construct a Stein Unbiased Risk Estimator (SURE) for the drift of Gaussian processes using their local and occupation times. By almost-sure minimization of the SURE risk of shrinkage estimators we derive an estimation and de-noising procedure for an input signal perturbed…
Among the plethora of techniques devised to curb the prevalence of noise in medical images, deep learning based approaches have shown the most promise. However, one critical limitation of these deep learning based denoisers is the requirement of high-quality noiseless ground truth images that are difficult to obtain in…
A weakly-supervised learning framework named as complementary-label learning has been proposed recently, where each sample is equipped with a single complementary label that denotes one of the classes the sample does not belong to. However, the existing complementary-label learning methods cannot learn from the easily …
Estimates conversion probabilities from click sequences with privacy constraints.
problem Training models in advertising with limited direct click-conversion links.
method Formalizes learning from attribution sets, constructs unbiased estimator, applies Empirical Risk Minimization.
result Empirical Risk Minimization achieves generalization guarantees and robustness against prior errors.
This paper studies the problem of learning with augmented classes (LAC), where augmented classes unobserved in the training data might emerge in the testing phase. Previous studies generally attempt to discover augmented classes by exploiting geometric properties, achieving inspiring empirical performance yet lacking t…
Paper proposes unbiased learning for recommendation causal effects.
problem Estimating the causal effect of recommendation when the ground truth is unobservable.
method Inverse propensity scoring technique to construct unbiased estimators, followed by empirical risk minimization with propensity capping.
result The proposed method outperforms other biased learning methods in various settings.
Bayesian method improves extreme quantile estimation with zero coverage error.
problem Estimating extreme quantiles with zero coverage error in small samples.
method Bayesian quantile estimation using Jeffreys prior.
result Bayesian method results in zero coverage error, unlike maximum likelihood.
Develops new instance-optimality concepts in differential privacy.
problem Improving privacy guarantees in statistical estimation.
method Introduces local minimax risk and unbiased mechanisms, and develops inverse sensitivity mechanisms.
result Inverse sensitivity mechanisms are nearly instance optimal for a wide range of functions.
In this paper, we obtain a property of the expectation of the inverse of compound Wishart matrices which results from their orthogonal invariance. Using this property as well as results from random matrix theory (RMT), we derive the asymptotic effect of the noise induced by estimating the covariance matrix on computing…
Reinforcement learning agents are faced with two types of uncertainty. Epistemic uncertainty stems from limited data and is useful for exploration, whereas aleatoric uncertainty arises from stochastic environments and must be accounted for in risk-sensitive applications. We highlight the challenges involved in simultan…
The paper analyzes the risk of CV-tuned regularized estimators and connects it to SURE.
problem Understanding the risk of CV-tuned regularized estimators.
method Derives asymptotic risk function of CV-tuned estimators and connects it to SURE.
result The risk function provides a more detailed picture of predictive performance than uniform bounds.
Stochastic gradient descent (SGD), which dates back to the 1950s, is one of the most popular and effective approaches for performing stochastic optimization. Research on SGD resurged recently in machine learning for optimizing convex loss functions and training nonconvex deep neural networks. The theory assumes that on…
A new framework for bilevel optimization tackles stochastic and global variance reduction.
problem Bilevel optimization challenges in large-scale empirical risk minimization.
method Introducing a novel framework where inner and main variables evolve simultaneously, leading to unbiased estimates and global variance reduction algorithms.
result SABA algorithm achieves $O( rac{1}{T})$ convergence rate and linear convergence under Polyak-Lojasciewicz assumption.
GARCH-UGH improves VaR estimation for financial risk management.
problem Dynamic estimation of extreme VaR in financial time series.
method AR-GARCH filtering followed by a bias-reduced extreme value estimator.
result GARCH-UGH estimates are more accurate than conventional methods.
In this paper we address the problem of pool based active learning, and provide an algorithm, called UPAL, that works by minimizing the unbiased estimator of the risk of a hypothesis in a given hypothesis space. For the space of linear classifiers and the squared loss we show that UPAL is equivalent to an exponentially…
PPAT uses predictions to improve risk estimation in active testing.
problem Exploiting informative predictions from black-box models for efficient risk estimation.
method Combines LURE estimator with prediction-powered control variate.
result PPAT outperforms existing methods in risk estimation and uncertainty quantification.
MUSE provides unbiased stopping estimates for optimal problems.
problem Estimating the utility of optimal stopping problems.
method Backward recursive construction of the Multilevel Unbiased Stopping Estimator (MUSE).
result MUSE achieves ε-accuracy with O(1/ε^2) computational cost.
SUMO provides unbiased log marginal likelihood estimation for latent variable models.
problem Biased estimates of log marginal likelihood in latent variable models.
method Randomized truncation of infinite series for unbiased estimation.
result Models trained with SUMO give better test-set likelihoods than standard methods.
The problem of estimating a high-dimensional sparse vector θ∈Rn from an observation in i.i.d. Gaussian noise is considered. The performance is measured using squared-error loss. An empirical Bayes shrinkage estimator, derived using a Bernoulli-Gaussian prior, is analyzed and compared with the…
Proposes unbiased estimators for training mixture of experts models.
problem Efficiently training large-scale mixture of experts models on modern hardware.
method Two unbiased estimators based on principled stochastic assignment procedures.
result Both estimators are more effective and robust than biased alternatives.
New algorithm for risk-sensitive reinforcement learning with natural policy gradients.
problem Risk-sensitive reinforcement learning with downside risk constraints.
method Introduce a new Bellman equation to estimate the lower partial moment of returns, use natural policy gradients, and extend Reward Constrained Policy Optimization.
result Sample-efficient estimation of partial moments and effective risk-sensitive control.
Many applied settings in empirical economics involve simultaneous estimation of a large number of parameters. In particular, applied economists are often interested in estimating the effects of many-valued treatments (like teacher effects or location effects), treatment effects for many groups, and prediction models wi…
Unbiased gradient estimation for Markov chains
problem Estimating gradients of stationary means in Markov chains
method Propose new unbiased estimators
result Improves efficiency for slow mixing Markov chains
Computing partition functions, the normalizing constants of probability distributions, is often hard. Variants of importance sampling give unbiased estimates of a normalizer Z, however, unbiased estimates of the reciprocal 1/Z are harder to obtain. Unbiased estimates of 1/Z allow Markov chain Monte Carlo sampling of "d…
New method uses SURE to denoise signals, outperforming NPMLE.
problem Learning to optimally denoise signals corrupted by Gaussian noise.
method Hyvärinen's score matching (SM) is shown equivalent to SURE minimization.
result SURE achieves nearly parametric rates of convergence in empirical Bayes settings.
New unbiased gradient estimators for complex optimization problems.
problem Unbiased and variance-limited gradient estimation for conditional stochastic optimization.
method Developed multilevel Monte Carlo gradient estimators for conditional stochastic optimization problems.
result Unbiased and finite variance gradient estimators for conditional stochastic optimization problems.
New theory of sensitivity for unbiased estimators using Wasserstein geometry.
problem Estimating the instability of estimators under small perturbations.
method Developed a new theory based on Wasserstein geometry, analogous to classical Cramér-Rao theory.
result Wasserstein-Cramér-Rao lower bound for sensitivity of unbiased estimators.
ENSURE framework trains deep image recon algorithms without clean data.
problem Lack of clean, fully sampled ground-truth data for deep learning image reconstruction.
method Introduces ENSURE framework, a generalization of SURE and GSURE to random sampling patterns.
result ENSURE loss function is an unbiased estimate for true mean-square error.
This paper evaluates different methods to estimate S&P 500 volatility.
problem Accurately estimating the volatility of the S&P 500 index.
method Historical volatility, GARCH model, and implied volatility methods were compared.
result Implied volatility is the best estimator of real volatility.
Paper develops unbiased gradient estimator for continuous-time models.
problem Estimating unbiased gradient of log-likelihood for continuous-time models.
method Doubly randomized scheme with coupled conditional particle filter (CCPF).
result Unbiased gradient estimate facilitates gradient-based algorithms.
In this paper, we introduce a new approach to constructing unbiased estimators when computing expectations of path functionals associated with stochastic differential equations (SDEs). Our randomization idea is closely related to multi-level Monte Carlo and provides a simple mechanism for constructing a finite variance…
Optimal Gaussian noise mechanisms achieve nearly optimal error in unbiased mean estimation.
problem Efficiently estimating the mean of high-dimensional data while preserving privacy.
method Differential privacy mechanisms with Gaussian noise, focusing on optimal covariance.
result Gaussian noise mechanisms achieve nearly optimal error among all private unbiased mean estimation mechanisms.
SVR analyzed within RQ framework for risk management.
problem Risk management in stochastic optimization.
method Risk Quadrangle (RQ) theory applied to SVR.
result SVR formulations as minimization of Vapnik error and CVaR norm.
Developed unbiased estimators for Heston model with stochastic interest rates.
problem Estimating the Heston model with stochastic interest rates.
method Combined unbiased estimators with the Heston model and developed a semi-exact log-Euler scheme.
result Convergence rate of O(h) in the L2 norm for a wide range of models. The paper shows how to audit fairness in decisions with hidden risk factors.
problem Estimating fairness in decisions influenced by hidden, unobservable risk factors.
method Derives unbiased estimates of risk using historical data and audits existing decision-making systems.
result One can compute meaningful bounds on treatment rates for high-risk individuals, even with hidden confounders.