The paper studies how more data affects prediction risk in high-dimensional models.
problem The impact of increasing data on prediction risk in high-dimensional models.
method Derives central limit theorem and provides finite-sample distribution and confidence interval for prediction risk.
result Demonstrates 'more data hurt' phenomenon in high-dimensional least squares estimation.
ERTS uses Thompson sampling for Gaussian entropic risk bandits, achieving regret bounds.
problem Risk in decision making complicates reward maximization in MAB problems.
method ERTS (Entropic Risk Thompson Sampling) using Thompson sampling with an entropic risk measure.
result Regret bounds for ERTS under entropic risk measure provided.
This paper unifies risk-averse Thompson sampling for continuous risk functionals.
problem Designing and analyzing risk-averse Thompson sampling algorithms for continuous risk functionals.
method Developed analytical toolkits to prove asymptotically optimal regret bounds for various risk measures.
result Proved asymptotic optimality of ρ ρ ρ -MTS for Bernoulli distributions and a class of risk measures. Paper proposes efficient method for estimating risk measures in complex models.
problem Accurately estimating distortion risk measures in computationally expensive models.
method Integrates importance sampling and machine learning for efficient Monte Carlo estimation.
result Demonstrates significant reduction in computational cost for estimating risk measures.
Study risk-sensitive reinforcement learning with entropic risk measures and generative models.
problem Risk-sensitive reinforcement learning in discounted MDPs with recursive entropic risk measures.
method Introduced Model-Based ERM Q Q Q -Value Iteration (MB-RS-QVI) and derived PAC bounds on sample complexity for value and policy learning. result PAC bounds show exponential dependence on ∣ β ∣ / ( 1 − γ ) |β|/(1-γ) ∣ β ∣/ ( 1 − γ ) , with tight bounds in S S S and A A A . The paper explains how importance sampling can be used for optimization of rare events.
problem Minimizing tail risks in stochastic optimization formulations.
method Importance sampling for reducing sample requirements in estimating rare events.
result Effective importance sampling techniques for optimization of rare events.
Paper proposes an algorithm to optimize CVaR using retrospective approximation and importance sampling.
problem Optimizing risk-averse problems with large sample requirements for CVaR.
method Retrospective approximation combined with importance sampling, tailored for CVaR optimization.
result The proposed algorithm reduces variance efficiently and is computationally efficient.
Improved sample complexity for diffusion models without needing empirical risk minimizers.
problem Theoretical limitations in sample complexity for diffusion models.
method Structured decomposition of score estimation error, eliminating dependence on neural network parameters.
result Achieved sample complexity bound of O(ε^(-4)) without empirical risk minimizer access.
New method improves privacy risk evaluation of machine learning models.
problem Machine learning models can be vulnerable to membership inference attacks.
method Proposed new inference attack method based on prediction entropy, and introduced privacy risk score metric.
result Existing defense approaches are not as effective as previously reported.
The paper analyzes risk estimation methods and derives bounds for OCE risk.
problem Estimating the Optimized Certainty Equivalent (OCE) risk from samples.
method Derives mean-squared error and concentration bounds for SAA of OCE, and analyzes an efficient stochastic approximation-based estimator.
result Finite sample bounds and mis-identification probability bounds for the efficient estimator.
Paper proposes CVaR-TS for risk-constrained MAB problems.
problem Risk in decision-making complicates reward maximization in MAB problems.
method Risk measure CVaR is used, and Thompson Sampling is adapted for CVaR.
result CVaR-TS outperforms other L/UCB-based algorithms in risk-constrained MAB settings.
The study examines how choice of risk measure and volatility estimator affects procyclicality.
problem Understanding the factors affecting procyclicality in risk measure estimation.
method Examined three risk measures (Value-at-Risk, Expected Shortfall, Expectile), realized volatility estimators (sample variance, mean absolute deviation), and two models (iid and GARCH).
result Procyclicality is always present regardless of the choice of risk measure and realized volatility estimator.
Study ridge ensembles in proportional feature-to-sample size regime, proving risk equivalence and GCV consistency.
problem Characterizing and optimizing ridge ensembles in proportional feature-to-sample size regimes.
method Proportional asymptotics analysis, GCV for tuning, proving risk equivalence.
result Risk of optimal full ridgeless ensemble matches optimal ridge predictor's risk.
Paper presents efficient IS for tail risk estimation with machine learning features.
problem Estimating Value at Risk and Conditional Value at Risk with black-box access.
method Efficient Importance Sampling algorithm with self-structuring transformation.
result Asymptotically optimal variance reduction in logarithmic scale.
Paper analyzes cyber risk classifications for forecasting performance.
problem Lack of effective out-of-sample forecasting performance in current cyber risk classifications.
method Rolling window analysis using threshold weighted scoring functions.
result Dynamic and impact-based cyber risk classifiers outperform others in forecasting future cyber risk losses.
Corrects sample selection bias in empirical risk minimization using importance sampling.
problem Statistical learning with biased training data.
method Weighted empirical risk minimization using importance sampling.
result Generalization capacity preserved with estimated importance weights.
Paper develops tighter risk certificates for contrastive learning models.
problem Statistical theory for contrastive learning is lacking, especially for practical models like SimCLR.
method Develops non-vacuous PAC-Bayesian risk certificates considering practical SimCLR factors.
result Risk certificates for contrastive loss and downstream prediction are much tighter than previous results.
Nonparametric Thompson Sampling achieves optimal regret for risk-averse bandits with sub-Gaussian rewards.
problem Optimizing risk-averse bandit problems with sub-Gaussian rewards.
method Anchor-free nonparametric Thompson Sampling algorithm ρ e x t − N P T S S G ρ ext{-}NPTS_{\mathrm{SG}} ρ e x t − N P T S SG . result Achieves regret matching the instance-dependent lower bound to leading order in log n \log n log n . RandALO speeds up risk estimation for large datasets.
problem Estimating out-of-sample risk for large, high-dimensional models.
method RandALO: a randomized approximate leave-one-out estimator.
result RandALO is a computationally efficient risk estimator in high dimensions.
Study optimizes sampling to avoid extreme tail risks in unknown heavy-tailed distributions.
problem Identify optimal alternative with minimal extreme tail risk from unknown heavy-tailed distributions.
method Data-driven sequential sampling policies to maximize likelihood of selecting the optimal alternative.
result Proposed methods outperform existing approaches in identifying the optimal alternative.
Estimates spectral risk measures from i.i.d. samples.
problem Estimating spectral risk measures from limited data.
method Numerical integration method for SRM estimation.
result Estimate concentrates exponentially for bounded support distributions.
Paper develops Monte-Carlo estimators for CoVaR, a key risk measure.
problem Estimating CoVaR, a critical risk measure in finance.
method Developed Monte-Carlo and importance-sampling estimators for CoVaR.
result Optimal rates of convergence for both estimators: n − 1 / 3 n^{-1/3} n − 1/3 and n − 1 / 2 n^{-1/2} n − 1/2 . Improved MLMC method boosts risk estimation efficiency.
problem Estimating risk measures like Value-at-Risk in financial risk management.
method Novel MLMC parametrization and antithetic sampling.
result Significantly improved performance in practical settings.
We give improved constants for data dependent and variance sensitive confidence bounds, called empirical Bernstein bounds, and extend these inequalities to hold uniformly over classes of functionswhose growth function is polynomial in the sample size n. The bounds lead us to consider sample variance penalization, a nov…
Develops Thompson Sampling algorithms for mean-variance bandits.
problem Risk in online decision making systems.
method Thompson Sampling algorithms for mean-variance MAB with comprehensive regret analyses.
result Achieves best known regret bounds for mean-variance MABs and information-theoretic bounds in some regimes.
New research shows that binary classification can be done with noisy data, but only if there are clean samples available.
problem Learning binary classification with instance and label dependent label noise.
method Theoretical analysis and empirical risk minimization.
result Empirical risk minimization achieves the optimal excess risk bound without additional assumptions.
Importance-weighting is a popular and well-researched technique for dealing with sample selection bias and covariate shift. It has desirable characteristics such as unbiasedness, consistency and low computational complexity. However, weighting can have a detrimental effect on an estimator as well. In this work, we empi…
The paper studies the convergence of SAA for systemic risk measures.
problem Theoretical convergence of SAA for set-valued systemic risk measures.
method General theory and specific case study with mixed-integer programming formulations.
result Theoretical convergence results for SAA under Wijsman and Hausdorff topologies.
Paper develops a two-population model to assess longevity basis risk.
problem Mismatch between hedger's liability and hedging instrument causes longevity basis risk.
method Develops a two-population mortality model using Lee-Carter model and renewal process.
result Proposed model provides significant risk reduction when mortality jumps and sampling risk are considered.
In this paper, we model dependence between operational risks by allowing risk profiles to evolve stochastically in time and to be dependent. This allows for a flexible correlation structure where the dependence between frequencies of different risk categories and between severities of different risk categories as well …
This work analyzes IRM and ERM from sample complexity perspective, revealing different behaviors under various distribution shifts.
problem Choosing between IRM and ERM for OOD generalization.
method Sample complexity analysis comparing IRM and ERM under different data generation mechanisms.
result IRM is preferred over ERM for certain distribution shifts, leading to better OOD generalization.
It is shown that the axioms for coherent risk measures imply that whenever there is an asset in a portfolio that dominates the others in a given sample (which happens with finite probability even for large samples), then this portfolio cannot be optimized under any coherent measure on that sample, and the risk measure …
Proposes a new framework for learning from labeled and unlabeled data.
problem Learning from unlabeled and multi-label samples with arbitrary loss functions.
method Multi-complementary and unlabeled learning framework.
result Effective estimation of classification risk with optimal convergence rate.
We study the task of learning from non-i.i.d. data. In particular, we aim at learning predictors that minimize the conditional risk for a stochastic process, i.e. the expected loss of the predictor on the next point conditioned on the set of training samples observed so far. For non-i.i.d. data, the training set contai…
PDTS improves robustness in sequential decision-making.
problem Robust active task sampling for efficient and reliable decision-making.
method Characterizes robust active task sampling as a Markov decision process, proposes PDTS method.
result Significantly improves zero-shot and few-shot adaptation robustness.
Local SGD proves efficient in overparameterized linear regression.
problem Efficiently learning overparameterized linear models in distributed settings.
method Distributed SGD (DSGD) with overparameterized models.
result Excess risk of SGD is smaller than ridge regression in the same sample complexity.
This paper investigates robust versions of the general empirical risk minimization algorithm, one of the core techniques underlying modern statistical methods. Success of the empirical risk minimization is based on the fact that for a "well-behaved" stochastic process { f ( X ) , f ∈ F } \left\{ f(X), \ f\in \mathcal F\right\} { f ( X ) , f ∈ F } indexed b…
We propose an computational framework for real-time risk assessment and prioritizing for random outcomes without prior information on probability distributions. The basic model is built based on satisficing measure (SM) which yields a single index for risk comparison. Since SM is a dual representation for a family of r…
New method approximates CVaR with less data for heavy-tailed risks.
problem Lack of data for accurate CVaR approximation in heavy-tailed distributions.
method Importance sampling based extrapolation for heavy-tailed distributions.
result Statistically consistent approximations with reduced data requirements.
Privacy affects how much data is needed for CVaR optimization.
problem Privacy constraints impact the effective sample size for CVaR optimization.
method Analyzes the privacy-relevant sample size and decomposes CVaR excess risk.
result The effective private tail sample size is εnτ, affecting CVaR learning rates.
Study non-monotonic loss functions in CRC, achieving valid risk control with large calibration samples.
problem Non-monotonic loss functions in CRC, violating existing theory's monotonicity assumption.
method Finite grid selection, calibration sample size analysis, Lipschitz continuity, monotonicity, distribution shift.
result Valid CRC achieved with large calibration samples, optimal excess risk rate of log ( m ) / n \sqrt{\log(m)/n} log ( m ) / n . Exact minimax risk derived for linear prediction with sample covariance analysis.
problem Understanding the minimax risk in linear prediction under various covariate distributions.
method Exact minimax risk analysis, leveraging statistical leverage scores and PAC-Bayes techniques.
result The minimax risk is of order d / ( n − d + 1 ) d/(n-d+1) d / ( n − d + 1 ) for any covariate distribution, nearly matching the risk for Gaussian design. SAA method solves insurance portfolio optimization with CVaR constraints.
problem Optimal allocation under CVaR constraint in insurance.
method Sample Average Approximation (SAA) method applied to CVaR constrained portfolio optimization.
result Convergence of SAA method and solution uniqueness proved under mild assumptions.
Risk aversion is a key element of utility maximizing hedge strategies; however, it has typically been assigned an arbitrary value in the literature. This paper instead applies a GARCH-in-Mean (GARCH-M) model to estimate a time-varying measure of risk aversion that is based on the observed risk preferences of energy hed…
New model optimizes portfolios over multiple periods using predictive control.
problem Optimizing multi-period portfolios with risk and variance objectives.
method Model Predictive Control with Mean-Variance and Risk Parity.
result 30x faster and more robust solutions compared to single period models.
Empirical risk minimization is the main tool for prediction problems, but its extension to relational data remains unsolved. We solve this problem using recent ideas from graph sampling theory to (i) define an empirical risk for relational data and (ii) obtain stochastic gradients for this empirical risk that are autom…
The paper analyzes risk bounds and Rademacher complexity in batch RL.
problem Estimating/minimizing Bellman error with general value function approximation.
method Characterizes generalization performance using Rademacher complexities of function classes.
result Risk bounds and Rademacher complexities provide insights into batch RL.
Plotting a learner's average performance against the number of training samples results in a learning curve. Studying such curves on one or more data sets is a way to get to a better understanding of the generalization properties of this learner. The behavior of learning curves is, however, not very well understood and…