Examines optimal risk sharing with realistic risk attitudes, finding risk seeking in certain subdomains.
problem Optimal risk sharing with empirically realistic risk attitudes.
method Allows for risk-seeking agents, generalizes expected utility, and uses counter-monotonic improvement theorem.
result First empirical results on optimal risk sharing with realistic risk attitudes.
New approach avoids excess empirical risk in domain generalization.
problem Learning models that generalize to unseen distributions from diverse data sets.
method Minimizes penalty under constraint of optimal empirical risk, leveraging rate-distortion theory.
result Significant improvements in domain generalization performance across multiple methods.
A new method approximates expected empirical loss for stochastic deep learning tasks.
problem Determining optimal step sizes for stochastic gradient descent in deep learning.
method Applying one-dimensional function fitting to noisy losses of vertical cross sections to approximate expected empirical loss.
result The method leads to a robust and straightforward optimization method that performs well across datasets and architectures.
We solve robust optimization problems using Wasserstein balls and apply it to mean-CVaR optimization.
problem Distributionally robust optimization with Wasserstein ambiguity sets.
method Transformed robust optimization into non-robust with penalty term, selecting ambiguity set size.
result Impressive results in robust mean-CVaR optimization compared to other strategies.
This paper tightens the law of the iterated logarithm for empirical KL_inf, applicable to unbounded data.
problem Developing nonasymptotic concentration bounds for empirical KL_inf with optimal constants and rates.
method Presenting a tight law of the iterated logarithm for empirical KL_inf, applicable to unbounded data.
result A tight law of the iterated logarithm for empirical KL_inf, applicable to unbounded data.
Optimizes bilevel empirical risk minimization with improved oracle calls.
problem Optimizing bilevel empirical risk minimization problems.
method Proposes a bilevel extension of the SARAH algorithm.
result Demonstrates improved oracle calls to achieve stationarity.
The optimal approach is to theorize after examining data, not before.
problem Optimal sequencing of theory and empirical analysis for economic questions.
method Formalized a Bayesian model to trade off Darwinian and Statistical Learning.
result Post hoc theorizing is typically optimal in modern economics.
Paper develops efficient algorithms for robust optimization across multiple groups.
problem Minimizing maximal empirical risk across distinct groups in robust optimization.
method Develops ALEG and ALEM algorithms for two-level finite-sum convex-concave minimax optimization.
result Achieves ε-accuracy with complexity O(m√(nlnm/ε)) and outperforms state-of-the-art methods.
Estimates policy value from off-policy data in contextual bandits.
problem Estimating policy value from limited off-policy data in contextual bandits.
method Empirical likelihood techniques for optimization.
result Improves over previous methods in finite sample regimes.
New algorithms achieve uniform stability for empirical risk minimization.
problem Designing uniformly stable optimization algorithms for empirical risk minimization.
method Black-box conversion of smooth optimization algorithms and development of Mirror Descent for smooth optimization.
result Optimal algorithms with uniform stability and convergence rates for smooth optimization.
New measure quantifies function similarity for optimization.
problem Measuring similarity between functions for optimization.
method Quantifies sub-optimality gaps and operation rules.
result Unified measure for various functional similarities.
The study provides theoretical guarantees for the statistical performance of optimal decision trees.
problem Theoretical limits on the statistical performance of globally optimal decision trees.
method Sharp oracle inequalities and uniform concentration framework based on Rademacher complexity.
result Derivation of minimax optimal rates for piecewise sparse heterogeneous anisotropic Besov space.
We develop an approach to risk minimization and stochastic optimization that provides a convex surrogate for variance, allowing near-optimal and computationally efficient trading between approximation and estimation error. Our approach builds off of techniques for distributionally robust optimization and Owen's empiric…
New algorithm closes empirical gap in PFSGD performance.
problem Empirical performance gap between tuned SGD and PFSGD.
method Parameter-free algorithm based on Coin-Betting ODE updates.
result New algorithm outperforms tuned baselines and matches optimal performance.
Optimizes SGLD noise structure for better generalization bounds.
problem Improving generalization bounds for large models trained with SGLD.
method Manipulates the noise structure in SGLD to optimize information-theoretical bounds.
result Optimal noise covariance is the square root of the expected gradient covariance under certain constraints.
Empirical study finds robust optimization can improve portfolio performance in Indian markets.
problem Comparing robust optimization to Markowitz model for portfolio performance.
method Three robust optimization models (box, ellipsoidal, separable uncertainty sets) tested on Indian market data.
result Robust optimization can be a viable alternative to Markowitz model in real market setups.
We consider distributed convex optimization problems originated from sample average approximation of stochastic optimization, or empirical risk minimization in machine learning. We assume that each machine in the distributed computing system has access to a local empirical loss function, constructed with i.i.d. data sa…
Disputes the empirical Fisher approximation for natural gradient descent.
problem The empirical Fisher approximation fails to capture second-order information in general.
method Comparison of empirical Fisher and Fisher information matrices.
result The empirical Fisher does not generally approximate the Fisher or Hessian.
We study the rates of convergence from empirical surrogate risk minimizers to the Bayes optimal classifier. Specifically, we introduce the notion of \emph{consistency intensity} to characterize a surrogate loss function and exploit this notion to obtain the rate of convergence from an empirical surrogate risk minimizer…
A new SGD framework reduces empirical risk by favoring higher loss observations.
problem Minimizing empirical risk in machine learning problems.
method Develops a biased gradient estimator for stochastic optimization.
result Minimizes an ordered modification of the empirical average loss.
New method improves model risk prediction using cross-audit projection.
problem Over-optimism in K K K -fold CV for binary classification. method Cross-audit projection (CAP) procedure combining resampling and asymptotic bias correction.
result CAP estimator achieves second-order asymptotic unbiasedness.
Robust portfolio optimization considers uncertainty in market probabilities.
problem Uncertainty in market probabilities in multiperiod portfolio selection.
method Robust mean-variance optimization using Wasserstein ball centered at empirical data.
result Numerical simulations show improved performance compared to other strategies.
New DNN method accelerates image processing optimization.
problem Optimizing large-scale inverse problems in image processing.
method Trains a deep neural network to learn parameters for scaled gradient projection method.
result Significantly improves convergence rate of optimization methods.
Enhances flexibility in data reweighting with optimal transport and maximum entropy principles.
problem Adapting empirical distributions to predefined constraints on moments, tail behavior, etc.
method Nonparametric distributional constraints, maximum entropy principle, optimal transport.
result Maximum entropy weight adjusted empirical distribution close to a specified distribution in optimal transport metric.
New bounds show empirical EOT adapts to simpler measure.
problem Statistical performance of empirical EOT estimators.
method Novel statistical bounds, empirical process theory, dual formulation.
result Empirical EOT and its unregularized version follow lower complexity adaptation.
Study sharp convergence rates of empirical UOT for spatio-temporal point processes.
problem Statistical analysis of UOT for spatio-temporal point processes.
method Empirical plug-in estimators for Kantorovich-Rubinstein distance between intensity measures.
result Sharp convergence rates of empirical UOT in terms of intrinsic dimensions of measures.
Dual optimization connects ERM-fDR to normalization function.
problem Empirical risk minimization with f-divergence regularization.
method Dual formulation, Legendre-Fenchel transform, implicit function theorem, nonlinear ODE.
result Computational method to calculate normalization function efficiently.
We propose a unified data-driven framework based on inverse optimal transport that can learn adaptive, nonlinear interaction cost function from noisy and incomplete empirical matching matrix and predict new matching in various matching contexts. We emphasize that the discrete optimal transport plays the role of a varia…
Optimal private ERM and SCO with subquadratic gradient complexity.
problem Private optimization of non-smooth convex functions.
method Subquadratic gradient complexity algorithm using subsampling and smoothing.
result Achieved optimal excess empirical risk and population loss.
Empirical analysis of gradient descent optimizers in Deep RL.
problem Performance degradation in gradient descent methods for Deep RL.
method Analysis of various gradient descent optimizers and their hyperparameters.
result Adaptive optimizers have a narrow effective learning rate window, diverging in other cases.
Empirical study shows Randomized Signature Methods improve portfolio optimization in financial markets.
problem Drift estimation in non-linear, non-parametric financial markets is challenging.
method Applied Randomized Signature Methods for non-linear, non-parametric drift estimation in multi-variate financial markets.
result Randomized Signature Methods provide features on the same scale and improve portfolio optimization in real-world settings.
New algorithm optimizes unimodal bandits using empirical divergence.
problem Optimizing decisions in multi-armed bandit problems with unimodal distributions.
method Indexed Minimum Empirical Divergence (IMED) adapted for unimodal structure.
result IMED-UB algorithm optimally exploits unimodal structure.
Wide deep neural networks are easy to optimize without constraints.
problem Optimizing wide deep neural networks.
method Analysis of optimization landscapes and empirical-risk minimization.
result Wide neural networks have no confined points, making optimization easier.
Proposes OBS, a method to adaptively combine Bayesian models online.
problem Learning optimal combinations of Bayesian models in online learning.
method Empirical Bayes lens, Online Bayesian Stacking (OBS).
result Establishes a novel connection between OBS and portfolio selection.
Understanding optimal prompts for binary sequence predictors is challenging.
problem Finding good prompts for binary sequence predictors is difficult.
method Viewing prompting as finding the best conditioning sequence on a near-optimal sequence predictor, using empirical and statistical analysis.
result Optimal prompts can be better understood given the pretraining distribution, which is not usually available.
We theoretically and empirically study portfolio optimization under transaction costs and establish a link between turnover penalization and covariance shrinkage with the penalization governed by transaction costs. We show how the ex ante incorporation of transaction costs shifts optimal portfolios towards regularized …
NHGD solves bilevel optimization problems with reduced computational time.
problem Solving bilevel optimization problems with high computational cost.
method Exploits statistical structure of inner optimization to use empirical Fisher matrix as Hessian surrogate, enabling parallel optimization and approximation.
result NHGD achieves error bounds and sample complexity guarantees matching state-of-the-art methods, with significantly reduced computational time.
Optimal algorithms for online convex optimization with missing sub-gradient observations.
problem Online convex optimization with noisy or missing sub-gradient observations.
method Adaptive algorithms using sub-gradient descent with minimax optimal regret guarantees.
result Achieves tight minimax optimal regret bounds with empirical property estimation.
The study bounds the utility of empirically optimal portfolios using stock return data.
problem Maximizing expected ratio of portfolio utility to best asset utility.
method High probability utility bounds derived from Lipschitz or Hölder continuous utility functions.
result Utility bounds depend on utility function, number of assets, and observations.
A new sequential method estimates Poisson means in streaming data, achieving optimality and efficiency.
problem Estimating Poisson means in a streaming, or online, framework.
method A quasi-Bayesian approach based on Newton's algorithm for a sequential estimate.
result Established frequentist guarantees including consistency and asymptotic optimality.
Study shows optimal rates for estimating Wasserstein metric and measures.
problem Minimax optimal estimation of Wasserstein metric between probability measures.
method Analyzes the minimax rates for estimating the Wasserstein-1 metric and probability measures.
result Minimax optimal rates for estimating Wasserstein metric and measures are multiplicatively equivalent.
The paper improves Bayesian optimization by estimating unknown Gaussian process parameters.
problem The challenge of unknown parameters in Bayesian optimization.
method Adopting empirical Bayes to estimate Gaussian process prior and constructing unbiased estimators.
result Achieves near-zero regret bound, decreasing to a constant proportional to observational noise.
The paper analyzes high-dimensional linear regression using parametric empirical Bayes methods.
problem Estimation of i.i.d. priors in high-dimensional Bayesian linear regression with random design.
method Parametric empirical Bayes estimation, variational lower bound maximization, phase transition analysis.
result The vEB estimator is information theoretically optimal up to p = o ( n 2 / 3 ) p=o(n^{2/3}) p = o ( n 2/3 ) but sub-optimal in higher dimensions. The paper improves methods for generating prediction intervals in regression.
problem Uncertainty quantification in regression models.
method Formalizes prediction interval generation as an optimization problem, studying generalization and calibration.
result Empirical demonstration of improved testing performances compared to existing methods.
Private algorithms minimize population loss with optimal rate.
problem Private optimization of convex functions with stochastic samples.
method Differentially private algorithms based on algorithmic stability.
result Optimal rate of 1 / n 1/\sqrt{n} 1/ n for population loss. Simplified screening tests for data points in optimization.
problem Discarding irrelevant data points in empirical risk minimization.
method Designing loss functions and regularizing convex losses to induce sparsity, using ellipsoidal approximations.
result Automatic discarding of data samples without losing optimization guarantees.
We study the problem of empirical minimization for variance-type functionals over functional classes. Sharp non-asymptotic bounds for the excess variance are derived under mild conditions. In particular, it is shown that under some restrictions imposed on the functional class fast convergence rates can be achieved incl…
Paper tackles heavy-tailed data without finite variance, proposing robust risk minimization.
problem Empirical risk minimization under heavy-tailed data with finite p p p -th moment. method Minimizes risk values robustly estimated via Catoni's method, using generalized generic chaining.
result Shows better performance of optimizer based on empirical risks via Catoni-style estimation.