Study minimax linear regression under quantile risk, improving existing bounds and providing new results.
problem Designing minimax procedures in linear regression under quantile risk.
method Analyzes realizable setting with Gaussian noise, extends to all p-th power error functions, develops new lower and upper bounds.
result Proves minimaxity of a variant of the min-max regression procedure for all p-th power error functions.
Study on estimating invertible functions with minimax analysis.
problem Minimizing risk of estimating invertible functions on a plane.
method Introduce two types of L 2 L^2 L 2 -risks, derive lower and upper rates for minimax values, develop an asymptotically almost everywhere invertible estimator. result Invertibility does not reduce the complexity of the estimation problem in terms of the rate.
Paper improves risk bounds for nonconvex-strongly-concave minimax problems.
problem Achieving sharper risk bounds for nonconvex-strongly-concave minimax problems.
method Using uniform localized convergence to derive high probability generalization error bounds.
result Derives n times faster excess primal risk bounds for popular algorithms.
New estimator achieves minimax optimal risk in transfer learning.
problem Nonparametric regression with transfer learning.
method Confidence thresholding estimator and data-driven adaptive algorithm.
result Adaptive algorithm achieves minimax risk up to a logarithmic factor.
The paper optimizes risk-sensitive RL with CVaR, achieving near-minimax-optimal results.
problem Optimizing risk-sensitive reinforcement learning with CVaR objective.
method Developed algorithms for multi-arm bandits and online RL in MDPs, achieving near-minimax-optimal regret.
result Achieved near-minimax-optimal regret of O ( τ − 1 S A K ) O(τ^{-1}\sqrt{SAK}) O ( τ − 1 S A K ) for constant τ τ τ . MRCpy implements minimax risk classifiers with performance guarantees and distribution shift adaptability.
problem Classical risk minimization approaches are not robust to distribution shifts.
method Robust risk minimization approach for minimax risk classifiers.
result MRCs provide performance guarantees and adapt to distribution shifts.
Develops estimators for near-optimal linear regression under distribution shift.
problem Linear regression under distribution shift with scarce target domain data.
method Minimax linear risk estimators covering various transfer learning settings.
result Achieves near-optimal risk for linear regression problems under distribution shift.
Paper explores generalization of minimax learners, proposing a new metric.
problem Understanding how minimax learners perform on unseen data.
method Proposes a new metric, the primal gap, to study generalization of minimax learners.
result Derives generalization error bounds for the primal gap in nonconvex-concave settings.
Since its inception, the modus operandi of multi-task learning (MTL) has been to minimize the task-wise mean of the empirical risks. We introduce a generalized loss-compositional paradigm for MTL that includes a spectrum of formulations as a subfamily. One endpoint of this spectrum is minimax MTL: a new MTL formulation…
Study tests uniformity of categorical data against missing-ball alternatives, finding chi-squared test outperforms.
problem Testing uniformity of categorical data against missing-ball alternatives.
method Characterizes minimax risk, uses collisions and chi-squared test, reduces to structured subset of alternatives.
result Minimax test outperforms chi-squared test under least favorable alternative.
We consider random-design linear prediction and related questions on the lower tail of random matrices. It is known that, under boundedness constraints, the minimax risk is of order d / n d/n d / n in dimension d d d with n n n samples. Here, we study the minimax expected excess risk over the full linear class, depending on the dist…
Develops high-probability minimax quantile bounds for statistical problems.
problem Statistical procedures often lose information about tail behavior when reduced to expectations.
method Introduces minimax quantiles, develops high-probability variants of minimax methods, and converts risk lower bounds to quantile lower bounds.
result Obtains high-probability minimax quantile lower bounds for various statistical problems.
Deep neural network with l_1-regularization achieves nearly optimal risk bounds.
problem Achieving optimal risk bounds in deep learning.
method Empirical risk minimization with l_1-regularization.
result Adaptively nearly-minimax risk bound across various function classes.
Local asymptotic minimax risk bounds in a locally asymptotically mixture of normal family of distributions have been investigated under asymmetric loss functions and the asymptotic distribution of the optimal estimator that attains the bound has been obtained.
Paper proposes ZO-SMD for MERO, achieving optimal convergence rates.
problem Minimizing excess risk across all test distributions.
method Zeroth-order stochastic mirror descent algorithm for both smooth and non-smooth MERO.
result Converges at optimal rates of O ( 1 / t ) \mathcal{O}(1/\sqrt{t}) O ( 1/ t ) for estimates and optimization errors. Study on kernel regression risk in high dimensions using Pinsker bound.
problem Kernel regression risk in high-dimensional inner product spaces.
method Investigation of Pinsker bound for kernel regression on sphere S d \mathbb{S}^{d} S d with sample size n = α d γ ( 1 + o d ( 1 ) ) n = αd^γ(1+o_{d}(1)) n = α d γ ( 1 + o d ( 1 )) . result Exact minimax risk and Pinsker constant identified for kernel regression.
Here we propose a general theoretical method for analyzing the risk bound in the presence of adversaries. Specifically, we try to fit the adversarial learning problem into the minimax framework. We first show that the original adversarial learning problem can be reduced to a minimax statistical learning problem by intr…
Paper presents an efficient algorithm for learning minimax risk classifiers with large-scale data.
problem Efficient learning of minimax risk classifiers for large-scale data with multiple classes.
method Combination of constraint and column generation for efficient learning.
result 10x speedup for general large-scale data and 100x speedup with many classes.
Score attack method provides a lower bound on privacy-constrained minimax risk.
problem Characterizing the optimality of privacy-constrained statistical models.
method Score attack based on tracing attack concept.
result Optimally lower bounds the minimax risk of estimating unknown model parameters.
Efficient learning of minimax risk classifiers in high dimensions.
problem Efficient learning of classifiers in high-dimensional data.
method Iterative algorithm leveraging constraint generation methods for minimax risk classifiers.
result The algorithm provides efficient learning and feature selection in high-dimensional scenarios.
The paper explores the information-theoretic nature of excess risk in machine learning.
problem Understanding the excess risk in machine learning models.
method Formulates the minimax excess risk as a zero-sum game and modifies it to allow swapping of the order of play.
result Proves that under certain conditions, the duality gap is zero, allowing for the application of Bayesian results to provide bounds on minimax excess risk.
Paper presents adaptive minimax risk classifiers for multidimensional concept drift.
problem Multidimensional concept drift in supervised classification.
method Adaptive minimax risk classifiers (AMRCs) tracking multivariate and high-order distribution changes.
result AMRCs provide computable tight performance guarantees and improve classification.
An algorithm learns from multiple models to match an oracle's risk.
problem Learning from multiple noisy models to estimate a target parameter.
method Elimination rounds algorithm for adaptive learning.
result Risk of weak-oracle learner matches that of an oracle in multiple source case.
Develops a robust learning method for unknown context distributions.
problem Learning from data in different, unknown contexts.
method Focuses on excess risks, constructs distribution sets with statistical coverage.
result Shows robustness in worst-case scenarios without sacrificing nominal performance.
Paper analyzes minimax risks of personalized federated learning algorithms.
problem Statistical heterogeneity among clients in federated learning.
method Minimax analysis of FedAvg and local training approaches.
result Threshold for optimality between FedAvg and local training depends on data heterogeneity.
MRCs minimize worst-case expected 0-1 loss and provide performance guarantees.
problem Minimizing expected 0-1 loss in classification.
method Minimizes worst-case expected 0-1 loss over uncertainty sets defined by linear constraints.
result Achieves efficient learning and generalization with performance guarantees.
We present an iterative Markov chainMonte Carlo algorithm for computingreference priors and minimax risk forgeneral parametric families. Ourapproach uses MCMC techniques based onthe Blahut-Arimoto algorithm forcomputing channel capacity ininformation theory. We give astatistical analysis of the algorithm,bounding the n…
The paper develops a new algorithm for constructing minimax estimators using online learning techniques.
problem Designing minimax estimators for probability distribution parameters.
method Viewing the problem as a zero-sum game and using online learning with non-convex losses to find a Nash equilibrium.
result The algorithm constructs both a minimax estimator and a least favorable prior.
Study minimax risk of score estimation for log-concave distributions.
problem Minimizing risk in score estimation for log-concave distributions.
method Developed subclasses of log-concave densities and constructed a locally adaptive, multiscale estimator.
result Established minimax rates for score estimation over specific subclasses of log-concave densities.
Reinforcement learning (RL) has achieved remarkable performance in numerous sequential decision making and control tasks. However, a common problem is that learned nearly optimal policy always overfits to the training environment and may not be extended to situations never encountered during training. For practical app…
The paper analyzes how optimization algorithms affect the generalization of minimax models.
problem The generalization performance of minimax models trained with different optimization algorithms.
method Analysis of gradient descent ascent (GDA) and proximal point method (PPM) algorithms under convex concave and non-convex non-concave settings.
result The PPM algorithm ensures a bounded excess risk in convex concave problems, while GDA's generalization depends on solving subproblems simultaneously.
Paper tackles linear models with missing values, achieving minimax optimal results.
problem Missing values in real-world data complicate linear model learning.
method Proposes a rigorous setting and a new algorithm leveraging missing data distribution.
result Derives minimax optimal adaptive risk bounds for predictions with missing values.
Choquet and minimax expectations are equivalent in European option pricing.
problem Pricing European options in incomplete markets.
method Comparing Choquet and minimax expectations in the context of European options.
result Choquet and minimax expectations are equal for European options.
End-to-end portfolio system accounts for model risk.
problem Model risk in portfolio selection.
method Distributionally robust optimization with convex duality.
result Explicitly accounts for model risk in portfolio selection.
GD with large, adaptive stepsizes achieves optimal risk in logistic regression.
problem Optimizing logistic regression with large stepsizes.
method Gradient Descent with adaptive stepsizes.
result GD achieves minimax optimal convergence in logistic regression.
Proposes a fairness criterion for multi-objective optimization in classification.
problem Ensuring fairness in classification models across different groups.
method Formulates a minimax Pareto fairness criterion and provides an optimization algorithm.
result Demonstrates improved fairness compared to existing methods on various real-world datasets.
Bayesian approach to robust risk measures under model uncertainty.
problem Representing robust risk measures as a single probability measure.
method Introducing two types of risk measures and analyzing their relation to robust risk measures.
result Robust risk measures can be represented by a mixture probability measure, a Bayesian approach.
Study dual representations for quasiconvex systemic risk measures.
problem Finding dual representations for quasiconvex systemic risk measures.
method Abstract infinite-dimensional setting, explicit formula for penalty function, nonstandard minimax inequality.
result Explicit formula for the penalty function of quasiconvex compositions.
Study non-stationary distributions, proving risk bounds for density estimation.
problem Estimating current distribution under gradual changes.
method Proves tight minimax risk bounds for nonparametric density estimation under drift.
result Generalizes previous results on agnostic learning under drift.
Paper introduces MRCs that minimize worst-case 0-1 loss, providing tight performance guarantees.
problem Minimizing worst-case 0-1 loss in classification.
method MRCs that minimize worst-case 0-1 loss with uncertainty sets of distributions.
result MRCs provide tight performance guarantees and are strongly universally consistent.
The study establishes minimax bounds for estimating operators from noisy samples.
problem Estimating unknown operators between Hilbert spaces from noisy data.
method Developed a minimax theory for uniformly bounded Lipschitz operators, proving lower and upper bounds.
result Sharp characterizations of minimax risk for generic Lipschitz operators, showing a curse of sample complexity.
Paper develops MRCs for supervised classification using generalized maximum entropy.
problem Developing robust classifiers for decision problems.
method Generalized maximum entropy principle applied to minimax risk classifiers.
result Learning techniques for determining MRCs with performance guarantees.
Study on statistical estimation over Gaussian MAC, comparing analog and digital schemes.
problem Distributed minimax statistical estimation over a Gaussian MAC.
method Developed analog joint estimation-communication schemes and derived information-theoretic lower bounds.
result Achieved risk within a logarithmic factor of information-theoretic lower bounds.
Study improves distributional regression evaluation with CRPS, finding optimal rates of convergence.
problem Improving probabilistic forecasts in meteorology using distributional regression.
method Extends theoretical properties of CRPS evaluation to include covariates and finite sample sizes, analyzing convergence rates for different methods.
result Optimal minimax rate of convergence for distributional regression methods is achieved by k-nearest neighbor and kernel methods.
Sharp risk bounds for early-stopping in Gaussian linear regression are derived.
problem Minimizing in-sample mean squared error in high-dimensional Gaussian linear regression.
method Early-stopped mirror descent (ESMD) with local Gaussian width bounds.
result Sharp risk bounds extend to early-stopped mirror descent for least squares estimator (LSE).
Study on clustering in high dimensions with anisotropic Gaussian mixtures, showing interpolation can be optimal and robust.
problem Clustering in high-dimensional anisotropic Gaussian mixtures.
method Derive minimax bounds, analyze ℓ 2 \ell_2 ℓ 2 -regularized classifiers, and investigate interpolation's robustness. result Interpolating solutions can be optimal and robust under certain conditions.
Optimal nonparametric regression estimator adapts to unknown smoothness.
problem Nonparametric regression with unknown smoothness.
method Constructs an interpolating estimator that adapts to unknown smoothness.
result Minimax optimal rates achieved on Hölder classes.
New fairness concept extends minimax fairness to lexicographic fairness.
problem Fairness in supervised learning, especially lexicographic fairness.
method Introduced approximate lexifairness, derived algorithms for finding solutions, and proved generalization bounds.
result Proved that approximate lexifairness on training data implies approximate lexifairness on true distribution.