A novel decentralized algorithm improves minimax optimization in federated learning.
problem Minimax optimization in federated learning with data heterogeneity.
method Decentralized Gradient Tracking (K-GT-Minimax) for nonconvex-strongly-concave optimization.
result Demonstrates superior convergence rate for NC-SC minimax optimization.
New estimator achieves minimax optimal risk in transfer learning.
problem Nonparametric regression with transfer learning.
method Confidence thresholding estimator and data-driven adaptive algorithm.
result Adaptive algorithm achieves minimax risk up to a logarithmic factor.
New algorithm solves federated minimax optimization problems.
problem Federated minimax optimization challenges.
method Federated Stochastic Smoothed Gradient Descent Ascent (FESS-GDA).
result FESS-GDA uniformly solves federated minimax problems.
Many tasks in modern machine learning can be formulated as finding equilibria in \emph{sequential} games. In particular, two-player zero-sum sequential games, also known as minimax optimization, have received growing interest. It is tempting to apply gradient descent to solve minimax optimization given its popularity a…
Optimally estimate distances on surfaces using reconstructed meshes.
problem Estimating intrinsic distances on smooth submanifolds.
method Reconstruction of the surface using a tangential Delaunay complex, and Isomap variant.
result Minimax optimality achieved for distance estimation.
New algorithms solve nonconvex-nonconcave minimax optimization problems.
problem Solving minimax optimization problems in machine learning.
method Two novel Newton-type algorithms for nonconvex-nonconcave minimax optimization.
result Proved local convergence at strict local minimax points.
We investigate the use of Minimax distances to extract in a nonparametric way the features that capture the unknown underlying patterns and structures in the data. We develop a general-purpose and computationally efficient framework to employ Minimax distances with many machine learning methods that perform on numerica…
This paper explores minimax-Bayes solutions for reinforcement learning problems.
problem How to select appropriate priors for decision making under uncertainty in sequential decision making.
method Study of minimax-Bayes solutions for various reinforcement learning problems.
result Minimax policies are more robust than standard priors.
The paper develops a new algorithm for constructing minimax estimators using online learning techniques.
problem Designing minimax estimators for probability distribution parameters.
method Viewing the problem as a zero-sum game and using online learning with non-convex losses to find a Nash equilibrium.
result The algorithm constructs both a minimax estimator and a least favorable prior.
Since its inception, the modus operandi of multi-task learning (MTL) has been to minimize the task-wise mean of the empirical risks. We introduce a generalized loss-compositional paradigm for MTL that includes a spectrum of formulations as a subfamily. One endpoint of this spectrum is minimax MTL: a new MTL formulation…
Adversarial meta-learning computes Gamma-minimax estimators for vague prior knowledge.
problem Estimating parameters with vague prior knowledge.
method Adversarial meta-learning algorithms for Gamma-minimax estimators.
result Convergence guarantees and neural network class for selection.
New adaptive learning rate for FTRL reduces regret to Θ(T^2/3).
problem Minimax regret of Θ(T^2/3) in online learning.
method Adaptive learning rate framework matching stability, penalty, and bias terms.
result Improves Best-of-Both-Worlds (BOBW) regret upper bounds.
This paper analyzes how machine learning models resist adversarial attacks in nonparametric regression.
problem Adversarial attacks on machine learning models in nonparametric regression.
method Theoretical analysis of minimax rates of convergence under adversarial sup-norm.
result The minimax rate under adversarial attacks is the sum of two terms: standard rate and deviation of true function.
Study minimax optimal RL in factored MDPs with bonus exploration.
problem Optimal reinforcement learning in episodic factored MDPs.
method Proposes two model-based algorithms with bonus exploration for minimax optimal regret.
result Achieves minimax optimal regret guarantees for rich factored structures.
The paper analyzes how optimization algorithms affect the generalization of minimax models.
problem The generalization performance of minimax models trained with different optimization algorithms.
method Analysis of gradient descent ascent (GDA) and proximal point method (PPM) algorithms under convex concave and non-convex non-concave settings.
result The PPM algorithm ensures a bounded excess risk in convex concave problems, while GDA's generalization depends on solving subproblems simultaneously.
A new decentralized method solves minimax problems with reduced communication and sample complexity.
problem Solving minimax optimization problems in a distributed setting.
method Decentralized stochastic gradient descent ascent with variance reduction.
result Achieved optimal sample and communication complexities for nonconvex-strongly-concave problems.
New analysis improves understanding of bilevel optimization stability and generalization.
problem Understanding how well bilevel optimization algorithms generalize.
method Algorithmic stability arguments and generalization bounds for three bilevel minimax solvers.
result Precise trade-off between algorithmic stability, generalization gaps, and practical settings.
The paper optimizes k-NN for distributed learning with minimax optimal performance.
problem Minimizing error rates in classification, regression, and density estimation.
method Optimal aggregation of fixed-k nearest neighbors from multiple subsets of data.
result Achieves minimax optimal error rates up to a logarithmic factor.
Develops estimators for near-optimal linear regression under distribution shift.
problem Linear regression under distribution shift with scarce target domain data.
method Minimax linear risk estimators covering various transfer learning settings.
result Achieves near-optimal risk for linear regression problems under distribution shift.
This work establishes distribution-free upper and lower bounds on the minimax label complexity of active learning with general hypothesis classes, under various noise models. The results reveal a number of surprising facts. In particular, under the noise model of Tsybakov (2004), the minimax label complexity of active …
Paper analyzes minimax risks of personalized federated learning algorithms.
problem Statistical heterogeneity among clients in federated learning.
method Minimax analysis of FedAvg and local training approaches.
result Threshold for optimality between FedAvg and local training depends on data heterogeneity.
Study on estimating invertible functions with minimax analysis.
problem Minimizing risk of estimating invertible functions on a plane.
method Introduce two types of L 2 L^2 L 2 -risks, derive lower and upper rates for minimax values, develop an asymptotically almost everywhere invertible estimator. result Invertibility does not reduce the complexity of the estimation problem in terms of the rate.
Efficient sampling reduces memory usage for Minimax distance analysis.
problem Quadratic memory requirement for existing Minimax distance methods.
method Proposes a novel sampling technique with linear space complexity.
result Demonstrates significant reduction in memory usage for Minimax distances.
Paper improves risk bounds for nonconvex-strongly-concave minimax problems.
problem Achieving sharper risk bounds for nonconvex-strongly-concave minimax problems.
method Using uniform localized convergence to derive high probability generalization error bounds.
result Derives n times faster excess primal risk bounds for popular algorithms.
Paper proposes an algorithm to solve complex minimax problems efficiently.
problem Stochastic nonconvex-concave minimax problems in various fields.
method Accelerated first-order regularized momentum descent ascent algorithm (FORMDA).
result Achieves best-known complexity bound of i l d e O ( ε − 6.5 ) ilde{\mathcal{O}}(\varepsilon ^{-6.5}) i l d e O ( ε − 6.5 ) for single-loop algorithms. Unified framework for structure learning via conditional independence testing.
problem Optimal structure learning and conditional independence testing.
method Established a fundamental connection and reduction between structure learning and conditional independence testing.
result Optimal rates for structure learning are determined by conditional independence testing rates.
Paper tackles adversarial attacks on nonparametric regression models.
problem Vulnerability of machine learning models to adversarial attacks in nonparametric regression.
method Establishes minimax rate and proposes adaptive estimators for robust nonparametric regression under adversarial L q L_q L q -risks. result Achieves minimax optimality and provides adaptive estimators for robust nonparametric regression.
New RL method nearly optimally learns policies with generative models.
problem Finding optimal policies in reinforcement learning with generative models.
method Mirror descent value iteration with KL divergence and entropy regularization.
result The method is nearly minimax-optimal for small ε \varepsilon ε -optimal policies. The study establishes minimax bounds for estimating operators from noisy samples.
problem Estimating unknown operators between Hilbert spaces from noisy data.
method Developed a minimax theory for uniformly bounded Lipschitz operators, proving lower and upper bounds.
result Sharp characterizations of minimax risk for generic Lipschitz operators, showing a curse of sample complexity.
Here we propose a general theoretical method for analyzing the risk bound in the presence of adversaries. Specifically, we try to fit the adversarial learning problem into the minimax framework. We first show that the original adversarial learning problem can be reduced to a minimax statistical learning problem by intr…
This paper tackles convex-submodular minimax problems in mixed continuous-discrete domains.
problem Convex-submodular minimax problems in mixed continuous-discrete domains.
method Introduces new notions of optimality and proposes iterative algorithms combining discrete and continuous optimization.
result Characterizes convergence rates, computational complexity, and quality of solutions for convex and monotone-submodular minimax problems.
The paper optimizes risk-sensitive RL with CVaR, achieving near-minimax-optimal results.
problem Optimizing risk-sensitive reinforcement learning with CVaR objective.
method Developed algorithms for multi-arm bandits and online RL in MDPs, achieving near-minimax-optimal regret.
result Achieved near-minimax-optimal regret of O ( τ − 1 S A K ) O(τ^{-1}\sqrt{SAK}) O ( τ − 1 S A K ) for constant τ τ τ . MRCpy implements minimax risk classifiers with performance guarantees and distribution shift adaptability.
problem Classical risk minimization approaches are not robust to distribution shifts.
method Robust risk minimization approach for minimax risk classifiers.
result MRCs provide performance guarantees and adapt to distribution shifts.
Study confirms optimal minimax rate for nonlocal interaction kernel estimation.
problem Estimating nonlocal interaction kernels in interacting particle systems.
method Introduced tamed least squares estimator (tLSE) achieving optimal convergence rate.
result Optimal minimax rate of convergence confirmed for β ≥ 1 / 4 β \geq 1/4 β ≥ 1/4 . Paper explores generalization of minimax learners, proposing a new metric.
problem Understanding how minimax learners perform on unseen data.
method Proposes a new metric, the primal gap, to study generalization of minimax learners.
result Derives generalization error bounds for the primal gap in nonconvex-concave settings.
Efficient strategies for online learning against bandit algorithms solve minimax problems.
problem Solving min-max problems in convex-linear settings with empirical distributions.
method Designing online learning algorithms that play against bandit algorithms, leveraging properties of the set of empirical distributions.
result High-probability convergence guarantees to minimax values for a specific family of sets.
Paper presents an efficient algorithm for learning minimax risk classifiers with large-scale data.
problem Efficient learning of minimax risk classifiers for large-scale data with multiple classes.
method Combination of constraint and column generation for efficient learning.
result 10x speedup for general large-scale data and 100x speedup with many classes.
ConquerNet smooths quantile regression for deep learning with minimax guarantees.
problem Optimization challenges in quantile regression for deep models.
method ConquerNet uses convolution-smoothed quantile ReLU neural networks.
result ConquerNet provides minimax guarantees and outperforms standard quantile neural networks.
TiAda adapts adaptive gradient methods for nonconvex minimax optimization.
problem Nonconvex minimax optimization challenges in achieving convergence.
method TiAda is a time-scale adaptive GDA algorithm for nonconvex minimax optimization.
result TiAda achieves near-optimal complexities in deterministic and stochastic settings.
The paper analyzes reinforcement learning methods for estimating weights and quality functions with fast convergence rates.
problem Estimating weights and quality functions in reinforcement learning with function approximation.
method The paper uses minimax methods for estimating marginal importance weights and q-functions.
result The minimax approach enables fast rates of convergence for weights and quality functions, achieving first-order efficiency.
In this paper, we give a new sharp generalization bound of lp-MKL which is a generalized framework of multiple kernel learning (MKL) and imposes lp-mixed-norm regularization instead of l1-mixed-norm regularization. We utilize localization techniques to obtain the sharp learning rate. The bound is characterized by the d…
The paper develops a minimax optimal method for high-dimensional regression using auxiliary data.
problem High-dimensional additive regression with heavy-tailed errors and transfer learning.
method Smooth backfitting estimator with local linear smoothing, followed by a two-stage estimation method.
result The method achieves the minimax optimal rate under certain conditions.
Minimax optimization has found extensive applications in modern machine learning, in settings such as generative adversarial networks (GANs), adversarial training and multi-agent reinforcement learning. As most of these applications involve continuous nonconvex-nonconcave formulations, a very basic question arises---"w…
New findings suggest minimax optimality doesn't guarantee distribution learning for GANs.
problem Understanding when GANs can truly learn the underlying distribution.
method Using cryptographic assumptions and ReLU network generators, the paper shows that achieving minimax optimality is insufficient for distribution learning.
result Achieving minimax optimality is insufficient for distribution learning in the usual statistical sense.
Efficient learning of minimax risk classifiers in high dimensions.
problem Efficient learning of classifiers in high-dimensional data.
method Iterative algorithm leveraging constraint generation methods for minimax risk classifiers.
result The algorithm provides efficient learning and feature selection in high-dimensional scenarios.
New fairness concept extends minimax fairness to lexicographic fairness.
problem Fairness in supervised learning, especially lexicographic fairness.
method Introduced approximate lexifairness, derived algorithms for finding solutions, and proved generalization bounds.
result Proved that approximate lexifairness on training data implies approximate lexifairness on true distribution.
Proves minimax sample complexity for turn-based stochastic games.
problem Proving theoretical guarantees for reinforcement learning in turn-based stochastic games.
method Developing absorbing TBSG and reward perturbation techniques to handle statistical dependence.
result Empirical Nash equilibrium strategy approximates true Nash equilibrium in turn-based stochastic games.
New RL algorithm minimizes distributional learning error.
problem Improving distributional reinforcement learning for better error minimization.
method Proposes a new model-based algorithm with theoretical minimax optimality.
result Proves minimax optimality for approximating return distributions.