A novel decentralized algorithm improves minimax optimization in federated learning.
problem Minimax optimization in federated learning with data heterogeneity.
method Decentralized Gradient Tracking (K-GT-Minimax) for nonconvex-strongly-concave optimization.
result Demonstrates superior convergence rate for NC-SC minimax optimization.
New algorithm solves federated minimax optimization problems.
problem Federated minimax optimization challenges.
method Federated Stochastic Smoothed Gradient Descent Ascent (FESS-GDA).
result FESS-GDA uniformly solves federated minimax problems.
Study minimax optimal RL in factored MDPs with bonus exploration.
problem Optimal reinforcement learning in episodic factored MDPs.
method Proposes two model-based algorithms with bonus exploration for minimax optimal regret.
result Achieves minimax optimal regret guarantees for rich factored structures.
New findings show many popular bandit algorithms are unstable, contradicting minimax optimality.
problem Challenges in statistical inference from bandit algorithms due to adaptive, non-i.i.d. nature.
method Analysis of stability properties of optimism-based bandit algorithms.
result Widely used minimax-optimal UCB-style algorithms are unstable.
The paper optimizes k-NN for distributed learning with minimax optimal performance.
problem Minimizing error rates in classification, regression, and density estimation.
method Optimal aggregation of fixed-k nearest neighbors from multiple subsets of data.
result Achieves minimax optimal error rates up to a logarithmic factor.
Study minimax-optimal rates for offline decision-making with function approximation.
problem Statistical complexity of offline decision-making with function approximation.
method Near minimax-optimal rates for stochastic contextual bandits and Markov decision processes, using pseudo-dimension and behavior policy.
result Established performance limits and new characterization of behavior policy.
We study the minimax optimal rate for estimating the Wasserstein- 1 1 1 metric between two unknown probability measures based on n n n i.i.d. empirical samples from them. We show that estimating the Wasserstein metric itself between probability measures, is not significantly easier than estimating the probability measures u…
New algorithm AG-OG optimizes separable convex-concave problems efficiently.
problem Efficiently solving separable convex-concave minimax optimization problems.
method Leverages Nesterov acceleration and optimistic gradient on component and coupling parts of the problem.
result Achieves optimal convergence rate for various settings including bilinearly coupled problems.
New RL method nearly optimally learns policies with generative models.
problem Finding optimal policies in reinforcement learning with generative models.
method Mirror descent value iteration with KL divergence and entropy regularization.
result The method is nearly minimax-optimal for small ε \varepsilon ε -optimal policies. We study online convex optimization under stochastic sub-gradient observation faults, where we introduce adaptive algorithms with minimax optimal regret guarantees. We specifically study scenarios where our sub-gradient observations can be noisy or even completely missing in a stochastic manner. To this end, we propose…
We find the optimal error for a constrained regression model under a linear model.
problem Minimizing error while adhering to demographic parity constraints.
method Proposed a minimax optimal error analysis for a demographic parity-constrained regression problem within a linear model.
result The minimax optimal error is characterized by $Θ( rac{dM}{n})$ .
TiAda adapts adaptive gradient methods for nonconvex minimax optimization.
problem Nonconvex minimax optimization challenges in achieving convergence.
method TiAda is a time-scale adaptive GDA algorithm for nonconvex minimax optimization.
result TiAda achieves near-optimal complexities in deterministic and stochastic settings.
New algorithms solve nonconvex-nonconcave minimax optimization problems.
problem Solving minimax optimization problems in machine learning.
method Two novel Newton-type algorithms for nonconvex-nonconcave minimax optimization.
result Proved local convergence at strict local minimax points.
Two-stage nonconvex algorithm and convex relaxation both achieve optimal accuracy in noisy blind deconvolution.
problem Solving bilinear systems of equations with random noise under different designs.
method Two-stage nonconvex algorithm and convex relaxation.
result Both methods achieve minimax-optimal accuracy in the presence of random noise.
Classical scaling is shown to be optimal under various noisy conditions.
problem Consistency of classical scaling under general noise conditions.
method Established using finite fourth moments of noise, derived convergence rates, and matching minimax lower bounds.
result Classical scaling achieves minimax optimality in recovering true configuration from noisy dissimilarities.
Many tasks in modern machine learning can be formulated as finding equilibria in \emph{sequential} games. In particular, two-player zero-sum sequential games, also known as minimax optimization, have received growing interest. It is tempting to apply gradient descent to solve minimax optimization given its popularity a…
Paper develops a minimax optimal test for goodness-of-fit using kernel Stein discrepancy.
problem Developing a robust goodness-of-fit test for general domains.
method Kernel Stein Discrepancy (KSD) with spectral regularization and adaptive testing.
result Proposed regularized test achieves minimax optimality up to a logarithmic factor.
Optimally estimate distances on surfaces using reconstructed meshes.
problem Estimating intrinsic distances on smooth submanifolds.
method Reconstruction of the surface using a tangential Delaunay complex, and Isomap variant.
result Minimax optimality achieved for distance estimation.
New findings suggest minimax optimality doesn't guarantee distribution learning for GANs.
problem Understanding when GANs can truly learn the underlying distribution.
method Using cryptographic assumptions and ReLU network generators, the paper shows that achieving minimax optimality is insufficient for distribution learning.
result Achieving minimax optimality is insufficient for distribution learning in the usual statistical sense.
Kernel estimator optimally recovers function from noisy exponential Radon transform.
problem Inverting noisy exponential Radon transform of a function.
method Proposed a kernel estimator to estimate the true function.
result The estimator converges to the true function at minimax optimal rate.
Efficient algorithm for global optimization of multivariate Lipschitz functions.
problem Global optimization of multivariate Lipschitz continuous functions.
method Proposes an efficient minimax optimal algorithm using a predetermined query creation rule.
result Achieves an average regret bound of O ( L n T − 1 n ) O(L\sqrt{n}T^{-\frac{1}{n}}) O ( L n T − n 1 ) , minimax optimal. Optimal nonparametric regression estimator adapts to unknown smoothness.
problem Nonparametric regression with unknown smoothness.
method Constructs an interpolating estimator that adapts to unknown smoothness.
result Minimax optimal rates achieved on Hölder classes.
New RL algorithm minimizes distributional learning error.
problem Improving distributional reinforcement learning for better error minimization.
method Proposes a new model-based algorithm with theoretical minimax optimality.
result Proves minimax optimality for approximating return distributions.
GNA optimally identifies the best arm with small gaps.
problem Best arm identification in fixed-budget settings.
method Generalized Neyman Allocation (GNA) for asymptotically locally minimax optimal BAI.
result GNA's worst-case bounds match the lower and upper bounds in the small-gap regime.
The paper analyzes generalization bounds for NC-SC/NC-C stochastic minimax optimization.
problem Generalization analysis of nonconvex-(strongly)-concave stochastic minimax optimization.
method Established algorithm-agnostic and algorithm-dependent generalization bounds via uniform convergence and stability arguments.
result Sample complexities and generalization bounds for NC-SC and NC-C settings.
Optimal algorithm for linear bandits on ellipsoids with minimax regret bound.
problem Linear stochastic bandits on ellipsoids.
method Novel sequential procedure to estimate norm of parameters, followed by an explore-and-commit strategy.
result Regret bound matches minimax lower bound with multiplicative constant.
New minimax optimal learner for robust predictors against adversarial examples.
problem Learning robust predictors against adversarial examples.
method Global perspective and new algorithmic ideas.
result Characterizes classes of predictors that are robustly learnable.
GP-UCB performs suboptimally under certain conditions, as shown by a new regret lower bound.
problem The suboptimality of GP-UCB under polynomial effective optimism.
method Analysis of effective optimism level and new regret lower bound.
result GP-UCB is not minimax optimal under polynomial growth of effective optimism.
Improved gap-dependent bounds for reinforcement learning with linear approximations.
problem Achieving nearly minimax-optimal performance with linear function approximation.
method Developed and analyzed the LSVI-UCB++ algorithm and its concurrent variant.
result First gap-dependent regret bound for nearly minimax-optimal algorithm LSVI-UCB++.
Meta-theorems validate fair regression algorithms under demographic parity constraints.
problem Regression under demographic parity constraints.
method Meta-theorems and post-processing methods.
result Fair minimax optimal regression can be achieved through post-processing.
UCRL2-VTR achieves nearly optimal regret for learning MDPs with linear function approximation.
problem Learning infinite-horizon average-reward MDPs with linear function approximation.
method UCRL2-VTR algorithm with Bernstein-type bonus.
result Achieves a regret of i l d e O ( d D T ) ilde{O}(d\sqrt{DT}) i l d e O ( d D T ) with matching lower bound. Paper tackles gradient-free minimax optimization with variance reduction for faster convergence.
problem Gradient-free minimax optimization problems in machine learning.
method Variance reduction technique to design a novel zeroth-order gradient descent ascent algorithm.
result Achieves the best known query complexity of O(κ(d₁ + d₂)ε⁻³), outperforming previous methods.
Randomization is minimax-optimal for variance in experimental design, even with structure.
problem Designing optimal randomized experiments for variance minimization.
method Analyzing permutation symmetric and non-symmetric sets of outcomes, proposing inference-constrained MSOD.
result Randomization is minimax-optimal for variance, even with structure, and requires uniformity constraints for Fisher's exact test.
Paper improves algorithms for convex-concave minimax optimization problems.
problem Minimizing convex-concave functions with strong convexity and concavity properties.
method Proposes a new algorithm with improved gradient complexity.
result Improves gradient complexity upper bound for minimax optimization.
Paper shows MoM is optimal under adversarial contamination for certain distributions.
problem Optimality of MoM under adversarial contamination.
method Upper and lower bounds for MoM's error under adversarial contamination.
result MoM is (minimax) optimal for distributions with finite variance and infinite variance with finite absolute moments.
Optimizes quantile and semi-adversarial regret with novel root-logarithmic regularizers.
problem Minimizes regret in adversarial and semi-adversarial online learning.
method FTRL with root-logarithmic regularizers for quantile and semi-adversarial settings.
result Achieves minimax optimal regret bounds in both paradigms.
The paper develops a minimax optimal method for high-dimensional regression using auxiliary data.
problem High-dimensional additive regression with heavy-tailed errors and transfer learning.
method Smooth backfitting estimator with local linear smoothing, followed by a two-stage estimation method.
result The method achieves the minimax optimal rate under certain conditions.
Study on DiTs' rates of approximation and estimation under various data assumptions.
problem Investigating statistical rates of conditional diffusion transformers.
method Discretization and Taylor expansion of conditional diffusion score function under Hölder smooth data assumption.
result Establishes statistical limits for conditional and unconditional DiTs, offering practical guidance.
Develops locally private methods for nonparametric contextual bandits.
problem Privacy concerns in sequential decision-making on sensitive data.
method Uniform-confidence-bound-type estimator and jump-start scheme.
result Minimax optimality of proposed methods supported by lower bounds.
This work provides a simplified proof of the statistical minimax optimality of (iterate averaged) stochastic gradient descent (SGD), for the special case of least squares. This result is obtained by analyzing SGD as a stochastic process and by sharply characterizing the stationary covariance matrix of this process. The…
New algorithms improve convergence of minimax optimization.
problem Minimax optimization convergence issues in nonconvex problems.
method Established new convergence results for two single-loop algorithms.
result Improved convergence rates for minimax optimization.
Unified analysis of efficient local training methods for distributed variational inequalities.
problem Efficient distributed/federated learning for variational inequality problems.
method Unified convergence analysis of communication-efficient local training methods.
result First local gradient descent-accent algorithms with improved communication complexity.
Transformers can efficiently approximate nonparametric regression with minimal parameters and sequences.
problem Efficiently approximating nonparametric regression functions with transformers.
method Kernel-weighted polynomial basis and gradient descent.
result Achieves minimax optimal rate of convergence with fewer parameters and sequences.
A federated minimax framework for heterogeneous clients.
problem Training with edge devices having different datasets and capabilities.
method Proposes a federated minimax optimization framework with normalized updates.
result Improves convergence and communication complexity for nonconvex functions.
New algorithm for offline RL with linear approx in MDPs and MGs, nearly optimal.
problem Offline RL with linear function approximation in MDPs and MGs.
method Pessimism-based algorithm with uncertainty decomposition via reference function.
result Nearly minimax optimal performance in offline RL for MDPs and MGs.
Shallow neural networks can represent polynomials efficiently.
problem Representing polynomials using shallow neural networks.
method Using shallow neural networks of width 2 ( R + d ) d 2(R+d)^d 2 ( R + d ) d to represent d d d -variate polynomials of degree R R R . result Derives minimax optimal convergence rate for shallow networks to unknown univariate regression functions.
The paper explores how control variates can reduce variance in Monte Carlo simulations, especially for Sobolev functions.
problem Efficiency of control variates in reducing variance for Monte Carlo simulations.
method Study of a specific quadrature rule using nonparametric regression-adjusted control variates.
result A specific quadrature rule can improve the Monte Carlo rate and achieve the minimax optimal rate under sufficient smoothness assumptions.
Diffusion models achieve nearly optimal distribution estimation in various spaces.
problem Theoretical limitations of diffusion modeling for distribution estimation.
method Analysis of approximation and generalization abilities of diffusion models in Besov spaces.
result Diffusion models achieve nearly minimax optimal estimation rates in total variation and Wasserstein distances.