New methods solve complex optimization problems without strong convexity assumptions.
problem Complex bilevel optimization problems with minimax lower-level structures.
method Penalty-based first-order methods for bilevel minimax optimization.
result Achieves ε ε ε -KKT point with improved oracle complexity. New combinatorial dimension VCL refines learning curve theory.
problem Explaining the behavior of learning curves for specific distributions.
method Introducing combinatorial dimension VCL to characterize strong minimax lower bounds.
result Learning rate can be decomposed into linear and exponential components.
Paper improves algorithms for convex-concave minimax optimization problems.
problem Minimizing convex-concave functions with strong convexity and concavity properties.
method Proposes a new algorithm with improved gradient complexity.
result Improves gradient complexity upper bound for minimax optimization.
An algorithm learns from multiple models to match an oracle's risk.
problem Learning from multiple noisy models to estimate a target parameter.
method Elimination rounds algorithm for adaptive learning.
result Risk of weak-oracle learner matches that of an oracle in multiple source case.
Paper sets fundamental limits for distributed covariance estimation with constrained communication.
problem Estimating high-dimensional covariance matrices in a feature-split setting with limited communication.
method Developed a Conditional Strong Data Processing Inequality (C-SDPI) to establish minimax lower bounds and an optimal estimation protocol.
result Achieved nearly optimal estimation protocol with sample and communication requirements matching lower bounds up to logarithmic factors.
Paper solves minimax optimization gap with near-optimal algorithms.
problem Designing efficient algorithms for smooth and strongly-convex-strongly-concave minimax problems.
method Accelerated proximal point method and accelerated solver for minimax proximal steps.
result First algorithm with gradient complexity matching the lower bound up to logarithmic factors.
New algorithms for private GLM estimation with minimax lower bounds.
problem Privacy in generalized linear models.
method Differentially private algorithms using projected gradient descent.
result Nearly rate-optimal performance with privacy-constrained minimax lower bounds.
Study minimax rates for binary classifier estimation with margin conditions.
problem Estimating binary classifiers with geometric margin conditions.
method Derive lower bounds for worst-case learning rates over various function classes.
result Identify optimal rates close to O ( n − 1 ) \mathcal{O}(n^{-1}) O ( n − 1 ) for different function classes. In statistical inference problems, we wish to obtain lower bounds on the minimax risk, that is to bound the performance of any possible estimator. A standard technique to obtain risk lower bounds involves the use of Fano's inequality. In an information-theoretic setting, it is known that Fano's inequality typically doe…
Study minimax linear regression under quantile risk, improving existing bounds and providing new results.
problem Designing minimax procedures in linear regression under quantile risk.
method Analyzes realizable setting with Gaussian noise, extends to all p-th power error functions, develops new lower and upper bounds.
result Proves minimaxity of a variant of the min-max regression procedure for all p-th power error functions.
Paper finds a lower bound for estimating low-rank matrices in logistic regression.
problem Estimating low-rank coefficient matrices in logistic regression.
method Derives a minimax lower bound on the risk.
result The bound depends on matrix dimensions, rank, and sample size.
Develops high-probability minimax quantile bounds for statistical problems.
problem Statistical procedures often lose information about tail behavior when reduced to expectations.
method Introduces minimax quantiles, develops high-probability variants of minimax methods, and converts risk lower bounds to quantile lower bounds.
result Obtains high-probability minimax quantile lower bounds for various statistical problems.
New algorithm optimally evaluates policies with linear approximations.
problem Policy evaluation with linear function approximation.
method Accelerated, variance-reduced fast temporal difference algorithm (VRFTD).
result VRFTD matches both deterministic and stochastic lower bounds.
New research sets the minimax lower bound for KSD estimation at sqrt(n).
problem Estimating goodness-of-fit using Kernel Stein Discrepancy (KSD) on high-dimensional spaces.
method Two complementary results proving the minimax lower bound of KSD estimation.
result The minimax lower bound of KSD estimation is n^(-1/2), indicating exponential difficulty with dimensionality.
We prove non-asymptotic lower bounds on the expectation of the maximum of d d d independent Gaussian variables and the expectation of the maximum of d d d independent symmetric random walks. Both lower bounds recover the optimal leading constant in the limit. A simple application of the lower bound for random walks is an (…
Exact minimax risk derived for linear prediction with sample covariance analysis.
problem Understanding the minimax risk in linear prediction under various covariate distributions.
method Exact minimax risk analysis, leveraging statistical leverage scores and PAC-Bayes techniques.
result The minimax risk is of order d / ( n − d + 1 ) d/(n-d+1) d / ( n − d + 1 ) for any covariate distribution, nearly matching the risk for Gaussian design. Score attack method provides a lower bound on privacy-constrained minimax risk.
problem Characterizing the optimality of privacy-constrained statistical models.
method Score attack based on tracing attack concept.
result Optimally lower bounds the minimax risk of estimating unknown model parameters.
Estimates sparse topic models with improved efficiency and adaptability.
problem Estimating sparse topic models with unknown sparsity and number of topics.
method Proposes a new algorithm for efficient estimation of sparse topic models with non-negative matrices.
result Upper bound matches minimax lower bound, demonstrating optimal performance.
Study on Q Q Q -function estimation for continuous state-action MDPs, deriving rates and conditions.
problem Estimating Q Q Q -function in off-policy evaluation for continuous state-action Markov decision processes. method Reformulated as nonparametric instrumental variables (NPIV) problem, derived minimax lower bounds, proposed sieve two-stage least squares estimator.
result First minimax lower bounds for Q Q Q -function and its derivatives in sup-norm and L 2 L^2 L 2 -norm, same as classical nonparametric regression. Study uses SGD to learn operators in Hilbert spaces with convergence analysis.
problem Learning operators in general Hilbert spaces with SGD.
method Proposes weak and strong regularity conditions for convergence analysis.
result SGD converges to best linear approximation of nonlinear operators.
Kernel method improves instrumental variable regression rates.
problem Nonparametric instrumental variable regression with weak instruments.
method Kernel-based two-stage least-squares method, strong L 2 L_2 L 2 convergence analysis. result Minimax optimal rates for instrumental regression under standard assumptions.
New research shows that binary classification can be done with noisy data, but only if there are clean samples available.
problem Learning binary classification with instance and label dependent label noise.
method Theoretical analysis and empirical risk minimization.
result Empirical risk minimization achieves the optimal excess risk bound without additional assumptions.
Paper analyzes minimax regret in constrained online convex optimization with limited switching opportunities.
problem Minimizing regret in online convex optimization with limited switching opportunities.
method Introduced fugal game relaxation and mini-batching algorithm to establish minimax regret bounds.
result Minimax regret of switching-constrained OCO is Θ(T / √K).
Novel methods for accelerating optimization in complex bilevel and minimax problems.
problem Optimization challenges in bilevel and minimax problems, especially when strong convexity assumptions are not met.
method Accelerated fully first-order methods for Bilevel Optimization (BLO) and Minimax Optimization (NCSC).
result State-of-the-art complexity for finding approximate second-order stationary points in BLO and NCSC.
New method generates private synthetic data with optimal utility for smooth queries.
problem Achieving strong utility guarantees for meaningful downstream analysis of sensitive datasets.
method Proposes a polynomial-time algorithm for generating ( ε , δ ) (\varepsilon,δ) ( ε , δ ) -differentially private synthetic data with minimax optimal error rates for smooth queries. result Achieves a minimax error rate of O k , d ( n − min { 1 , k d } ) O_{k,d}(n^{-\min \{1, \frac{k}{d}\}}) O k , d ( n − m i n { 1 , d k } ) for k k k -smooth queries, up to a log ( n ) \log(n) log ( n ) factor. GP-UCB performs suboptimally under certain conditions, as shown by a new regret lower bound.
problem The suboptimality of GP-UCB under polynomial effective optimism.
method Analysis of effective optimism level and new regret lower bound.
result GP-UCB is not minimax optimal under polynomial growth of effective optimism.
Study tackles contamination and heterogeneity in multi-task learning, improving robustness and personalization.
problem Challenges in integrating related tasks due to contamination and heterogeneity.
method Proposes a filtering-based robust multi-task gradient descent method to estimate global and clean task-specific minimizers.
result Demonstrates improved robustness and personalization compared to existing methods.
Study finds optimal regret bound for multi-armed bandit problem with expert advice.
problem Optimizing decision-making in a multi-armed bandit problem with expert advice.
method Proved a tight lower bound matching the upper bound of Kale (2014) for minimax expected regret.
result The minimax optimal expected regret is Θ(√(T K log (N/K))) for the problem.
Study on statistical estimation over Gaussian MAC, comparing analog and digital schemes.
problem Distributed minimax statistical estimation over a Gaussian MAC.
method Developed analog joint estimation-communication schemes and derived information-theoretic lower bounds.
result Achieved risk within a logarithmic factor of information-theoretic lower bounds.
Dictionary learning is the problem of estimating the collection of atomic elements that provide a sparse representation of measured/collected signals or data. This paper finds fundamental limits on the sample complexity of estimating dictionaries for tensor data by proving a lower bound on the minimax risk. This lower …
Sparse covariance estimation in the vertical-split model achieves exponential improvement over dense estimates.
problem Minimax estimation error for distributed covariance matrix estimation in the vertical-split setting.
method Elementwise s s s -sparsity is shown to reduce communication and sample complexity. result Minimax lower bounds for 1 1 1 -sparse cross-covariance estimation are established. The paper tackles deep learning from dependent data, achieving optimal performance.
problem Deep learning from strongly mixing observations, especially with regularization and optimality.
method Sparse-penalized regularization for deep neural networks, oracle inequality for expected excess risk.
result Deep neural network estimator achieves minimax optimal rate for nonparametric autoregression.
Unified framework for corruption-robust linear bandits with optimal gap-dependent misspecification bounds.
problem Effective learning in linear bandits with corrupted rewards across different corruption models.
method Unified framework for analyzing strong and weak corruption, connection to gap-dependent misspecification, and specialized algorithm.
result Optimal bounds for gap-dependent misspecification in linear bandits.
Study minimax regret in sequential probability assignment with and without side information.
problem Minimax regret analysis in sequential probability assignment.
method Upper and lower bounds on minimax regret using square-root entropy.
result Lower bound matches upper bound for Donsker classes, up to log factors.
Rejection Sampling is a fundamental Monte-Carlo method. It is used to sample from distributions admitting a probability density function which can be evaluated exactly at any given point, albeit at a high computational cost. However, without proper tuning, this technique implies a high rejection rate. Several methods h…
Paper explores generalization of minimax learners, proposing a new metric.
problem Understanding how minimax learners perform on unseen data.
method Proposes a new metric, the primal gap, to study generalization of minimax learners.
result Derives generalization error bounds for the primal gap in nonconvex-concave settings.
The study establishes minimax bounds for estimating operators from noisy samples.
problem Estimating unknown operators between Hilbert spaces from noisy data.
method Developed a minimax theory for uniformly bounded Lipschitz operators, proving lower and upper bounds.
result Sharp characterizations of minimax risk for generic Lipschitz operators, showing a curse of sample complexity.
New minimax optimal learner for robust predictors against adversarial examples.
problem Learning robust predictors against adversarial examples.
method Global perspective and new algorithmic ideas.
result Characterizes classes of predictors that are robustly learnable.
Develops locally private methods for nonparametric contextual bandits.
problem Privacy concerns in sequential decision-making on sensitive data.
method Uniform-confidence-bound-type estimator and jump-start scheme.
result Minimax optimality of proposed methods supported by lower bounds.
Paper explores fair classification with bounded disparity using finite datasets.
problem Ensuring fairness in binary classification with protected groups.
method Minimax optimal approach with fairness constraints and demographic disparity control.
result Proposes FairBayes-DDP+ method that achieves minimax lower bound on fairness-aware excess risk.
New tensor model reduces GLM estimation error and sample complexity.
problem Estimating GLM coefficients with reduced sample complexity.
method Developed LSR tensor model and block coordinate descent algorithm.
result Minimax lower bound on estimation error, suggesting lower sample complexity.
Transductive learning considers a training set of m m m labeled samples and a test set of u u u unlabeled samples, with the goal of best labeling that particular test set. Conversely, inductive learning considers a training set of m m m labeled samples drawn iid from P ( X , Y ) P(X,Y) P ( X , Y ) , with the goal of best labeling any future sample…
We consider a sequential learning problem with Gaussian payoffs and side information: after selecting an action i i i , the learner receives information about the payoff of every action j j j in the form of Gaussian observations whose mean is the same as the mean payoff, but the variance depends on the pair ( i , j ) (i,j) ( i , j ) (and may…
Detecting a planted submatrix in random matrices with non-asymptotic methods.
problem Detecting a planted submatrix in random matrices with non-zero entries.
method Established minimax lower bounds and derived optimal tests for distinguishing the null and alternative hypotheses.
result Non-asymptotic upper and lower bounds match for any configuration of matrix dimensions.
The cost-sensitive classification problem plays a crucial role in mission-critical machine learning applications, and differs with traditional classification by taking the misclassification costs into consideration. Although being studied extensively in the literature, the fundamental limits of this problem are still n…
Spectral algorithms on manifolds using diffusion kernels improve convergence rates.
problem The limitations of existing spectral algorithms in RKHSs for data on manifolds.
method Integrating manifold structure into spectral algorithms using heat kernel diffusion spaces.
result Spectral algorithms converge to the target function and its derivatives in a strong sense, with rates dependent on manifold intrinsic dimension.
We study the linear contextual bandit problem with finite action sets. When the problem dimension is d d d , the time horizon is T T T , and there are n ≤ 2 d / 2 n \leq 2^{d/2} n ≤ 2 d /2 candidate actions per time period, we (1) show that the minimax expected regret is Ω ( d T ( log T ) ( log n ) ) Ω(\sqrt{dT (\log T) (\log n)}) Ω ( d T ( log T ) ( log n ) ) for every algorithm, and (2) introduce a V…
An important class of distance metrics proposed for training generative adversarial networks (GANs) is the integral probability metric (IPM), in which the neural net distance captures the practical GAN training via two neural networks. This paper investigates the minimax estimation problem of the neural net distance ba…