CD algorithm achieves near-optimal convergence rate for unnormalized models.
problem Training unnormalized models with high efficiency.
method Non-asymptotic analysis of contrastive divergence algorithm.
result CD can achieve O ( n − 1 / 2 ) O(n^{-1 / 2}) O ( n − 1/2 ) convergence rate under regularity assumptions. ROOT-SGD solves convex optimization problems with optimal nonasymptotic and near-optimal asymptotic performance.
problem Solving strongly convex and smooth unconstrained optimization problems using stochastic first-order algorithms.
method ROOT-SGD: Recursive One-Over-T SGD, averaging past stochastic gradients.
result Achieves state-of-the-art performance in both nonasymptotic and asymptotic senses.
New asymptotic e-values improve inference by eliminating data-dependent scaling inefficiency.
problem Data-dependent scaling inefficiency in existing asymptotic e-values.
method Drawing on Bentkus's near-optimal concentration inequalities, introduce Bentkus-type asymptotic e-values.
result Bentkus-type asymptotic e-values consistently deliver sharper inference than existing alternatives.
New algorithm achieves near optimal sample complexity for 1-identification problem.
problem Determining if an arm's mean reward is at least a known threshold with high probability.
method Design of Sequential-Exploration-Exploitation (SEE) algorithm with non-asymptotic analysis.
result Achieves near optimality in sample complexity, matching upper and lower bounds up to a polynomial logarithmic factor.
Near-optimal tests and confidence sequences for non-parametric data.
problem Flexible statistical inference and decision-making with non-parametric data.
method Classic delayed-start normal-mixture sequential probability ratio tests with asymptotic guarantees.
result Asymptotically optimal type-I error and expected rejection time guarantees.
New algorithm detects changes quickly without knowing parameters, near optimally.
problem Quickest change detection with unknown parameters.
method Leverages theoretical asymptotic properties to derive a scalable approximate algorithm with near optimal performance.
result Detects changes in constant complexity with near optimal performance.
Near-optimal rates for multi-task learning with shared representations.
problem Approximation and statistical complexity of learning multiple operators.
method Multiple Neural Operators (MNO) architecture and comparison with DeepONet.
result Near-optimal upper and lower bounds for approximation and generalization.
Near-optimal private tests for simple and MLR hypotheses developed under Gaussian differential privacy.
problem Developing private tests for simple and MLR hypotheses under Gaussian differential privacy.
method A private mean estimator with data-driven clamping bounds, constructing private test statistics.
result Private tests achieve the same asymptotic relative efficiency as non-private most powerful tests.
In this paper, we consider the problem of learning an unknown graph via queries on groups of nodes, with the result indicating whether or not at least one edge is present among those nodes. While learning arbitrary graphs with n n n nodes and k k k edges is known to be hard in the sense of requiring $Ω( \min\{ k^2 \log n, …
Unified method for estimating properties of large domain distributions efficiently.
problem Estimating properties of distributions over large domains efficiently.
method Piecewise-polynomial approximation technique for constructing sample- and time-efficient estimators.
result Near-linear-time computable estimators with optimal and highly-concentrated approximation values.
Paper proposes a debiased estimator for adaptive linear regression.
problem Non-normal asymptotic behavior of OLS estimator in adaptive linear regression.
method Adaptive linear estimating equations to construct debiased estimator.
result Established asymptotic normality of the debiased estimator.
In the context of clustering, we consider a generative model in a Euclidean ambient space with clusters of different shapes, dimensions, sizes and densities. In an asymptotic setting where the number of points becomes large, we obtain theoretical guaranties for a few emblematic methods based on pairwise distances: a si…
New algorithm reduces regret by focusing on near-optimal actions.
problem Regret minimization in multi-armed bandits leads to over-exploitation.
method Proposes a lenient regret criterion and a new algorithm, ε-TS.
result ε-TS achieves asymptotic optimality in lenient regret.
In this paper, we study the trace regression when a matrix of parameters B* is estimated via the convex relaxation of a rank-regularized regression or via regularized non-convex optimization. It is known that these estimators satisfy near-optimal error bounds under assumptions on the rank, coherence, and spikiness of B…
Proposes online debiasing estimators for adaptive linear regression.
problem Adaptive data collection leads to non-normal asymptotic behavior in simple methods.
method Online debiasing estimators that correct distributional anomalies.
result Asymptotic normality and minimax lower bound for proposed estimators.
We provide a general theory of the expectation-maximization (EM) algorithm for inferring high dimensional latent variable models. In particular, we make two contributions: (i) For parameter estimation, we propose a novel high dimensional EM algorithm which naturally incorporates sparsity structure into parameter estima…
Fast robust subspace tracking in sparse data-dependent noise with near-optimal delay.
problem Robustly tracking time-varying subspaces in the presence of sparse outliers.
method Introduces a fast mini-batch robust ST solution under mild assumptions.
result Provably correct subspace tracking with near-optimal delay and same time complexity as simple PCA.
Paper establishes statistical inference for pairwise comparison models.
problem Statistical inference for pairwise comparison models when the number of subjects diverges.
method Identifies Fisher information matrix as a weighted graph Laplacian for asymptotic normality.
result Near-optimal asymptotic normality result for maximum likelihood estimator.
Stochastic optimization naturally arises in machine learning. Efficient algorithms with provable guarantees, however, are still largely missing, when the objective function is nonconvex and the data points are dependent. This paper studies this fundamental challenge through a streaming PCA problem for stationary time s…
Bayesian method estimates dynamics from near-optimal trajectories.
problem Estimating dynamics from near-optimal expert trajectories in reinforcement learning.
method Constraint-based Bayesian approach integrating expert near-optimality.
result Significant improvements in decision-making and transfer success.
Paper tackles non-stationary kernelized bandits with near-optimal algorithm.
problem Minimizing regret in a time-varying reward function.
method Near-optimal algorithm with a novel restarting phased elimination with random permutation (R-PERP).
result Regret upper bound matches the lower bound, making the algorithm near-optimal.
This study analyzes AdaGrad's stability and convergence in non-convex optimization.
problem Lack of theoretical analysis for AdaGrad in non-convex optimization.
method Novel stopping time-based techniques from probability theory.
result Established stability and derived convergence rates for AdaGrad.
UCB algorithm provides stable sample means for sequential data.
problem Challenges in inferential tasks with sequential data.
method Stability property of UCB algorithm for multiarmed bandit problems.
result UCB algorithm ensures asymptotically normal sample means.
MINTS uses a minimalist Bayesian framework to tackle multi-armed bandits with structural constraints.
problem Sequential decision-making under uncertainty with complex structural constraints.
method Minimalist Bayesian framework with profile likelihood to eliminate nuisance parameters.
result MINTS achieves near-optimal regret guarantees and adapts to unimodal structure.
We study the worst-case adaptive optimization problem with budget constraint that is useful for modeling various practical applications in artificial intelligence and machine learning. We investigate the near-optimality of greedy algorithms for this problem with both modular and non-modular cost functions. In both case…
Efficient streaming algorithms for robust statistics with near-optimal memory.
problem High-dimensional robust statistics tasks in streaming model.
method First efficient streaming algorithms with near-optimal memory requirements.
result Near-optimal error guarantees and space complexity nearly-linear in the dimension for robust mean estimation.
Paper proposes a novel method to improve matrix completion with median loss for large datasets.
problem Matrix completion with absolute deviation loss for large-scale data.
method Proposes a refinement step using pseudo data to improve inefficient estimators of median matrix completion.
result Turns inefficient estimators into a rate (near-)optimal matrix completion procedure.
We study the problem of detecting a change in the mean of one-dimensional Gaussian process data. This problem is investigated in the setting of increasing domain (customarily employed in time series analysis) and in the setting of fixed domain (typically arising in spatial data analysis). We propose a detection method …
Algorithm learns linear systems from partial observations with near-optimal rate.
problem Identifying linear dynamical systems from partial observations, especially those with long-term memory.
method Multi-scale low-rank approximation using SVD on Hankel matrices of increasing sizes, combined with Fourier domain concentration bounds.
result Near-optimal rate of $\widetilde O\left(\sqrt\frac{d}{T}
ight)$ in H 2 \mathcal{H}_2 H 2 error, with logarithmic dependence on memory length. The paper offers simple, near-optimal algorithms for multi-group learning.
problem Learning predictors within subgroups of a population, addressing fairness and hidden stratification.
method Studies the structure of solutions and provides simple, near-optimal algorithms.
result Simple and near-optimal algorithms for multi-group learning.
Active Reinforcement Learning (ARL) is a twist on RL where the agent observes reward information only if it pays a cost. This subtle change makes exploration substantially more challenging. Powerful principles in RL like optimism, Thompson sampling, and random exploration do not help with ARL. We relate ARL in tabular …
Efficiently learns Gaussian tree models with near-optimal sample complexity.
problem Learning tree-structured Gaussian distributions efficiently.
method Conditional mutual information tester for Gaussian variables, near-optimal sample complexity.
result Near-optimal sample complexity for structure learning of Gaussian tree models.
We study the problem of regret minimization for distributed bandits learning, in which M M M agents work collaboratively to minimize their total regret under the coordination of a central server. Our goal is to design communication protocols with near-optimal regret and little communication cost, which is measured by the…
Paper tackles clustering with ordinal comparisons, achieving near-optimal results.
problem Clustering with ordinal comparisons when similarity measures are not available.
method Two-step procedure: estimate similarity matrix from comparisons, then apply SDP clustering.
result Near-optimal recovery of planted clustering using near-optimal number of comparisons.
New algorithm achieves near-optimal performance in dueling bandit problem.
problem Optimizing decision-making in dueling bandit problems with limited adaptive rounds.
method Developed a batched algorithm that matches the asymptotic regret bounds of sequential algorithms under the Condorcet condition.
result Asymptotic regret of O ( K 2 log 2 ( K ) ) + O ( K log ( T ) ) O(K^2\log^2(K)) + O(K\log(T)) O ( K 2 log 2 ( K )) + O ( K log ( T )) in O ( log ( T ) ) O(\log(T)) O ( log ( T )) rounds. Develops first optimal algorithm for logistic bandits.
problem Pure exploration in logistic bandits.
method Logistic track-and-stop (Log-TS) algorithm.
result Asymptotically matches lower bound for expected sample complexity.
New policy combines Thompson sampling with best challenger rule for best arm identification.
problem Best arm identification in bandit framework with fixed confidence.
method Combines Thompson sampling with best challenger rule.
result Asymptotically optimal for any two-armed bandit problems, near optimal for general K-armed bandit problems.
Efficiently find near-optimal medical treatments with less trial and error.
problem Finding effective medical treatments through trial and error.
method Formalizes the problem, uses a causal inference framework, and proposes model-based dynamic programming and greedy algorithms.
result Our methods compare favorably to model-free reinforcement learning, offering a more transparent trade-off between search time and treatment efficacy.
New DP algorithms achieve near-optimal regret bounds for online learning problems.
problem Online learning problems with zero-loss solutions and differential privacy constraints.
method Developed new Differentially Private algorithms with near-optimal regret bounds.
result Achieved near-optimal regret bounds for various online prediction and convex optimization problems.
Algorithm extsc{Pedel} learns near-optimal policies efficiently on specific problems.
problem Learning near-optimal policies in linear MDPs with minimal samples.
method Online experiment design to focus exploration on relevant directions.
result Achieves instance-dependent complexity, outperforming minimax-optimal algorithms.
Improved algorithm for adaptive dueling bandits with near-optimal regret bound.
problem Non-stationary dueling bandits with unknown number of preference changes.
method Elimination-based rescheduling algorithm for adaptive dynamic regret.
result Near-optimal i l d e O ( S e x t t t C W T ) ilde{O}(\sqrt{S^{ exttt{CW}} T}) i l d e O ( S e x ttt C W T ) dynamic regret bound. Estimates and infers multi-stage stationary treatment policies with variable selection.
problem Valid inference for multi-stage stationary treatment policies with high-dimensional feature variables.
method Estimate the value function using augmented inverse probability weighted estimator, apply penalty for variable selection, construct one-step improvements for valid inference.
result Improved estimators are asymptotically normal, valid inference for policy parameters demonstrated.
A single policy suffices for near-optimal parallel exploration in RL.
problem Quantitative effects of parallel exploration in reward-free RL.
method Using a single policy to guide exploration across all agents.
result Near-linear speedup and near-minimax optimality for linear MDPs.
New algorithms for best arm identification in bandits robust to misspecified parameters.
problem Inconsistent learning performance of traditional MAB algorithms when parameters are misspecified.
method Proposes two classes of asymptotically near-optimal algorithms for statistically robust MAB under fixed-budget pure exploration.
result Establishes fundamental performance limits and proposes algorithms that are asymptotically near-optimal.
Short proof for ideal polygons with near optimal orthogeodesic decomposition.
problem Decomposing ideal polygons into orthogeodesics.
method Short proof with orthogeodesic decomposition of length at most 2 log ( n ) 2 \log(n) 2 log ( n ) . result Optimal orthogeodesic decomposition of ideal polygons with length 2 log ( n ) 2 \log(n) 2 log ( n ) . A new UCB algorithm for heavy-tailed bandits with near-optimal regret.
problem Sequential decision making in uncertain environments with heavy-tailed rewards.
method Data-driven, distribution-free UCB algorithm combining resampled median-of-means and UCB.
result Near-optimal regret bound for heavy-tailed distributions.
New algorithm reduces bias in trained models, near-optimal performance proven.
problem Reduction of bias in trained machine learning models.
method Scalable post-processing algorithm for debiasing trained models, including deep neural networks (DNNs).
result Proven to be near-optimal by bounding its excess Bayes risk.
A new method finds diverse near-optimal portfolios using quality-diversity.
problem Optimizing financial portfolios with robustness to input parameter uncertainties.
method Quality-Diversity (QD) optimization using CVT-MAP-Elites algorithm.
result Diverse set of near-optimal portfolios identified.