Study shows robust method for estimating density ratios even with heavy contamination.
problem Estimating density ratios in the presence of heavy contamination.
method Weighted density ratio estimation (DRE) with doubly strong robustness.
result Weighted DRE achieves sparse consistency under heavy contamination.
The paper studies reward concentration in MDPs, covering asymptotic and non-asymptotic settings.
problem Reward concentration in Markov Decision Processes (MDPs).
method Unified approach to reward concentration in MDPs, including asymptotic and non-asymptotic bounds.
result Rate-equivalent definitions of regret for learning policies.
Novel framework for uncertainty quantification in metric spaces.
problem Uncertainty quantification in regression models with metric responses.
method Developed algorithms for large datasets, agnostic to predictive models, with asymptotic and non-asymptotic guarantees.
result Asymptotic and non-asymptotic guarantees for special cases, demonstrated in clinical applications.
This work analyzes DP-SGD for online LDP problems with practical convergence rates.
problem Analyzing DP-SGD for online LDP problems with practical convergence rates.
method Developed a general framework for online LDP model in stochastic optimization problems, conducted non-asymptotic convergence analysis.
result Comprehensive non-asymptotic convergence analysis of the proposed estimators in finite-sample situations.
New framework predicts AMP behavior in spiked models for finite iterations.
problem Understanding AMP dynamics in high-dimensional spiked models.
method Developed a non-asymptotic framework for AMP in spiked matrix estimation.
result Predicted AMP behavior for up to O(polylognn) iterations in Z2 synchronization. Develops a generalized version of Chung's Lemma for stochastic optimization methods.
problem Establishing asymptotic convergence rates for stochastic optimization methods under various step size rules.
method Generalized version of Chung's Lemma for a broader family of step size rules.
result Demonstrates tight non-asymptotic convergence rates for various stochastic methods.
New SGD method uses adaptive sampling to converge faster in non-convex problems.
problem Non-convex optimization problems with noisy gradients.
method Adaptive coordinate sampling in stochastic gradient descent (SGD).
result Almost sure convergence and non-asymptotic bounds established.
Develops new bounds for deterministic samplers in diffusion models.
problem Analyzing deterministic samplers in diffusion generative models.
method Operational interpretation of deterministic sampling; restoration and degradation steps.
result First polynomial convergence bounds for DDIM-type samplers.
We propose a unified framework to speed up the existing stochastic matrix factorization (SMF) algorithms via variance reduction. Our framework is general and it subsumes several well-known SMF formulations in the literature. We perform a non-asymptotic convergence analysis of our framework and derive computational and …
Conditional diffusion models improve data generation with non-asymptotic convergence bounds.
problem Lack of non-asymptotic properties in conditional diffusion models.
method Integrates a pre-trained model into the diffusion model framework to capture conditional distributions.
result Established upper error bounds for the convergence between original and generated conditional distributions.
This paper analyzes error bounds for biased SMC samplers in conditional sampling.
problem Analyzing error bounds for biased SMC samplers in conditional sampling.
method Develops a non-asymptotic error analysis for SMC samplers with biased mutation kernels.
result Derives the first non-asymptotic error bound for conditional sampling with score-based diffusion models.
Constructs non-asymptotic confidence regions for unknown functions in RKHS.
problem Global probabilistic confidence regions for unknown functions in RKHS.
method Reduces confidence region construction to estimating RKHS norm.
result Valid confidence regions can be constructed non-asymptotically.
We provide non-asymptotic convergence rates of the Polyak-Ruppert averaged stochastic gradient descent (SGD) to a normal random vector for a class of twice-differentiable test functions. A crucial intermediate step is proving a non-asymptotic martingale central limit theorem (CLT), i.e., establishing the rates of conve…
New schemes improve error estimates for sampling from non-log-concave distributions.
problem Improving sampling from non-log-concave distributions with super-linear drift growth.
method Developed tamed Euler and randomized Euler schemes with error estimates.
result Near-optimal error bounds for sampling and optimization problems.
New algorithm samples superlinearly growing log-gradient distributions.
problem Sampling from distributions with superlinearly growing log-gradient.
method Proposes a novel taming Langevin-based scheme called sTULA.
result Derives non-asymptotic convergence bounds in KL, TV, and W2 distances.
New framework improves text watermark detection under imperfect pseudorandomness.
problem Structured dependence in generated text from language models causes Type I error control issues.
method Hierarchical two-layer partition, minimal units, non-asymptotic efficiency measure, minimax hypothesis testing.
result Closed-form optimal rules for watermark detection under imperfect pseudorandomness.
Value aggregation is a general framework for solving imitation learning problems. Based on the idea of data aggregation, it generates a policy sequence by iteratively interleaving policy optimization and evaluation in an online learning setting. While the existence of a good policy in the policy sequence can be guarant…
New framework finds periodic policies in reset-free MDPs with sublinear regret.
problem Reset-free reinforcement learning with unknown dynamics and terminal law constraints.
method Periodic framework, periodic policies, periodic regret.
result First non-asymptotic guarantees for reset-free learning in multi-agent settings.
Particle-optimization-based sampling (POS) is a recently developed effective sampling technique that interactively updates a set of particles. A representative algorithm is the Stein variational gradient descent (SVGD). We prove, under certain conditions, SVGD experiences a theoretical pitfall, {\it i.e.}, particles te…
New quasi-Newton method guarantees global superlinear convergence.
problem Global convergence and superlinear convergence of quasi-Newton methods.
method Hybrid proximal extragradient method with online learning for Hessian approximation.
result First globally convergent quasi-Newton method with explicit superlinear convergence rate.
New streaming methods improve convergence rates for optimization problems.
problem Optimizing large-scale, sequential data problems.
method Time-varying mini-batches and Polyak-Ruppert averaging for gradient-based algorithms.
result Time-varying mini-batches and averaging achieve optimal convergence and variance reduction.
Efficient tensor decomposition for count data models achieves near-optimal multiway analysis.
problem Efficient tensor decomposition for count data models.
method Rank-constrained maximum-likelihood estimator for tensor decomposition.
result Achieves multiway analysis with variance matching Cramér-Rao Lower Bound up to constants and logarithmic factors.
A tutorial on non-asymptotic system identification methods.
problem Identifying system parameters in linear models.
method Covering technique, Hanson-Wright Inequality, method of self-normalized martingales.
result Streamlined proofs of least-squares based estimator performance.
We prove non-asymptotic lower bounds on the expectation of the maximum of d independent Gaussian variables and the expectation of the maximum of d independent symmetric random walks. Both lower bounds recover the optimal leading constant in the limit. A simple application of the lower bound for random walks is an (…
We study the constrained linear quadratic regulator with unknown dynamics, addressing the tension between safety and exploration in data-driven control techniques. We present a framework which allows for system identification through persistent excitation, while maintaining safety by guaranteeing the satisfaction of st…
Paper analyzes ensemble Kalman updates for effective dimension and localization.
problem Why small ensemble sizes work well in inverse problems and data assimilation.
method Non-asymptotic analysis of ensemble Kalman updates, focusing on effective dimension and localization.
result Rigorously explains why a small ensemble size is sufficient when prior covariance has moderate effective dimension.
New method quantifies reliability of neural network image segmentation.
problem Assessing statistical reliability of neural network-based image segmentation results.
method Selective inference framework to compute exact p-values for DNN-driven hypotheses.
result Proposed method successfully controls false positive rate and provides good results for medical image data.
Paper characterizes gradient descent in high-dimensional learning problems.
problem Understanding gradient descent dynamics in high-dimensional statistical learning.
method Non-asymptotic joint distributional characterization of gradient descent iterates and debiased statistics.
result Gradient descent iterates approximate normality after debiasing correction.
The paper analyzes methods for estimating linear functionals from observational data, proving upper bounds and showing optimal procedures.
problem Estimating linear functionals from observational data in causal inference and bandit literature.
method Two-stage procedures that first estimate treatment effect function, then use it to estimate the linear functional.
result Proves non-asymptotic upper bounds on mean-squared error for two-stage procedures and shows instance-dependent optimality.
VRPG algorithm optimizes convex constraints with non-asymptotic guarantees.
problem Stochastic convex optimization under convex constraints.
method Natural variance reduced proximal gradient (VRPG) algorithm.
result VRPG achieves local minimax lower bound up to constants and log factor of N. Optimizes prediction error method for time-varying models.
problem Achieving optimal prediction error rates for time-varying models.
method Nonlinear least squares method for time-varying parametric models.
result First rate-optimal non-asymptotic analysis for time-varying models.
We study the efficiency of V-fold cross-validation (VFCV) for model selection from the non-asymptotic viewpoint, and suggest an improvement on it, which we call ``V-fold penalization''. Considering a particular (though simple) regression problem, we prove that VFCV with a bounded V is suboptimal for model selection, be…
We introduce an asymmetric distance in the space of learning tasks, and a framework to compute their complexity. These concepts are foundational for the practice of transfer learning, whereby a parametric model is pre-trained for a task, and then fine-tuned for another. The framework we develop is non-asymptotic, captu…
The paper analyzes the training dynamics of a transformer for next-token prediction.
problem Understanding the non-asymptotic performance of transformers in next-token prediction.
method Characterizes training dataset properties, designs a two-stage training algorithm, and analyzes attention gradient properties.
result Trained transformers converge sub-linearly to max-margin solutions and exhibit linear convergence in cross-entropy loss.
Paper approximates risk measures using SGD with Langevin dynamics.
problem Approximating arbitrary law invariant risk measures.
method Stochastic Gradient Langevin Dynamics (SGD-Langevin) for general risk measures.
result Non-asymptotic convergence rates of the approximation algorithm.
Stochastic particle-optimization sampling (SPOS) is a recently-developed scalable Bayesian sampling framework that unifies stochastic gradient MCMC (SG-MCMC) and Stein variational gradient descent (SVGD) algorithms based on Wasserstein gradient flows. With a rigorous non-asymptotic convergence theory developed recently…
Study on stochastic approximation with Polyak-Ruppert averaging for linear systems.
problem Understanding the asymptotic and non-asymptotic properties of stochastic approximation procedures.
method Detailed analysis of linear stochastic approximation with Polyak-Ruppert averaging, focusing on asymptotic and non-asymptotic properties.
result Proves CLT and non-asymptotic concentration inequality for averaged iterates, providing refined understanding of linear stochastic approximation.
We propose a new algorithmic framework for sequential hypothesis testing with i.i.d. data, which includes A/B testing, nonparametric two-sample testing, and independence testing as special cases. It is novel in several ways: (a) it takes linear time and constant space to compute on the fly, (b) it has the same power gu…
Study optimizes prediction error for growing-dimensional PFLM models.
problem Optimizing prediction error for growing-dimensional PFLM models.
method Penalized least-squares approach in RKHS with effective dimension consideration.
result Shows exact upper bound for excess prediction risk in non-asymptotic form.
In this paper, we are concerned with a non-asymptotic analysis of sampling algorithms used in nonconvex optimization. In particular, we obtain non-asymptotic estimates in Wasserstein-1 and Wasserstein-2 distances for a popular class of algorithms called Stochastic Gradient Langevin Dynamics (SGLD). In addition, the afo…
A new algorithm reduces bias in estimating model parameters.
problem Efficient estimation of model parameters in non-linear state-space models.
method Parisian particle Gibbs (PPG) algorithm for bias reduction in online learning.
result Non-asymptotic bounds on bias and variance for PPG.
Analyzes learning and applying preconditioners in MCMC for efficiency.
problem Improving efficiency of MCMC algorithms.
method Non-asymptotic analysis of schemes that learn preconditioners.
result Established non-asymptotic guarantees for preconditioned ULA.
New algorithm achieves instance-optimality in decision making.
problem Develop adaptive algorithms for interactive decision making.
method Introduce Allocation-Estimation Coefficient (AEC) and develop AE2 algorithm. result First non-asymptotic instance-optimal performance guarantees.
This paper improves non-asymptotic bounds for denoising diffusions, focusing on the Ornstein-Uhlenbeck process.
problem Improving non-asymptotic bounds for denoising diffusions, especially for the Ornstein-Uhlenbeck process.
method Explicit non-asymptotic bounds on forward diffusion error in total variation, considering multi-modal data distributions.
result The Ornstein-Uhlenbeck process cannot be significantly improved in terms of reducing terminal time T for multi-modal data distributions. Recent works have derived non-asymptotic upper bounds for convergence of underdamped Langevin MCMC. We revisit these bound and consider introducing scaling terms in the underlying underdamped Langevin equation. In particular, we provide conditions under which an appropriate scaling allows to improve the error bounds in…
This paper provides performance guarantees for neural estimation of statistical distances.
problem Developing performance guarantees for neural estimation of statistical distances.
method Non-asymptotic error bounds using function approximation theorems and empirical process theory.
result Established a fundamental tradeoff between approximation and estimation errors in neural estimation of statistical distances.
DALMC provides non-asymptotic error bounds for generative models.
problem Efficiently generating samples from complex data distributions.
method Analysis of diffusion paths and Langevin Monte Carlo.
result Theoretical guarantees for a class of generative models.
Motivated by the study of Q-learning algorithms in reinforcement learning, we study a class of stochastic approximation procedures based on operators that satisfy monotonicity and quasi-contractivity conditions with respect to an underlying cone. We prove a general sandwich relation on the iterate error at each time,…