Improves understanding of stochastic NGVI convergence rates.
problem Lack of knowledge about non-asymptotic convergence rates in stochastic NGVI.
method Proved non-asymptotic convergence rates for conjugate likelihoods and showed implicit optimization for non-conjugate likelihoods.
result First O(T1) non-asymptotic convergence rate for stochastic NGVI in conjugate likelihoods. Paper proposes a debiased estimator for adaptive linear regression.
problem Non-normal asymptotic behavior of OLS estimator in adaptive linear regression.
method Adaptive linear estimating equations to construct debiased estimator.
result Established asymptotic normality of the debiased estimator.
Paper derives convergence rates and confidence intervals for LSA with Markovian noise.
problem Analyzing convergence rates and constructing confidence intervals for LSA with Markovian noise.
method Derives non-asymptotic Berry-Esseen bounds and multiplier block bootstrap procedure.
result Provides O(n−1/4) convergence rates and guarantees consistent inference. Paper improves Bayesian inference in federated learning with new algorithm VR-FALD*.
problem Bayesian inference in federated learning with communication bottlenecks and statistical heterogeneity.
method Federated Averaging Langevin Dynamics (FALD) and VR-FALD*.
result VR-FALD* corrects client drift due to statistical heterogeneity, improving convergence.
In this paper, we are concerned with a non-asymptotic analysis of sampling algorithms used in nonconvex optimization. In particular, we obtain non-asymptotic estimates in Wasserstein-1 and Wasserstein-2 distances for a popular class of algorithms called Stochastic Gradient Langevin Dynamics (SGLD). In addition, the afo…
The paper improves confidence set construction for statistical inference.
problem Constructing reliable confidence sets in statistical inference.
method Establishes a finite-sample bound using effective dimension and generalized self-concordance.
result Developed a confidence set adapted to optimization landscapes.
The paper analyzes methods for estimating linear functionals from observational data, proving upper bounds and showing optimal procedures.
problem Estimating linear functionals from observational data in causal inference and bandit literature.
method Two-stage procedures that first estimate treatment effect function, then use it to estimate the linear functional.
result Proves non-asymptotic upper bounds on mean-squared error for two-stage procedures and shows instance-dependent optimality.
Paper characterizes gradient descent in high-dimensional learning problems.
problem Understanding gradient descent dynamics in high-dimensional statistical learning.
method Non-asymptotic joint distributional characterization of gradient descent iterates and debiased statistics.
result Gradient descent iterates approximate normality after debiasing correction.
The study of private inference has been sparked by growing concern regarding the analysis of data when it stems from sensitive sources. We present the first method for private Bayesian inference in exponential families that properly accounts for noise introduced by the privacy mechanism. It is efficient because it work…
Efficient tensor decomposition for count data models achieves near-optimal multiway analysis.
problem Efficient tensor decomposition for count data models.
method Rank-constrained maximum-likelihood estimator for tensor decomposition.
result Achieves multiway analysis with variance matching Cramér-Rao Lower Bound up to constants and logarithmic factors.
Paper stabilizes bandit learning with regularization, improving inference under adaptive sampling.
problem Challenges in statistical inference with adaptive sampling.
method Refined stability condition for online algorithms, using regularized stochastic-mirror-descent-style methods.
result Derives precise regret bounds and asymptotic normality, showing necessity of regularization for valid inference.
A new particle algorithm improves mean-field variational inference.
problem Efficiently approximating nonparametric posterior distributions in machine learning.
method Introduces PArticle VI (PAVI), a novel particle-based algorithm for nonparametric mean-field approximation.
result Obtains non-asymptotic error bounds for PArticle VI, providing the first end-to-end guarantee for particle-based MFVI.
New method aligns diffusion models for inference-time properties without retraining.
problem Aligning pre-trained diffusion models for desired inference-time properties.
method Variationally stable Doob's matching for provable guidance estimation.
result Consistent estimator of guidance with non-asymptotic convergence guarantees.
Estimates treatment effects in rare extreme events using EVT.
problem Estimating treatment effects in rare, impactful events like extreme climate events.
method Introduces a novel framework using EVT and multivariate regular variation for consistent treatment effect estimation.
result Developed a consistent estimator for extreme treatment effects with rigorous non-asymptotic analysis.
Paper explores weighted averaging schemes for SGD, achieving asymptotic normality and optimality.
problem Improving convergence of SGD in various settings.
method Develops a general weighted averaging scheme for SGD and establishes asymptotic normality.
result Establishes asymptotic normality and optimality of weighted averaged SGD solutions.
Exact inference method for Wasserstein distance with finite-sample coverage.
problem Asymptotic approximation methods for Wasserstein distance lack finite-sample validity.
method Selective Inference inspired approach for exact inference.
result Valid confidence interval for Wasserstein distance with finite-sample coverage.
The paper reviews and improves concentration inequalities for statistical inference.
problem Analyzing statistical inference in various settings with high-dimensional data.
method Review and improvement of concentration inequalities for different types of random variables and statistical measures.
result Fresh new results and improved bounds with sharper constants.
New method quantifies reliability of neural network image segmentation.
problem Assessing statistical reliability of neural network-based image segmentation results.
method Selective inference framework to compute exact p-values for DNN-driven hypotheses.
result Proposed method successfully controls false positive rate and provides good results for medical image data.
Paper develops robust methods for large-scale testing without tuning parameters.
problem Heavy-tailed data in high-dimensional settings.
method Revisits Hodges-Lehmann estimator for robust inference without tuning parameters.
result Develops confidence intervals and controls false discovery proportion.
We study the problem of detecting change points (CPs) that are characterized by a subset of dimensions in a multi-dimensional sequence. A method for detecting those CPs can be formulated as a two-stage method: one for selecting relevant dimensions, and another for selecting CPs. It has been difficult to properly contro…
Develops a framework for inferring causal relationships in networked data with uncertainty quantification.
problem Extracting reliable inference from complex Hawkes network data with uncertainty.
method Statistical inference framework based on maximum likelihood estimation and concentration inequalities of continuous-time martingales.
result Provides a non-asymptotic confidence set for uncertainty quantification.
New model handles complex non-linear relationships with hidden graph structures.
problem Modeling non-linear relationships with hidden graph-structured interactions.
method Block-diagonal localized mixture of polynomial experts (BLoMPE) regression model with penalized maximum likelihood selection criterion.
result Strong theoretical guarantee for finite-sample oracle inequality.
New method tightens sub-Gaussian concentration inequalities.
problem Estimating variance-type parameters of sub-Gaussian distributions.
method Using sub-Gaussian intrinsic moment norm to maximize normalized moments.
result Provides tighter sub-Gaussian concentration inequalities.
PPI++ outperforms gold-standard labels only if pseudo-labels are highly correlated.
problem Optimizing statistical estimation using noisy pseudo-labels.
method Exact finite-sample analysis of PPI++ on mean estimation problem.
result PPI++ has provably worse estimation error than gold-standard labels alone in some settings.
We study the problem of cooperative inference where a group of agents interact over a network and seek to estimate a joint parameter that best explains a set of observations. Agents do not know the network topology or the observations of other agents. We explore a variational interpretation of the Bayesian posterior de…
Geometric analysis improves convergence of variational inference.
problem Challenges in analyzing convergence of variational inference due to non-convexity and non-smoothness.
method Exploits exponential family structure and Bregman divergences to geometrically analyze the optimization landscape.
result Establishes non-asymptotic convergence rates for gradient descent algorithms.
Paper develops methods for PCA inference with missing data and heteroskedastic noise.
problem Constructing confidence regions for PCA in high dimensions with missing data and heteroskedastic noise.
method Proposes HeteroPCA and develops non-asymptotic distributional guarantees for valid inference.
result Valid inference on principal subspace and spiked covariance matrix with missing data.
Study shows robust method for estimating density ratios even with heavy contamination.
problem Estimating density ratios in the presence of heavy contamination.
method Weighted density ratio estimation (DRE) with doubly strong robustness.
result Weighted DRE achieves sparse consistency under heavy contamination.
Data augmentation, by the introduction of auxiliary variables, has become an ubiquitous technique to improve convergence properties, simplify the implementation or reduce the computational time of inference methods such as Markov chain Monte Carlo ones. Nonetheless, introducing appropriate auxiliary variables while pre…
A framework uses a mixture of predictors for semi-supervised inference.
problem Limited labeled data, abundant unlabeled data.
method Mixture of Experts (MOE) for semi-supervised inference.
result MOE-powered inference framework achieves smallest possible variance.
Study quantifies information borrowing in hierarchical Bayesian models.
problem Impact of shared hyperparameters on posterior inference.
method Non-asymptotic framework, nested hierarchical prior distribution, integrated risk measure.
result Deeper hierarchical models outperform nested ones under certain conditions.
Constructs non-asymptotic confidence regions for unknown functions in RKHS.
problem Global probabilistic confidence regions for unknown functions in RKHS.
method Reduces confidence region construction to estimating RKHS norm.
result Valid confidence regions can be constructed non-asymptotically.
We provide non-asymptotic convergence rates of the Polyak-Ruppert averaged stochastic gradient descent (SGD) to a normal random vector for a class of twice-differentiable test functions. A crucial intermediate step is proving a non-asymptotic martingale central limit theorem (CLT), i.e., establishing the rates of conve…
New method combines experimental and observational data for causal inference.
problem Combining internal validity of experiments and larger sample sizes of observations.
method Empirical risk minimization (ERM) framework with cross-validation.
result Efficacy and reliability demonstrated on real and synthetic data.
The paper studies how to improve language model inference using particle filtering.
problem Understanding the accuracy-cost tradeoffs of inference-time methods for large language models.
method Introduces particle filtering algorithms like Sequential Monte Carlo (SMC) to study language model inference.
result Identifies criteria enabling non-asymptotic guarantees for SMC and fundamental limits faced by all particle filtering methods.
This paper offers a distribution-free method for post-detection changepoint localization.
problem Locating the exact time of a change in distribution after a sequential detection procedure.
method A distribution-free framework using conformal test martingales for sequential change detection and post-detection inference.
result Valid post-detection coverage guarantees and non-asymptotic bounds on confidence set size.
The paper studies reward concentration in MDPs, covering asymptotic and non-asymptotic settings.
problem Reward concentration in Markov Decision Processes (MDPs).
method Unified approach to reward concentration in MDPs, including asymptotic and non-asymptotic bounds.
result Rate-equivalent definitions of regret for learning policies.
The cold posterior effect is explored through PAC-Bayes bounds for small sample sizes.
problem The cold posterior effect in approximate Bayesian inference for small datasets.
method Investigation through PAC-Bayes generalization bounds, focusing on temperature parameter λ.
result The temperature parameter λ in PAC-Bayes bounds captures the cold posterior effect.
A tutorial on non-asymptotic system identification methods.
problem Identifying system parameters in linear models.
method Covering technique, Hanson-Wright Inequality, method of self-normalized martingales.
result Streamlined proofs of least-squares based estimator performance.
Develops methods for inference after detecting a change in sequential data.
problem Inference after a detected change in sequential data.
method General framework for constructing confidence sets using only data up to a stopping time.
result First general method for sequential changepoint localization with theoretical guarantees.
We prove non-asymptotic lower bounds on the expectation of the maximum of d independent Gaussian variables and the expectation of the maximum of d independent symmetric random walks. Both lower bounds recover the optimal leading constant in the limit. A simple application of the lower bound for random walks is an (…
New framework improves variational inference with Markov chain methods.
problem Challenges of minimizing KL divergence with stochastic gradient descent.
method Markov chain score ascent (MCSA) methods, including parallel MCSA (pMCSA).
result Improved theoretical and empirical performance of MCSA methods.
Study on statistical inference for nonlinear stochastic approximation with Markovian data.
problem Statistical inference for nonlinear stochastic approximation algorithms with Markovian data.
method Established a functional central limit theorem for the partial-sum process of the target parameter estimate, providing asymptotic pivotal statistics for constructing confidence intervals.
result Valid and efficient asymptotic inference method for nonlinear stochastic approximation algorithms with Markovian data.
The posteriors over neural network weights are high dimensional and multimodal. Each mode typically characterizes a meaningfully different representation of the data. We develop Cyclical Stochastic Gradient MCMC (SG-MCMC) to automatically explore such distributions. In particular, we propose a cyclical stepsize schedul…
Paper introduces online tensor inference for real-time data analysis.
problem Real-time processing of high-dimensional tensor data.
method Stochastic Gradient Descent (SGD) for efficient online inference.
result Establishes non-asymptotic convergence and optimal estimation error rate.
BBVI with STL converges geometrically under perfect specification, with quadratic variance bound.
problem Convergence rate of BBVI with STL estimator.
method Proved geometric convergence rate with quadratic variance bound for BBVI with STL estimator.
result BBVI with STL converges geometrically under perfect variational family specification.
The paper analyzes two ISGD modes for statistical inference, deriving error bounds and confidence intervals.
problem Statistical inference with implicit SGD for smooth convex functions.
method Proximal Robbins-Monro (proxRM) and proximal Polyak-Ruppert (proxPR) procedures for ISGD.
result Derives non-asymptotic error bounds and confidence interval estimators for model parameters.
This paper addresses the problem of inferring sparse causal networks modeled by multivariate auto-regressive (MAR) processes. Conditions are derived under which the Group Lasso (gLasso) procedure consistently estimates sparse network structure. The key condition involves a "false connection score." In particular, we sh…