Nonasymptotic error bounds and strong consistency rates for survival analysis methods.
problem Establishing reliable error bounds and consistency rates for survival analysis methods.
method Nonasymptotic error bounds for Kaplan-Meier-based nearest neighbor and kernel survival probability estimators in metric spaces.
result Rates of strong consistency match existing lower bounds for conditional CDF estimation.
Optimized AIS scheme reduces bias and MSE for general proposals.
problem Performing Monte Carlo integration with general proposals.
method Global optimization of χ²-divergence using stochastic gradient Langevin dynamics.
result Explicit theoretical guarantees for uniform-in-time MSE reduction.
SGHMC improves sampling and optimization under local conditions.
problem Nonconvex optimization and sampling under local conditions.
method Nonasymptotic analysis of SGHMC convergence.
result SGHMC provides high-precision results uniformly in iterations.
Develops a new algorithm for estimating model parameters using interacting particle systems.
problem Estimating parameters of latent variable models.
method Interacting Particle Langevin Algorithm (IPLA) based on Langevin diffusion.
result Nonasymptotic optimisation error bounds for the estimator.
This paper tightens the law of the iterated logarithm for empirical KL_inf, applicable to unbounded data.
problem Developing nonasymptotic concentration bounds for empirical KL_inf with optimal constants and rates.
method Presenting a tight law of the iterated logarithm for empirical KL_inf, applicable to unbounded data.
result A tight law of the iterated logarithm for empirical KL_inf, applicable to unbounded data.
FIEM accelerates EM for large datasets with nonasymptotic convergence bounds.
problem Efficiently optimizing large datasets using EM framework.
method FIEM recasts EM in Stochastic Approximation framework and provides nonasymptotic convergence bounds.
result Nonasymptotic bounds for convergence in expectation as a function of n and $\kmax$. Sampling from various kinds of distributions is an issue of paramount importance in statistics since it is often the key ingredient for constructing estimators, test procedures or confidence intervals. In many situations, the exact sampling from a given distribution is impossible or computationally expensive and, there…
New method estimates gradients accurately with sharp bounds.
problem Accurate gradient estimation in regression problems.
method Nearest-neighbor based pointwise estimate of gradients.
result Sharp nonasymptotic bounds for gradient estimation.
ROOT-SGD solves convex optimization problems with optimal nonasymptotic and near-optimal asymptotic performance.
problem Solving strongly convex and smooth unconstrained optimization problems using stochastic first-order algorithms.
method ROOT-SGD: Recursive One-Over-T SGD, averaging past stochastic gradients.
result Achieves state-of-the-art performance in both nonasymptotic and asymptotic senses.
Bayesian inference typically requires the computation of an approximation to the posterior distribution. An important requirement for an approximate Bayesian inference algorithm is to output high-accuracy posterior mean and uncertainty estimates. Classical Monte Carlo methods, particularly Markov Chain Monte Carlo, rem…
New confidence intervals improve treatment effect estimation in randomized experiments.
problem Improving confidence intervals for treatment effects in randomized experiments.
method Systematic exploitation of negative dependence or variance adaptivity.
result Achieved nonasymptotic confidence intervals with the same effective sample size as asymptotic ones.
Gradient descent optimally trains RNNs without overparameterization.
problem Training recurrent neural networks (RNNs) with gradient descent.
method Nonasymptotic analysis of gradient descent for RNNs with diagonal weight matrices.
result Gradient descent can achieve optimality in RNNs with a network size scaling logarithmically with the number of samples.
Study Q-learning with averaging for reinforcement learning, proving efficient inference and error bounds.
problem Efficient inference and error bounds for Q-learning with averaging.
method Functional central limit theorem and asymptotic linear estimator for optimal Q-value function.
result Standardized partial-sum process converges weakly to a rescaled Brownian motion, matching instance-dependent lower bound for error.
Paper improves confidence intervals and variance estimation for deep learning models.
problem Improving confidence intervals and variance estimation in deep learning models.
method Residual-based framework for conditional variance estimation; robust bootstrap procedure for confidence intervals.
result First non-asymptotic bounds for variance estimation using ReLU networks.
In this paper, we study the problem of sampling from a given probability density function that is known to be smooth and strongly log-concave. We analyze several methods of approximate sampling based on discretizations of the (highly overdamped) Langevin diffusion and establish guarantees on its error measured in the W…
The paper develops tests for comparing means in high dimensions with unknown covariance.
problem Testing if the mean of a high-dimensional distribution is close to zero or different from another.
method Develops nonasymptotic tests using concentration inequalities and operator norms.
result Obtains bounds on the minimal separation distance for controlling Type I and Type II errors.
We study sparse principal components analysis in high dimensions, where p (the number of variables) can be much larger than n (the number of observations), and analyze the problem of estimating the subspace spanned by the principal eigenvectors of the population covariance matrix. We introduce two complementary not…
This paper introduces time-uniform CLT-based confidence intervals for statistical inference.
problem Developing valid statistical inference methods for sequential data.
method Time-uniform central limit theory and strong invariance principles.
result Asymptotic confidence sequences (CSs) that are uniformly valid over time.
Phase retrieval refers to the problem of recovering real- or complex-valued vectors from magnitude measurements. The best-known algorithms for this problem are iterative in nature and rely on so-called spectral initializers that provide accurate initialization vectors. We propose a novel class of estimators suitable fo…
Paper analyzes SGLD for nonconvex optimization with local conditions.
problem Analyzing sampling algorithms for nonconvex optimization.
method Non-asymptotic estimates for SGLD under local conditions.
result Establishes error bounds for expected excess risk.
Unified framework for learning from unlabeled and labeled data in knowledge graphs.
problem Scarcity of relation-specific labeled triples and ad hoc scoring functions limit model training and generalizability.
method Two-stage procedure: unsupervised pretraining over heterogeneous corpora followed by supervised learning with multiple relation types.
result Established a nonasymptotic risk bound quantifying the benefit of large-scale unlabeled data.
The paper creates nonparametric confidence bands for band-limited functions.
problem Estimating confidence bands for band-limited functions with finite samples and unknown noise.
method Uses Paley-Wiener reproducing kernel Hilbert spaces and gradient-perturbation methods.
result Non-asymptotic guarantees for confidence regions without assuming a parametric model.
This research provides theoretical guarantees for hyperparameter estimation in complex network dynamical systems.
problem Theoretical guarantees for hyperparameter estimation in large, inhomogeneous complex network dynamical systems.
method Formulating the system's evolution in a measure transport perspective, proposing a theoretical framework for estimating hyperparameters with mean-type observations.
result A nonasymptotic bound for the deviation of hyperparameter estimates in inhomogeneous complex network dynamical systems with respect to network population size.
The Rasch model is widely used for item response analysis in applications ranging from recommender systems to psychology, education, and finance. While a number of estimators have been proposed for the Rasch model over the last decades, the available analytical performance guarantees are mostly asymptotic. This paper p…
Study identifies three quantization regimes for ReLU networks.
problem Approximation of Lipschitz functions by ReLU networks with finite-precision weights.
method Established through nonasymptotic tight lower and upper bounds on minimax approximation error.
result Memory-optimality achieved in proper quantization regime for deep networks.
The paper improves nonparametric confidence bands for band-limited functions.
problem Constructing nonparametric simultaneous confidence bands with nonasymptotic and distribition-free guarantees.
method Based on Paley-Wiener reproducing kernel Hilbert spaces, the paper relaxes assumptions, improves noise estimation, and tightens constraints.
result Enhanced confidence bands with improved efficiency and tighter constraints.
Many modern big data applications feature large scale in both numbers of responses and predictors. Better statistical efficiency and scientific insights can be enabled by understanding the large-scale response-predictor association network structures via layers of sparse latent factors ranked by importance. Yet sparsit…
Method solves nonconvex constrained optimization problems with a new augmented Lagrangian approach.
problem Nonconvex composite functional constraints with inequality constraints.
method First-order augmented Lagrangian method with smoothed prox-linear reformulation.
result Explicit convergence rates for the proposed method in terms of KKT residual.
CSI method learns conditional distributions by estimating flow equations.
problem Learning conditional distributions in generative models.
method Estimates probability flow equations to transport reference to target distribution.
result Derives explicit expressions for conditional drift and score functions.
Study shows how to effectively predict functions on manifolds using kernel methods.
problem Regression on manifolds with limited data.
method Reproducing kernel Hilbert space methods, Weyl law, effective dimension.
result Kernel regression estimator yields minimax-optimal error bounds controlled by effective dimension.
The paper analyzes SGD with Richardson-Romberg extrapolation for convex optimization problems.
problem Solving strongly convex and smooth minimization problems efficiently.
method Combining SGD with Polyak-Ruppert averaging and Richardson-Romberg extrapolation.
result An expansion of the mean-squared error of the estimator with respect to the number of iterations.
Efficient algorithm for sparse PCA reduces data complexity.
problem Sparse PCA for high-dimensional data with non-convex optimization issues.
method Convex FPS formulation, gradient-based optimization, online learning extension.
result Explicit bounds on optimization error and statistical accuracy.
We developed a novel statistical method to identify structural differences between networks characterized by structural equation models. We propose to reparameterize the model to separate the differential structures from common structures, and then design an algorithm with calibration and construction stages to identif…
New method tests independence using ROC analysis and bipartite ranking.
problem Testing independence of two random variables with unknown marginals.
method Nonparametric framework based on ROC analysis and bipartite ranking.
result The method detects small departures from independence in high dimensions.
The study analyzes the performance of a nonparametric estimator for dynamical systems.
problem Analyzing the performance of a nonparametric estimator for dynamical systems.
method Nonparametric least squares estimator (LSE) and information-theoretic methods.
result Rate-optimal error bounds for nonparametric hypotheses classes.
New theory explains how overparametrized neural networks generalize well without bias-variance trade-off.
problem Overparametrized neural networks generalize well despite classical bias-variance trade-off.
method Nonasymptotic generalization theory for two-layer neural networks with ReLU activation, incorporating scaled variation regularization.
result Prediction bounds for all network widths reproduce the double descent phenomenon, and overparametrized models are nearly minimax optimal.
Paper develops sparse learning for heavy-tailed time series with locally stationary dynamics.
problem Sparse learning for high-dimensional heavy-tailed locally stationary time series.
method Additive modeling with kernel smoothing, sparsity-inducing penalized estimation.
result Prediction-error bounds and convergence rates for different sparsity structures.
The paper provides bounds for regression schemes using nonstationary training samples.
problem Developing confidence intervals for nonparametric regression with nonstationary data.
method The approach involves Rademacher and Vapnik-Chervonenkis theories to analyze the cost and optimality of regression schemes.
result The paper establishes nonasymptotic bounds for regression schemes and optimality in L2-distance. Proposes a method to improve regression model performance with limited target data using fused-regularizer.
problem Model shifts and covariate shifts in high-dimensional regression.
method Two-step method with fused-regularizer to leverage source data for target task.
result Robust to covariate shifts, minimax-optimal under certain conditions, and validated by numerical tests.
In this work, we study the problem of reconstructing shapes from simple nonasymptotic densities measured only along shape boundaries. The particular density we study is also known as the integral area invariant and corresponds to the area of a disk centered on the boundary that is also inside the shape. It is easy to s…
Study tests whether trade-off functions are above or below benchmarks using finite samples.
problem Testing trade-off functions between unknown distributions.
method Identifies a condition for nontrivial testing, constructs a test with error guarantees, and inverts the test for confidence bands.
result Finite-sample testing is possible under specific structural assumptions about rejection regions.
Kernel method estimates long-term effects from short-term data.
problem Estimating long-term effects from short-term data in continuous actions.
method Kernel ridge regression to embed and extrapolate long-term effects.
result Uniform consistency and nonasymptotic error bounds for the estimator.
RFM uses tangent vector fields to match data on manifolds, analyzing TV convergence for Euler discretization.
problem Matching data on curved manifolds using flow-based models.
method Developed a nonasymptotic TV convergence analysis for RFM samplers using Euler discretization.
result Explicit bounds on TV convergence separating numerical discretization and learning errors.
Regularized linear regression under the ℓ1 penalty, such as the Lasso, has been shown to be effective in variable selection and sparse modeling. The sampling distribution of an ℓ1-penalized estimator β^ is hard to determine as the estimator is defined by an optimization problem that in general can only…
New algorithm samples from log concave distributions efficiently.
problem Sampling from log concave distributions efficiently.
method Stochastic Proximal Langevin Algorithm (SPLA) with potential splitting.
result Established nonasymptotic convergence rates for SPLA.
The paper improves confidence regions for band-limited functions using tighter norm bounds and majority voting.
problem Constructing reliable confidence regions for band-limited functions from noisy data.
method Improved norm bounds using Hoeffding's inequality and empirical Bernstein bound, majority voting to aggregate intervals.
result Confidence intervals retain their simultaneous coverage guarantee even when aggregated from random subsamples.
New method uses machine learning to improve statistical inference.
problem Performing inference on conditional functionals with scarce labeled data.
method Combines localization with prediction-based variance reduction.
result Valid and sharp confidence intervals for conditional functionals.
New strategy identifies best Markovian arm with fixed confidence.
problem Identifying the best arm in Markovian bandit models with fixed confidence.
method Analyzed the Track-and-Stop strategy and derived a concentration inequality for Markov chains.
result The Track-and-Stop strategy is at most a factor of four apart from the lower bound for asymptotic performance.