This work uses sampling theory to analyze smoothness and error bounds of finite neural networks.
problem Analyzing the function space of finite neural networks and providing error bounds.
method Applying sampling theory to finite neural networks with non-expansive activation functions, considering both deterministic and random sampling.
result Novel error bounds for univariate neural networks under band-limited input assumption, highlighting the advantage of deterministic uniform sampling.
The study sets lower bounds on MMSE for inferring sensitive features from noisy data.
problem Estimating sensitive features from noisy observations of correlated features.
method Adversarial evaluation framework based on MMSE estimation with theoretical lower bounds.
result Derives closed-form bounds for linear models, showing optimality in noise variance.
New bounds show current methods overestimate system parameter errors.
problem Current bounds overestimate parameter errors in system identification.
method Utilized asymptotic normality and second-order decomposition.
result Obtained finite-sample bounds matching optimal rates up to constants.
Estimates barycenter in geodesic spaces with finite sample bounds.
problem Estimating the barycenter of a distribution in geodesic spaces.
method Finite sample error bounds, Hoeffding- and Bernstein-type concentration inequalities, efficient algorithms.
result Statistical guarantees for efficient barycenter computation.
The paper analyzes LOCV for high-dimensional risk estimation, proving error bounds.
problem Estimating out-of-sample prediction error in high-dimensional settings.
method Theoretical analysis of leave-one-out cross validation (LOCV) in penalized regression.
result Finite sample upper bounds on LOCV error, showing it converges to zero as n,p → ∞.
This paper analyzes error bounds for biased SMC samplers in conditional sampling.
problem Analyzing error bounds for biased SMC samplers in conditional sampling.
method Develops a non-asymptotic error analysis for SMC samplers with biased mutation kernels.
result Derives the first non-asymptotic error bound for conditional sampling with score-based diffusion models.
End-to-end algorithm for controlling bilinear systems with probabilistic noise.
problem Controlling bilinear systems with noisy data.
method Proposes an end-to-end algorithm using statistical learning theory and robust controller design.
result Derived finite sample identification error bounds and structurally suitable for control.
We identify linear models from nonlinear systems with initialization constraints.
problem Identifying linear models from nonlinear systems with initialization constraints.
method Multiple trajectories-based deterministic data acquisition algorithm followed by regularized least squares.
result We provide a finite sample error bound on the learned linearized dynamics.
CNFs learn distributions from samples with error bounds.
problem Learning probability distributions from finite samples.
method Continuous normalizing flows with linear interpolation and flow matching objective function.
result Non-asymptotic error bounds for distribution estimator in Wasserstein-2 distance.
We learn linear models from nonlinear systems using multiple trajectories and regularization.
problem Identifying linear models from data when the underlying dynamics are nonlinear.
method Multiple trajectories data acquisition followed by regularized least squares.
result Learn linearized dynamics with arbitrarily small error given enough samples.
Sparse feature selection improves batch RL efficiency.
problem High-dimensional batch RL with many features.
method Sparse linear function approximation, Lasso, group Lasso, fitted Q-evaluation, fitted Q-iteration.
result Sparse feature selection makes batch RL more sample efficient.
This study analyzes LTS in sparse models with finite sample error bounds.
problem Robust regression in high-dimensional sparse models with limited data.
method Non-asymptotic analysis of LTS error bounds.
result Established finite sample error bounds for LTS in sparse models.
Paper analyzes online tensorial ICA convergence with stochastic approximation.
problem Online tensorial ICA convergence analysis.
method Stochastic approximation for nonconvex optimization.
result Sharp finite-sample error bound of O ~ ( d / T ) \tilde{O}(\sqrt{d/T}) O ~ ( d / T ) . Unified framework for finite-sample RL algorithms using Lyapunov theory.
problem Finite-sample convergence guarantees of asynchronous RL algorithms.
method Reformulate RL algorithms as Markovian SA, develop Lyapunov analysis.
result Mean-square error bounds and convergence for various RL algorithms.
The paper provides bounds on estimation error in a distributed online learning setting.
problem Estimating an unknown parameter in a distributed and online manner with finite sample guarantees.
method Proposes a distributed online estimation algorithm that improves accuracy through communication, providing non-asymptotic bounds on estimation error.
result Demonstrates a trade-off between estimation error and communication costs, and determines a stopping time for communication based on desired accuracy.
The paper analyzes GTD algorithms with finite-sample bounds.
problem Convergence rate analysis of GTD family of algorithms.
method Formulated as stochastic gradient algorithms and analyzed using saddle-point error.
result Obtained finite-sample bounds on GTD performance.
Detect changes in noisy dynamical systems using empirical approximations and finite-sample bounds.
problem Change detection in noisy dynamical systems
method Partition-based empirical approximations and finite-state stationary distribution stability
result Finite-sample bound for empirical stationary density
GANs learn distributions well from samples, with rates depending on intrinsic dimension.
problem Learning distributions from samples using GANs.
method Oracle inequality, Hölder functions approximation, neural network approximation, integral probability metrics.
result Convergence rates of GANs depend on intrinsic dimension, not ambient dimension.
Paper derives PAC-Bayesian bounds for LTI systems learning from empirical data.
problem Characterizing predictive power of LTI systems learned from data.
method PAC-Bayesian bounds for LTI stochastic dynamical systems with inputs.
result Finite-sample error bounds for learning algorithms of LTI systems.
Study identifies and analyzes three types of errors in learning Fourier operators.
problem Statistical, discretization, and truncation errors in learning Fourier operators.
method Analysis of a Discrete Fourier Transform (DFT) based least squares estimator.
result Established upper and lower bounds on statistical, discretization, and truncation errors.
In this paper, we analyze the finite sample complexity of stochastic system identification using modern tools from machine learning and statistics. An unknown discrete-time linear system evolves over time under Gaussian noise without external inputs. The objective is to recover the system parameters as well as the Kalm…
Consider the task of estimating a 3-order n × n × n n \times n \times n n × n × n tensor from noisy observations of randomly chosen entries in the sparse regime. We introduce a similarity based collaborative filtering algorithm for estimating a tensor from sparse observations and argue that it achieves sample complexity that nearly matc…
Despite the increasing interest in multi-agent reinforcement learning (MARL) in multiple communities, understanding its theoretical foundation has long been recognized as a challenging problem. In this work, we address this problem by providing a finite-sample analysis for decentralized batch MARL with networked agents…
Learn dynamics of a system using auxiliary data from similar systems.
problem Learning dynamics of a linear system with limited data.
method Weighted least squares approach, incorporating auxiliary data.
result Auxiliary data can help reduce intrinsic error due to noise.
The paper provides guarantees for feedback control with sensor errors.
problem Certifying performance and safety in feedback control systems with sensor errors.
method Solving a supervised learning problem to characterize sensor errors and providing uniform error bounds.
result Finite-time convergence rate on sub-optimality of using a regressor in closed-loop for waypoint tracking.
Principal component analysis (PCA) has been a prominent tool for high-dimensional data analysis. Online algorithms that estimate the principal component by processing streaming data are of tremendous practical and theoretical interests. Despite its rich applications, theoretical convergence analysis remains largely ope…
Study error bounds in evaluating distributional computational graphs.
problem Error analysis in evaluating graphs with inputs as probability distributions.
method Establish non-asymptotic error bounds using Wasserstein-1 distance.
result Non-asymptotic error bounds for discretization errors in distributional computational graphs.
The study establishes minimax bounds for estimating operators from noisy samples.
problem Estimating unknown operators between Hilbert spaces from noisy data.
method Developed a minimax theory for uniformly bounded Lipschitz operators, proving lower and upper bounds.
result Sharp characterizations of minimax risk for generic Lipschitz operators, showing a curse of sample complexity.
New bounds derived for KG algorithm's performance in finite time.
problem Best arm identification problem in multi-armed bandit.
method Theoretical analysis of finite-time performance, deriving bounds for sample allocation, error probability, and regret.
result Upper and lower bounds for the probability of error and simple regret of the KG algorithm.
We consider least squares estimation in a general nonparametric regression model. The rate of convergence of the least squares estimator (LSE) for the unknown regression function is well studied when the errors are sub-Gaussian. We find upper bounds on the rates of convergence of the LSE when the errors have uniformly …
FQE with deep neural networks achieves asymptotic normality and finite-sample bounds.
problem Theoretical understanding of FQE with general differentiable function approximators.
method Z-estimation theory applied to FQE with deep neural networks.
result FQE estimation error is asymptotically normal with explicit variance.
New framework improves EM algorithm convergence under log-Sobolev inequality.
problem Improving convergence of the EM algorithm.
method Extending gradient flow techniques to EM algorithm, using free energy representation.
result Exponential convergence of EM algorithm under log-Sobolev inequality.
Adapts RKHS methods to estimate density ratios with optimal error.
problem Estimating density ratios from limited data.
method Minimizes regularized Bregman divergence in RKHS, with Lepskii type parameter choice.
result Adaptive minimax optimal error rate for quadratic loss.
The paper analyzes the probabilistic structure of DDPMs and bounds their sampling error.
problem Understanding and controlling errors in discrete-time DDPMs.
method Structural analysis of score functions, Schrödinger's problem, and FBSDEs.
result Explicit upper bound for total variation distance between sampling and target distributions.
Paper improves off-policy evaluation for reinforcement learning with asymptotically efficient estimators.
problem Estimating target policy performance using offline data collected by a different policy.
method Developed a modified marginalized importance sampling (MIS) estimator that achieves asymptotically efficient error bounds.
result Proved that a simple modification to the MIS estimator can achieve a Cramer-Rao lower bound in mean square error.
We consider the dynamics of a linear stochastic approximation algorithm driven by Markovian noise, and derive finite-time bounds on the moments of the error, i.e., deviation of the output of the algorithm from the equilibrium point of an associated ordinary differential equation (ODE). We obtain finite-time bounds on t…
The paper introduces a frequency-domain estimator for low-order systems from noisy data.
problem Estimating frequency responses of low-order systems from noisy measurements.
method Uses a quadratic data-fitting term regularized by the nuclear norm of a Loewner matrix, subject to a convex stability constraint.
result Proves a finite-sample error bound and extends it to all frequencies through rational interpolation.
Estimates time-series drifts from i.i.d. data using a direct Nadaraya-Watson plug-in method.
problem Nonparametric estimation of Schrödinger bridge drifts from single time interval data.
method Direct Nadaraya-Watson plug-in estimator based on kernelized numerator and denominator terms.
result Uniform non-asymptotic bound, CLT under undersmoothing, and adaptive bandwidth selector.
Sharp bounds derived for test error of finite-rank kernel ridge regression.
problem Loose bounds on test error for finite-rank kernels in machine learning.
method Sharp non-asymptotic upper and lower bounds for KRR test error.
result Tighter bounds on finite-rank KRR test error, valid for any regularization parameters.
In this paper, we study the performance of extremum estimators from the perspective of generalization ability (GA): the ability of a model to predict outcomes in new samples from the same population. By adapting the classical concentration inequalities, we derive upper bounds on the empirical out-of-sample prediction e…
The paper improves methods for estimating set size using samples.
problem Estimating the size of a set from a uniform sample.
method Refines estimators using the birthday problem and maximum of sample.
result Develops a general theory for non-asymptotic error bounds.
The OLS estimator optimally identifies stable linear systems with a finite number of samples.
problem Identifying stable linear systems with a finite number of samples.
method Finite-time analysis of the Ordinary Least Squares (OLS) estimator for stable linear systems.
result The OLS estimator achieves optimal sample complexity for stable systems, matching existing lower bounds up to universal factors.
New method improves understanding of machine learning model performance.
problem Understanding how well machine learning models generalize from training data to unseen data.
method Auxiliary Distribution Method to derive new generalization error bounds.
result Upper bounds on generalization errors are tighter and more applicable.
Paper reduces sample complexity for bilinear systems identification to nearly constant.
problem Identifying discrete-time bilinear systems under bounded disturbances.
method Uses trajectory-dependent regressors and polynomial mean-square state growth analysis.
result Proves sample complexity of O ~ ( 1 / ε ) \widetilde{\mathcal O}(1/ε) O ( 1/ ε ) for estimation error ε ε ε . We introduce a simulation method for dynamic portfolio valuation and risk management building on machine learning with kernels. We learn the dynamic value process of a portfolio from a finite sample of its cumulative cash flow. The learned value process is given in closed form thanks to a suitable choice of the kernel.…
New algorithm learns changing discrete distributions with minimal drift error.
problem Learning discrete distributions that change over time with limited past samples.
method Adaptive algorithm using data-dependent bounds to balance statistical and drift errors.
result Tighter statistical error bounds for drifting distributions with or without finite support.
Study tests whether trade-off functions are above or below benchmarks using finite samples.
problem Testing trade-off functions between unknown distributions.
method Identifies a condition for nontrivial testing, constructs a test with error guarantees, and inverts the test for confidence bands.
result Finite-sample testing is possible under specific structural assumptions about rejection regions.
Improved stochastic optimization outperforms standard methods.
problem Optimizing smooth, strongly convex functions with noisy data.
method Variance reduction strategy called VISOR.
result VISOR achieves optimal sample complexity and oracle complexity.