New method optimizes tail dependence coefficient estimation.
problem Estimating tail dependence in nonparametric data.
method Optimal threshold selection combining mean squared error and copula estimation.
result Improved accuracy in tail dependence coefficient estimation.
Random Forests adapted for dependent data using GLS.
problem Random Forests' local nature fails for dependent data.
method RF-GLS: GLS loss for correlated data.
result Consistency of RF-GLS under beta-mixing processes.
Study problem-dependent rates in statistical learning theory, achieving optimal generalization error bounds.
problem Generalization error in statistical learning theory.
method Uniform localized convergence framework.
result Optimal generalization error bounds for various learning problems.
This research examines how the error rate of nearest neighbor classifiers varies with dataset size.
problem The scaling of classification error rates with dataset size is not uniform.
method Theoretical analysis of nearest neighbor classifiers, focusing on early and late phases of dataset size.
result The error rate of nearest neighbor classifiers can have fine-grained rates depending on the dataset size and data distribution.
DP models misspecify LF dependencies, leading to significant performance errors.
problem Misspecification of LF dependencies in DP models.
method Theoretical bounds and empirical analysis of modeling errors.
result Modeling errors can be substantial, even with sensible LF structures.
Study shows accuracy of neural networks depends more on error location than percentage of error.
problem Effect of noise on accuracy in incremental learning neural networks.
method Empirical study using Perceptron, Feed Forward Neural Network, and Radial Basis Function Neural Network.
result Accuracy of neural networks is more dependent on error location than the percentage of error.
Study on learning properties of scale-dependent kernels controlling stability and error.
problem Understanding the learning properties of scale-dependent kernels in nonparametric ridge-less least squares.
method Combines probabilistic results with interpolation theory to analyze stability and error.
result Different regimes of learning error depending on sample size and data dimension.
Study on linear regression with dependent covariates, proving universality and error characterization.
problem Linear regression with dependent covariates in high-dimensional settings.
method Analysis of ridge regression performance, Gaussian universality theorem, spectral properties of covariance matrices.
result Asymptotic performance of ridge regression is invariant under non-Gaussian covariates with preserved mean and covariance.
Despite its potential to improve sample complexity versus model-free approaches, model-based reinforcement learning can fail catastrophically if the model is inaccurate. An algorithm should ideally be able to trust an imperfect model over a reasonably long planning horizon, and only rely on model-free updates when the …
This paper examines error bounds for deep learning classifiers with noisy labels.
problem Understanding the performance of classifiers trained on noisy data.
method Derives error bounds for excess risk, decomposing it into statistical and approximation errors. Uses independent block construction for statistical dependencies and vector-valued setting for approximation error.
result Established theoretical results for error bounds in deep learning with noisy labels, mitigating the impact of high-dimensional input spaces.
Proposes a new test for validating multivariate dynamic regression models.
problem Inadequate exogeneity conditions for conventional model specification tests in dynamic systems.
method Develops a generalized Durbin estimator for multiple-equation systems with dynamic dependencies, and constructs Wald tests.
result Bootstrap-based Wald tests improve finite-sample size control and validate the null hypothesis in multifactor models.
Paper introduces new bounds linking data compressibility to generalization error.
problem Establishing data-dependent generalization bounds.
method Variable-size compressibility framework linking generalization error to compression rate of input data.
result New bounds depend on empirical data measure, subsuming existing PAC-Bayes and intrinsic dimension bounds.
This paper analyzes RMR under Markov-dependent samples, improving understanding of its generalization error.
problem Understanding the generalization error of RMR in Markov-dependent settings.
method Established the upper bound for RMR estimator under Markov-dependent samples, providing a learning rate.
result Markov dependence affects the generalization error, reducing it by a multiplicative factor of the spectral gap.
New framework for network regression models accounting for community structure.
problem Inaccurate modeling of residual dependencies in network regression models.
method Modeling errors as community-based and exploiting exchangeability properties.
result Parsimonious standard errors for regression parameters.
A new learning rule consistently reduces error over data samples.
problem Finding a learning rule that consistently reduces error over all data distributions.
method A deterministic, data-dependent partitioning rule that only partitions cyclic intervals with sufficient empirical diversity of labels.
result The expected error is monotone non-increasing with the sample size under every data distribution.
The paper offers error bounds for quantized dynamical models.
problem Accuracy of dynamical models from dependent data sequences.
method Developed uniform error bounds for quantized models and imperfect optimization algorithms.
result Unified bounds for slow and fast rates, scaling with model encoding bits.
Full-batch GD achieves generalization close to any stationary point with fewer assumptions.
problem Generalization and excess risk bounds for smooth losses, including non-Lipschitz and nonconvex cases.
method Path-dependent analysis of GD's generalization error, focusing on optimization error and stability.
result Generalization error is tightly bound in terms of optimization error and iteration count, bypassing common assumptions.
Analyzes error sources in global feature effect estimation methods.
problem Unexplored error sources in global feature effect estimation methods.
method Systematic, estimator-level analysis of bias and variance.
result Holdout data is theoretically cleanest, but estimation variance depends on sample size and model characteristics.
Fairness-aware learning is a novel framework for classification tasks. Like regular empirical risk minimization (ERM), it aims to learn a classifier with a low error rate, and at the same time, for the predictions of the classifier to be independent of sensitive features, such as gender, religion, race, and ethnicity. …
Paper analyzes deep neural networks with dependent data, establishing convergence rates and error bounds.
problem Statistical analysis of deep neural networks under dependent data.
method Establishes rates of convergence and L 2 \mathcal{L}^{2} L 2 -error bounds for nonparametric sieve estimators of DNNs. result Non-asymptotic probability bounds on L 2 \mathcal{L}^{2} L 2 -errors for DNN estimators under stationary β \beta β -mixing data. Investigates probability of error in structured thresholding bandit problems.
problem Probability of misclassifying arms in structured thresholding bandit problems.
method Analyzes two shape constraints: monotonic increasing and concave sequences of arm means.
result Upper and lower bounds for the probability of error match up to constants in the problem dependent regime.
The Gibbs algorithm's generalization error is bounded, improving with prior volume in low temperatures.
problem Bounding the generalization error of the Gibbs algorithm in low temperature regimes.
method Analyzes the Gibbs algorithm's performance, extending known high-temperature bounds to low-temperature scenarios.
result With high probability, the generalization error decreases with the total prior volume of similar hypotheses.
This paper reviews methods for constructing confidence intervals for error rates in 1:1 matching tasks.
problem Challenges in assessing uncertainty of error rates in matching algorithms, especially when data are dependent and error rates are low.
method Derives and examines statistical properties of methods for constructing confidence intervals for error rates in 1:1 matching tasks.
result Coverage and interval width vary with sample size, error rates, and data dependence.
Paper proposes deep neural networks for nonparametric regression from dependent data.
problem Nonparametric regression from strongly mixing observations.
method Minimum error entropy principle applied to deep neural networks.
result Deep neural networks achieve minimax optimal convergence rates for Gaussian errors.
The study characterizes diffusion model generalization using data-dependent ridge manifolds.
problem Understanding where diffusion model-generated samples lie when not memorizing the training set.
method Introduced a time-dependent family of log-density ridge manifolds to characterize reverse-time inference.
result Generated samples evolve by a reach-align-slide mechanism, controlled by normal and tangential components of training error.
In this paper, we consider the problem of column subset selection. We present a novel analysis of the spectral norm reconstruction for a simple randomized algorithm and establish a new bound that depends explicitly on the sampling probabilities. The sampling dependent error bound (i) allows us to better understand the …
In this paper, we study the minimax optimization problem in the smooth and strongly convex-strongly concave setting when we have access to noisy estimates of gradients. In particular, we first analyze the stochastic Gradient Descent Ascent (GDA) method with constant stepsize, and show that it converges to a neighborhoo…
In this paper, we prove that some Gaussian structural equation models with dependent errors having equal variances are identifiable from their corresponding Gaussian distributions. Specifically, we prove identifiability for the Gaussian structural equation models that can be represented as Andersson-Madigan-Perlman cha…
Metric learning enhances combinatorial coverage metrics' ability to predict classification errors.
problem Dataset dependence of combinatorial coverage metrics in anticipating classification errors.
method Metric learning to improve latent space separation of data classes.
result Metric learning increases SDCCMs' ability to distinguish between correctly and incorrectly classified data.
Analysis of non-asymptotic estimation error and structured statistical recovery based on norm regularized regression, such as Lasso, needs to consider four aspects: the norm, the loss function, the design matrix, and the noise model. This paper presents generalizations of such estimation error analysis on all four aspe…
We introduce two versions of a new sketch for approximately embedding the Gaussian kernel into Euclidean inner product space. These work by truncating infinite expansions of the Gaussian kernel, and carefully invoking the RecursiveTensorSketch [Ahle et al. SODA 2020]. After providing concentration and approximation pro…
Improved sample complexity for training diffusion models.
problem How many samples are needed to train an accurate diffusion model?
method Analyzing the sample complexity of training diffusion models using neural networks.
result Exponential improvement in the dependence on Wasserstein error and depth, along with improved dependencies on other parameters.
Study of estimation errors in surrogate loss minimizers, providing stronger guarantees than existing methods.
problem Estimation errors in surrogate loss minimizers for various hypothesis sets.
method Detailed study of H \mathscr{H} H -consistency estimation error bounds, proving general theorems for distribution-dependent and independent settings. result Explicit bounds for zero-one and adversarial losses, showing enhancements under distributional assumptions.
GANs learn distributions well from samples, with rates depending on intrinsic dimension.
problem Learning distributions from samples using GANs.
method Oracle inequality, Hölder functions approximation, neural network approximation, integral probability metrics.
result Convergence rates of GANs depend on intrinsic dimension, not ambient dimension.
Unified framework for blending ML and mechanistic models in dynamical systems.
problem Learning dynamical systems from noisy, partially observed data.
method A unifying framework that combines mechanistic and machine learning approaches.
result Proves that hybrid models can learn memory-dependent model error.
Improved analysis for diffusion models reduces KL divergence error dependence on data dimension and discretization step size.
problem Analyze the convergence of diffusion-based generative models under minimal assumptions.
method Model the generation process as a composition of reverse ODE and noising steps, leveraging Wasserstein-type error control and noise addition.
result Achieved a linear dependence on data dimension and improved dependence on discretization step size for KL divergence error.
Study on RNNs' ability to approximate past-dependent Hölder functions and their application to regression.
problem Understanding and optimizing the approximation capacity of RNNs for regression tasks.
method Derivation of upper bounds on RNN approximation error for Hölder smooth functions and application to regression.
result Achievement of minimax optimal prediction error bounds for RNNs under various data assumptions.
ECI improves time series prediction uncertainty quantification by smoothing miscoverage error.
problem Challenges in uncertainty quantification for time series prediction due to temporal dependence and distribution shift.
method Error-quantified Conformal Inference (ECI) by smoothing quantile loss function and introducing adaptive feedback scale.
result ECI achieves valid miscoverage control and tighter prediction sets than existing methods.
Algorithm-dependent generalization error bounds are central to statistical learning theory. A learning algorithm may use a large hypothesis space, but the limited number of iterations controls its model capacity and generalization error. The impacts of stochastic gradient methods on generalization error for non-convex …
New algorithm for multi-player bandits with collision-dependent rewards.
problem Stochastic multi-player multi-armed bandits with collision-dependent reward distributions.
method Error-Correction Collision Communication (EC3) algorithm.
result EC3 algorithm achieves optimal regret approaching centralized MP-MAB regret.
Max-rank improves multiple testing in conformal prediction.
problem Simultaneous testing of multiple hypotheses in scientific inquiries.
method Introduces max-rank, a novel correction for positive dependencies in simultaneous testing.
result Max-rank efficiently controls family-wise error rate and improves predictive uncertainty estimates.
Graphical lasso models ASR utterance dependencies for consistent WER estimation.
problem Modeling dependent structure among ASR utterances for accurate significance analysis.
method Graphical lasso for dependency modeling, followed by blockwise bootstrap resampling.
result Statistically consistent variance estimator of WER under mild conditions.
A fast Monte Carlo method for additive processes and option pricing.
problem Efficiently pricing path-dependent options with additive processes.
method Developed a fast Monte Carlo scheme for additive processes, analyzing and reducing numerical error sources.
result Shows significant reduction in error (1 bp or below) for pricing path-dependent options.
SGD-trained deep nets have bounds on their generalization error.
problem Bounding generalization error for deep neural networks trained by SGD.
method Combining dynamical control of parameter norms and Rademacher complexity estimates.
result Explicit bounds depend on loss trajectory, work for various architectures.
The paper improves transformer generalization bounds using rank-dependent covering number bounds.
problem Improving generalization bounds for transformers.
method Introducing rank-dependent covering number bounds for linear function classes and applying them to transformers.
result Generalization error bounds for transformers decay as O ( 1 / n ) O(1/\sqrt{n}) O ( 1/ n ) and O ( log r w ) O(\log r_w) O ( log r w ) , improving existing bounds. We study the generalization performance of online learning algorithms trained on samples coming from a dependent source of data. We show that the generalization error of any stable online algorithm concentrates around its regret--an easily computable statistic of the online performance of the algorithm--when the underl…
This paper studies the statistical theory of batch data reinforcement learning with function approximation. Consider the off-policy evaluation problem, which is to estimate the cumulative value of a new target policy from logged history generated by unknown behavioral policies. We study a regression-based fitted Q iter…
The paper improves tensor completion bounds using spectral gap.
problem Theoretical limitations in tensor completion, especially for deterministic sampling.
method Bounding the generalization error of tensor completion methods using spectral gap.
result Improved bounds on tensor completion error, reducing rank dependence.