Study GLS estimator properties in multivariate regression with heteroskedastic and autocorrelated errors.
problem Asymptotic properties of GLS estimator in multivariate regression with specific error structures.
method Derive Wald statistics for linear restrictions and assess their performance.
result Wald statistics remain robust to heteroskedasticity and autocorrelation.
In this paper, we obtain asymptotic formulas with error estimates for the implied volatility associated with a European call pricing function. We show that these formulas imply Lee's moment formulas for the implied volatility and the tail-wing formulas due to Benaim and Friz. In addition, we analyze Pareto-type tails o…
New weighted Lasso estimates improve logistic regression performance with measurement error.
problem Improper Lasso estimates in sparse logistic regression with equal penalties.
method Proposed weighted Lasso estimates using McDiarmid inequality for non-asymptotic oracle inequalities.
result Finite sample behavior illustrated by non-asymptotic oracle inequalities for estimation and prediction errors.
The most important aspect of any classifier is its error rate, because this quantifies its predictive capacity. Thus, the accuracy of error estimation is critical. Error estimation is problematic in small-sample classifier design because the error must be estimated using the same data from which the classifier has been…
The paper analyzes methods for estimating linear functionals from observational data, proving upper bounds and showing optimal procedures.
problem Estimating linear functionals from observational data in causal inference and bandit literature.
method Two-stage procedures that first estimate treatment effect function, then use it to estimate the linear functional.
result Proves non-asymptotic upper bounds on mean-squared error for two-stage procedures and shows instance-dependent optimality.
We revisit resampling procedures for error estimation in binary classification in terms of U-statistics. In particular, we exploit the fact that the error rate estimator involving all learning-testing splits is a U-statistic. Thus, it has minimal variance among all unbiased estimators and is asymptotically normally dis…
Estimates error for robust M-estimators with convex penalties.
problem Estimating out-of-sample error for robust M-estimators in high-dimensional linear regression.
method Proposes a generic out-of-sample error estimate for robust M-estimators with convex penalties, using observed data and derivatives. result The out-of-sample error estimate has a relative error of order n−1/2 under certain conditions. Study non-asymptotic bounds for robust estimators under misspecified models.
problem Evaluate performance of robust estimators under adversarial conditions.
method Propose a general approach to adversarial risk analysis, including investigations on generalization and approximation errors.
result Establish non-asymptotic upper bounds for adversarial excess risk under Lipschitz loss functions.
The statistical analysis of Randomized Numerical Linear Algebra (RandNLA) algorithms within the past few years has mostly focused on their performance as point estimators. However, this is insufficient for conducting statistical inference, e.g., constructing confidence intervals and hypothesis testing, since the distri…
The paper proves a non-asymptotic test error approximation for KRR.
problem Understanding the test error of Kernel Ridge Regression.
method Established a non-asymptotic deterministic approximation for test error of KRR.
result The test error of KRR can be approximated by a closed-form estimate derived from the spectrum of the kernel operator.
New method calibrates asynchronous, error-prone covariates for longitudinal data.
problem Estimation biases and slow convergence in analyzing time-varying covariates with measurement error.
method Functional calibration approach based on functional principal component analysis.
result Asymptotically unbiased and consistent estimators for time-invariant coefficients; optimal convergence rate for time-varying coefficients.
Estimates point counts in Teichmüller space for mapping class groups.
problem Counting points in Teichmüller space under mapping class group actions.
method Quantitative estimates with power saving error terms for Teichmüller metric balls.
result Effectivizes asymptotic counting results of Athreya et al.
The paper provides bounds on estimation error in a distributed online learning setting.
problem Estimating an unknown parameter in a distributed and online manner with finite sample guarantees.
method Proposes a distributed online estimation algorithm that improves accuracy through communication, providing non-asymptotic bounds on estimation error.
result Demonstrates a trade-off between estimation error and communication costs, and determines a stopping time for communication based on desired accuracy.
In this paper, we obtain generic bounds on the variances of estimation and prediction errors in time series analysis via an information-theoretic approach. It is seen in general that the error bounds are determined by the conditional entropy of the data point to be estimated or predicted given the side information or p…
This study analyzes LTS in sparse models with finite sample error bounds.
problem Robust regression in high-dimensional sparse models with limited data.
method Non-asymptotic analysis of LTS error bounds.
result Established finite sample error bounds for LTS in sparse models.
Optimal kernel improves estimation accuracy in modal statistical methods.
problem Estimation accuracy of kernel-based modal statistical methods depends on the kernel used.
method The study theoretically shows an optimal kernel that minimizes asymptotic error criterion.
result An optimal kernel minimizes the error criterion when using an optimal bandwidth.
New metric explains neural network performance, simplifying generalization error calculation.
problem Precise characterization of neural network generalization error.
method Introducing Representation Gap, linking to intrinsic dimension and equivariant diffusion models.
result Asymptotic equivalent of Representation Gap is governed by intrinsic dimension, easy to estimate.
Estimates proper calibration errors and refinement terms in probabilistic predictions.
problem Lack of a general estimator for proper calibration errors and refinement terms with known statistical properties.
method Proposes a method for consistent, asymptotically unbiased estimation of proper calibration errors and refinement terms.
result Proves the relation between refinement and f-divergences, implying information monotonicity in neural networks.
Study on ridge regression in convolutional models shows double descent error behavior.
problem Understanding generalization and estimation error in over-parameterized convolutional models.
method Analysis of ridge estimators for convolutional linear models, derivation of exact error formulae.
result Ridge estimators exhibit double descent error behavior in high-dimensional convolutional models.
We study how the round-off (or discretization) error changes the statistical properties of a Gaussian long memory process. We show that the autocovariance and the spectral density of the discretized process are asymptotically rescaled by a factor smaller than one, and we compute exactly this scaling factor. Consequentl…
Study robust linear regression with outliers, providing exact asymptotics for ERM performance.
problem Robust linear regression in high-dimension with outliers.
method Analyzes ℓ2, ℓ1, and Huber losses, providing asymptotic performance metrics. result Optimally-regularised ERM is asymptotically consistent with simple calibration, but Huber loss requires norm calibration.
Study on Transfer Elastic Net error bounds and grouping effect.
problem Estimation error and grouping effect in Transfer Elastic Net.
method Derives non-asymptotic error bound and examines grouping effect scenarios.
result Effective error bounds and grouping effect observed in Transfer Elastic Net.
This paper provides performance guarantees for neural estimation of statistical distances.
problem Developing performance guarantees for neural estimation of statistical distances.
method Non-asymptotic error bounds using function approximation theorems and empirical process theory.
result Established a fundamental tradeoff between approximation and estimation errors in neural estimation of statistical distances.
Improved estimator reduces bias in statistical learning models.
problem Asymptotic bias in classic WDRO estimator.
method Adjusted Wasserstein distributionally robust estimator.
result Asymptotic unbiased estimator with smaller MSE.
We derive formulas for F measures' standard error and confidence intervals.
problem Estimating F measures' accuracy with confidence.
method Analytic formulas based on asymptotic normality.
result Valid formulas for sample size planning.
Optimizes shortfall risk using gradient-based methods.
problem Optimizing utility-based shortfall risk measures.
method Gradient-based stochastic optimization, non-asymptotic bounds derivation.
result Non-asymptotic convergence rate for optimizing UBSR.
New method selects best HTE estimator without ground-truth treatment effects.
problem Selecting best HTE estimator from multiple candidates.
method Cross-fitted, exponentially weighted test statistic with two-way sample splitting.
result Empirically, reliable error control and reduced false selections.
Study on linear regression with dependent covariates, proving universality and error characterization.
problem Linear regression with dependent covariates in high-dimensional settings.
method Analysis of ridge regression performance, Gaussian universality theorem, spectral properties of covariance matrices.
result Asymptotic performance of ridge regression is invariant under non-Gaussian covariates with preserved mean and covariance.
Double Q-learning has the same mean-squared error as Q-learning under certain conditions.
problem Comparing the mean-squared error of Double Q-learning and Q-learning.
method Theoretical analysis based on Lyapunov equations for both tabular and linear function approximation settings.
result The asymptotic mean-squared error of Double Q-learning is exactly equal to that of Q-learning under specific conditions.
PPI++ outperforms gold-standard labels only if pseudo-labels are highly correlated.
problem Optimizing statistical estimation using noisy pseudo-labels.
method Exact finite-sample analysis of PPI++ on mean estimation problem.
result PPI++ has provably worse estimation error than gold-standard labels alone in some settings.
Corrects local error estimates for UBU integrator in SDEs, improving complexity guarantees.
problem Improper local error estimates in UBU integrator for SDEs.
method Reconciles theory with practice by correcting local error estimates.
result Stronger assumptions needed for O(d1/4ε−1/2) steps in Wasserstein-2 distance. The paper improves quantization error estimates on Riemannian manifolds.
problem Improving quantization error estimates on Riemannian manifolds.
method Using covering growth estimates of spheres instead of curvature bounds.
result Provides a more general integral condition for quantization error.
Hierarchical statistical models are widely employed in information science and data engineering. The models consist of two types of variables: observable variables that represent the given data and latent variables for the unobservable labels. An asymptotic analysis of the models plays an important role in evaluating t…
SS-GEN simulates rare events in heavy and light-tailed data.
problem Estimating probabilities of extreme events in multivariate data.
method Self-Similar Generative Estimation (SS-GEN) decomposes tail distribution into radial and angular components.
result SS-GEN generates representative extreme scenarios and estimates rare-event probabilities beyond observed data.
New estimator adapts to various error distributions.
problem Adapting to different error distributions in nonparametric regression.
method Introduces outrigger local polynomial estimator with modified weighted least squares.
result Minimax optimal over Hölder classes with multiplicative factor.
The paper improves confidence intervals for test error using cross-validation.
problem Improving confidence intervals for test error in machine learning.
method Develops central limit theorems and consistent estimators for cross-validation.
result Provides asymptotically-exact confidence intervals and hypothesis tests.
Efficient classifier error estimation without re-training.
problem Estimating classifier error without re-training.
method Generalized resubstitution based on empirical measures.
result Consistent and asymptotically unbiased error estimation.
Paper proposes a new UCB approach for estimating maximum mean.
problem Estimating the maximum mean in various applications.
method Upper Confidence Bound (UCB) approach with adaptive sampling.
result LSA estimator shows faster bias decay compared to GA.
Sharp estimates on 2-step nilpotent Lie groups' metrics and cones.
problem Estimating asymptotic metrics in 2-step nilpotent Lie groups.
method Developed a novel technique to perturb rectifiable curves.
result Every 2-step nilpotent Riemannian Lie group is at bounded distance from its asymptotic cone.
Deep neural networks enforce non-crossing quantile regression curves.
problem Estimating quantile regression curves without crossing.
method Penalized deep ReQU neural networks with a non-crossing penalty.
result Established non-asymptotic risk and error bounds for the estimated QRP.
New estimator optimizes black-box model errors in semiparametric estimation.
problem How nuisance estimation errors affect low-dimensional target parameters in semiparametric models.
method Proposed a new estimator achieving a sharper rate of convergence.
result The first-order stochastic error of nuisance estimation can be eliminated.
Optimal convex loss function improves regression coefficient estimation.
problem Asymptotic variance improvement in linear regression estimation.
method Score matching extension for log-concave projection.
result Semiparametric estimator attains minimal asymptotic covariance.
In this paper, we provide explicit formulas, in terms of the covariances of sample covariances or sample correlations, for the asymptotic covariances of unrotated factor loading estimates and unique variance estimates. These estimates are extracted from least square, principal, iterative principal component, alpha or i…
New method uses online learning to improve AIPW estimators for adaptively collected data.
problem Estimating treatment effects with adaptively collected data.
method Online learning to minimize sequentially weighted estimation error.
result Local minimax lower bound shows optimality of AIPW estimator.
New method balances covariates for stable causal survival effect estimation.
problem Estimating causal survival effects in data with conditionally-independent censoring.
method Covariate-balancing approach to empirically stable and asymptotically efficient estimation.
result Validated theoretical results in synthetic and semi-synthetic data.
Operational risk models commonly employ maximum likelihood estimation (MLE) to fit loss data to heavy-tailed distributions. Yet several desirable properties of MLE (e.g. asymptotic normality) are generally valid only for large sample-sizes, a situation rarely encountered in operational risk. In this paper, we study how…
Random forests have proven to be reliable predictive algorithms in many application areas. Not much is known, however, about the statistical properties of random forests. Several authors have established conditions under which their predictions are consistent, but these results do not provide practical estimates of ran…
We propose an estimator of prediction error using an approximate message passing (AMP) algorithm that can be applied to a broad range of sparse penalties. Following Stein's lemma, the estimator of the generalized degrees of freedom, which is a key quantity for the construction of the estimator of the prediction error, …