The paper improves methods for estimating set size using samples.
problem Estimating the size of a set from a uniform sample.
method Refines estimators using the birthday problem and maximum of sample.
result Develops a general theory for non-asymptotic error bounds.
New method for MMD with unequal sample sizes improves test power.
problem Existing MMD methods assume equal sample sizes, discarding valuable data.
method Extended generalized U-statistics to handle unequal sample sizes.
result New asymptotic distributions and power optimization for MMD with unequal sample sizes.
Estimates sample size for subgroup analysis in randomized experiments.
problem Determining sample size for accurate subgroup analysis.
method Turns inference problem into simultaneous inference, calculates sample size based on confidence level and margin of error.
result Allows inversion of sample size to feasible number of treatment arms or partition complexity.
Simple private estimators for mean and covariance outperform existing methods.
problem Private estimation of mean and covariance at small sample sizes.
method Differentially private estimators for multivariate sub-Gaussian data.
result Asymptotic error rates match theoretical bounds and outperform previous methods.
pmsims R package uses Gaussian process for flexible sample size estimation in clinical models.
problem Determining adequate sample size for clinical prediction models.
method Simulation-based Gaussian process search for flexible sample size estimation.
result Gaussian process-based method produces more stable sample size estimates, especially in challenging settings.
Estimates neural representation dimensionality from small sample sizes.
problem Estimating neural representation dimensionality from limited data.
method Proposed a bias-corrected estimator for participation ratio of eigenvalues.
result The estimator is more accurate with finite samples and noise.
Regularized EM algorithm improves clustering performance with small sample sizes.
problem Performance reduction in EM algorithm due to small sample size and poorly conditioned covariance matrices.
method Regularized EM algorithm that uses prior knowledge to ensure positive definiteness of covariance matrices.
result The regularized EM algorithm outperforms standard EM in clustering tasks with small sample sizes.
Most real-world networks are too large to be measured or studied directly and there is substantial interest in estimating global network properties from smaller sub-samples. One of the most important global properties is the number of vertices/nodes in the network. Estimating the number of vertices in a large network i…
In the information-based paradigm of inference, model selection is performed by selecting the candidate model with the best estimated predictive performance. The success of this approach depends on the accuracy of the estimate of the predictive complexity. In the large-sample-size limit of a regular model, the predicti…
Downsampling can improve generalization in ridgeless linear regression, especially with optimal sketching size.
problem Improving generalization in ridgeless linear regression with limited data.
method Investigating the effects of downsampling on the sketched ridgeless least square estimator in the proportional regime.
result Optimal sketching size minimizes out-of-sample prediction risks and stabilizes risk curves.
Estimates set overlap and similarity using random samples.
problem Estimating set overlap and similarity with limited data.
method Binomial model for predicting set overlap, comparing to previous methods.
result Binomial model provides better estimates with small sample sizes.
Extended study improves covariance matrix estimation for portfolio managers.
problem Limited sample sizes and poor performance of PCA estimator in high-dimensional returns.
method Developed a more general shrinkage framework targeting further information.
result Improves the PCA estimator of beta by shrinking it toward a target.
Learn2Evaluate uses learning curves to estimate high-dimensional prediction performance.
problem Estimating test performance in high-dimensional data settings is challenging.
method Learn2Evaluate uses learning curves to estimate test performance at the total sample size.
result Learn2Evaluate provides a lower confidence bound for performance estimation.
Deep learning speeds up IFA estimation for large datasets.
problem Slow MML estimation for large-scale IFA models.
method Importance-weighted autoencoder (IWAE) for fast VI.
result IWAE yields accurate estimates faster than MH-RM.
Kurtosis is seen as a measure of the discrepancy between the observed data and a Gaussian distribution and is defined when the 4th moment is finite. In this work an empirical study is conducted to investigate the behaviour of the sample estimate of kurtosis with respect to sample size and the tail index when applied to…
Enhanced DFO using adaptive batch-based FD estimates.
problem Derivative-free optimization with imprecise gradient estimates.
method Adaptive batch-based finite difference estimation and dynamic sampling strategy.
result Algorithm achieves convergence rate similar to KW and SPSA methods.
Bagging reduces variance in LID estimation by preserving local distribution of NN distances.
problem High estimation variance from limited data in small neighborhoods.
method Subbagging to preserve local distribution of NN distances, combined with ensemble size.
result Bagging significantly reduces variance and MSE in LID estimation.
The paper analyzes the risk of bagging regularized M-estimators under proportional asymptotics.
problem Characterizing the risk of ensemble estimators trained with subsamples and regularizers.
method Developed a consistent estimator for the risk of ensemble estimators under proportional asymptotics.
result Optimal subsample size k⋆ tends to be in the overparameterized regime for the full-ensemble estimator. Privacy affects how much data is needed for CVaR optimization.
problem Privacy constraints impact the effective sample size for CVaR optimization.
method Analyzes the privacy-relevant sample size and decomposes CVaR excess risk.
result The effective private tail sample size is εnτ, affecting CVaR learning rates.
We consider the problem of providing nonparametric confidence guarantees for undirected graphs under weak assumptions. In particular, we do not assume sparsity, incoherence or Normality. We allow the dimension D to increase with the sample size n. First, we prove lower bounds that show that if we want accurate infe…
Sample measures of top centile contributions to the total (concentration) are downward biased, unstable estimators, extremely sensitive to sample size and concave in accounting for large deviations. It makes them particularly unfit in domains with power law tails, especially for low values of the exponent. These estima…
SFCNeXt estimates brain age from small MRI datasets.
problem Efficiently estimating brain age from limited MRI data.
method Simple fully convolutional network (SFCNeXt) with SPEC and HRL.
result SFCNeXt outperforms complex models in small sample size scenarios.
The contour map of estimation error of Expected Shortfall (ES) is constructed. It allows one to quantitatively determine the sample size (the length of the time series) required by the optimization under ES of large institutional portfolios for a given size of the portfolio, at a given confidence level and a given esti…
Analyzes error sources in global feature effect estimation methods.
problem Unexplored error sources in global feature effect estimation methods.
method Systematic, estimator-level analysis of bias and variance.
result Holdout data is theoretically cleanest, but estimation variance depends on sample size and model characteristics.
Estimates unknown population sizes using the hypergeometric distribution.
problem Estimating discrete distributions with unknown population sizes and category sizes.
method Proposes a novel solution using the hypergeometric likelihood, accounting for a data generating process with a latent variable.
result Empirically demonstrates superior performance in estimating population sizes and learning latent spaces compared to other methods.
In this paper, we propose a novel approach to automatically determine the batch size in stochastic gradient descent methods. The choice of the batch size induces a trade-off between the accuracy of the gradient estimate and the cost in terms of samples of each update. We propose to determine the batch size by optimizin…
A fast method estimates Gaussian mixture components without iterative fitting.
problem Estimating the number of components in high-dimensional Gaussian mixtures.
method Center data, compute singular values, and count above a threshold.
result The estimator consistently recovers the true number of components under mild separation condition.
Paper proposes a method to estimate variance reduction in DNN training using importance sampling.
problem Challenges in assessing variance reduction during DNN training using importance sampling.
method Proposes a method for estimating variance reduction using minibatches sampled under importance sampling.
result Demonstrates consistent reduction in variance, improved training efficiency, and enhanced model accuracy.
LoCoV reduces portfolio optimization errors from sample covariance matrices.
problem Large errors in sample covariance matrix for optimal portfolio weights.
method LoCoV (low dimension covariance voting) algorithm to reduce these errors.
result LoCoV outperforms classical methods in portfolio optimization experiments.
Density-Softmax improves uncertainty estimation and robustness without sampling, reducing model size and latency.
problem Sampling-based uncertainty estimation methods suffer from large model size and high latency.
method Combines a Lipschitz-constrained feature extractor with the softmax layer to create a sampling-free deterministic framework.
result Density-Softmax reduces over-confidence under distribution shifts and achieves competitive results in uncertainty and robustness.
Optimistic estimate predicts best fitting performance of nonlinear models.
problem Evaluating the potential of nonlinear models in fitting.
method Proposes an optimistic estimate to quantify the smallest sample size for fitting nonlinear models.
result Predicts specific subsets of targets that can be fitted at overparameterization.
We consider non-parametric estimation and inference of conditional moment models in high dimensions. We show that even when the dimension D of the conditioning variable is larger than the sample size n, estimation and inference is feasible as long as the distribution of the conditioning variable has small intrinsic…
Efficiently estimate risk of large portfolios using MLMC and sub-sampling.
problem Estimating risk of large portfolios with high computational cost.
method Apply Multilevel Monte Carlo (MLMC) with adaptive inner sampling and sub-sampling strategy.
result Sub-sampling strategy reduces computational complexity without portfolio size increase.
This study analyzes LTS in sparse models with finite sample error bounds.
problem Robust regression in high-dimensional sparse models with limited data.
method Non-asymptotic analysis of LTS error bounds.
result Established finite sample error bounds for LTS in sparse models.
The best-known and most commonly used distribution-property estimation technique uses a plug-in estimator, with empirical frequency replacing the underlying distribution. We present novel linear-time-computable estimators that significantly "amplify" the effective amount of data available. For a large variety of distri…
Kernel adaptive filters (KAF) are a class of powerful nonlinear filters developed in Reproducing Kernel Hilbert Space (RKHS). The Gaussian kernel is usually the default kernel in KAF algorithms, but selecting the proper kernel size (bandwidth) is still an open important issue especially for learning with small sample s…
New method reduces confidence interval sizes for causal inference.
problem Inaccurate propensity scores and extreme scores cause large confidence intervals.
method Data-dependent Coarse IPW (CIPW) estimators.
result Robust CIPW estimators reduce confidence interval sizes to ε+1/√n.
Uniform consistency proven for spatial distribution and depth estimators in any dimension.
problem Uniform consistency of spatial distribution and depth estimators in arbitrary dimensions.
method Proof of uniform L1-consistency using sample size n as the only dependency. result Consistency rate is independent of dimension d and sample size n. Sparse Polyak improves high-dimensional statistical estimation.
problem High-dimensional statistical estimation problems with growing problem dimension.
method Sparse Polyak modifies Polyak's adaptive step size to estimate restricted Lipschitz smoothness.
result Sparse Polyak achieves optimal statistical precision with fewer iterations.
The paper investigates model collapse in language models from a probabilistic perspective.
problem Understanding and preventing model collapse in language model training.
method Investigates recursive parametric model training from a probabilistic standpoint, characterizing conditions for model collapse and proposing mitigation strategies.
result Progressively increasing sample size is necessary to prevent model collapse, with a superlinear growth rate required in the asymptotic regime.
This study calculates the maximum error of a famous estimation method.
problem Estimating rare items not seen in a sample.
method Characterizes the maximal mean-squared error of the Good-Turing estimator.
result Characterizes the maximal mean-squared error of the Good-Turing estimator.
In this paper, we develop a general theory of truncated inverse binomial sampling. In this theory, the fixed-size sampling and inverse binomial sampling are accommodated as special cases. In particular, the classical Chernoff-Hoeffding bound is an immediate consequence of the theory. Moreover, we propose a rigorous and…
We study three fundamental statistical-learning problems: distribution estimation, property estimation, and property testing. We establish the profile maximum likelihood (PML) estimator as the first unified sample-optimal approach to a wide range of learning tasks. In particular, for every alphabet size k and desired…
New algorithm groups variables by ancestral relationships to improve causal graph estimation accuracy.
problem Difficulty in estimating causal graphs with small sample sizes relative to variables.
method CAG algorithm groups variables based on ancestral relationships, reducing complexity and improving accuracy.
result CAG outperforms existing methods in estimation accuracy and computation time.
Study ridge ensembles in proportional feature-to-sample size regime, proving risk equivalence and GCV consistency.
problem Characterizing and optimizing ridge ensembles in proportional feature-to-sample size regimes.
method Proportional asymptotics analysis, GCV for tuning, proving risk equivalence.
result Risk of optimal full ridgeless ensemble matches optimal ridge predictor's risk.
Tabular foundation models outperform other methods in conditional density estimation across various datasets.
problem Estimating the full conditional distribution of a response given tabular covariates, especially in settings with heteroscedasticity, multimodality, or asymmetric uncertainty.
method Benchmarked three tabular foundation model variants (TabPFN and TabICL) against six CDE baselines on 39 real-world datasets.
result Tabular foundation models achieve the best CDE loss, log-likelihood, and CRPS across all sample sizes, outperforming other methods.
Study shows annealing with adaptive schedule reduces mode collapse in NFs for parameter estimation.
problem Mode collapse in normalizing flows for multimodal distributions.
method Annealing with an adaptive schedule based on effective sample size (ESS).
result Our approach reduces mode collapse and converges marginal likelihood faster than MCMC methods.
A new method calculates intrinsic effective sample size for manifold-valued data.
problem Challenges in choosing effective sample size for manifold-valued data.
method Proposes an intrinsic effective sample size based on kernel discrepancy.
result Establishes an exact finite-sample risk interpretation and consistency of the estimator.