GANs learn distributions well from samples, with rates depending on intrinsic dimension.
problem Learning distributions from samples using GANs.
method Oracle inequality, Hölder functions approximation, neural network approximation, integral probability metrics.
result Convergence rates of GANs depend on intrinsic dimension, not ambient dimension.
Analyzes generalization error in distributed linear regression.
problem Understanding generalization performance in distributed learning.
method Analytical characterization of generalization error in linear regression with distributed learning.
result Generalization error increases dramatically when nodes estimate close to the number of observations.
Study on distributed linear regression performance, focusing on generalization error.
problem Performance of distributed learning in large-scale linear regression.
method Statistical learning approach, focusing on generalization error.
result Generalization error of distributed solution can be higher than centralized solution.
New generalization concept considers distribution of errors, not just average error.
problem Classical generalization fails to capture distributional differences in classifier outputs.
method Formal conjectures about distributional generalization based on model architecture, training procedure, and data distribution.
result Distributional generalization can be expected in specific conditions, as evidenced by empirical results.
Study error bounds in evaluating distributional computational graphs.
problem Error analysis in evaluating graphs with inputs as probability distributions.
method Establish non-asymptotic error bounds using Wasserstein-1 distance.
result Non-asymptotic error bounds for discretization errors in distributional computational graphs.
The paper connects three machine learning methods to reduce generalization errors.
problem Reducing generalization errors in machine learning models.
method Distributionally robust optimization, Bayesian methods, and regularization.
result Machine learning models can be characterized using distributional uncertainty and robustness measures.
Study shows data heterogeneity affects distributed learning's generalization error.
problem Effect of data heterogeneity on distributed learning performance.
method Established bounds on generalization error using information-theoretic rate-distortion theory.
result Data heterogeneity improves generalization error for distributed learning.
This study analyzes how well GANs approximate distributions from small samples.
problem Understanding how well GANs approximate distributions from limited data.
method Analysis of GANs using integral probability metrics and Hölder classes.
result GANs can adaptively learn low-dimensional structures or Hölder densities.
New framework identifies and reduces errors in machine learning under distribution shift.
problem Errors in machine learning models when distributions change.
method Developed a principled framework to characterize and eliminate epistemic errors in imperfect multitask learning.
result Provided a decompositional epistemic error bound for general settings of distribution shift.
New method improves understanding of machine learning model performance.
problem Understanding how well machine learning models generalize from training data to unseen data.
method Auxiliary Distribution Method to derive new generalization error bounds.
result Upper bounds on generalization errors are tighter and more applicable.
Enhances reinforcement learning uncertainty estimation with a generalized Gaussian error model.
problem Inaccurate error representations and compromised uncertainty estimation in conventional uncertainty-aware TD learning.
method Introduces a novel framework for generalized Gaussian error modeling in deep reinforcement learning, incorporating higher-order moments, particularly kurtosis, to improve uncertainty estimation and mitigation.
result Significant performance gains in policy gradient algorithms with the proposed framework.
New bounds on generalization error for distributed learning using rate-distortion theory.
problem Establishing upper bounds on generalization error for distributed learning algorithms.
method Using rate-distortion theory, the paper introduces new bounds that depend on the compressibility of each client's algorithm.
result The bounds suggest that the generalization error of the distributed setting decays faster than that of the centralized one with a factor of O ( log ( K ) / K ) \mathcal{O}(\log(K)/\sqrt{K}) O ( log ( K ) / K ) . New method uses extreme value theory to estimate neural network errors.
problem Quantifying the error of neural networks, especially for large values.
method Applying extreme value theory to approximate the distribution of error.
result Developed a new estimator for the shape parameter of the Pareto distribution.
A form of generalisation error known as Off Training Set (OTS) error was recently introduced in [Wolpert, 1996b], along with a theorem showing that small training set error does not guarantee small OTS error, unless assumptions are made about the target function. Here it is shown that the applicability of this theorem …
New findings show privacy affects generalization error in a non-monotonic way.
problem Privacy and robustness in distributed learning.
method Theoretical analysis and matching lower/upper bounds on algorithmic stability.
result Generalization error is non-monotonically affected by privacy, depending on noise level.
This paper provides a mathematical framework for understanding distribution learning models.
problem The paradox between memorization and generalization in distribution learning models.
method A unified mathematical framework to derive various distribution learning models.
result The models enjoy implicit regularization, avoiding the curse of dimensionality and resolving the paradox.
Langevin dynamics fails to produce accurate samples even with small score function errors.
problem Robustness of Langevin dynamics to score function errors.
method Analysis of Langevin dynamics and score function errors.
result Langevin dynamics produces a distribution far from the target distribution in TV distance even with small L 2 L^2 L 2 errors in the score function. Sharp analysis of out-of-distribution error in overparameterized models with importance weights.
problem Understanding and quantifying the degradation of performance in overparameterized models when faced with underrepresented data.
method Sharp analysis of an overparameterized Gaussian mixture model with spurious features and cost-sensitive interpolating solutions incorporating importance weights.
result Characterization of a novel tradeoff between worst-case robustness and average accuracy as a function of importance weight magnitude.
CNFs learn distributions from samples with error bounds.
problem Learning probability distributions from finite samples.
method Continuous normalizing flows with linear interpolation and flow matching objective function.
result Non-asymptotic error bounds for distribution estimator in Wasserstein-2 distance.
DFM models are analyzed for generating distributions with provable convergence.
problem Training DFM models to generate distributions that match true data.
method Theoretical analysis decomposes error into approximation and estimation errors.
result DFM models converge to true data distribution as training set size increases.
Study of estimation errors in surrogate loss minimizers, providing stronger guarantees than existing methods.
problem Estimation errors in surrogate loss minimizers for various hypothesis sets.
method Detailed study of H \mathscr{H} H -consistency estimation error bounds, proving general theorems for distribution-dependent and independent settings. result Explicit bounds for zero-one and adversarial losses, showing enhancements under distributional assumptions.
Paper studies distributed learning with limited communication bits, achieving optimal error exponents.
problem Distributed hypothesis testing with constant communication bits.
method Geometric approach in distribution spaces, encoding empirical distributions to transmission bits.
result Optimal achievable error exponents and coding schemes for various communication constraints.
ADB framework improves OOD generalization by increasing ID bias during training.
problem Machine learning models degrade on new data distributions.
method ADB framework introduces controlled statistical diversity during training.
result Higher in-distribution bias leads to better out-of-distribution generalization.
This research examines how the error rate of nearest neighbor classifiers varies with dataset size.
problem The scaling of classification error rates with dataset size is not uniform.
method Theoretical analysis of nearest neighbor classifiers, focusing on early and late phases of dataset size.
result The error rate of nearest neighbor classifiers can have fine-grained rates depending on the dataset size and data distribution.
Unified error analysis for discrete flow models.
problem Error analysis of discrete flow models.
method Stochastic calculus theory, Girsanov theorem, generator matching, uniformization.
result First error analysis for discrete flow models.
Bayesian method identifies causal DAG structure from non-Gaussian errors.
problem Learning causal structure from non-Gaussian errors in Bayesian networks.
method Bayesian hierarchical model with DAG prior for non-Gaussian errors.
result Posterior DAG selection consistency achieved under mild assumptions.
Estimates model performance under distribution shift using domain-invariant predictors.
problem Poor performance of models on test distributions different from training distributions.
method Uses domain-invariant predictors as a proxy for unknown target labels.
result Shows that the complexity of latent representations influences target risk.
Confidence measures for the generalization error are crucial when small training samples are used to construct classifiers. A common approach is to estimate the generalization error by resampling and then assume the resampled estimator follows a known distribution to form a confidence set [Kohavi 1995, Martin 1996,Yang…
Paper presents a method to reduce prediction variance of DNNs for unknown systems.
problem Uncertainty in DNN predictions due to high variance.
method Ensemble averaging of multiple DNN models trained independently.
result Reduction in variance of DNN predictions, improving reliability.
Efficiently estimates mean of symmetric distributions without moments.
problem Estimating mean of symmetric distributions without moment assumptions.
method Generalization of filtering technique, Huber-loss-based techniques, SoS proofs.
result Achieves optimal error bounds for various symmetric distributions.
New error bound for diffusion models without dimensionality constraints.
problem Error estimation in diffusion generative models without dimensionality constraints.
method Derive dimension-free error bound using a smooth test functional.
result Explicit, dimension-free bound on generated vs true data distributions.
The study finds that memorization is necessary or harmful depending on the prior distribution and noise level.
problem The impact of memorization on generalization in overparameterized models.
method An overparameterized linear model with general priors in a Bayesian setup.
result Explicit conditions for optimal generalization based on the prior distribution and noise level.
Unified framework for error quantification in off-policy evaluation with distributional shift.
problem Establishing high-confidence CI for target policy value from offline data.
method Unified error analysis quantifying misspecification and sampling errors.
result Achieves tightest possible CI and robustness against distributional shifts.
New algorithm improves distributed SGD with random sparsification for better convergence and generalization.
problem Communication bottleneck in distributed deep learning.
method Proposes detached error feedback (DEF) algorithm to improve convergence and generalization of communication-efficient distributed SGD.
result Shows better convergence and generalization bounds than existing methods.
Paper presents robust clustering methods for general mixture models.
problem Clustering with sub-Gaussian error assumptions often invalid in practice.
method Hybrid clustering with robust centroid estimate and data-driven initialization.
result Provably near-optimal mislabeling guarantees for general error distributions.
DALMC provides non-asymptotic error bounds for generative models.
problem Efficiently generating samples from complex data distributions.
method Analysis of diffusion paths and Langevin Monte Carlo.
result Theoretical guarantees for a class of generative models.
New error bounds for flow matching methods using deterministic sampling.
problem Improving the accuracy of flow matching methods for generating probability distributions.
method Derived error bounds for flow matching methods under deterministic sampling conditions.
result Presented error bounds for flow matching methods using L 2 L^2 L 2 loss and regularity conditions. This paper studies clustering of data sequences using the k-medoids algorithm. All the data sequences are assumed to be generated from \emph{unknown} continuous distributions, which form clusters with each cluster containing a composite set of closely located distributions (based on a certain distance metric between di…
Training on some out-of-distribution data improves generalization error before it deteriorates.
problem Generalization error improves with some out-of-distribution data but deteriorates with more.
method Synthetic datasets and deep networks on computer vision benchmarks.
result Non-monotonic trend in generalization error with OOD samples.
Modified training direction reduces generalization error in neural networks.
problem Reducing generalization error in neural networks.
method Theoretical analysis of modified natural gradient descent in function space.
result Modifying training direction in function space reduces total generalization error.
LMC improves sampling from complex distributions using quasi-random sequences.
problem Sampling from complex high-dimensional distributions with high accuracy.
method Using completely uniformly distributed (CUD) sequences in Langevin Monte Carlo (LMC) to generate Gaussian perturbations.
result LMC with low-discrepancy CUD sequences achieves smaller estimation error than standard LMC.
Sharp bounds found on expert error in binary advice aggregation.
problem Aggregating binary advice from conditionally independent experts.
method Sharp upper and lower bounds on optimal error probability in asymmetric case.
result Sharp bounds recover and sharpen known results in symmetric case.
WES improves neural network regression by stretching distribution error.
problem Improving prediction performance in neural-network-based regression.
method Proposed weighted empirical stretching (WES) loss function.
result WES outperforms existing loss functions, especially in extreme domains.
Study finds simple model-agreement scores perform well in various error estimation scenarios.
problem Evaluating model performance on unseen distributions using disparate scoring functions.
method Rigorously studied popular scoring functions (confidence, local manifold smoothness, model agreement) independently of mechanism choice.
result Simple model-agreement scores outperform confidence- and smoothness-based scores in realistic settings with compromised training data.
Bayesian framework improves robustness in nonlinear regression models.
problem Measurement error, model misspecification, and distributional misspecification in regression analyses.
method Joint Dirichlet process prior on latent covariate-response distribution, updating with posterior pseudo-samples.
result Improved stability and consistency in estimators under increasing measurement error.
New method reveals good classifiers are common in over-parameterized models.
problem Understanding how over-parameterized models generalize well.
method Developed a methodology to compute the full distribution of test errors.
result Test errors concentrate around a small typical value ε* rather than the worst-case model.
The paper analyzes prediction error in nonstationary settings using weighted risk minimization.
problem Prediction under distribution drift and nonstationary conditions.
method General decomposition of excess risk into learning and drift terms, proving oracle inequalities under mixing conditions.
result Oracle inequalities for the learning error, providing bounds that hold uniformly over arbitrary weight classes.
DCMA uses generative models to analyze treatment effects on entire outcome distributions.
problem Traditional mediation analysis focuses on summary contrasts, missing complex distributional changes.
method DCMA learns conditional generative models for mediators and outcome, reconstructing interventional distributions via Monte Carlo simulation.
result DCMA captures both summary effects and rich distributional contrasts like energy distance and Wasserstein distance.