The study examines how equivariance in networks affects generalization error using PAC-Bayesian bounds.
problem Understanding how equivariance in networks impacts generalization error.
method Utilized PAC-Bayesian analysis for equivariant networks, deriving norm-based bounds for generalization error.
result The bound indicates that using larger group size in the model improves generalization error.
PAC-Bayesian bounds for stochastic LTI systems derived.
problem Error bounds for stochastic LTI systems.
method PAC-Bayesian theory applied to autonomous stochastic LTI models.
result Error bounds for stochastic LTI systems derived.
Paper improves PAC-Bayesian bounds for linear regression.
problem Improving PAC-Bayesian error bounds for linear regression.
method Proposes a tighter error bound for linear regression that converges to the generalization loss with a well-chosen temperature parameter. Also applies to non-i.i.d. training data.
result Error bound applies to certain time series generated by dynamical models.
The paper connects three machine learning methods to reduce generalization errors.
problem Reducing generalization errors in machine learning models.
method Distributionally robust optimization, Bayesian methods, and regularization.
result Machine learning models can be characterized using distributional uncertainty and robustness measures.
Paper calculates the exact error of LDA models.
problem Bayesian generalization error in Latent Dirichlet Allocation (LDA).
method Theoretical analysis of learning coefficient using algebraic geometry.
result Exact asymptotic form of LDA's generalization error.
Study clarifies Bayesian generalization error in CBM for 3-layered linear neural networks.
problem Understanding the generalization error in concept bottleneck models.
method Mathematical analysis of Bayesian generalization error and free energy in CBM for 3-layered linear neural networks.
result CBM significantly alters the parameter region and Bayesian generalization error compared to standard models.
Bayesian framework improves robustness in nonlinear regression models.
problem Measurement error, model misspecification, and distributional misspecification in regression analyses.
method Joint Dirichlet process prior on latent covariate-response distribution, updating with posterior pseudo-samples.
result Improved stability and consistency in estimators under increasing measurement error.
Paper derives PAC-Bayesian bounds for LTI systems learning from empirical data.
problem Characterizing predictive power of LTI systems learned from data.
method PAC-Bayesian bounds for LTI stochastic dynamical systems with inputs.
result Finite-sample error bounds for learning algorithms of LTI systems.
Paper analyzes the free energy of CNNs with skip connections in Bayesian learning.
problem Dependency of CNNs with skip connections on the number of parameters.
method Examines the Bayesian free energy of CNNs with and without skip connections.
result The upper bound of free energy of Bayesian CNN with skip connections does not depend on overparametrization.
The paper improves error bounds for Bayesian quadrature in noisy settings.
problem Improving error bounds for Bayesian quadrature in noisy settings.
method Develops a two-step meta-algorithm to relate average-case quadrature error to L 2 L^2 L 2 -function approximation error. result Provides new average-case results for various kernels and noise settings.
Non-negative matrix factorization (NMF) is a new knowledge discovery method that is used for text mining, signal processing, bioinformatics, and consumer analysis. However, its basic property as a learning machine is not yet clarified, as it is not a regular statistical model, resulting that theoretical optimization me…
Bayesian regression underestimates parameter uncertainties in noisy models.
problem Parameter uncertainties are underestimated in Bayesian regression for imperfect models.
method Analyzed and designed an ansatz to correct for misspecification in near-deterministic surrogate models.
result Posterior distributions must cover all training points to avoid divergent generalization error.
The paper connects ABC to GBI, suggesting ABC as a robustification strategy.
problem Approximate Bayesian Computation struggles with tractability in complex simulators.
method Reinterpreting ABC as an implicitly defined error model and suggesting GBI.
result ABC can be seen as a robustification strategy for approximating Bayesian posteriors.
Bayesian method identifies causal DAG structure from non-Gaussian errors.
problem Learning causal structure from non-Gaussian errors in Bayesian networks.
method Bayesian hierarchical model with DAG prior for non-Gaussian errors.
result Posterior DAG selection consistency achieved under mild assumptions.
Bayesian free energy remains bounded for deep ReLU networks in overparametrized cases.
problem Understanding the generalization performance of deep ReLU neural networks.
method Analyzes Bayesian free energy in overparametrized deep ReLU neural networks.
result Bayesian free energy is bounded even in overparametrized deep ReLU networks.
PCBM improves neural network generalization by partially observing concepts.
problem Decreased generalization performance due to observing all concepts in CBM.
method Developed a theoretical analysis of PCBM's Bayesian generalization error.
result PCBM's generalization error is lower than CBM's due to partial concept observation.
New framework identifies and reduces errors in machine learning under distribution shift.
problem Errors in machine learning models when distributions change.
method Developed a principled framework to characterize and eliminate epistemic errors in imperfect multitask learning.
result Provided a decompositional epistemic error bound for general settings of distribution shift.
Bayesian framework tackles measurement error in covariates.
problem Misleading inference due to corrupted covariates.
method Bayesian Nonparametric Learning framework robust to misspecification.
result General framework for Classical and Berkson error models.
Bayesian deep learning accounts for input uncertainty using Errors-in-Variables models.
problem Uncertainty in deep regression models, especially from input data.
method Bayesian treatment with Errors-in-Variables model to decompose predictive uncertainty.
result The approach yields more complete and consistent uncertainty estimates.
This study uses neural networks to approximate Bayesian filtering problems.
problem Estimating latent time-series signal statistics from observation sequences.
method Formulated a generic recurrent neural network framework to learn recursive mappings directly.
result Approximation error bounds for filtering in non-compact domains and strong time-uniform bounds.
Paper derives an error bound for stochastic LTI systems.
problem Stochastic LTI systems with inputs in control engineering and econometrics.
method PAC-Bayesian-Like error bound derivation.
result Derived an error bound for stochastic LTI systems.
Bayesian method recovers causal structure in SEMs with equal error variances.
problem Recovering causal structure in SEMs with equal error variances.
method Bayesian DAG selection method using g-priors and the key property of minimum expected squared errors.
result The method consistently recovers the true graph without additional distributional assumptions.
A new error bound improves safety in Bayesian optimization.
problem Ensuring safety in Bayesian optimization with probabilistic models.
method Introducing a novel error bound using Wiener kernel regression for Gaussian processes and noise.
result The new error bound provides larger safety regions than previous methods.
Recently the generalization error of deep neural networks has been analyzed through the PAC-Bayesian framework, for the case of fully connected layers. We adapt this approach to the convolutional setting.
Improved BO algorithms reduce prediction error under Gaussian noise.
problem Reducing prediction error in Bayesian optimization with Gaussian noise.
method Established new prediction error bounds for Gaussian process under frequentist setting.
result Proved improved convergence rates of cumulative regret for GP-UCB and GP-TS.
A novel hierarchical Bayesian approach to Federated Learning reduces data exposure and improves convergence rates.
problem Data privacy and convergence in Federated Learning.
method Hierarchical Bayesian modeling and block-coordinate descent optimization.
result The proposed algorithm converges to an optimal solution with a rate of O ( 1 / t ) O(1/\sqrt{t}) O ( 1/ t ) and guarantees vanishing generalization error. Paper tackles SBI under model misspecification, presenting robust strategies.
problem Challenges in SBI under model misspecification.
method Three key strategies: robust summary statistics, generalised Bayesian inference, and error modelling.
result Empirical results show vulnerabilities of SBI and effectiveness of misspecification-robust alternatives.
Paper extends Bayesian Cramér-Rao bound with geometric considerations.
problem Estimation of covariance matrices with geometric structures.
method Intrinsic Bayesian Cramér-Rao bound with Riemannian geometry.
result Performance bounds for covariance matrix estimation.
Unified framework for deriving generalization bounds in supervised learning.
problem Generalization error bounds in supervised learning.
method Data Processing Inequality PAC-Bayesian framework.
result Unified bounds on binary Kullback-Leibler generalization gap for various divergences.
Study analyzes EU in variational inference for Bayesian deep learning.
problem Analyzing epistemic uncertainty in variational inference for Bayesian deep learning.
method Theoretical analysis and derivation of relations between generalization error and EU measurements. Proposing a new objective function for VI.
result Proposed new objective function significantly improves EU evaluation over existing VI methods.
The study finds that memorization is necessary or harmful depending on the prior distribution and noise level.
problem The impact of memorization on generalization in overparameterized models.
method An overparameterized linear model with general priors in a Bayesian setup.
result Explicit conditions for optimal generalization based on the prior distribution and noise level.
Paper introduces a new method for error estimation in classification tasks with limited data.
problem Challenges in designing accurate classifiers and evaluating their performance with limited training data.
method Introduces a novel Bayesian MMSE estimator for optimal Bayesian transfer learning (OBTL) using Monte Carlo importance sampling.
result Proposed OBTL error estimation scheme outperforms standard methods, especially in small-sample settings.
Proposes a new model to handle noisy data in scientific research.
problem Measurement error in noisy data settings.
method Measurement error BART (meBART) integrates measurement error in Bayesian additive regression trees.
result meBART provides more accurate parameter estimation, robust uncertainty quantification, and superior predictive performance.
Bayesian method improves extreme quantile estimation with zero coverage error.
problem Estimating extreme quantiles with zero coverage error in small samples.
method Bayesian quantile estimation using Jeffreys prior.
result Bayesian method results in zero coverage error, unlike maximum likelihood.
Study proposes a stopping criterion for active learning based on error stability.
problem Improving predictive performance in active learning by adaptively annotating samples.
method Proposes a stopping criterion based on error stability for Bayesian active learning.
result Demonstrates the proposed criterion stops active learning at the appropriate timing for various models and datasets.
Generative models improve inverse problems by providing tailored priors.
problem Analyzing the error in inverse problems solved with generative priors.
method Quantitative error bounds for minimum Wasserstein-2 generative models.
result The error in the posterior due to the generative prior is bounded by the prior's error in Wasserstein-1 distance.
New method improves experimental design under model misspecification.
problem Improving experimental design under time and budget constraints with model misspecification.
method Developed a new acquisition function (R-IDeA) that targets representativeness, informativeness, and de-amplification.
result The new method outperforms methods focusing on representativeness or informativeness alone.
Bayesian Additive Regression Trees (BART) is a fully Bayesian approach to modeling with ensembles of trees. BART can uncover complex regression functions with high dimensional regressors in a fairly automatic way and provide Bayesian quantification of the uncertainty through the posterior. However, BART assumes IID nor…
Bayesian inference typically requires the computation of an approximation to the posterior distribution. An important requirement for an approximate Bayesian inference algorithm is to output high-accuracy posterior mean and uncertainty estimates. Classical Monte Carlo methods, particularly Markov Chain Monte Carlo, rem…
The most important aspect of any classifier is its error rate, because this quantifies its predictive capacity. Thus, the accuracy of error estimation is critical. Error estimation is problematic in small-sample classifier design because the error must be estimated using the same data from which the classifier has been…
This paper studies a specific blow-up algorithm for sop polynomials and their RLCT.
problem Determining the RLCT of sum-of-products polynomials through blow-up.
method Investigates a specific blow-up algorithm for sop polynomials to resolve their singularities.
result It is possible to resolve the singularities of sop polynomials using a specific blow-up algorithm.
Paper improves particle variational inference by optimizing generalization error bound.
problem Improving the diversity of models in particle variational inference to enhance generalization.
method Develops a new second-order Jensen inequality with a repulsion term based on the loss function, leading to a tighter generalization error bound.
result The proposed PVI optimizes the generalization error bound directly, improving performance compared to existing methods.
This paper connects RND, deep ensembles, and Bayesian inference, providing a unified theoretical perspective.
problem Uncertainty quantification in deep learning models.
method Analysis of Random Network Distillation (RND) within the neural tangent kernel framework.
result The uncertainty signal from RND is equivalent to the predictive variance of a deep ensemble and can be made to mirror the centered posterior predictive distribution of Bayesian inference.
Bayesian method improves segmentation accuracy with noisy labels.
problem Annotation errors in semantic segmentation due to mislabeling and spatial correlations.
method Approximate Bayesian estimation with spatially correlated discrete distributions and variational inference.
result The method achieves performance comparable to clean labels under moderate noise levels.
New bounds on learning algorithm generalization error derived using information density.
problem Bounding the generalization error of learning algorithms.
method Exponential inequalities and information density/conditional information density.
result Novel bounds on average and tail probability of generalization error.
BSG learns dynamic network spillovers and uncertainty quantification.
problem Identifying indirect spillovers and systemic risk in dynamic networks.
method Bayesian Spillover Graphs using FEVD and Bayesian time series models.
result Significant performance gains over baselines in identifying source and sink nodes.
PClean automates Bayesian data cleaning for specific datasets.
problem Bayesian inference for diverse and complex data cleaning.
method Domain-specific probabilistic programming language with custom models and inference.
result PClean programs outperform general-purpose PPLs in accuracy and runtime.
This study examines how large language models learn in-context and provides insights into their performance and architecture.
problem Understanding how large language models learn in-context and their performance metrics.
method Bayesian model averaging, parameterization, and statistical analysis of transformer architecture.
result Transformer architecture enables in-context learning through attention mechanisms and fine-grained statistical analysis.