Deep learning improves Bayes factor computation for likelihood-free models.
problem Computing Bayes factors for likelihood-free models is challenging.
method Proposes a deep learning estimator of Bayes factors using simulated data.
result Establishes consistency of the Deep Bayes Factor estimator.
The paper derives a formula for factorizing categorical data to improve Bayes classifiers.
problem Improving the accuracy of Bayes classifiers by effectively factoring multidimensional data.
method Derives an explicit formula for calculating the marginal likelihood of a factorized categorical dataset.
result The derived formula can be used to select the best factorization for constructing a Bayes classifier.
Improved Naive Bayes for text classification with small datasets.
problem Poor performance of Naive Bayes in small training datasets.
method Introducing a correlation factor to Naive Bayes estimator.
result Our method achieves better accuracy than traditional Naive Bayes.
Bayes factors and relative belief ratios are compared as measures of statistical evidence.
problem Which measure of evidence is more appropriate: Bayes factors or relative belief ratios?
method Comparison of Bayes factors and relative belief ratios, considering properties and restrictions.
result Relative belief ratio has better properties as a measure of evidence.
Bayes Factor Surprise enables rapid adaptation to changing environments.
problem Learning in volatile, non-stationary stochastic environments.
method Bayesian inference in a hierarchical model with a Bayes Factor Surprise probability ratio.
result Novel surprise-based algorithms improve parameter estimation and performance.
Bayesian inference improves neural network pruning efficiency.
problem Reducing computational and memory demands of large neural networks.
method Utilizes Bayesian inference to calculate Bayes factors for iterative pruning.
result Achieves desired levels of sparsity while maintaining competitive accuracy.
New method for hyperparameter tuning in sparse matrix factorization.
problem Hyperparameter tuning in sparse matrix factorization.
method Numerical method based on evaluating the zero point of normalization factor in sparse matrix prior.
result Our method outperforms existing algorithms in ground-truth sparse matrix reconstruction.
Bayes-Factor-VAE models improve disentanglement of latent factors in data.
problem Disentangling latent factors in data using standard Gaussian priors is suboptimal.
method Introduced hierarchical Bayesian deep auto-encoder models with hyper-priors on latent variances.
result Bayes-Factor-VAEs outperform existing methods in latent disentanglement.
We presented Bayesian portfolio selection strategy, via the k factor asset pricing model. If the market is information efficient, the proposed strategy will mimic the market; otherwise, the strategy will outperform the market. The strategy depends on the selection of a portfolio via Bayesian multiple testing methodol…
In this document we are going to derive the equations needed to implement a Variational Bayes i-vector extractor. This can be used to extract longer i-vectors reducing the risk of overfittig or to adapt an i-vector extractor from a database to another with scarce development data. This work is based on Patrick Kenny's …
A framework uses variational Bayes for solving inverse problems efficiently.
problem Solving inverse problems in various dimensions with flexibility and accuracy.
method Variational Bayes approximations with message passing and factor graph approach.
result Efficient algorithm updates for higher dimensions and computational advantage over MCMC.
NPMLE improves Gaussian denoising without prior knowledge of clusters.
problem Estimating Gaussian location mixtures from noisy data.
method Nonparametric Maximum Likelihood Estimator (NPMLE) for convex optimization.
result Empirical Bayes estimates perform nearly optimally in Gaussian denoising.
NBMF improves recommendation precision by modeling count data dispersion.
problem Predicting user preferences in recommender systems with over-dispersed data.
method NBMF extends PF with a multiplicative term to handle over-dispersion, skipping binarization.
result NBMF predicts user tastes more accurately than Poisson matrix factorization.
Researchers estimate optimal PAC-Bayes bounds using Hamiltonian Monte Carlo.
problem Estimating tight PAC-Bayes bounds with restricted posterior families.
method Sampling from optimal Gibbs posterior using Hamiltonian Monte Carlo, estimating KL divergence, and proposing high-probability bounds.
result Significant tightness gaps in PAC-Bayes bounds, up to 5-6% in some cases.
A new framework for private Bayesian tests maintains interpretability and computational efficiency.
problem Lack of interpretability and inability to quantify evidence in confidential data.
method Differentially private Bayesian tests based on test statistics.
result Established results on Bayes factor consistency under the proposed framework.
Unified Bayesian framework improves clinical trial hypothesis testing.
problem Lack of transparency and inability to quantify evidence in traditional P-values.
method Interval null hypothesis framework combined with Bayes factor-based tests.
result Bayesian interval hypothesis testing ensures frequentist error control and interpretability.
Evidence Networks simplify Bayesian model comparison for complex models.
problem Bayesian model comparison challenges with intractable likelihoods or priors.
method Loss functions and neural networks for fast, amortized estimation of Bayes factors.
result Evidence Networks provide accurate and scalable Bayes factor estimation.
Proposes CSG model to separate semantic and variation factors for OOD prediction.
problem Out-of-distribution examples cause conventional models to mix semantic and variation factors, leading to poor performance.
method Causal Semantic Generative model (CSG) based on causal reasoning, using variational Bayes for efficient learning and prediction.
result CSG can identify semantic factor and improve OOD prediction performance.
New index measures class imbalance impact on classification performance.
problem Class imbalance affects classifier performance beyond imbalance ratio.
method Theoretical study of Bayes optimal classifier, proposing IBI3 and BI3 measures. result Demonstrates the extent of imbalance impact on classification performance.
Adaptive variational Bayes framework improves inference adaptively.
problem Lack of general and computationally tractable variational Bayes method for adaptive inference.
method Proposes a novel adaptive variational Bayes framework combining variational posteriors over individual models.
result Adaptive variational Bayes achieves optimal contraction rates adaptively under general conditions.
Bayesian method tests Granger causality in functional time series.
problem Testing Granger causality between functional time series.
method Bayesian dynamic linear models (DLM) and Bayes Factor.
result Captures Granger causality between yield curves and weather conditions.
New method improves Bayesian inference for parametric models, robust to misspecification.
problem Inference can be untrustworthy when parametric models are wrong.
method Adaptive nonparametric corrections for parametric Bayesian models using generalized Bayes.
result The method achieves robustness and efficiency, converging fast when the parametric model is close to true.
Lower bounds on Bayes risk for realizable models derived using information theory.
problem Deriving lower bounds on Bayes risk for realizable machine learning models.
method Information-theoretic analysis using rate-distortion theory and mutual information.
result Lower bounds on Bayes risk for realizable models, matching known bounds up to logarithmic factors.
VB approximates posterior mean perfectly in linear Gaussian VAR models.
problem Unknown approximation error of VB in VAR models.
method Derive approximation error in terms of mean, mode, variance, predictive density, and KL divergence.
result VB approximates posterior mean perfectly.
New decision-theoretic characterization separates belief and decision posteriors.
problem Understanding the conditions under which loss-based updating coincides with Bayesian updating.
method Decision-theoretic approach to distinguish belief and decision posteriors.
result Generalized Bayes coincides with ordinary Bayesian updating only if the loss is proportional to negative log-likelihood.
Adaptive framework improves NB accuracy by fusing two index categories.
problem Challenges in attribute weighted NB, especially fusion of two indexes.
method Proposes ATFNB framework using switching factor to fuse two index categories.
result ATFNB outperforms basic NB and state-of-the-art models.
New algorithms for high-dimensional HMMs reduce complexity by discarding non-local factors.
problem High-dimensional HMMs are computationally expensive to filter and smooth.
method Approximate filtering and smoothing via locality in factor graphs, avoiding exponential cost.
result Error bounds in local total variation norm are dimension-free, improving scalability.
Efficiently identifies important variables in binary outcomes using variational Bayes.
problem Bayesian variable selection for binary outcomes with computational challenges.
method Mean-field variational Bayes approximation with closed-form updates and efficient inference algorithm.
result Successfully identifies important variables and is orders of magnitude faster than MCMC.
Unified approach to neural network learning with PAC-Bayes bounds.
problem Overfitting and extrapolation issues in neural networks.
method Combines variational inference and PAC-Bayes for scalable learning.
result Validates theory and shows better generalization on high-dimensional tasks.
Paper proposes C-STM for multimodal neuroimaging data classification.
problem Multimodal neuroimaging data fusion for better classification.
method Coupled Support Tensor Machine (C-STM) using latent factors from ACMTF.
result C-STM achieves better classification performance than single-mode classifiers.
This thesis develops efficient computational schemes for Bayesian inference in digital receivers.
problem Efficient inference in digital receivers to minimize expected loss.
method Two schemes: exact (GDL) and approximate (VB and TVB).
result GDL guarantees reduction in operators and TVB improves performance for correlated models.
IIC provides a PAC-Bayes bound for interpolating models, revealing factors affecting generalization.
problem Theoretical challenges in understanding overparameterized models and their performance.
method PAC-Bayesian perspective applied to the Interpolating Information Criterion (IIC).
result Test error for overparameterized models achieving zero training error depends on various factors.
Deep neural networks are optimal for dependent data using PAC-Bayes bounds.
problem Optimizing deep neural networks for dependent data.
method PAC-Bayes oracle inequalities and Bernstein inequality.
result Upper and lower bounds match, proving minimax optimality.
We analyse the matrix factorization problem. Given a noisy measurement of a product of two matrices, the problem is to estimate back the original matrices. It arises in many applications such as dictionary learning, blind matrix calibration, sparse principal component analysis, blind source separation, low rank matrix …
Oracle inequality for sparse neural nets adapts to unknown structure.
problem Sparse deep neural nets in nonparametric regression.
method Gibbs posterior distribution with Metropolis-adjusted Langevin algorithms and mixture of uniform priors.
result Oracle inequality showing adaptation to unknown regularity and structure, achieving minimax-optimal rate of convergence.
Bayesian method optimizes interventions for causal discovery.
problem Active interventions are needed for causal discovery when observational data is insufficient.
method Bayesian optimization-based approach using observational data and pre-experimental evaluation of interventions.
result Demonstrated effectiveness through various experiments.
Improved IFA with Generative Adversarial Networks for high-dimensional latent variables.
problem Limited expressiveness of traditional VAEs in high-dimensional latent variable modeling.
method Introducing Adversarial Variational Bayes (AVB) and Importance-weighted Adversarial Variational Bayes (IWAVB) algorithms.
result IWAVB demonstrated superior expressiveness and higher likelihood compared to IWAE.
Flexible estimator synthesizes noisy experiments and covariates for optimal effect estimation.
problem Simultaneous analysis of many noisy experiments with rich covariate information.
method Plug-in empirical Bayes estimator that synthesizes noisy experimental results and covariates.
result Within a constant factor of minimax for a simple data-generating model, and robust convergence guarantees hold under generality.
Proposes a method to enforce structural constraints in auto-encoding variational Bayes.
problem Difficulty in imposing structural constraints on approximate posterior of generative models.
method Uses kernel-based measures of independence (dHSIC) to enforce independence between latent representations and nuisance factors.
result Shows superior performance in learning interpretable representations, especially in scRNA-seq.
Thermodynamic integration (TI) for computing marginal likelihoods is based on an inverse annealing path from the prior to the posterior distribution. In many cases, the resulting estimator suffers from high variability, which particularly stems from the prior regime. When comparing complex models with differences in a …
Paper examines M-SVM for multi-task learning, showing reliability and pre-convergence-rate factor improvements.
problem Whether MTL always provides reliable results and how MTL outperforms independent learning.
method Regularized multi-task learning (MTL) based on SVM models (M-SVM).
result M-SVM is Bayes risk consistent in large sample size, improving pre-convergence-rate factor (PCR) for small data.
New estimator reduces risk in slate bandits by leveraging Bayes risk criterion.
problem Evaluating slate policies using logged data when policies factorize over slots.
method Developed a new estimator using a control variate approach, showing risk improvement over existing methods.
result The new estimator has lower risk than the pseudoinverse estimator in slate bandit problems.
Study on generalization in reparameterizable RL, deriving new guarantees.
problem Understanding generalization in reparameterizable RL.
method Using supervised learning and transfer learning theory, derived guarantees on the gap between expected and empirical return.
result Generalization capability of reparameterizable RL is related to multiple factors including smoothness of the environment transition, reward, and policy function class.
LPF provides formal guarantees for aggregating multi-evidence in probabilistic tasks.
problem Lack of formal guarantees for multi-evidence reasoning in AI.
method LPF uses variational autoencoders and Sum-Product Networks to aggregate evidence items.
result Proves multiple formal guarantees including calibration preservation and error decay.
Efficient Bayesian LMM framework for high-dimensional longitudinal data.
problem Scalability and dependence in high-dimensional longitudinal data.
method Partitioned empirical Bayes ECM algorithm for scalable MAP estimation.
result Identification of genes and clinical factors associated with a lupus biomarker.
New framework shows cross-attention improves multi-modal in-context learning.
problem Understanding multi-modal in-context learning in neural networks.
method Mathematical framework and linearized cross-attention mechanism.
result Cross-attention mechanism is provably optimal for multi-modal in-context learning.
In this paper we review the concepts of Bayesian evidence and Bayes factors, also known as log odds ratios, and their application to model selection. The theory is presented along with a discussion of analytic, approximate and numerical techniques. Specific attention is paid to the Laplace approximation, variational Ba…
The CAP slope is Bayes' theorem in cumulative coordinates, unlocking the weight of evidence, Somers' D, and Gini coefficient.
problem Calibration and weight of evidence
method Identifying the CAP slope as Bayes' theorem
result Revealing the CAP slope as Bayes' theorem