The method infers Markov process parameters from steady state snapshots.
problem Inferring parameters from non-equilibrium steady states without Boltzmann distribution.
method Propagator likelihood based on fictitious transitions.
result Efficient reconstruction of parameters in various systems.
Improves hyperparameter learning in GP models with non-conjugate likelihoods.
problem Hyperparameter learning entangled with approximate inference in GP models.
method Hybrid training procedure combining VI for inference and EP-like marginal likelihood approximation for hyperparameter learning.
result Empirically demonstrates the effectiveness of the proposed training procedure across various data sets.
DP-SEP privatizes EP by refining a single factor per data point.
problem Private inference of complex models with limited memory.
method Stochastic Expectation Propagation with differential privacy.
result DP-SEP provides better posterior estimates with guaranteed privacy.
A method for converting NIW parameters for better estimation.
problem Estimating parameters of multivariate normal distribution.
method Convergent procedure for converting mean parameters to natural parameters in NIW family.
result Maximum likelihood estimation of natural parameters from observed statistics.
Combines VI and EP for better Gaussian process hyperparameter learning.
problem Improving hyperparameter learning in Gaussian processes for better performance.
method Hybrid training procedure combining Variational Inference (VI) for posterior inference and Expectation Propagation (EP) for hyperparameter learning.
result The hybrid training procedure provides a better learning objective and generalizes better than using only VI or EP.
A new method improves fitting neural data with spiking network models.
problem Fitting spiking network models to neural activity does not produce realistic data.
method Augment log-likelihood with dissimilarity terms measured by summary statistics and optimized via back-propagation.
result The new method generates more realistic neural activity statistics and improves network connectivity inference.
The study examines how side information quality and quantity affect community recovery in graphs.
problem Recovering a hidden community of size K=o(n) in a graph of size n. method Maximum likelihood detection and belief propagation are used to calculate necessary and sufficient conditions for exact and weak recovery. A local voting procedure is also designed and analyzed.
result Tight necessary and sufficient conditions for exact and weak recovery are derived, showing how side information needs to evolve with n to improve recovery thresholds. Meta-embeddings improve PLDA model's accuracy for speaker recognition.
problem Improving PLDA model's accuracy for speaker recognition.
method Proposed Gaussian meta-embeddings (GMEs) for heavy-tailed PLDA models.
result GMEs with variable precisions propagate uncertainty, leading to up to 20% more accurate results.
In this paper, we address the inverse problem, or the statistical machine learning problem, in Markov random fields with a non-parametric pair-wise energy function with continuous variables. The inverse problem is formulated by maximum likelihood estimation. The exact treatment of maximum likelihood estimation is intra…
Improved WBP decoding with simple scaling and SNR adaptation.
problem Efficiently decoding weighted Tanner graphs with reduced complexity.
method Simple-scaling models with machine learning for edge weights, and parameter adapter networks.
result Simple scaling with few parameters can achieve near-maximum-likelihood performance.
This paper presents a Bayesian image segmentation model based on Potts prior and loopy belief propagation. The proposed Bayesian model involves several terms, including the pairwise interactions of Potts models, and the average vectors and covariant matrices of Gauss distributions in color image modeling. These terms a…
Efficient GP models with non-Gaussian likelihoods using state space methods.
problem Modeling non-Gaussian likelihoods in Gaussian Process (GP) regression.
method State space formulation for efficient GP models, combining LA, VB, ADF, and EP schemes.
result Efficient inference methods for non-Gaussian likelihoods in GP models.
Learn invariances in models using the marginal likelihood.
problem Generalizing well in supervised learning tasks.
method Learn invariances in model structure using the marginal likelihood.
result Demonstrated for Gaussian process models, reducing complexity of invariant models.
Many models of interest in the natural and social sciences have no closed-form likelihood function, which means that they cannot be treated using the usual techniques of statistical inference. In the case where such models can be efficiently simulated, Bayesian inference is still possible thanks to the Approximate Baye…
This paper describes an expectation propagation (EP) method for multi-class classification with Gaussian processes that scales well to very large datasets. In such a method the estimate of the log-marginal-likelihood involves a sum across the data instances. This enables efficient training using stochastic gradients an…
Variational methods have been recently considered for scaling the training process of Gaussian process classifiers to large datasets. As an alternative, we describe here how to train these classifiers efficiently using expectation propagation. The proposed method allows for handling datasets with millions of data insta…
BBPL uses block updates to learn Markov random fields without full inference.
problem Training Markov random fields requires inference over all variables, scaling with model size.
method Block-coordinate updates of approximate marginals to compute approximate gradients.
result BBPL converges to the same solution as full inference, despite approximations.
New algorithm tackles hidden confounders in causal discovery.
problem Hidden confounders make causal discovery difficult.
method LFOICA estimates mixing matrix directly without parametric assumptions.
result Computational efficiency makes causal discovery more feasible.
New algorithm for Gaussian process classification using posterior linearisation.
problem Improving Gaussian process classification performance.
method Posterior linearisation to approximate posterior density iteratively, accounting for linearisation error.
result PL has better performance than EP in experimental data.
New method improves neural network robustness with brain-like learning.
problem Traditional methods struggle with robustness in adversarial attacks.
method Generalized likelihood ratio method with brain-like learning mechanisms.
result Significant improvement in robustness of neural networks.
Restricted Boltzmann machines~(RBMs) and conditional RBMs~(CRBMs) are popular models for a wide range of applications. In previous work, learning on such models has been dominated by contrastive divergence~(CD) and its variants. Belief propagation~(BP) algorithms are believed to be slow for structured prediction on con…
Flowification enriches neural networks with an inverse pass and likelihood monitoring.
problem Neural networks lack an inverse pass and likelihood monitoring, limiting their generative capabilities.
method Introduce flowification, enriching neural networks with a stochastic inverse pass and likelihood monitoring.
result Certain neural network architectures can be enriched to fall under the generalized notion of a normalizing flow.
TGD improves conditional sampling by concentrating computation on promising trajectories.
problem Efficiently training-free conditional sampling with diffusion priors.
method Tempered Guided Diffusion (TGD) using annealed sequential Monte Carlo.
result TGD yields a consistent particle approximation to the posterior as the number of particles grows.
A new method for Bayesian neural networks using probabilistic backpropagation.
problem Approximating posterior distributions in Bayesian neural networks.
method Variational Expectation Propagation (VEP) with probabilistic backpropagation.
result Efficient algorithm for approximate integration over posterior distributions.
Smooth generative model of shapes with uncertainty from silhouette images.
problem Challenges in modeling shapes represented as silhouette images due to intractable posteriors.
method Gaussian Process Deep Belief Networks (GPDBN) that learn from small data and propagate uncertainty.
result Proposed model provides favorable results compared to state-of-the-art models.
PIPPS solves deep learning's exploding gradient problem by reparameterization gradients.
problem Exploding gradients in deep learning and model-based RL.
method Develops PIPPS framework, a flexible policy search method robust to chaos-like gradients.
result PIPPS improves over reparameterization gradients by up to 10^6 times.
We present a global optimization algorithm for clustering data given the ratio of likelihoods that each pair of data points is in the same cluster or in different clusters. To define a clustering solution in terms of pairwise relationships, a necessary and sufficient condition is that belonging to the same cluster sati…
This work improves neural network robustness to symbol substitutions using formal verification.
problem Neural networks' vulnerability to adversarial attacks, especially under discrete text perturbations.
method Formal verification using Interval Bound Propagation on a simplex model of input perturbations.
result Models show improved verified accuracy under perturbations with formal guarantees.
Paper models COVID-19 spread as spatio-temporal point processes.
problem Understanding complex spacetime propagation of COVID-19.
method Generative and intensity-free model using adversarial imitation learning.
result Imitation learning framework for scalable model inference.
A fast method approximates likelihood scores for noisy linear inverse problems.
problem Solving noisy linear inverse problems efficiently.
method Proposes a simple closed-form approximation to the likelihood score for diffusion and flow-based models.
result Significantly faster than baseline methods while maintaining competitive or better reconstruction performances.
Bayesian LSTM for outlier detection reduces overfitting.
problem Overfitting and lack of uncertainty in LSTM networks.
method Approximate Bayesian estimation with Ensemble Kalman Filter and maximum likelihood.
result The method reduces overfitting and provides uncertainty estimates.
Paper improves spectral learning of HMMs to avoid local optima and improve robustness.
problem Spectral learning of HMMs can get stuck in local optima and degrade due to unchecked error propagation.
method Developed a novel algorithm (PSHMM) and online learning variants to mitigate error propagation and nonstationarity.
result PSHMM provides more robust estimation and forecasting compared to SHMM and B-W algorithm.
Proposes a method to adapt DNNs to drift in data distribution.
problem Adapting to out-of-distribution data and shifting objectives.
method Bayesian Inference, Variational Density Propagation, Evidence Lower Bound (ELBO), Minimum Description Length (MDL) Principle.
result Minimizes catastrophic forgetting by approximating MDL principle.
Improved model-based estimation through tempered Bayes filter.
problem Improving predictive accuracy in partially-observable stochastic systems.
method Developed tempered Bayes filter combining likelihood and full posterior tempering.
result Tempered Bayes filter achieves improved predictive performance over the Bayes filter baseline.
Analytical method approximates ELBO gradient in clutter problem.
problem Clutter problem in Bayesian networks with Gaussian likelihood.
method Reparameterization trick, local approximation of likelihood factors.
result Good accuracy and linear computational complexity compared to classical methods.
EP algorithm for efficient feature selection in binary classification.
problem Sparse feature selection in binary classification.
method Statistical mechanics inspired expectation propagation (EP) on a diluted Bayesian classifier.
result EP is a robust and competitive algorithm in terms of variable selection, estimation accuracy, and computational complexity.
New method uses machine learning to estimate sensitivity without binning.
problem Estimating sensitivity of high-dimensional data sets without binning.
method Combines machine-learning classification with likelihood-based inference tests using Kernel Density Estimators.
result Significance estimation is not sensitive to non-smooth probability distributions.
We consider probabilistic multinomial probit classification using Gaussian process (GP) priors. The challenges with the multiclass GP classification are the integration over the non-Gaussian posterior distribution, and the increase of the number of unknown latent variables as the number of target classes grows. Expecta…
Survey on community detection methods and their theoretical properties.
problem Consistent estimation of community labels in networks.
method Various community detection methods and their theoretical properties.
result Review of community detection methods and their theoretical properties.
This paper introduces the Markov-Switching Multifractal Duration (MSMD) model by adapting the MSM stochastic volatility model of Calvet and Fisher (2004) to the duration setting. Although the MSMD process is exponential β-mixing as we show in the paper, it is capable of generating highly persistent autocorrelation. W…
This paper presents a fast Bayesian filtering technique for state estimation.
problem Bottleneck in Bayesian inference for state estimation from noisy sensor data.
method Processor-native uncertainty tracking for uncertainty propagation and inference.
result Deterministic approximate filtering with up to 805x speedup and competitive accuracy.
GumBolt extends Gumbel trick for Boltzmann priors in VAEs.
problem Non-differentiability of discrete units in Boltzmann machines prevents using the reparameterization trick.
method Proposes GumBolt, extending Gumbel trick to Boltzmann priors in VAEs.
result Significantly simpler than recent methods and outperforms them.
Generative AI decodes quantum codes without labeled data.
problem Efficient decoding of quantum error-correcting codes.
method Generative Transformers learn logical operators from unsupervised syndromes.
result Significantly better decoding accuracy than traditional methods.
Algorithm recovers communities in preferential attachment graphs.
problem Recovering communities in graphs generated by preferential attachment models.
method Message passing algorithm based on belief propagation and vertex attachment order.
result Probability of correct classification depends on vertices' arrival times.
Improved susceptibility propagation for Markov random fields using diagonal matching.
problem Approximate computation of Markov random fields with robustness across network structures.
method Combines belief propagation and linear response method with diagonal matching for inverse Ising problems.
result Proposed method reduces to standard susceptibility propagation and Thouless-Anderson-Palmer equation in specific cases.
Generalized belief propagation converges to optimal solutions on graphs with motifs.
problem Understanding belief propagation on loopy graphs.
method Study of generalized belief propagation on graphs with motifs.
result Generalized belief propagation converges to the global optimum of the Bethe free energy.
A new method for Bayesian neural networks improves prediction uncertainty.
problem Estimating uncertainty in neural network predictions.
method Adversarial α-divergence minimization for approximate Bayesian inference.
result The method often gives better performance in terms of test log-likelihood and squared error in regression problems.
Proposes a new method for learning MRFs without sampling.
problem Learning deep undirected graphical models (MRFs) efficiently.
method Optimizes a saddle-point objective derived from the Bethe free energy approximation, using trained inference networks to amortize the optimization.
result The method compares favorably with loopy belief propagation and achieves better held-out log likelihood.