Approximate probabilistic inference algorithms are central to many fields. Examples include sequential Monte Carlo inference in robotics, variational inference in machine learning, and Markov chain Monte Carlo inference in statistics. A key problem faced by practitioners is measuring the accuracy of an approximate infe…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Inference Algorithms”
A new particle algorithm improves mean-field variational inference.
problem Efficiently approximating nonparametric posterior distributions in machine learning.
method Introduces PArticle VI (PAVI), a novel particle-based algorithm for nonparametric mean-field approximation.
result Obtains non-asymptotic error bounds for PArticle VI, providing the first end-to-end guarantee for particle-based MFVI.
We review three algorithms for Latent Dirichlet Allocation (LDA). Two of them are variational inference algorithms: Variational Bayesian inference and Online Variational Bayesian inference and one is Markov Chain Monte Carlo (MCMC) algorithm -- Collapsed Gibbs sampling. We compare their time complexity and performance.…
Probabilistic inference procedures are usually coded painstakingly from scratch, for each target model and each inference algorithm. We reduce this effort by generating inference procedures from models automatically. We make this code generation modular by decomposing inference algorithms into reusable program-to-progr…
The paper reclassifies RL algorithms using inference concepts.
problem To provide a unified perspective on RL algorithms.
method Using the control as inference framework, the paper classifies RL algorithms as amortised and iterative inference.
result A wide range of RL algorithms can be classified in this new manner.
The paper parallelizes HMM inference for efficient long-term computations.
problem Efficiently computing inference in long-term hidden Markov models.
method Parallelization using associative elements and operators for sum-product and max-product algorithms.
result The proposed parallel algorithms are computationally efficient for long time horizons.
Graphical models use graphs to compactly capture stochastic dependencies amongst a collection of random variables. Inference over graphical models corresponds to finding marginal probability distributions given joint probability distributions. In general, this is computationally intractable, which has led to a quest fo…
Approximate inference algorithm is one of the fundamental research fields in machine learning. The two dominant theoretical inference frameworks in machine learning are variational inference (VI) and Markov chain Monte Carlo (MCMC). However, because of the fundamental limitation in the theory, it is very challenging to…
New algorithm for aggregate inference in HMMs with continuous observations.
problem Inference in large populations with indistinguishable individuals and continuous measurements.
method Continuous observation collective forward-backward algorithm extending existing discrete case algorithm.
result Efficacy demonstrated through numerical experiments.
A new algorithm estimates aggregate marginals from noisy data in an online manner.
problem Estimating aggregate marginals of a Markov chain from noisy aggregate observations.
method Sliding window Sinkhorn belief propagation (SW-SBP) algorithm.
result Demonstrated improved performance on inferring population flow.
BaMANI uses ensemble learning to improve Bayesian network inference.
problem Bayesian network inference's reliance on specific algorithms can obscure causal relationships.
method Developed an ensemble learning approach to marginalize algorithm impact.
result Improved accuracy and reliability of causal network predictions.
Paper develops an online EM algorithm for graph signal inference from streaming data.
problem Joint inference and clustering of graph signals with non-white excitation.
method Mixture model with low-rank plus sparse prior, online EM algorithm.
result Proposed online EM algorithm converges to MAP solution.
Study uses online bootstrap for RL inference, showing effectiveness.
problem Statistical inference for RL parameters in online settings.
method Online bootstrap method applied to TD and GTD algorithms in RL.
result Method is distributionally consistent for policy evaluation inference.
Paper analyzes inclusive KL inference using Wasserstein gradient flows.
problem Analyzing inclusive KL inference with mathematical tools.
method Gradient flows derived from PDE analysis.
result Unified view of existing sampling algorithms as inclusive-KL inference.
Variational inference is a scalable technique for approximate Bayesian inference. Deriving variational inference algorithms requires tedious model-specific calculations; this makes it difficult to automate. We propose an automatic variational inference algorithm, automatic differentiation variational inference (ADVI). …
New algorithms improve causal direction inference accuracy using parallel ensemble methods.
problem Stability of causal direction inference results from observational data.
method Parallel ensemble frameworks to map and improve inference accuracy.
result Significant improvement in accuracy of causal direction inference.
New algorithm for collective Gaussian hidden Markov models inference.
problem Inference of collective Gaussian hidden Markov models from aggregate data.
method Collective Gaussian forward-backward algorithm, extending Sinkhorn belief propagation.
result Convergence guarantee and applicability to single individual Kalman filter.
Deep learning and genetic algorithms speed up cosmological Bayesian inference.
problem Substantial computational demands in Bayesian inference for cosmological parameter estimation.
method Deep learning using feedforward neural networks to approximate likelihood functions dynamically, optimized with genetic algorithms.
result Significant speed-up in Bayesian inference process for cosmological models and datasets.
The paper tackles fairness in data and algorithms, expanding on prior work.
problem Discrimination and disparate treatment in data and algorithms.
method Targeted learning for nonparametric inference of fairness in the data generating process.
result Derivation and validation of estimators for fairness metrics like demographic parity and equal opportunity.
Improved Bayesian inference for neuronal ensemble inference reduces computational cost.
problem Efficient inference of neuronal ensembles from activity data.
method Modified MCMC algorithm with simulated annealing for hyperparameter control.
result Our method reduces computational cost while maintaining or improving inference accuracy.
A benchmark for simulation-based inference methods.
problem Lack of a public benchmark for 'likelihood-free' algorithms.
method Provided a benchmark with tasks and performance metrics, including neural networks and ABC methods.
result State-of-the-art algorithms have room for improvement, and neural network-based approaches generally perform better.
A new predictive coding algorithm improves machine learning performance.
problem Improving performance of variational inference algorithms in high-dimensional, structured problems.
method Divide-and-Conquer Predictive Coding (DCPC) that respects generative model correlation structure and performs maximum-likelihood updates.
result DCPC achieves better numerical performance and accurate inference in structured generative models.
Inference of stochastic time series with missing dataphysics.data-an
Method infers dynamics from incomplete time series data.
problem Challenges in inferring stochastic dynamics from time series with missing data.
method Expectation Maximization (EM) algorithm that iterates between E-step and M-step.
result The EM algorithm effectively recovers missing data points and infers underlying network models from real neuronal activities.
Tree-AMP simplifies inference in complex tree-structured models.
problem Inference in high-dimensional tree-structured models.
method Approximate Message Passing algorithms for various machine learning tasks.
result Theoretical performance predictions and automated entropy estimation.
To model modern large-scale datasets, we need efficient algorithms to infer a set of unknown model parameters from noisy measurements. What are fundamental limits on the accuracy of parameter inference, given finite signal-to-noise ratios, limited measurements, prior information, and computational tractability …
This work frames active inference through control as inference, offering robust control algorithms.
problem Active inference framework lacks practical sensorimotor control algorithms.
method Frame active inference through control as inference, presenting trajectory optimization as inference.
result AI may be framed as partially-observed CaI when the cost function is defined in observation states.
Coordinate ascent variational inference is an important algorithm for inference in probabilistic models, but it is slow because it updates only a single variable at a time. Block coordinate methods perform inference faster by updating blocks of variables in parallel. However, the speed and stability of these algorithms…
Recent efforts on combining deep models with probabilistic graphical models are promising in providing flexible models that are also easy to interpret. We propose a variational message-passing algorithm for variational inference in such models. We make three contributions. First, we propose structured inference network…
In this paper, we consider the sigmoid Gaussian Hawkes process model: the baseline intensity and triggering kernel of Hawkes process are both modeled as the sigmoid transformation of random trajectories drawn from Gaussian processes (GP). By introducing auxiliary latent random variables (branching structure, Pólya-Gamm…
An autonomous variational inference algorithm for arbitrary graphical models requires the ability to optimize variational approximations over the space of model parameters as well as over the choice of tractable families used for the variational approximation. In this paper, we present a novel combination of graph part…
Variational inference methods for latent variable statistical models have gained popularity because they are relatively fast, can handle large data sets, and have deterministic convergence guarantees. However, in practice it is unclear whether the fixed point identified by the variational inference algorithm is a local…
Two novel distributed VB algorithms improve Bayesian inference in sensor networks.
problem Efficient inference in Bayesian frameworks for sensor networks.
method Two novel distributed VB algorithms for general Bayesian inference, using stochastic natural gradient and ADMM.
result Distributed algorithms perform nearly as well as centralized ones, demonstrating excellent performance.
New method for inferring network topology from partial data.
problem Inferring network topology from limited node data.
method Vector autoregressive model and Gaussian mixture algorithm.
result The proposed method converges to the network combination matrix in probability.
Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriorscs.LG
DDVI uses diffusion models for variational inference, improving latent variable model performance.
problem Improving variational inference in latent variable models.
method Introduces diffusion-based variational posteriors trained with a regularized ELBO.
result Outperforms alternative variational posteriors on various benchmarks and a biology task.
Unified reinforcement learning methods using hybrid inference.
problem Combining model-based and model-free reinforcement learning approaches.
method Control as Hybrid Inference (CHI) framework.
result CHI algorithm balances model-based and model-free learning.
New algorithm improves CRF inference and learning.
problem Efficient inference and learning for dense CRFs.
method Regularized Frank-Wolfe algorithm for nonconvex CRF optimization.
result Regularized Frank-Wolfe outperforms mean field and CNN baselines.
A new algorithm infers causal networks from data using topological thresholds.
problem Inferring causal networks from data.
method Two methods for determining topological thresholds: one to leave no disconnected nodes, the other to find a causal large connected component.
result The novel algorithm is faster and more accurate than the PC algorithm.
Prediction-Powered Inferencestat.ML
Valid inference from data and predictions.
problem Valid statistical inference with machine learning predictions.
method Framework for valid inference using machine learning predictions.
result Valid confidence intervals without assumptions on predictions.
CIfly simplifies causal inference tasks with linear-time reachability primitives.
problem Efficiently solving complex causal inference problems.
method Formalizes reachability as a core operation, builds on state-space graphs, and uses rule tables.
result CIfly algorithms run in linear time, outperforming existing methods.
Boosting Variational Inference improves posterior approximations with adaptive step-sizes.
problem Limited resources hinder the widespread adoption of Boosting Variational Inference.
method Characterized global curvature impact, introduced local curvature, and developed an approximate backtracking algorithm.
result New theoretical convergence rates and experimental validation demonstrate improved performance.
Bayesian inference provides an attractive online-learning framework to analyze sequential data, and offers generalization guarantees which hold even with model mismatch and adversaries. Unfortunately, exact Bayesian inference is rarely feasible in practice and approximation methods are usually employed, but do such met…
Efficiently estimates marginal posteriors for complex simulations.
problem Bayesian inference in high-dimensional, intractable likelihood scenarios.
method Simulates and estimates low-dimensional marginal posteriors, using truncated indicators.
result Simulator efficiency and robustness testing of inference results.
A fundamental computation for statistical inference and accurate decision-making is to compute the marginal probabilities or most probable states of task-relevant variables. Probabilistic graphical models can efficiently represent the structure of such complex data, but performing these inferences is generally difficul…
Unified approach to path planning using probabilistic inference on factor graphs.
problem Path planning problems using probabilistic inference.
method Unified framework using probabilistic factor graphs and message composition rules.
result Unified approach includes various algorithms like Sum-product, Max-product, Dynamic programming, and mixed criteria.
Over the past decade there has been considerable interest in spectral algorithms for learning Predictive State Representations (PSRs). Spectral algorithms have appealing theoretical guarantees; however, the resulting models do not always perform well on inference tasks in practice. One reason for this behavior is the m…
Paper addresses regret minimization and inference in high-dimensional online decision-making.
problem Regret minimization and statistical inference in high-dimensional online decision-making.
method Integrates ε-greedy bandit algorithm with hard thresholding for sparse bandit parameters and debiasing method for inference.
result Achieves either regret or -consistent inference, with trade-off between exploration and exploitation.
Stochastic variational inference for collapsed models has recently been successfully applied to large scale topic modelling. In this paper, we propose a stochastic collapsed variational inference algorithm in the sequential data setting. Our algorithm is applicable to both finite hidden Markov models and hierarchical D…
In theory, Bayesian nonparametric (BNP) models are well suited to streaming data scenarios due to their ability to adapt model complexity with the observed data. Unfortunately, such benefits have not been fully realized in practice; existing inference algorithms are either not applicable to streaming applications or no…