A new probabilistic framework improves deep learning performance.
problem Improving deep learning models and understanding their limitations.
method Developed a probabilistic framework based on DRMM, a generative model capturing latent variables.
result DRMM outperforms DCNs in classification tasks, achieving state-of-the-art results.
Convex message passing algorithms converge to a fixed point.
problem Understanding convergence properties of convex message passing methods.
method Proving convergence of coordinate descent applied to piecewise-affine convex objectives, and showing this applies to various message passing methods.
result The iterates converge to a fixed point of the method, and the algorithm terminates in a known number of iterations.
Identifies a sub-matrix with maximal sum in large data matrices.
problem Finding a rectangular sub-matrix with the highest sum of entries.
method Proposes two algorithms: CP-GC and MILP, leveraging problem characteristics.
result CPGC approach tends to be the fastest to produce a good solution.
We present a global optimization algorithm for clustering data given the ratio of likelihoods that each pair of data points is in the same cluster or in different clusters. To define a clustering solution in terms of pairwise relationships, a necessary and sufficient condition is that belonging to the same cluster sati…
For the problem of binary linear classification and feature selection, we propose algorithmic approaches to classifier design based on the generalized approximate message passing (GAMP) algorithm, recently proposed in the context of compressive sensing. We are particularly motivated by problems where the number of feat…
Deep neural networks improve MIMO detection performance.
problem Improving detection accuracy in massive MIMO systems.
method Introduced a neural network architecture based on BP algorithms, optimized with deep learning techniques.
result DNN MIMO detectors achieve lower bit error rates compared to other detectors.
Transformers learn to use induction heads or shortcuts based on data diversity.
problem How data diversity influences the behavior of transformers.
method Gradient-based training of a single-layer transformer on a minimal task.
result Data diversity steers transformers toward induction heads or shortcuts.
Study on H-consistency bounds for machine learning surrogates.
problem Estimating target loss error relative to surrogate loss error in machine learning.
method Developed H-consistency bounds for various surrogates and loss functions. result Stronger guarantees than existing methods, offering distribution-dependent and -independent bounds.
Iterative models improve inference efficiency in deep latent variable models.
problem Inference models in deep latent variable models are computationally inefficient and have an amortization gap.
method Proposes iterative models that learn to perform inference optimization through repeated encoding of gradients.
result Iterative models outperform standard inference models on benchmark data sets of images and text.
AIDE measures the accuracy of probabilistic inference algorithms.
problem Measuring the accuracy of approximate inference algorithms on specific data sets.
method AIDE is an algorithm based on viewing inference algorithms as probabilistic models and auxiliary variables.
result AIDE captures the qualitative behavior of inference algorithms and detects failure modes.
This work frames active inference through control as inference, offering robust control algorithms.
problem Active inference framework lacks practical sensorimotor control algorithms.
method Frame active inference through control as inference, presenting trajectory optimization as inference.
result AI may be framed as partially-observed CaI when the cost function is defined in observation states.
Simformer uses transformer models to perform flexible Bayesian inference.
problem Current simulation-based inference methods are inflexible and require fixed priors.
method Trains a probabilistic diffusion model with transformer architectures.
result Outperforms state-of-the-art methods on various benchmarks.
PE-SVI reduces SVI inference complexity by finding a suitable start point.
problem Complex posterior inference in graphical models leads to suboptimal learning.
method PE-SVI uses a pseudo-encoded start point to reduce gradient steps and step sizes.
result PE-SVI achieves the same ELBo objective as SVI with less than 1% of the required steps.
Proposes a new algorithm for efficient probabilistic inference.
problem Efficient probabilistic inference in deep models with graphical structures.
method Structured inference networks and variational message-passing algorithm.
result Enables fast and efficient natural-gradient inference for deep structured models.
Improved Bayesian inference for neuronal ensemble inference reduces computational cost.
problem Efficient inference of neuronal ensembles from activity data.
method Modified MCMC algorithm with simulated annealing for hyperparameter control.
result Our method reduces computational cost while maintaining or improving inference accuracy.
New framework for improved inference algorithms.
problem Challenges in improving VI and MCMC methods.
method Ergodic Inference framework based on ergodic transformations.
result Established theoretical foundation for practical algorithms.
Improves probabilistic inference with new weighting method.
problem Improving variational inference for probabilistic models.
method Importance Weighted Variational Inference (IWVI) using augmented variational inference.
result IWVI is a practical technique for probabilistic inference.
Two new methods for variational inference without tractable densities.
problem Challenges in variational inference due to computationally intractable probability density functions.
method Introduces wild variational inference methods that do not require tractable density functions.
result Significant improvement in stochastic gradient Langevin dynamics (SGLD) step size adjustment.
Adding metadata abruptly changes network inference outcomes.
problem Understanding the impact of metadata on network inference.
method Investigated the effect of metadata on network inference problems.
result Metadata causes abrupt transitions in inference outcomes.
SNVI combines likelihood estimation with variational inference for efficient Bayesian inference.
problem Bayesian inference in models with intractable likelihoods.
method Sequential Neural Variational Inference (SNVI) that combines likelihood-estimation with variational inference.
result SNVI is more computationally efficient than previous algorithms without sacrificing accuracy.
Paper introduces a diagnostic for approximate inference methods.
problem Estimating errors in probabilistic inference algorithms, especially for approximate methods.
method Repeatedly simulate datasets from the prior and perform inference on each, estimating a symmetric KL-divergence.
result A diagnostic for approximate inference methods can be estimated using symmetric KL-divergence.
AIR improves VAE generalization by controlling inference smoothness.
problem Overly expressive inference models harm VAE performance.
method Amortized inference regularization (AIR) controls inference smoothness.
result AIR improves VAE generalization on inference and generative tasks.
Probabilistic inference procedures are usually coded painstakingly from scratch, for each target model and each inference algorithm. We reduce this effort by generating inference procedures from models automatically. We make this code generation modular by decomposing inference algorithms into reusable program-to-progr…
A new method for safer statistical inference after predictions.
problem Statistical inference with pseudo-outcomes from machine learning predictions.
method Prediction De-Correlated Inference (PDC) framework.
result PDC consistently outperforms supervised methods and can adapt to any model.
Paper proposes Walsh-Hadamard Variational Inference for efficient approximate inference in large models.
problem Over-regularization in variational inference for large models.
method Walsh-Hadamard factorization strategies to reduce parameterization, accelerate computations, and increase posterior expressiveness.
result Efficient approximate inference achieved in over-parameterized models.
Bayesian interpolants explain neural network inferences concisely.
problem Understanding neural network inferences.
method Adapting Craig interpolants for neural networks.
result Produces precise, understandable explanations.
ptype infers data types robustly in real-world data.
problem Type inference fails with missing data and anomalies.
method Probabilistic robust type inference method.
result Outperforms existing methods.
Paper presents efficient variational inference for Pachinko Allocation Machines.
problem Difficult approximate inference in Pachinko Allocation Machines.
method Amortized variational inference using deep inference network.
result Produces more coherent topics and is faster than state-of-the-art methods.
Neurally-Guided Structure Inference combines search and data-driven methods for efficient, robust structure inference.
problem Combining the advantages of exhaustive search and data-driven methods for structure inference.
method Neurally-Guided Structure Inference (NG-SI) uses a neural network to guide hierarchical search over structures.
result NG-SI outperforms search-based and data-driven methods on probabilistic matrix decomposition and symbolic program parsing.
Paper shows how to infer hidden states in neural networks analytically.
problem Intractability of Bayesian inference for neural networks.
method Leverage tractable approximate Gaussian inference (TAGI) for hidden states inference.
result Demonstrates inference of hidden states through constraints for various applications.
New comparison shows differences in how value is incorporated in AIF and CAI.
problem Clarifying the relationship between Active Inference and Control-as-Inference.
method Formal comparison of AIF and CAI frameworks.
result Primary difference is how value is incorporated into generative models.
Post-ADC inference corrects bias in statistical inference after active data collection.
problem Bias in inference after active data collection.
method Post-ADC inference framework that corrects bias from both ADC process and data-driven target construction.
result Valid inference for data collected by SMBO methods like GP-UCB and TPE.
Variational Inference shows promise for Bayesian GARCH model estimation.
problem Bayesian estimation of GARCH-family models using Monte Carlo sampling.
method Variational Inference as an alternative to Monte Carlo sampling.
result Variational Inference is a reliable and competitive method for Bayesian learning in GARCH-like models.
Meta-learn Bayesian inference for task-specific BNNs using amortised inference.
problem Efficiently learning Bayesian inference for small-scale probabilistic meta-learning.
method Replace global inducing points with actual data to create a set of approximate likelihoods, train a meta-model to learn these parameters across related datasets.
result Meta-learned inference can be applied to task-specific BNNs, improving efficiency and scalability.
A new particle algorithm improves mean-field variational inference.
problem Efficiently approximating nonparametric posterior distributions in machine learning.
method Introduces PArticle VI (PAVI), a novel particle-based algorithm for nonparametric mean-field approximation.
result Obtains non-asymptotic error bounds for PArticle VI, providing the first end-to-end guarantee for particle-based MFVI.
Wasserstein variational inference uses optimal transport for stable likelihood-free training.
problem Approximate Bayesian inference with stability and flexibility for implicit distributions.
method Optimal transport theory, Sinkhorn iterations, and backpropagation.
result Stable likelihood-free training method for autoencoders and probabilistic programs.
Bayesian method infers contextual bandit policies robustly.
problem Inference of contextual bandit policies in small sample sizes.
method Empirical likelihood for Bayesian inference.
result Accurate uncertainty measurements and policy comparison.
Exact selective inference with randomization for Gaussian regression models.
problem Exact selective inference in Gaussian regression models.
method Introduces a pivot for exact selective inference with randomization, reducing the problem to a bivariate truncated Gaussian distribution.
result Our pivot leads to exact inference and produces narrower confidence intervals than related methods.
Paper introduces probabilistic module interface for complex models and inference.
problem Handling complex probabilistic models with latent variables and custom inference methods.
method Develops a platform-agnostic interface for encapsulating models and inference programs, allowing sound approximate inference algorithms for networks of modules.
result Sound approximate inference algorithms can be constructed for networks of probabilistic modules.
BOED improves SBI by optimizing experimental designs and inference functions.
problem Efficiently use experimental resources for better inference on complex models.
method Link mutual information bounds between SBI and BOED, optimizing both design and inference.
result BOED improves inference in real-world simulators in epidemiology and biology.
Valid inference from data and predictions.
problem Valid statistical inference with machine learning predictions.
method Framework for valid inference using machine learning predictions.
result Valid confidence intervals without assumptions on predictions.
EFI automates statistical inference for big data.
problem Statistical inference for model parameters based on observations.
method EFI uses stochastic gradient Markov chain Monte Carlo and sparse deep neural networks.
result EFI provides higher fidelity in parameter estimation and automates the inference process.
Scalable Bayesian inference for state space models using variational inference and SMC.
problem Performing fully Bayesian inference in state space models.
method Combining variational methods with sequential Monte Carlo sampling.
result Fully Bayesian inference of dynamic latent states and static parameters.
Machine learning infers fluid dynamics from data without prior knowledge.
problem Inferring complex fluid behavior from limited data.
method Reservoir computing for partial and full inference of fluid variables and energy functions.
result Reservoir systems can infer long-term fluid dynamics and energy spectra from past data.
HuMaINs combines human and machine strengths for better inference tasks.
problem Improving inference performance through human-machine collaboration.
method Novel signal processing and machine learning solutions for HuMaINs architecture.
result HuMaINs achieves higher performance than either humans or machines individually.
Derives time-averaged active inference from control principles.
problem Finite-horizon or discounted-surprise problems in active inference.
method Derives infinite-horizon, average-surprise active inference from optimal control principles.
result Unified objective functional for sensorimotor control.
UA-SABI uses surrogates to speed up Bayesian inference for expensive models.
problem Inference for computationally expensive models is slow and uncertain.
method Combines surrogate modeling with Amortized Bayesian Inference (ABI) to propagate uncertainties.
result Reliable, fast, and repeated Bayesian inference for expensive models is achieved.
Adversarial robustness of amortized Bayesian inference is studied, showing it can be improved.
problem Adversarial robustness of amortized Bayesian inference.
method Simulation-based estimation, regularization scheme based on Fisher information.
result Adversarial robustness can be improved with a regularization scheme.