We propose a new Bayesian Neural Net formulation that affords variational inference for which the evidence lower bound is analytically tractable subject to a tight approximation. We achieve this tractability by (i) decomposing ReLU nonlinearities into the product of an identity and a Heaviside step function, (ii) intro…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Sample-Free Inference”
VarDeepPCA: A Sampling-Free Variational DNN Plugin for OOD Segmentation with Uncertainty Estimation
problem Deep neural networks (DNNs) fail to generalize to out-of-distribution (OOD) medical images due to variations in scanners and acquisition protocols.
method VarDeepPCA is a lightweight variational DNN framework that learns a distribution of valid anatomical geometries using small in-distribution datasets.
result VarDeepPCA restores segmentation maps produced by existing methods on OOD data to improve anatomical plausibility and reduce errors.
We present a sampling-free approach for computing the epistemic uncertainty of a neural network. Epistemic uncertainty is an important quantity for the deployment of deep neural networks in safety-critical applications, since it represents how much one can trust predictions on new data. Recently promising works were pr…
A new framework for lightweight BNNs learns heteroscedastic uncertainties efficiently.
problem Learning heteroscedastic uncertainties from BNNs for lightweight networks.
method Embedding heteroscedastic variances into BNN parameters and using moment propagation for inference.
result Improves predictive performance for lightweight BNNs without increasing parameter count.
Efficient inference for multimodal Gaussian mixture models of interacting dynamical systems.
problem Efficient inference for multimodal distributions in stochastic dynamical systems.
method Graph neural networks with moment matching for sample-free inference and structured covariance approximations.
result Sample-free inference with improved efficiency and stability compared to Monte Carlo alternatives.
Improved uncertainty estimation in neural networks with VBLL.
problem Improving uncertainty estimation in neural networks.
method Deterministic variational formulation for training Bayesian last layer neural networks.
result Improves predictive accuracy, calibration, and out-of-distribution detection.
New method for efficient probabilistic deep state-space models.
problem Efficient inference for probabilistic deep state-space models.
method Deterministic inference algorithm for ProDSSM with neural network weights.
result Superior balance between predictive performance and computational budget.
Real-Time Uncertainty Estimation in Computer Vision via Uncertainty-Aware Distribution Distillationcs.CV
Real-time uncertainty estimation for computer vision tasks.
problem Real-time inference of uncertainty in deep learning models.
method Uncertainty-Aware Distribution Distillation method for fast inference.
result Significantly reduced inference time with improved uncertainty and predictive performance.
VarDeepPCA refines medical image segmentation from small datasets, improving anatomical plausibility and reducing errors.
problem Medical image segmentation fails on out-of-distribution data due to variations in scanners and protocols.
method VarDeepPCA learns valid anatomical geometries using only small in-distribution datasets, providing uncertainty estimates.
result VarDeepPCA restores segmentation maps to OOD data, improving anatomical plausibility and reducing errors.
Proposes a new method to approximate Gaussian inference in classification tasks.
problem Uncertainty quantification in classification tasks using softmax functions.
method Develops a new formalism to approximate Gaussian distributions over logit space and proposes using element-wise normCDF or sigmoid instead of softmax.
result Improves uncertainty quantification compared to softmax Monte Carlo sampling.
There has recently been a concerted effort to derive mechanisms in vision and machine learning systems to offer uncertainty estimates of the predictions they make. Clearly, there are enormous benefits to a system that is not only accurate but also has a sense for when it is not sure. Existing proposals center around Ba…
Density-Softmax improves uncertainty estimation and robustness without sampling, reducing model size and latency.
problem Sampling-based uncertainty estimation methods suffer from large model size and high latency.
method Combines a Lipschitz-constrained feature extractor with the softmax layer to create a sampling-free deterministic framework.
result Density-Softmax reduces over-confidence under distribution shifts and achieves competitive results in uncertainty and robustness.
New method for privacy amplification without sampling for matrix factorization.
problem Privacy amplification for differentially private model training with matrix factorization.
method Sampling-free bounds based on Rényi divergence and conditional composition.
result Stronger privacy guarantees for small ε, applicable to various matrices.
Proposes a new method to approximate Bayesian predictive uncertainty.
problem Bayesian uncertainty quantification in model predictions.
method Self-supervised learning approach to approximate posterior predictive distribution.
result SSLA and ASSLA outperform classical Laplace approximations in predictive calibration.
Proposes a new tensor decomposition method for functional temporal data with adaptive complexity.
problem Challenges in temporal tensor decomposition for general tensor data with continuous indexes.
method Encodes continuous spatial indexes as learnable Fourier features and uses neural ODEs for temporal trajectories. Introduces a sparsity-inducing prior for complexity adaptation.
result Significantly outperforms existing methods in prediction performance and robustness against noise.
Paper presents a method to accurately quantify neural network uncertainty without sampling.
problem Uncertainty quantification in neural networks for reliability and robustness.
method Sample-free moment propagation technique for mean vectors and covariance matrices.
result Analytic solution for covariance of nonlinear activation functions.
Paper proposes BQNs for efficient Bayesian quantized neural networks.
problem Learning with well-calibrated uncertainty in neural networks.
method Bayesian quantized networks (BQNs) with efficient algorithms for learning and prediction without sampling.
result BQNs achieve lower predictive errors and better-calibrated uncertainties than E-QNN with less than 20% negative log-likelihood.
Labeling training data is one of the most costly bottlenecks in developing machine learning-based applications. We present a first-of-its-kind study showing how existing knowledge resources from across an organization can be used as weak supervision in order to bring development time and cost down by an order of magnit…
Unified framework for self-supervised learning via latent distribution matching.
problem Lack of a unifying theoretical framework for diverse SSL methods.
method Casting SSL as latent distribution matching (LDM): maximizing alignment and uniformity.
result Derives a Bayesian filtering model and proves identifiable latent representations.
Neural backdoor attack is emerging as a severe security threat to deep learning, while the capability of existing defense methods is limited, especially for complex backdoor triggers. In the work, we explore the space formed by the pixel values of all possible backdoor triggers. An original trigger used by an attacker …
New method uses TT approximations to solve HJB equations for efficient sampling.
problem Efficiently sampling from complex probability densities.
method Direct time integration of HJB equations using Tensor Train compression.
result Sample-free, dimensionality-avoiding integration method.
Inference models are a key component in scaling variational inference to deep latent variable models, most notably as encoder networks in variational auto-encoders (VAEs). By replacing conventional optimization-based inference with a learned model, inference is amortized over data examples and therefore more computatio…
Approximate probabilistic inference algorithms are central to many fields. Examples include sequential Monte Carlo inference in robotics, variational inference in machine learning, and Markov chain Monte Carlo inference in statistics. A key problem faced by practitioners is measuring the accuracy of an approximate infe…
This work frames active inference through control as inference, offering robust control algorithms.
problem Active inference framework lacks practical sensorimotor control algorithms.
method Frame active inference through control as inference, presenting trajectory optimization as inference.
result AI may be framed as partially-observed CaI when the cost function is defined in observation states.
Simformer uses transformer models to perform flexible Bayesian inference.
problem Current simulation-based inference methods are inflexible and require fixed priors.
method Trains a probabilistic diffusion model with transformer architectures.
result Outperforms state-of-the-art methods on various benchmarks.
PE-SVI reduces SVI inference complexity by finding a suitable start point.
problem Complex posterior inference in graphical models leads to suboptimal learning.
method PE-SVI uses a pseudo-encoded start point to reduce gradient steps and step sizes.
result PE-SVI achieves the same ELBo objective as SVI with less than 1% of the required steps.
Improved Bayesian inference for neuronal ensemble inference reduces computational cost.
problem Efficient inference of neuronal ensembles from activity data.
method Modified MCMC algorithm with simulated annealing for hyperparameter control.
result Our method reduces computational cost while maintaining or improving inference accuracy.
Adding metadata abruptly changes network inference outcomes.
problem Understanding the impact of metadata on network inference.
method Investigated the effect of metadata on network inference problems.
result Metadata causes abrupt transitions in inference outcomes.
SNVI combines likelihood estimation with variational inference for efficient Bayesian inference.
problem Bayesian inference in models with intractable likelihoods.
method Sequential Neural Variational Inference (SNVI) that combines likelihood-estimation with variational inference.
result SNVI is more computationally efficient than previous algorithms without sacrificing accuracy.
An Easy to Interpret Diagnostic for Approximate Inference: Symmetric Divergence Over Simulationscs.LG
Paper introduces a diagnostic for approximate inference methods.
problem Estimating errors in probabilistic inference algorithms, especially for approximate methods.
method Repeatedly simulate datasets from the prior and perform inference on each, estimating a symmetric KL-divergence.
result A diagnostic for approximate inference methods can be estimated using symmetric KL-divergence.
Probabilistic inference procedures are usually coded painstakingly from scratch, for each target model and each inference algorithm. We reduce this effort by generating inference procedures from models automatically. We make this code generation modular by decomposing inference algorithms into reusable program-to-progr…
Recent efforts on combining deep models with probabilistic graphical models are promising in providing flexible models that are also easy to interpret. We propose a variational message-passing algorithm for variational inference in such models. We make three contributions. First, we propose structured inference network…
A new method for safer statistical inference after predictions.
problem Statistical inference with pseudo-outcomes from machine learning predictions.
method Prediction De-Correlated Inference (PDC) framework.
result PDC consistently outperforms supervised methods and can adapt to any model.
Bayesian interpolants explain neural network inferences concisely.
problem Understanding neural network inferences.
method Adapting Craig interpolants for neural networks.
result Produces precise, understandable explanations.
Variational inference provides a powerful tool for approximate probabilistic in- ference on complex, structured models. Typical variational inference methods, however, require to use inference networks with computationally tractable proba- bility density functions. This largely limits the design and implementation of v…
New comparison shows differences in how value is incorporated in AIF and CAI.
problem Clarifying the relationship between Active Inference and Control-as-Inference.
method Formal comparison of AIF and CAI frameworks.
result Primary difference is how value is incorporated into generative models.
Paper shows how to infer hidden states in neural networks analytically.
problem Intractability of Bayesian inference for neural networks.
method Leverage tractable approximate Gaussian inference (TAGI) for hidden states inference.
result Demonstrates inference of hidden states through constraints for various applications.
Variational Inference shows promise for Bayesian GARCH model estimation.
problem Bayesian estimation of GARCH-family models using Monte Carlo sampling.
method Variational Inference as an alternative to Monte Carlo sampling.
result Variational Inference is a reliable and competitive method for Bayesian learning in GARCH-like models.
Post-ADC inference corrects bias in statistical inference after active data collection.
problem Bias in inference after active data collection.
method Post-ADC inference framework that corrects bias from both ADC process and data-driven target construction.
result Valid inference for data collected by SMBO methods like GP-UCB and TPE.
A new particle algorithm improves mean-field variational inference.
problem Efficiently approximating nonparametric posterior distributions in machine learning.
method Introduces PArticle VI (PAVI), a novel particle-based algorithm for nonparametric mean-field approximation.
result Obtains non-asymptotic error bounds for PArticle VI, providing the first end-to-end guarantee for particle-based MFVI.
Meta-learn Bayesian inference for task-specific BNNs using amortised inference.
problem Efficiently learning Bayesian inference for small-scale probabilistic meta-learning.
method Replace global inducing points with actual data to create a set of approximate likelihoods, train a meta-model to learn these parameters across related datasets.
result Meta-learned inference can be applied to task-specific BNNs, improving efficiency and scalability.
Bayesian method infers contextual bandit policies robustly.
problem Inference of contextual bandit policies in small sample sizes.
method Empirical likelihood for Bayesian inference.
result Accurate uncertainty measurements and policy comparison.
Exact selective inference with randomization for Gaussian regression models.
problem Exact selective inference in Gaussian regression models.
method Introduces a pivot for exact selective inference with randomization, reducing the problem to a bivariate truncated Gaussian distribution.
result Our pivot leads to exact inference and produces narrower confidence intervals than related methods.
Recent work used importance sampling ideas for better variational bounds on likelihoods. We clarify the applicability of these ideas to pure probabilistic inference, by showing the resulting Importance Weighted Variational Inference (IWVI) technique is an instance of augmented variational inference, thus identifying th…
BOED improves SBI by optimizing experimental designs and inference functions.
problem Efficiently use experimental resources for better inference on complex models.
method Link mutual information bounds between SBI and BOED, optimizing both design and inference.
result BOED improves inference in real-world simulators in epidemiology and biology.
Prediction-Powered Inferencestat.ML
Valid inference from data and predictions.
problem Valid statistical inference with machine learning predictions.
method Framework for valid inference using machine learning predictions.
result Valid confidence intervals without assumptions on predictions.
EFI automates statistical inference for big data.
problem Statistical inference for model parameters based on observations.
method EFI uses stochastic gradient Markov chain Monte Carlo and sparse deep neural networks.
result EFI provides higher fidelity in parameter estimation and automates the inference process.
Derives time-averaged active inference from control principles.
problem Finite-horizon or discounted-surprise problems in active inference.
method Derives infinite-horizon, average-surprise active inference from optimal control principles.
result Unified objective functional for sensorimotor control.