Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

55109164218 · Jun 202019922001200920172026
48 results for Bayesian reasoning

Bayesian framework integrates spectral deconvolution with expert reasoning for robust peak estimation.

problem Challenges in extracting meaningful peaks from noisy or complex spectra.
method Bayesian spectral deconvolution coupled with a physical-property regression layer.
result Recovery of weak peaks in poly(lactic acid) IR spectra related to degradation rates.

Transformers mimic Bayesian reasoning in controlled settings, revealing geometric mechanisms.

problem Verifying if transformers perform Bayesian reasoning rigorously in natural data.
method Constructing Bayesian wind tunnels with known posteriors and proving memorization impossibility.
result Transformers achieve 10310^{-3}-10410^{-4} bit accuracy in Bayesian posteriors, while MLPs fail.

R2-B2 optimizes game interactions with recursive reasoning.

problem Optimizing interactions between boundedly rational agents with unknown payoff functions.
method Recursive Reasoning-Based Bayesian Optimization (R2-B2) for repeated games.
result R2-B2 achieves faster asymptotic convergence to no regret than non-recursive methods.

The paper explores how to evaluate Bayesian approximations in neural networks.

problem The difficulty in evaluating Bayesian approximations in neural networks.
method Exploring the interactions between probabilistic models, approximating distributions, optimization algorithms, and datasets.
result The expected utility of the approximate posterior can measure inference quality.

In this work we investigate the reasons why Batch Normalization (BN) improves the generalization performance of deep networks. We argue that one major reason, distinguishing it from data-independent normalization methods, is randomness of batch statistics. This randomness appears in the parameters rather than in activa…

2018-11-01abs ↗pdf ↗

Proposes a new prior for complex models to improve prediction accuracy.

problem Difficulty in specifying priors for complex models like neural networks.
method Predictive complexity priors defined by comparing model predictions to a reference model, transferred to parameters via change of variables.
result Improves model predictions by reducing unintuitive effects of traditional priors.

Bayesian RL enhances LLMs to reflectively explore and correct errors.

problem LLMs trained via RL lack reflective behaviors like rethinking and error correction.
method Bayesian RL framework that optimizes expected return under posterior distribution over Markov decision processes.
result BARL algorithm improves LLM performance in reasoning tasks.

Bayesian framework explains diverse explanatory values.

problem Understanding and predicting human preferences for explanations.
method Developed a Bayesian account to integrate various explanatory values.
result Core values from psychology, statistics, and philosophy emerge from a common framework.

We showed how to use trained neural networks to perform Bayesian reasoning in order to solve tasks outside their initial scope. Deep generative models provide prior knowledge, and classification/regression networks impose constraints. The tasks at hand were formulated as Bayesian inference problems, which we approximat…

2020-01-29abs ↗pdf ↗

Bayesian network classifiers are used in many fields, and one common class of classifiers are naive Bayes classifiers. In this paper, we introduce an approach for reasoning about Bayesian network classifiers in which we explicitly convert them into Ordered Decision Diagrams (ODDs), which are then used to reason about t…

2012-10-19abs ↗pdf ↗

We review three algorithms for Latent Dirichlet Allocation (LDA). Two of them are variational inference algorithms: Variational Bayesian inference and Online Variational Bayesian inference and one is Markov Chain Monte Carlo (MCMC) algorithm -- Collapsed Gibbs sampling. We compare their time complexity and performance.…

2013-07-01abs ↗pdf ↗

We draw a formal connection between using synthetic training data to optimize neural network parameters and approximate, Bayesian, model-based reasoning. In particular, training a neural network using synthetic data can be viewed as learning a proposal distribution generator for approximate inference in the synthetic-d…

2017-03-02abs ↗pdf ↗

Recent reports have described that learning Bayesian networks are highly sensitive to the chosen equivalent sample size (ESS) in the Bayesian Dirichlet equivalence uniform (BDeu). This sensitivity often engenders some unstable or undesirable results. This paper describes some asymptotic analyses of BDeu to explain the …

2012-02-14abs ↗pdf ↗

The paper optimizes LLM accuracy by stopping early based on consistent answers.

problem Improving LLM accuracy in math and reasoning problems.
method Bayesian stopping policy to save on sampling costs, tracking only the L-1 most frequent answer counts.
result The L=3 stopping policy is sufficient for asymptotic optimality and significantly reduces inference costs.

We show that a neural network with arbitrary depth and non-linearities, with dropout applied before every weight layer, is mathematically equivalent to an approximation to a well known Bayesian model. This interpretation might offer an explanation to some of dropout's key properties, such as its robustness to over-fitt…

2015-06-06abs ↗pdf ↗

Bayesian methods enhance deep learning models by improving reliability and uncertainty.

problem Improving reliability and uncertainty awareness in deep learning models.
method Approximate Bayesian inference techniques, including SG-MCMC and VI, applied to deep learning models.
result Enhanced posterior inference for deep learning models, particularly in neural networks and generative models.

New analysis shows how cross-entropy training shapes attention in transformers.

problem Understanding how gradient-based learning creates the required internal geometry in transformers.
method Developed a first-order analysis of cross-entropy training effects on attention scores and values in a transformer attention head.
result Introduced an advantage-based routing law and responsibility-weighted update for attention scores and values, respectively.

Deep neural networks have achieved impressive results on a wide variety of tasks. However, quantifying uncertainty in the network's output is a challenging task. Bayesian models offer a mathematical framework to reason about model uncertainty. Variational methods have been used for approximating intractable integrals t…

2019-05-26abs ↗pdf ↗

Work proposes a new framework to improve uncertainty estimation in deep Bayesian models.

problem Traditional training procedures underestimate uncertainty in NLMs, leading to unreliable predictions.
method Introduces a novel training framework that captures useful predictive uncertainties for out-of-distribution inputs.
result Demonstrates that traditional methods for NLMs significantly underestimate uncertainty and propose a new framework to address this issue.

Bayesian methods for machine learning have been widely investigated, yielding principled methods for incorporating prior information into inference algorithms. In this survey, we provide an in-depth review of the role of Bayesian methods for the reinforcement learning (RL) paradigm. The major incentives for incorporati…

2016-09-14abs ↗pdf ↗

Dropout has recently emerged as a powerful and simple method for training neural networks preventing co-adaptation by stochastically omitting neurons. Dropout is currently not grounded in explicit modelling assumptions which so far has precluded its adoption in Bayesian modelling. Using Bayesian entropic reasoning we s…

2015-08-12abs ↗pdf ↗

Bayesian neural networks benefit from fully marginalizing over all modes to improve generalization.

problem Bayesian neural networks suffer from multimodal posterior distributions that can lead to suboptimal generalization.
method Use appropriate Bayesian sampling tools to fully marginalize over all posterior modes.
result Training with full marginalization improves the ability of the network to reason between multiple candidate solutions.

ABCI infers causal models and queries simultaneously using Bayesian active learning.

problem Inference of causal models and effects in a two-stage process is inefficient and unnatural.
method Active Bayesian Causal Inference (ABCI) using Gaussian processes for sequentially designing experiments.
result ABCI is more data-efficient and accurate in learning causal queries from fewer samples.

This work analyzes CoT prompting methods from a statistical estimation perspective.

problem Improving the effectiveness of LLMs in solving multi-step reasoning problems.
method Introducing a multi-step latent variable model to characterize CoT prompting from a statistical estimation viewpoint.
result The CoT estimator is equivalent to a Bayesian estimator when the pretraining dataset is large.

New method improves uncertainty estimation in Bayesian deep learning models.

problem Underestimation of predictive uncertainty in Neural Linear Models (NLMs).
method Proposes a novel training method to capture useful predictive uncertainties and incorporate domain knowledge.
result Traditional training procedures for NLMs can drastically underestimate uncertainty in data-scarce regions.

New method calibrates LLMs for safety-critical tasks with scalable Bayesian inference.

problem Overconfidence in LLMs after fine-tuning for specific tasks.
method Orthogonalized Low-Rank Adapters (PoLAR) with variational Bayesian inference.
result Scalable and well-calibrated uncertainty estimation for LLMs.

Unified Bayesian-AI framework improves epidemiological risk prediction and uncertainty quantification.

problem Lack of calibrated uncertainty in machine learning models for epidemiology.
method Combines Bayesian prediction with Bayesian hyperparameter optimization using logistic regression and Gaussian-process Bayesian optimization.
result Unified Bayesian-AI framework provides reliable coverage and improved calibration, enhancing epidemiological decision making.

In many fields observations are performed irregularly along time, due to either measurement limitations or lack of a constant immanent rate. While discrete-time Markov models (as Dynamic Bayesian Networks) introduce either inefficient computation or an information loss to reasoning about such processes, continuous-time…

2012-03-15abs ↗pdf ↗

Bayesian optimization enhanced with conformal prediction for better outcome reliability.

problem Uncertainty and model misspecification in Bayesian optimization.
method Conformal prediction to provide coverage guarantees and Bayesian optimization to select queries.
result Significant improvement in query coverage without sacrificing sample-efficiency.

We present a new method to approximate posterior probabilities of Bayesian Network using Deep Neural Network. Experiment results on several public Bayesian Network datasets shows that Deep Neural Network is capable of learning joint probability distri- bution of Bayesian Network by learning from a few observation and p…

2017-12-31abs ↗pdf ↗

Bayesian approach models neurodegenerative diseases without clinical labels.

problem Personalized, predictive modeling of neurodegenerative diseases.
method Probabilistic programmed deep kernel learning combining Gaussian processes and neural networks.
result Surpasses deep learning in accuracy and timeliness of predicting neurodegeneration.

The paper introduces a method to probabilistically select inducing points in sparse Gaussian processes.

problem The challenge is selecting the optimal number of inducing points in sparse Gaussian processes.
method A point process prior is applied to the inducing points, and the posterior is approximated using stochastic variational inference.
result The model learns which and how many inducing points to use, leading to fewer inducing points being preferred as they become less informative.

Methods for reasoning under uncertainty are a key building block of accurate and reliable machine learning systems. Bayesian methods provide a general framework to quantify uncertainty. However, because of model misspecification and the use of approximate inference, Bayesian uncertainty estimates are often inaccurate -…

2018-07-01abs ↗pdf ↗

Bayesian Additive Regression Trees (BART) is a fully Bayesian approach to modeling with ensembles of trees. BART can uncover complex regression functions with high dimensional regressors in a fairly automatic way and provide Bayesian quantification of the uncertainty through the posterior. However, BART assumes IID nor…

2018-06-29abs ↗pdf ↗