Early stopping methods reduce unnecessary reasoning steps in LLMs by monitoring uncertainty signals.
problem LLMs sometimes generate unnecessary reasoning steps, especially under uncertainty.
method Statistically principled early stopping methods that monitor uncertainty signals during generation.
result Uncertainty-aware early stopping improves efficiency and reliability in LLM reasoning, especially in math reasoning.
CoT-UQ improves LLM uncertainty quantification by integrating reasoning steps.
problem LLMs' overconfidence and lack of response-wise uncertainty quantification.
method Integrates LLMs' reasoning steps into uncertainty estimation.
result Significantly improves uncertainty quantification accuracy (5.9% AUROC improvement).
Neurosymbolic predictors fail to model uncertainty under independence assumption.
problem Neurosymbolic predictors' reliance on independence assumption limits their ability to model uncertainty.
method Formal analysis of NeSy predictors under independence assumption.
result Assuming independence among symbolic concepts prevents NeSy predictors from representing uncertainty.
Paper refines royalty determination using Bayesian methods.
problem Determining a reasonable royalty with risk and uncertainty.
method Bayesian Cost approach to refine Nash Bargaining Solution.
result Nash Bargaining Solution emerges as more reliable.
Bayesian Neural Networks improve uncertainty reasoning in NNs.
problem Frequentist implementation of NNs cannot reason about uncertainty in predictions.
method Introduces Bayesian Neural Networks and compares approximate inference methods.
result Future research can improve on current methods of inference.
A conformal procedure improves CoT reasoning by aggregating reasoning paths and calibrating abstention rules.
problem Aggregation uncertainty in chain-of-thought reasoning makes correct answers less reliable.
method Introduces a conformal procedure for CoT reasoning that uses weighted score aggregation and abstention rules.
result Achieves higher selective accuracy with abstention, reducing confident-error rate.
The paper outlines future work in random sets theory.
problem Developing a theory of statistical reasoning with random sets.
method Generalizing logistic regression, probability laws, and geometric uncertainty.
result A new geometric approach to uncertainty with general random sets.
Single-pass method estimates neural network uncertainty.
problem Uncertainty estimation in deep learning requires multiple passes.
method Probabilistic reasoning over neural network depths.
result Single forward pass for uncertainty estimation.
New method extracts aleatoric and epistemic uncertainties from regression-based neural networks.
problem Need for principled uncertainty reasoning in machine learning systems.
method Learning evidential distributions for aleatoric and epistemic uncertainties.
result Allows for the simultaneous extraction of both uncertainties without sampling or out-of-distribution data.
uGMM-NN integrates probabilistic reasoning into neural networks.
problem Capturing multimodality and uncertainty in neural network activations.
method Parameterizes activations as univariate Gaussian mixtures with learnable parameters.
result Competitive discriminative performance with probabilistic activations.
LiveTradeBench evaluates LLMs in live trading environments.
problem Static benchmarks fail to assess real-world trading ability.
method Live data streaming, portfolio management abstraction, multi-market evaluation.
result LLMs show distinct portfolio styles and adapt to live signals.
Paper introduces MCRP for estimating feature relevance uncertainty in neural networks.
problem Lack of uncertainty in feature relevance for neural network decisions.
method Monte Carlo Relevance Propagation (MCRP) method.
result Allows deeper understanding of neural networks' perception and reasoning.
CROP verifies clean prefixes in reasoning traces, improving downstream repair accuracy.
problem Uncertainty in reasoning traces prevents full certification of entire responses.
method CROP selects a calibrated threshold to certify the longest prefix with low risk proxies.
result CROP improves downstream repair accuracy by preserving valid reasoning and discarding misleading suffixes.
Sequence models quantify uncertainty over latent concepts.
problem Quantifying uncertainty in latent environments.
method Exchangeable sequence models, equivalent to empirical Bayes and posterior inference.
result Sequence prediction loss controls uncertainty quantification.
New research challenges the independence assumption in neurosymbolic learning, leading to overconfident predictions and unrepresentable uncertainty.
problem The independence assumption in neurosymbolic learning systems can lead to overconfident predictions and hinder uncertainty quantification.
method The study proves the limitations of the independence assumption and introduces new loss functions that are non-convex and difficult to optimise.
result Neurosymbolic learning systems using the independence assumption are prone to overconfidence and cannot represent uncertainty over multiple valid options.
The paper extends explainability methods to uncertainty-aware models, revealing feature impacts on predictive entropy and likelihood.
problem Understanding the factors contributing to uncertainty in probabilistic models.
method Adapting permutation feature importance, partial dependence plots, and individual conditional expectation plots to measure feature impacts on predictive entropy and likelihood.
result Novel insights into model behaviour and feature impacts on uncertainty are obtained.
Study improves AI's handling of uncertainty.
problem Uncertainty in AI models, especially with limited data.
method Integrates theories, latest developments, and practical applications.
result Novel definition of total uncertainty in AI.
Calibrated PRMs improve inference efficiency for LLMs by dynamically adjusting compute budgets.
problem Poor calibration of PRMs leads to overestimation of success probabilities in partial reasoning steps.
method Quantile regression for calibration, instance-adaptive scaling (IAS) framework.
result Calibrated PRMs reduce inference costs while maintaining accuracy, especially on confident problems.
Novel approach trains LLMs for inductive reasoning using probabilistic programs.
problem Training LLMs for inductive reasoning with sparse, ambiguous data.
method Program-based Posterior Training (PPT) using probabilistic inference.
result Significant improvement in estimation accuracy and alignment with human judgments.
Unified framework for hierarchical image classification with epistemic uncertainty.
problem Overconfident predictions and lack of logical consistency in deep learning models.
method Neurosymbolic approach with epistemic deep learning, using focal set reasoning and differentiable fuzzy logic.
result Maintains accuracy on par with transformer baselines while providing more calibrated and interpretable predictions.
Work proposes a new framework to improve uncertainty estimation in deep Bayesian models.
problem Traditional training procedures underestimate uncertainty in NLMs, leading to unreliable predictions.
method Introduces a novel training framework that captures useful predictive uncertainties for out-of-distribution inputs.
result Demonstrates that traditional methods for NLMs significantly underestimate uncertainty and propose a new framework to address this issue.
New method improves uncertainty estimation in Bayesian deep learning models.
problem Underestimation of predictive uncertainty in Neural Linear Models (NLMs).
method Proposes a novel training method to capture useful predictive uncertainties and incorporate domain knowledge.
result Traditional training procedures for NLMs can drastically underestimate uncertainty in data-scarce regions.
RACER optimizes LLM-as-judge accuracy with dynamic reasoning selection.
problem Balancing reasoning accuracy with computational cost in LLM-as-judge settings.
method Formulates routing as a constrained distributionally robust optimization problem, accounting for distribution shift via KL-divergence uncertainty set.
result RACER achieves superior accuracy-cost trade-offs under distribution shift.
TRUST improves structure learning with tractable uncertainty.
problem Capturing uncertainty in structure learning for causal DAGs.
method Probabilistic circuits for posterior inference.
result Probabilistic circuits enhance structure learning quality and uncertainty.
DER uses neural nets to better handle uncertainty in machine learning.
problem Need for principled uncertainty reasoning in safety-critical domains.
method Uncertainty-aware regression-based neural networks (NNs) with evidential distributions.
result DER shows promise over traditional methods but is a heuristic.
Paper proposes risk-averse reinforcement learning algorithms.
problem Managing model uncertainty in reinforcement learning.
method Entropic risk constrained policy gradient and actor-critic algorithms.
result Demonstrates usefulness of risk-averse algorithms on various domains.
Softmax confidence misrepresents uncertainty in neural networks.
problem Neural networks fail to increase uncertainty on out-of-distribution data.
method Investigates two implicit biases in softmax confidence.
result Softmax confidence correlates with epistemic uncertainty due to decision boundary structure and deep network filtering.
Paper introduces variance-based measures for second-order uncertainty quantification in classification problems.
problem Uncertainty in machine learning predictions and decision-making.
method Second-order uncertainty quantification using variance-based measures.
result Variance-based measures effectively quantify uncertainty on a class-based level and are competitive with entropy-based measures.
UACQR improves CQR by separating aleatoric and epistemic uncertainties.
problem Ineffective CQR for problems with varying quantile regressor performance.
method Integrates aleatoric and epistemic uncertainties in CQR.
result UACQR provides stronger conditional coverage in simulated and real-world data.
Trains neural nets for gamma hedging with model uncertainty.
problem Gamma hedging with model mismatch.
method Trains neural networks using loss functions that reward model uncertainty.
result Networks can learn optimal gamma hedging even with model mismatch.
Framework disentangles deep feature uncertainty for efficient inference.
problem Inference-time uncertainty estimation for reliable decision-making.
method Uncertainty-Guided Inference-Time Selection framework.
result Significantly tighter prediction intervals and 60% compute reduction.
Methods for reasoning under uncertainty are a key building block of accurate and reliable machine learning systems. Bayesian methods provide a general framework to quantify uncertainty. However, because of model misspecification and the use of approximate inference, Bayesian uncertainty estimates are often inaccurate -…
Uncertainty quantification for deep learning is a challenging open problem. Bayesian statistics offer a mathematically grounded framework to reason about uncertainties; however, approximate posteriors for modern neural networks still require prohibitive computational costs. We propose a family of algorithms which split…
This work provides uncertainty intervals for semantic latent variables in disentangled latent spaces.
problem Challenges in providing meaningful uncertainty quantification for semantic information in disentangled latent spaces.
method Uses quantile regression to output heuristic uncertainty intervals, calibrates these intervals to contain true latent values, and propagates them through the generator.
result Reliably communicates semantically meaningful, principled, and instance-adaptive uncertainty in image super-resolution and image completion.
Cryptocurrencies have heavy-tailed return distributions, requiring diversification.
problem Cryptocurrency returns do not follow Gaussian distributions.
method Applied econophysics and entropy measures to analyze returns.
result Portfolio diversification reduces return uncertainty.
Learning probability distributions on the weights of neural networks (NNs) has recently proven beneficial in many applications. Bayesian methods, such as Stein variational gradient descent (SVGD), offer an elegant framework to reason about NN model uncertainty. However, by assuming independent Gaussian priors for the i…
Proposes a simple method to explain aleatoric uncertainty in neural networks.
problem Lack of transparent explanations for uncertainty estimates in AI models.
method Adapting a neural network with Gaussian output to estimate predictive variance and applying explainers to the variance output.
result The proposed method explains uncertainty more reliably than complex approaches and outperforms them in most settings.
The paper critiques existing uncertainty concepts and proposes a new decision-theoretic approach.
problem Incoherence in existing discussions of aleatoric and epistemic uncertainty.
method Decision-theoretic perspective that relates uncertainty, predictive performance, and statistical dispersion.
result Popular information-theoretic quantities can be poor estimators but still useful for guiding data acquisition.
FinZero improves financial time series forecasting accuracy with multimodal modeling.
problem Lack of interpretability, uncertainty, and scalability in financial time series forecasting.
method Developed a multimodal pre-trained model FinZero using UARPO method for reasoning, prediction, and uncertainty analysis.
result FinZero achieves an approximate 13.48% improvement in prediction accuracy over GPT-4o in high-confidence group.
Proposes a Bayesian approach to explain, justify, and quantify uncertainty in DNNs.
problem Lack of transparency and confidence in DNNs for critical applications.
method Bayesian approach to extract explanations, justifications, and uncertainty estimates from black box DNNs.
result Improves interpretability and reliability of DNNs, validated on CIFAR-10.
ManifoldMind uses adaptive-curvature probabilistic spheres for trustworthy recommendations in semantic hierarchies.
problem Sparse and abstract recommendation domains where users explore diverse conceptual paths.
method Adaptive-curvature probabilistic spheres, soft multi-hop inference, and curvature-aware semantic kernel.
result Superior NDCG, calibration, and diversity compared to baselines on public benchmarks.
Improved measure of predictive uncertainty for machine learning models.
problem Current measure of predictive uncertainty assumes BMA predictive distribution is equivalent to true model's distribution.
method Introduced a new measure based on information theory to correct the assumption.
result Our measure behaves more reasonably in synthetic tasks and is advantageous in real-world applications.
New model reconstructs flow from sparse data with uncertainty quantification.
problem Reconstructing nonlinear flow from limited observations.
method Semi-Conditional Variational Autoencoder (SCVAE) for probabilistic flow reconstruction.
result SCVAE improves reconstruction accuracy compared to Gappy Proper Orthogonal Decomposition (GPOD).
Deep learning tools have gained tremendous attention in applied machine learning. However such tools for regression and classification do not capture model uncertainty. In comparison, Bayesian models offer a mathematically grounded framework to reason about model uncertainty, but usually come with a prohibitive computa…
UAG defends GNNs against adversarial attacks by quantifying and explaining uncertainties.
problem Lack of uncertainty quantification in GNNs makes them vulnerable to adversarial attacks.
method UAG uses Bayesian Uncertainty Technique (BUT) and Uncertainty-aware Attention Technique (UAT).
result UAG outperforms state-of-the-art solutions in defending adversarial attacks on GNNs.
UnKGCP generates prediction intervals for uncertain knowledge graphs with statistical guarantees.
problem Lack of quantified predictive uncertainty in existing UnKGE methods.
method Proposes extsc{UnKGCP} framework using conformal prediction with a novel nonconformity measure.
result Sharp prediction intervals effectively capture predictive uncertainty in diverse UnKGE methods.
Deep neural networks have achieved impressive results on a wide variety of tasks. However, quantifying uncertainty in the network's output is a challenging task. Bayesian models offer a mathematical framework to reason about model uncertainty. Variational methods have been used for approximating intractable integrals t…
An Artificial Intelligence (AI) system is an autonomous system which emulates human mental and physical activities such as Observe, Orient, Decide, and Act, called the OODA process. An AI system performing the OODA process requires a semantically rich representation to handle a complex real world situation and ability …