Early stopping methods reduce unnecessary reasoning steps in LLMs by monitoring uncertainty signals.
problem LLMs sometimes generate unnecessary reasoning steps, especially under uncertainty.
method Statistically principled early stopping methods that monitor uncertainty signals during generation.
result Uncertainty-aware early stopping improves efficiency and reliability in LLM reasoning, especially in math reasoning.
Kernel smoothing improves LLM reasoning efficiency.
problem Efficiently estimate value functions with limited samples for reinforcement learning.
method Kernelized advantage estimation using classical nonparametric statistics.
result Improved policy optimization with accurate value and gradient estimation.
The paper outlines future work in random sets theory.
problem Developing a theory of statistical reasoning with random sets.
method Generalizing logistic regression, probability laws, and geometric uncertainty.
result A new geometric approach to uncertainty with general random sets.
This work analyzes CoT prompting methods from a statistical estimation perspective.
problem Improving the effectiveness of LLMs in solving multi-step reasoning problems.
method Introducing a multi-step latent variable model to characterize CoT prompting from a statistical estimation viewpoint.
result The CoT estimator is equivalent to a Bayesian estimator when the pretraining dataset is large.
CausAdv detects adversarial examples using causal reasoning.
problem Vulnerability of CNNs to adversarial perturbations.
method Causal framework based on counterfactual reasoning.
result Adversarial examples exhibit different CI distributions compared to clean samples.
New method uses model comparison signals to improve LLM evaluation accuracy.
problem Limited benchmark sizes and model stochasticity in evaluating LLMs' mathematical reasoning.
method Combines standard labeled outcomes with model comparison signals to design a statistically efficient evaluation framework.
result Semiparametric estimator achieves the semiparametric efficiency bound and substantially improves ranking accuracy.
Intensive algorithmic efforts have been made to enable the rapid improvements of certificated robustness for complex ML models recently. However, current robustness certification methods are only able to certify under a limited perturbation radius. Given that existing pure data-driven statistical approaches have reache…
FinCARE combines financial data and AI reasoning to improve causal analysis of financial performance.
problem Correlation-based analysis fails to capture true causal relationships in financial performance.
method Hybrid framework integrating causal discovery algorithms with financial domain knowledge from SEC filings and LLM reasoning.
result KG+LLM-enhanced methods improve causal discovery across PC, GES, and NOTEARS by 36-366%.
Private variable selection method controls FDR with simulations showing reasonable power.
problem Performing variable selection with privacy constraints.
method Private knockoff filter using Gaussian and Laplace mechanisms.
result Achieves controlled false discovery rate (FDR) in variable selection.
Reasoning models generate differently based on problem difficulty, not just length.
problem Understanding how reasoning models handle different problem difficulties.
method Examined hidden-state trajectories across competitive programming, mathematics, and Boolean satisfiability.
result Corrected trajectory geometry shows difficulty-dependent differences in reasoning models, with stronger effects in the code domain.
Benchmark assesses LLMs' causal inference skills, revealing significant limitations.
problem Lack of rigorous evaluation of LLMs' causal inference capabilities.
method CausalPitfalls benchmark with structured challenges and grading rubrics.
result Significant limitations in current LLMs' statistical causal inference.
COLEP improves robustness of conformal prediction via probabilistic circuits.
problem Adversarial perturbations can undermine the coverage guarantees of conformal prediction.
method COLEP uses probabilistic circuits to learn and reason about different semantic concepts, providing certifiable coverage guarantees.
result COLEP achieves higher prediction coverage and accuracy than a single model, especially with non-trivial knowledge models.
We study the statistical behavior of reasoning probes in a stylized model of iterative computation inspired by neural algorithmic reasoning. The underlying computation is given by a looped Boolean circuit whose graph is a perfect ν-ary tree (ν≥2), with outputs recursively fed back as inputs across computation ro…
Paper cleans option price datasets by removing outliers.
problem Unusual option prices in datasets.
method Statistical techniques to identify and remove outliers.
result Removes option prices violating no arbitrage assumption.
AGENTICAITA uses AI agents to autonomously trade markets without human intervention.
problem Inability of traditional trading systems to adapt to market complexity.
method Introduces an agentic AI framework with specialized LLM agents reasoning, negotiating, and acting.
result Demonstrated operational correctness and non-trivial inter-agent negotiation in live market conditions.
Paper refines royalty determination using Bayesian methods.
problem Determining a reasonable royalty with risk and uncertainty.
method Bayesian Cost approach to refine Nash Bargaining Solution.
result Nash Bargaining Solution emerges as more reliable.
Statistical tests for fairness in admissions data reveal hidden patterns.
problem Simpson's paradox in admissions data hides true gender bias.
method Introduces a new statistical test based on Pearl's instrumental-variable inequalities.
result Statistical tests for fairness coincide with causal notions for the Berkeley admissions case.
The problem of replicating the flexibility of human common-sense reasoning has captured the imagination of computer scientists since the early days of Alan Turing's foundational work on computation and the philosophy of artificial intelligence. In the intervening years, the idea of cognition as computation has emerged …
CROP verifies clean prefixes in reasoning traces, improving downstream repair accuracy.
problem Uncertainty in reasoning traces prevents full certification of entire responses.
method CROP selects a calibrated threshold to certify the longest prefix with low risk proxies.
result CROP improves downstream repair accuracy by preserving valid reasoning and discarding misleading suffixes.
We extend Kirman's model by introducing variable event time scale. The proposed flexible time scale is equivalent to the variable trading activity observed in financial markets. Stochastic version of the extended Kirman's agent based model is compared to the non-linear stochastic models of long-range memory in financia…
Bayesian interpolants explain neural network inferences concisely.
problem Understanding neural network inferences.
method Adapting Craig interpolants for neural networks.
result Produces precise, understandable explanations.
This paper improves learning complex functions with CoT supervision, reducing sample complexity.
problem Learning complex functions with multi-step reasoning.
method Develops a statistical theory linking CoT risk and end-to-end risk, using CoT information measure.
result CoT supervision can achieve significantly faster learning rates compared to standard E2E supervision.
The book explores essential stats and psychology for quantitative trading.
problem Developing a quantitative trading system.
method Logical progression through articles on statistics, quantitative trading, and psychology.
result Essential elements for quantitative trading systems.
Enhanced TSFMs improve time series forecasting accuracy and reliability.
problem Variance, bias, and uncertainty in TSFMs' predictions on real data.
method Statistical and ensemble techniques including bagging, stacking, residual modeling, and prediction intervals.
result Hybrid models consistently outperform standalone TSFMs across multiple horizons.
Whether neural networks can learn abstract reasoning or whether they merely rely on superficial statistics is a topic of recent debate. Here, we propose a dataset and challenge designed to probe abstract reasoning, inspired by a well-known human IQ test. To succeed at this challenge, models must cope with various gener…
Unified framework for certifying LLM reliability without extra supervision.
problem Improving reliability of large language models without additional supervision.
method Unified framework using majority voting and Martingale Majority Certificate (MMC).
result Certifiable inference in LLMs with statistical guarantees and adaptive stopping rules.
This work clarifies different transport map constructions and their causal interpretations.
problem Identifying distinct transport map constructions and their equivalence.
method Comparative analysis of three transport map constructions: cyclically monotone, quantile-preserving, and triangular monotone.
result Conditions for equivalence of different transport map constructions.
Proposes a new prior for complex models to improve prediction accuracy.
problem Difficulty in specifying priors for complex models like neural networks.
method Predictive complexity priors defined by comparing model predictions to a reference model, transferred to parameters via change of variables.
result Improves model predictions by reducing unintuitive effects of traditional priors.
Tensor logic aims to unify AI types with scalable and transparent features.
problem Lack of a unified AI programming language with scalability and transparency.
method Introduces tensor logic, a new AI language based on tensor equations.
result Tensor logic enables key AI forms like transformers, formal reasoning, and graphical models.
In this work we investigate the reasons why Batch Normalization (BN) improves the generalization performance of deep networks. We argue that one major reason, distinguishing it from data-independent normalization methods, is randomness of batch statistics. This randomness appears in the parameters rather than in activa…
New causal versions of MaxEnt and PIR avoid paradoxical probability updates.
problem Paradoxical probability updates in causal MaxEnt and PIR.
method Separate constraints into cause-specific and mechanism-specific restrictions.
result Causal MaxEnt avoids paradoxical updates and aligns with Information Geometric Causal Inference.
New language model shows context length impacts generation quality and reasoning ability.
problem Analyzing the impact of context length and reasoning on autoregressive generation.
method Introduced synthetic hierarchical languages, used an exact k-gram ansatz, derived asymptotic predictions, and validated empirically.
result Reasoning models with limited context can generate sequences from the true language, improving exponentially over standard models.
There is a mismatch between the standard theoretical analyses of statistical machine learning and how learning is used in practice. The foundational assumption supporting the theory is that we can represent features and models using real-valued parameters. In practice, however, we do not use real numbers at any point d…
Bayesian framework explains diverse explanatory values.
problem Understanding and predicting human preferences for explanations.
method Developed a Bayesian account to integrate various explanatory values.
result Core values from psychology, statistics, and philosophy emerge from a common framework.
We present the Bayesian Case Model (BCM), a general framework for Bayesian case-based reasoning (CBR) and prototype classification and clustering. BCM brings the intuitive power of CBR to a Bayesian generative framework. The BCM learns prototypes, the "quintessential" observations that best represent clusters in a data…
Active-GRPO improves molecular optimization by actively deciding when to imitate or self-improve.
problem Training robust and efficient molecular optimization models with large language models.
method Active-GRPO combines imitation and reinforcement learning, upgrading references and policies dynamically.
result Improves molecular optimization performance, achieving statistically significant gains.
Quant strategies lost during market selloff.
problem Why equities Statistical Arbitrage strategies failed during the COVID-19 market downturn.
method Explains strategies' normal performance and market regimes, discusses their limitations.
result Quant strategies are vulnerable to extreme market events.
The main object of Bayesian statistical inference is the determination of posterior distributions. Sometimes these laws are given for quantities devoid of empirical value. This serious drawback vanishes when one confines oneself to considering a finite horizon framework. However, assuming infinite exchangeability gives…
We analyze the relationships between game theory and quantum mechanics and the extensions to statistical physics and information theory. We use certain quantization relationships to assign quantum states to the strategies of a player. These quantum states are contained in a density operator which describes the new quan…
LASER compresses recursive model activations by exploiting their low-dimensional structure.
problem Understanding and optimizing the geometric structure of recursive reasoning trajectories.
method Dynamic low-rank basis tracking via matrix-free subspace tracking with a fidelity-triggered reset mechanism.
result Recursive activations occupy a linear, low-dimensional subspace that can be compressed efficiently.
Study improves understanding of why agentic theorem provers succeed.
problem Understanding which components of agentic theorem provers improve proof success.
method Statistical provability theory and finite-horizon reachability MDP model.
result Bounds provability gap and explains components' effectiveness.
Deep Learning has managed to push boundaries in a wide variety of tasks. One area of interest is to tackle problems in reasoning and understanding, with an aim to emulate human intelligence. In this work, we describe a deep learning model that addresses the reasoning task of question-answering on categorical plots. We …
We show that a large class of Estimation of Distribution Algorithms, including, but not limited to, Covariance Matrix Adaption, can be written as a Monte Carlo Expectation-Maximization algorithm, and as exact EM in the limit of infinite samples. Because EM sits on a rigorous statistical foundation and has been thorough…
A new method combines DNN with simpler models for better interpretability and accuracy.
problem Combining interpretability and accuracy in machine learning models.
method A mixture of linear models guided by deep neural networks.
result Improved interpretability without sacrificing accuracy.
A neural network approach unifies Lasso for variable selection.
problem Combining statistical and machine learning techniques for variable selection.
method Representing Lasso through a neural network and developing a new optimization algorithm.
result The new optimization algorithm achieves better performance than previous methods.
Inference-Time Scaling can be extended to domains prone to systematic failure using intrinsic statistics.
problem Scaling inference time in domains prone to systematic failure
method Intrinsic Selection (iS), Intrinsic Particle Filtering (iPF), and Particle Distillation (dPF)
result Intrinsic Selection improves engineering design selection by 20% and pass@1 by 6.1 points on average.
Adversarial consistency depends on the uniqueness of adversarial Bayes classifiers.
problem Consistency of adversarial surrogate losses is not guaranteed.
method Connected consistency of adversarial surrogate losses to the uniqueness of adversarial Bayes classifiers.
result A convex surrogate loss is statistically consistent for adversarial learning if and only if the adversarial Bayes classifier is unique.
The paper explores how market-based returns depend on past trade values.
problem Improving accuracy in forecasting market-based average and volatility of returns.
method Derives the dependence of market-based volatility and higher statistical moments of returns on statistical moments and correlations of current and past trade values.
result Market-based statistical moments can be approximated by a finite number of moments, improving forecast reliability.