DAFT models attention as a dynamical system to make neural networks more interpretable.
problem Uninterpretable features learned by neural networks without human priors.
method DAFT models attention as a continuous dynamical system using neural ODEs.
result DAFT reduces the number of reasoning steps while maintaining similar performance.
End-to-end KBQA system learns from multiple reasoning paths without labeled paths.
problem Lack of labeled reasoning paths limits KBQA system performance.
method End-to-end KBQA system using multiple reasoning paths.
result Demonstrates strong performance on various KBQA datasets.
We study side-lengths of triangles in path metric spaces. We prove that unless such a space X is bounded, or quasi-isometric to line or half-line, every triple of real numbers satisfying the strict triangle inequalities, is realized by the side-lengths of a triangle in X. We construct an example of a complete path metr…
Transformers learn multi-step reasoning through gradient descent.
problem Understanding how transformers solve symbolic multi-step reasoning tasks.
method Theoretical analysis of gradient descent dynamics and multi-phase training.
result Trained one-layer transformers can solve both backward and forward reasoning tasks with generalization guarantees.
Handel and Mosher have proved that the free splitting complex FS for the free group is Gromov hyperbolic. This is a deep and much sought-after result, since it establishes FS as a good analogue of the curve complex for surfaces. We give a shorter alternative proof of this theorem, using surgery paths in Hatcher's spher…
Model learns brevity by exposing to easy problems, improving efficiency without explicit length penalties.
problem Excessive verbosity in step-by-step reasoning models trained with RLVR.
method Retaining and up-weighting moderately easy problems as implicit length regularizers.
result Model generates solutions that are, on average, nearly twice as short without explicit length penalties.
Improved RL for knowledge graph reasoning with entity types.
problem Challenges in path-based relational reasoning over knowledge graphs.
method Type-enhanced RL agent using GNN for neighborhood information.
result Outperforms state-of-the-art RL methods and discovers novel paths.
A conformal procedure improves CoT reasoning by aggregating reasoning paths and calibrating abstention rules.
problem Aggregation uncertainty in chain-of-thought reasoning makes correct answers less reliable.
method Introduces a conformal procedure for CoT reasoning that uses weighted score aggregation and abstention rules.
result Achieves higher selective accuracy with abstention, reducing confident-error rate.
Linear RNNs exhibit a bias towards shorter memory due to initialization variance.
problem Understanding the performance limitations of RNNs, especially linear ones.
method Kernel regime analysis to show equivalence to 1D-convolutional networks and analyze weightings.
result Linear RNNs with random initialization have a bias towards shorter memory periods.
Neural-symbolic model improves link prediction in knowledge graphs.
problem Effective relational learning and reasoning for AI systems.
method Neural-symbolic graph neural network that learns over all paths in knowledge graphs.
result Neural-symbolic model outperforms path-based approaches in link prediction.
Unified coagent network theory with execution paths.
problem Hierarchical reinforcement learning exploration-exploitation trade-off.
method Formalized coagent networks, introduced execution paths, revisited coagent network theory.
result Shorter proof of policy gradient theorem, generalizable to asynchronous coagents.
New RL approach builds short ancestral recombination graphs.
problem Building short ancestral recombination graphs (ARGs).
method Reinforcement Learning applied to genetic sequences.
result RL can build ARGs as short as heuristic algorithms.
Auto-CEI improves LLM reasoning by balancing assertiveness and conservativeness.
problem Hallucinations and laziness in LLM reasoning tasks.
method Expert Iteration explores reasoning trajectories, guiding incorrect paths back on track and promoting appropriate 'I don't know' responses.
result Auto-CEI achieves superior alignment in logical reasoning, mathematics, and planning tasks.
Cognitive KR model learns new facts from few examples.
problem Inferring new facts from small KG datasets.
method CogKR combines summary and reasoning modules with cognitive science principles.
result Significantly outperforms previous models on one-shot KG reasoning.
Neural networks struggle with reasoning tasks that require specialized structures.
problem Understanding why and when neural network structures generalize better.
method Developing a framework to characterize algorithmic alignment with reasoning tasks.
result Neural networks align better with dynamic programming (DP) for certain reasoning tasks.
For the convenience of readers of the article {\em No-arbitrage pricing under systemic risk: accounting for cross-ownership} (Fischer, 2012, arXiv:1005.0768), a full proof of Lemma A.5 and a shorter proof of Lemma A.6 of that paper are provided.
Theoretical work shows integrating coherent reasoning improves LLM performance and error correction.
problem Improving reasoning and error correction in large language models (LLMs) with few-shot prompting.
method Theoretical analysis and sensitivity experiments on transformer behavior with coherent reasoning and corrupted demonstrations.
result The transformer gains better error correction ability and more accurate predictions when coherent reasoning is integrated.
Tractograms are mathematical representations of the main paths of axons within the white matter of the brain, from diffusion MRI data. Such representations are in the form of polylines, called streamlines, and one streamline approximates the common path of tens of thousands of axons. The analysis of tractograms is a ta…
Improves bounds on surface decompositions.
problem Surface decomposition bounds
method Improved theorem of Bers for surfaces with boundary and closed surfaces.
result Slight improvement on best known bound for closed surfaces.
Shorter sides in geodesic triangles in hyperbolic plane.
problem Properties of geodesic triangles in hyperbolic surfaces.
method Analyzing lifts of a closed geodesic in hyperbolic 2-space.
result Sides of triangles formed by geodesics are shorter than the geodesic itself.
Proposes CLRS-Text, a new benchmark for evaluating LM reasoning capabilities.
problem Lack of transferable benchmarks for evaluating reasoning capabilities of language models.
method Developed a textual version of the CLRS benchmark, generating diverse algorithmic tasks.
result Demonstrates a novel challenge for the LM reasoning community and validates prior work.
ManifoldMind uses adaptive-curvature probabilistic spheres for trustworthy recommendations in semantic hierarchies.
problem Sparse and abstract recommendation domains where users explore diverse conceptual paths.
method Adaptive-curvature probabilistic spheres, soft multi-hop inference, and curvature-aware semantic kernel.
result Superior NDCG, calibration, and diversity compared to baselines on public benchmarks.
NSR enables neural networks to reason with continuous numbers and extrapolate.
problem Quantitative reasoning and extrapolation in neural networks.
method Proposes Neural Status Register (NSR) for continuous number reasoning.
result NSR achieves extrapolation to numbers many orders of magnitude larger than training data.
ERM uses energy-based selection to improve recursive reasoning.
problem Lack of principled inference mechanism in recursive models.
method Energy-guided Recursive Model (ERM) introduces Hopfield energies for trajectory selection.
result ERM achieves optimal solutions on various puzzles.
Reasoning models generate differently based on problem difficulty, not just length.
problem Understanding how reasoning models handle different problem difficulties.
method Examined hidden-state trajectories across competitive programming, mathematics, and Boolean satisfiability.
result Corrected trajectory geometry shows difficulty-dependent differences in reasoning models, with stronger effects in the code domain.
Knowledge base (KB) completion adds new facts to a KB by making inferences from existing facts, for example by inferring with high likelihood nationality(X,Y) from bornIn(X,Y). Most previous methods infer simple one-hop relational synonyms like this, or use as evidence a multi-hop relational path treated as an atomic f…
EORM boosts LLM accuracy with a lightweight, energy-based verifier.
problem Efficiently verifying mathematical reasoning in large language models.
method Energy-based framework to rank Chain-of-Thought solutions using simple outcome labels.
result EORM boosts LLM accuracy to 90.7% on GSM8k and 63.7% on MATH with only 55M parameters.
Simplified proof for a theorem about graphs.
problem Proving a generalized Conway--Gordon--Sachs theorem for complete graphs.
method Provided a shorter proof over integers.
result A simpler proof of the generalized theorem.
We establish causal semantics for SDEs and develop methods to reason about them.
problem Understanding causal relationships in systems modeled by stochastic differential equations.
method We introduce a causal graph framework, Markov properties, and do-calculus for SDEs.
result We prove the σ σ σ -separation Markov property and do-calculus for causal SDEs. We propose the Neural Logic Machine (NLM), a neural-symbolic architecture for both inductive learning and logic reasoning. NLMs exploit the power of both neural networks---as function approximators, and logic programming---as a symbolic processor for objects with properties, relations, logic connectives, and quantifier…
G1 uses RL to enhance LLMs' graph reasoning, improving performance on diverse tasks.
problem Limited graph reasoning abilities of LLMs, especially in synthetic graph-theoretic tasks.
method Curated synthetic graph dataset, RL training on LLMs.
result Significant improvements in graph reasoning, zero-shot generalization to unseen tasks.
Proof of Gromov's conjecture in 3D simplifies existing methods.
problem Gromov's dihedral rigidity conjecture on scalar curvature in 3D.
method Self-contained proof avoiding technical complications, shorter and more accessible than general case.
result Simplified proof of Gromov's conjecture in 3D.
Improved Markov models learn from their mistakes and adapt to problem complexity.
problem Limitations of standard masked discrete diffusion models in reasoning tasks.
method Learning a Markov transition kernel trained on its own outputs, allowing remasking and adaptation.
result Significant improvement in solving reasoning problems, especially Sudoku-Extreme and Countdown-4.
Language grounded image understanding tasks have often been proposed as a method for evaluating progress in artificial intelligence. Ideally, these tasks should test a plethora of capabilities that integrate computer vision, reasoning, and natural language understanding. However, rather than behaving as visual Turing t…
LaRT models LLMs' response accuracy and CoT length to evaluate reasoning ability and speed.
problem Valid evaluation of Large Language Models (LLMs) via response accuracy and chain-of-thought length.
method Introduces Latency-Response Theory (LaRT) to jointly model response accuracy and CoT length using latent ability and latent speed.
result LaRT yields higher estimation accuracy and shorter confidence intervals for latent traits compared to IRT.
FinReflectKG benchmarks financial QA by linking relevant context from a financial KG, improving model performance and efficiency.
problem Efficiently retrieving and navigating relevant financial information across diverse sources and years.
method A benchmark built on a temporally indexed financial KG, generating QA pairs via pattern-specific prompts and quality control, evaluating retrieval scenarios.
result KG-guided retrieval yields substantial gains in correctness and token utilization, improving model performance by 24%.
Proposes a copula-based method to interpret neural networks.
problem Neural networks lack interpretability due to ambiguity in decision-making.
method Uses pair copula correlation coefficients to reveal feature importance and decision paths.
result Consistent results with Random Forest's feature importance values.
Regularizes decision trees to reduce inference time by up to 4x with minimal accuracy loss.
problem Optimizing decision tree execution time on resource-constrained devices.
method Regularizes impurity computation during CART algorithm training to favor highly asymmetric distributions.
result Reduces inference time by up to 4x with minimal accuracy loss.
Chameleon optimizes neural network compilation for faster execution and shorter time.
problem Faster execution and shorter compilation time for neural networks.
method Adaptive code optimization using reinforcement learning and adaptive sampling.
result 4.45x speed up in optimization time over AutoTVM, 5.6% improvement in inference time.
DeepRF_SLR uses deep reinforcement learning to design shorter RF pulses.
problem Designing efficient RF pulses for MRI sequences.
method Optimizing the root pattern of SLR polynomial using deep reinforcement learning and greedy tree search.
result DeepRF_SLR generates shorter duration RF pulses in less time compared to conventional methods.
End-to-end multi-object tracking learns object interactions.
problem Object tracking ignores interactions between objects.
method End-to-end relational reasoning model MOHART.
result Relational reasoning improves tracking and prediction.
We give a shorter and simpler proof of the result of [2], which gives a necessary and sufficient condition for when a lattice diagram is the projection of a lattice link.
Fuelled by increasing computer power and algorithmic advances, machine learning techniques have become powerful tools for finding patterns in data. Since quantum systems produce counter-intuitive patterns believed not to be efficiently produced by classical systems, it is reasonable to postulate that quantum computers …
New algorithm reduces regret in stochastic shortest path problems.
problem Planning and control in environments with unknown dynamics and variable episode lengths.
method Developed an algorithm with a new regret bound of O ( B ⋆ ∣ S ∣ ∣ A ∣ K ) O(B_\star |S| \sqrt{|A| K}) O ( B ⋆ ∣ S ∣ ∣ A ∣ K ) . result Guaranteed a significant reduction in regret compared to previous methods.
ICQ improves high-dimensional similarity search without sacrificing precision.
problem High-dimensional similarity search is computationally expensive.
method Interleaved Composite Quantization (ICQ) reduces code length and quantization error.
result ICQ achieves fast similarity search without using shorter codes.
Improves many-shot learning by optimizing and generating influential examples.
problem Limited performance in many-shot in-context learning (ICL).
method Iterative optimization and generation of influential examples.
result Significant improvements across various tasks using BRIDGE.
A novel fact-checking method using debate dynamics on knowledge graphs.
problem Fact-checking on knowledge graphs with user comprehension and interactive reasoning.
method Reinforcement learning agents debate on paths in the graph to classify facts as true or false.
result Interactive reasoning and user understanding of AI decisions on knowledge graphs.
A method for reasoning on knowledge graphs using debate dynamics.
problem Automatic reasoning on knowledge graphs with interpretability.
method Reinforcement learning agents debate over facts, judge decides truth.
result Method outperforms baselines on triple classification and link prediction tasks.