Auto-CEI improves LLM reasoning by balancing assertiveness and conservativeness.
problem Hallucinations and laziness in LLM reasoning tasks.
method Expert Iteration explores reasoning trajectories, guiding incorrect paths back on track and promoting appropriate 'I don't know' responses.
result Auto-CEI achieves superior alignment in logical reasoning, mathematics, and planning tasks.
A new method for math reasoning that allows for iterative correction.
problem Standard reasoning models commit to each token and cannot recover from early errors.
method Generative framework with latent thought vectors for iterative self-correction.
result 30 rethinking iterations surpass baselines with 15 times more parameters.
Tab-TRM uses recursive model for insurance pricing on tabular data.
problem Insurance pricing on tabular data.
method Adapts recursive latent reasoning to insurance modeling using a compact, parameter-efficient network.
result Improves insurance pricing accuracy using iterative refinement of latent tokens.
XLVINs improve deep reinforcement learning by combining self-supervised learning and neural algorithmic reasoning.
problem Limitations of Value Iteration Networks (VINs) in deep reinforcement learning.
method Combining contrastive self-supervised learning, graph representation learning, and neural algorithmic reasoning.
result XLVINs match VIN-like models on discrete, fixed MDPs and significantly outperform model-free baselines.
MAC Net improves natural language question answering with data-driven reasoning.
problem Natural Language Question Answering requires complex reasoning.
method MAC Net architecture separates memory and control for iterative reasoning.
result MAC Net achieves high efficiency and interpretability in NLP tasks.
The paper develops a theory for iterative self-improvement of models, proving conditions for better performance with easy-to-hard curricula.
problem Lack of theoretical foundation for iterative self-improvement in practical settings.
method Modeling self-improvement as maximum-likelihood fine-tuning on reward-filtered distributions and proving finite-sample guarantees.
result Explicit feedback loop and conditions for better performance with easy-to-hard curricula.
We developed a caching method to speed up concept learning in complex knowledge bases.
problem Complex concept learning requires many instance retrieval calls, increasing runtime.
method Semantics-aware caching that links concepts to instances via crisp set operations.
result Our cache reduces concept retrieval and learning runtime by an order of magnitude.
FLOP algorithm speeds up causal structure learning for linear models.
problem Efficiently learning causal structures from discrete data.
method FLOP algorithm combines fast parent selection and iterative score updates.
result FLOP finds highly accurate causal structures with near-perfect recovery.
LASER compresses recursive model activations by exploiting their low-dimensional structure.
problem Understanding and optimizing the geometric structure of recursive reasoning trajectories.
method Dynamic low-rank basis tracking via matrix-free subspace tracking with a fidelity-triggered reset mechanism.
result Recursive activations occupy a linear, low-dimensional subspace that can be compressed efficiently.
XLVINs improve data efficiency in implicit planning by leveraging latent space.
problem Improving data efficiency in implicit planning algorithms.
method XLVINs use a high-dimensional latent space to perform planning computations, breaking the algorithmic bottleneck.
result XLVINs significantly improve data efficiency across various settings compared to value iteration-based implicit planners and model-free baselines.
Cognitive KR model learns new facts from few examples.
problem Inferring new facts from small KG datasets.
method CogKR combines summary and reasoning modules with cognitive science principles.
result Significantly outperforms previous models on one-shot KG reasoning.
We present novel minibatch stochastic optimization methods for empirical risk minimization problems, the methods efficiently leverage variance reduced first-order and sub-sampled higher-order information to accelerate the convergence speed. For quadratic objectives, we prove improved iteration complexity over state-of-…
Study reveals how depth of reasoning affects generalization in models.
problem Understanding scaling behavior of generalization with CoT depth.
method Theoretical model of CoT in linear regression using random matrix theory.
result Sharp phase transition between exponential and polynomial improvement, saturation, and overthinking.
Hybrid deep architectures with reasoning layers show promising convergence and generalization properties.
problem Understanding the theoretical foundations of hybrid deep architectures with reasoning layers.
method Analyzing the interplay between algorithm layers and neural components in deep architectures.
result Properties of algorithm layers are closely related to the approximation and generalization abilities of end-to-end models.
We study the statistical behavior of reasoning probes in a stylized model of iterative computation inspired by neural algorithmic reasoning. The underlying computation is given by a looped Boolean circuit whose graph is a perfect ν-ary tree (ν≥2), with outputs recursively fed back as inputs across computation ro…
This work investigates how multi-round reasoning improves LLM performance.
problem Improving problem-solving abilities in complex tasks with LLMs.
method Investigates approximation, learnability, and generalization properties of multi-round auto-regressive models.
result Transformers with finite context windows are universal approximators for Turing-computable functions and can approximate any Turing-computable sequence-to-sequence function through multi-round reasoning.
We introduce the value iteration network (VIN): a fully differentiable neural network with a `planning module' embedded within. VINs can learn to plan, and are suitable for predicting outcomes that involve planning-based reasoning, such as policies for reinforcement learning. Key to our approach is a novel differentiab…
Paper proposes a new method for effective network compression.
problem Limited resources for deep learning models.
method Iterative pruning guided by simulation.
result The proposed method achieves higher performance at the same pruning level.
Transformers solve parity problems efficiently with step-by-step reasoning.
problem Training transformers to solve complex, recursive problems like parity.
method Training a one-layer transformer to solve k-parity, incorporating intermediate parities into the loss function, and using teacher forcing or augmented data. result Transformers can learn parity in one gradient update with intermediate supervision or self-consistency checks.
We develop a privatised stochastic variational inference method for Latent Dirichlet Allocation (LDA). The iterative nature of stochastic variational inference presents challenges: multiple iterations are required to obtain accurate posterior distributions, yet each iteration increases the amount of noise that must be …
This short note contains an explicit proof of the Jacobi identity for variational Schouten bracket in Z2-graded commutative setup. For the reasoning to be rigorous, it refers to the product bundle geometry of iterated variations (see arXiv:1312.1262 [math-ph]); no ad hoc regularizations occur anywhere in this theory…
Iterative algorithms are ubiquitous in the field of data mining. Widely known examples of such algorithms are the least mean square algorithm, backpropagation algorithm of neural networks. Our contribution in this paper is an improvement upon this iterative algorithms in terms of their respective performance metrics an…
New benchmark for causal reasoning from human video descriptions.
problem Lack of diversity in event types and natural language descriptions, and differences from human judgments.
method Iterative event cloze task and data augmentation techniques.
result Improved data collection efficiency and diverse causal judgments.
Many commonly used learning algorithms work by iteratively updating an intermediate solution using one or a few data points in each iteration. Analysis of differential privacy for such algorithms often involves ensuring privacy of each step and then reasoning about the cumulative privacy cost of the algorithm. This is …
CausalEvolve improves efficiency and discovery in open-ended scientific tasks.
problem Lack of targeted guidance and knowledge organization in evolve-based agents.
method Causal scratchpad that identifies and reasons about guiding factors for evolution.
result Effective improvement in evolutionary efficiency and better solutions.
LLMs struggle with arithmetic tasks unless they use high numerical precision.
problem Improving arithmetical reasoning capabilities of LLMs.
method Theoretical analysis and empirical experiments on numerical precision.
result LLMs require high numerical precision to efficiently handle arithmetic tasks.
Paper proposes a self-training method to generate molecular targets.
problem Challenges in training generative models for complex molecular design.
method Iterative target augmentation using a property predictor and EM iterations.
result Significant gains in molecular design, outperforming previous methods.
Paper simplifies deep RL for mobile robots on consumer hardware.
problem Training complex reasoning agents in 3D environments.
method Used ViZDoom for fast simulation and varied difficulty scenarios.
result Robust baseline agent architecture trained on consumer hardware.
Method solves complex optimization problems with high probability bounds.
problem Nonlinear equality constrained stochastic optimization problems.
method Step-search sequential quadratic programming method.
result High-probability bound on iteration complexity for first-order stationarity.
This is a paper about geometry of (iterated) variations. We explain why no sources of divergence are built into the Batalin-Vilkovisky (BV) Laplacian, whence there is no need to postulate any ad hoc conventions such as "δ(0)=0" and "logδ(0)=0" within BV-approach to quantisation of gauge systems. Remarkably, the ge…
Iterative algorithms, like gradient descent, are common tools for solving a variety of problems, such as model fitting. For this reason, there is interest in creating differentially private versions of them. However, their conversion to differentially private algorithms is often naive. For instance, a fixed number of i…
We introduce an approach for deep reinforcement learning (RL) that improves upon the efficiency, generalization capacity, and interpretability of conventional approaches through structured perception and relational reasoning. It uses self-attention to iteratively reason about the relations between entities in a scene a…
We prove the existence and uniqueness of solutions of SDEs with Lipschitz coefficients, driven by continuous, model-free martingales. The main tool in our reasoning is Picard's iterative procedure and a model-free version of the Burkholder-Davis-Gundy inequality for integrals driven by model-free, continuous martingale…
A new algorithm reconstructs population dynamics from coarse samples.
problem Reconstructing population dynamics from unlabeled samples at coarse time intervals.
method Deep Momentum Multi-Marginal Schrödinger Bridge (DMSB) framework.
result Significantly outperforms baselines in synthetic and real-world datasets.
This work shows how approximate reward models can significantly improve inference-time scaling.
problem Improving the efficiency of inference for large language models.
method Identifying the Bellman error of approximate reward models and using Sequential Monte Carlo (SMC) for inference.
result Approximate reward models can reduce computational complexity from exponential to polynomial in T. We present a novel Newton-type method for distributed optimization, which is particularly well suited for stochastic optimization and learning problems. For quadratic objectives, the method enjoys a linear rate of convergence which provably \emph{improves} with the data size, requiring an essentially constant number of…
New algorithm reduces online learning iterations by a factor of T^2/3.
problem Efficiency in online learning with smooth cost functions.
method Follow-the-Perturbed-Leader method using online primal-dual framework.
result Guaranteed T^2/3 regret for general online convex optimization.
Improves many-shot learning by optimizing and generating influential examples.
problem Limited performance in many-shot in-context learning (ICL).
method Iterative optimization and generation of influential examples.
result Significant improvements across various tasks using BRIDGE.
Implicit models can match or exceed explicit models with more test-time compute.
problem Understanding the expressive power and scaling of implicit models.
method Nonparametric analysis of expressive power, mathematical characterization of implicit operators, and test-time scaling experiments.
result Implicit models can progressively express more complex mappings through iteration, matching a richer function class with test-time compute.
Enhances math problem-solving models with multi-turn preference learning.
problem Improving mathematical problem-solving capabilities of large language models.
method Introduces a multi-turn direct preference learning framework for tool-integrated mathematical reasoning tasks.
result Significant performance improvements in model accuracy on math datasets.
While the machine learning literature dedicated to fully automated reasoning algorithms is abundant, the number of methods enabling the inference process on the basis of previously defined knowledge structures is scanter. Fuzzy Cognitive Maps (FCMs) are neural networks that can be exploited towards this goal because of…
Paper proposes PPMM for fast estimation of large-scale OTM.
problem Estimation of large-scale optimal transport maps (OTM) is challenging due to the curse of dimensionality.
method Combines projection pursuit regression and sufficient dimension reduction to adaptively select projection directions.
result PPMM consistently estimates the most informative projection direction and weakly converges to the target OTM.
We study reinforcement learning under model misspecification, where we do not have access to the true environment but only to a reasonably close approximation to it. We address this problem by extending the framework of robust MDPs to the model-free Reinforcement Learning setting, where we do not have access to the mod…
Sparse coding is a core building block in many data analysis and machine learning pipelines. Typically it is solved by relying on generic optimization techniques, such as the Iterative Soft Thresholding Algorithm and its accelerated version (ISTA, FISTA). These methods are optimal in the class of first-order methods fo…
The paper addresses human-like decision-making in multi-agent systems using bounded risk-sensitive Markov Games.
problem Modeling human-like decision-making in multi-agent systems with risk-seeking and loss-aversion behaviors.
method Forward policy design and inverse reward learning with iterative reasoning and cumulative prospect theory.
result The proposed algorithms demonstrate both risk-averse and risk-seeking behaviors in multi-agent systems.
Deep-neural-network-based image reconstruction has demonstrated promising performance in medical imaging for under-sampled and low-dose scenarios. However, it requires large amount of memory and extensive time for the training. It is especially challenging to train the reconstruction networks for three-dimensional comp…
In "extreme" computational imaging that collects extremely undersampled or noisy measurements, obtaining an accurate image within a reasonable computing time is challenging. Incorporating image mapping convolutional neural networks (CNN) into iterative image recovery has great potential to resolve this issue. This pape…
A theoretical framework for deep learning is proposed to explain its effectiveness.
problem Lack of a comprehensive theory explaining deep learning's effectiveness.
method Integrates three characteristics into a graphical model called neurashed.
result Explains common empirical patterns in deep learning and provides insights into regularization and elasticity.