This paper improves learning complex functions with CoT supervision, reducing sample complexity.
problem Learning complex functions with multi-step reasoning.
method Develops a statistical theory linking CoT risk and end-to-end risk, using CoT information measure.
result CoT supervision can achieve significantly faster learning rates compared to standard E2E supervision.
Study learning from multiple thinkers providing step-by-step solutions to problems.
problem Learning from multiple, possibly different, thinkers providing step-by-step solutions to problems.
method Active learning algorithm that uses CoT data from multiple thinkers and end-result data.
result Learning can be hard from CoT supervision provided by two or a few different thinkers, but a generic algorithm can learn efficiently.
Transformers learn sparse Boolean functions through RL and SFT, revealing distinct learning behaviors.
problem Learning sparse Boolean functions with Transformers.
method Reinforcement Learning (RL) with process rewards and Supervised Fine-Tuning (SFT).
result RL learns the whole CoT chain simultaneously, while SFT learns step by step.
In this paper, we study the generative models of sequential discrete data. To tackle the exposure bias problem inherent in maximum likelihood estimation (MLE), generative adversarial networks (GANs) are introduced to penalize the unrealistic generated samples. To exploit the supervision signal from the discriminator, m…
Transformers solve parity problems efficiently with step-by-step reasoning.
problem Training transformers to solve complex, recursive problems like parity.
method Training a one-layer transformer to solve k k k -parity, incorporating intermediate parities into the loss function, and using teacher forcing or augmented data. result Transformers can learn parity in one gradient update with intermediate supervision or self-consistency checks.
Transformers with CoT don't enhance reasoning power across all tasks.
problem Does CoT enhance the reasoning power of transformers?
method Examined the memorization capabilities of fixed-precision transformers with and without CoT.
result Transformers with CoT cannot memorize all reasoning tasks, leading to a negative answer.
AI systems that explain their decisions can be monitored for harmful intentions.
problem Monitoring AI systems' decision-making processes for harmful intentions is imperfect and can miss some misbehavior.
method Monitoring the chain of thought (CoT) of AI systems that communicate in human language.
result CoT monitoring is a promising but fragile approach to AI safety.
CoT improves transformer sample efficiency by reducing input token dependencies and attention sparsity.
problem Transformer sample inefficiency in simple tasks.
method Demonstrated through parity-learning setup, showing CoT reduces required samples from exponential to polynomial.
result Transformer learns function within polynomial samples with CoT, requiring exponential samples without CoT.
Conditional COT-GAN predicts sequences using past data and kernel smoothing.
problem Predicting sequences given past data.
method Conditional COT-GAN with kernel smoothing.
result Improved convergence results for sequence prediction.
CoT enhances transformer accuracy on serial tasks by enabling serial computation.
problem Improving accuracy of large language models on inherently serial problems.
method Integrating a chain of thought (CoT) into decoder-only transformers to enable serial computation.
result Constant-depth transformers with CoT can solve problems in AC^0, surpassing TC^0 without CoT.
In this paper, we introduce two notions on a surface in a contact manifold. The first one is called degree of transversality (DOT) which measures the transversality between the tangent spaces of a surface and the contact planes. The second quantity, called curvature of transversality (COT), is designed to give a compar…
CoT-UQ improves LLM uncertainty quantification by integrating reasoning steps.
problem LLMs' overconfidence and lack of response-wise uncertainty quantification.
method Integrates LLMs' reasoning steps into uncertainty estimation.
result Significantly improves uncertainty quantification accuracy (5.9% AUROC improvement).
Fractured Sampling improves LLM reasoning efficiency by truncating CoT trajectories.
problem Efficiently scaling reasoning in large language models with limited tokens.
method Integrating truncated Chain-of-Thought (CoT) with Fractured Sampling across multiple dimensions.
result Fractured Sampling achieves superior accuracy-cost trade-offs compared to full CoT.
The paper explores how LLMs with CoT improve performance on complex tasks.
problem Understanding the mechanisms behind LLMs' improved performance with CoT.
method Using circuit complexity theory, the paper examines LLMs' expressivity in solving mathematical and decision-making problems.
result LLMs with CoT can generate correct solutions step-by-step, even for complex tasks.
Theoretical work shows integrating coherent reasoning improves LLM performance and error correction.
problem Improving reasoning and error correction in large language models (LLMs) with few-shot prompting.
method Theoretical analysis and sensitivity experiments on transformer behavior with coherent reasoning and corrupted demonstrations.
result The transformer gains better error correction ability and more accurate predictions when coherent reasoning is integrated.
This work analyzes CoT prompting methods from a statistical estimation perspective.
problem Improving the effectiveness of LLMs in solving multi-step reasoning problems.
method Introducing a multi-step latent variable model to characterize CoT prompting from a statistical estimation viewpoint.
result The CoT estimator is equivalent to a Bayesian estimator when the pretraining dataset is large.
COT-GAN generates sequential data with a causal optimal transport approach.
problem Generating sequential data with temporal causality constraints.
method Adversarial training with Causal Optimal Transport (COT) and entropic penalization.
result COT-GAN effectively learns time-dependent data distributions and generates stable time series data.
Study examines the impact of employment benefit costs on firm profitability.
problem Impact of employment benefit costs on firm profitability.
method Panel data regression analysis using E-Views.
result There is a significant positive relationship between employment benefit costs and firm profitability.
Using Kauffman's model of flat knotted ribbons, we demonstrate how all regular polygons of at least seven sides can be realised by ribbon constructions of torus knots. We calculate length to width ratios for these constructions thereby bounding the Ribbonlength of the knots. In particular, we give evidence that the clo…
Minimalist softmax attention learns constrained Boolean functions with supervision.
problem Learning k k k -bit Boolean functions, especially AND and OR. method Minimalist single-head softmax-attention mechanism with teacher forcing.
result Minimalist attention can solve Boolean tasks with supervision, but not without.
RNNs struggle with in-context retrieval, while Transformers excel.
problem In-context retrieval capability of RNNs.
method Theoretical analysis and experimental techniques (CoT, RAG, Transformer layer).
result Enhancing RNNs with techniques improves their in-context retrieval capability, closing the representation gap with Transformers.
Study reveals how depth of reasoning affects generalization in models.
problem Understanding scaling behavior of generalization with CoT depth.
method Theoretical model of CoT in linear regression using random matrix theory.
result Sharp phase transition between exponential and polynomial improvement, saturation, and overthinking.
FinRobot AI agent for equity research provides comprehensive insights.
problem Narrow focus and limited discretion in AI solutions for equity research.
method Multi-agent Chain of Thought system integrating quantitative and qualitative analyses.
result FinRobot delivers insights comparable to major brokerage firms.
LaRT models LLMs' response accuracy and CoT length to evaluate reasoning ability and speed.
problem Valid evaluation of Large Language Models (LLMs) via response accuracy and chain-of-thought length.
method Introduces Latency-Response Theory (LaRT) to jointly model response accuracy and CoT length using latent ability and latent speed.
result LaRT yields higher estimation accuracy and shorter confidence intervals for latent traits compared to IRT.
The study examines how extra compute during testing affects the performance of large language models.
problem Understanding the conditions under which test-time scaling improves model performance.
method An in-context weight prediction task for linear regression was used to train transformers. The performance was analyzed under varying levels of test-time compute.
result Training transformers on diverse, relevant, and hard tasks leads to the best performance for test-time scaling.
Transformers learn chain-of-thought reasoning for longer problems, proving length generalization.
problem Challenging problems require deeper reasoning, but how do models generalize this to longer tasks?
method Theoretical analysis of transformers on synthetic state-tracking tasks, proving length generalization through attention concentration.
result Transformers can learn chain-of-thought reasoning for longer problems, proving length generalization.
A conformal procedure improves CoT reasoning by aggregating reasoning paths and calibrating abstention rules.
problem Aggregation uncertainty in chain-of-thought reasoning makes correct answers less reliable.
method Introduces a conformal procedure for CoT reasoning that uses weighted score aggregation and abstention rules.
result Achieves higher selective accuracy with abstention, reducing confident-error rate.
Bounds on intersection number for right-angled triangles.
problem Estimating intersection numbers in Teichmüller spaces.
method Analyzing moduli spaces of translation surfaces and Teichmüller discs.
result Sharp bounds on KVol function for specific Teichmüller discs.
We prove the nontriviality, at all integral levels n, of the filtration, F_n, of the classical topological knot concordance group recently defined by the authors and Kent Orr [COT]. Recall that this filtration is significant not only because of it's strong connection to Whitney tower constructions of Casson and Freedma…
Prefix consistency improves model reliability by weighting answers based on their reproducibility.
problem Improving the reliability of large language models' reasoning traces.
method Use prefix consistency to weight candidate answers based on their reproducibility during regeneration.
result Prefix consistency is the best correctness predictor, reaching Standard MV plateau accuracy with up to 21x fewer tokens.
Two neural network methods approximate conditional optimal transport for Bayesian inference.
problem Approximating conditional optimal transport for Bayesian inference in high dimensions.
method Neural network approximations of conditional optimal transport maps.
result Improved scalability and modeling choices for conditional sampling and density estimation.
Enhances math problem-solving models with multi-turn preference learning.
problem Improving mathematical problem-solving capabilities of large language models.
method Introduces a multi-turn direct preference learning framework for tool-integrated mathematical reasoning tasks.
result Significant performance improvements in model accuracy on math datasets.
LaTRO optimizes latent reasoning in LLMs without external reward.
problem Training LLMs to perform complex reasoning tasks.
method Formulates reasoning as latent distribution sampling and optimizes via variational approaches.
result LLMs improve reasoning and evaluation quality through self-improvement.
The paper studies how search and distillation improve reasoning in large language models.
problem Improving reasoning capabilities of large language models.
method Viewing chain-of-thought generation as a metastable Markov process, proving benefits of search and distillation.
result Search protocol rewards sparse edges, reducing the expected number of steps to reach different clusters.
ChatGPT improves financial reasoning, overcoming biases in gold investment.
problem Improving financial reasoning and overcoming biases in investment decisions.
method Applied advanced prompt engineering and semantic news information to enhance LLMs' performance.
result ChatGPT with CoT prompt provides more explainable predictions and higher investment returns.
We study Kauffman's model of folded ribbon knots: knots made of a thin strip of paper folded flat in the plane. The ribbonlength is the length to width ratio of such a ribbon, and it turns out that the way the ribbon is folded influences the ribbonlength. We give an upper bound of n cot ( π / n ) n\cot(π/n) n cot ( π / n ) for the ribbonlength of $…
Summarizes financial news for better investment decisions.
problem Information overload from financial news hinders timely investment decisions.
method Personalized Chain-of-Thought summarization framework integrating user-specified keywords.
result Personalized summaries highlight relevant market signals, improving investment narratives.
We study bi-warped product submanifolds of nearly Kaehler manifolds which are the natural extension of warped products. We prove that every bi-warped product submanifold of the form M = M T × f 1 M ⊥ × f 2 M θ M=M_T\times_{f_1}\! M_\perp\times_{f_2}\! M_θ M = M T × f 1 M ⊥ × f 2 M θ in a nearly Kaehler manifold satisfies the following sharp inequality: $$\|h\|^2\geq 2p\|\…
New emulator bridges simulators using conditional optimal transport.
problem Bridging simulators with minimal distortion.
method Flow-based approach to learn likelihood transport, COT-FM for optimal matching.
result Emulator accurately captures full correction between simulators.
Let m>1 and n>1 be any pair of integers. In this paper we prove that if H is between the numbers \cot(\fracπ{m}) and b_{m,n}=\frac{(m^2-2)\sqrt{n-1}}{n\sqrt{m^2-1}}, then, there exists a non isoparametric, compact embedded hypersurface in S^{n+1} with constant mean curvature H that admits the group O(n)x Z_m in their g…
In this work, we first describe a framework for the application of Reinforcement Learning (RL) control to a radar system that operates in a congested spectral setting. We then compare the utility of several RL algorithms through a discussion of experiments performed on Commercial off-the-shelf (COTS) hardware. Each RL …
In this paper, we prove that there are no proper C R S CRS C R S bi-warped product submanifolds other than contact CR-biwarped products in Sasakian manifolds. On the other hand, we prove that if M M M is a C R S CRS C R S bi-warped product of the form M = N T × f 1 N ⊥ n 1 × f 2 N θ n 2 M=N_T \times_{f_1}N^{n_{1}}_\perp\times_{f_2} N^{n_{2}}_θ M = N T × f 1 N ⊥ n 1 × f 2 N θ n 2 in a cosymplectic manifold $\w…
Transformers can generalize to a large task family with only a few demonstrations.
problem Can learning from a small set of tasks generalize to a large task family?
method Investigating autoregressive compositional structure where each task is a composition of T T T operations, each from a finite family of D D D subtasks. result Transformers can generalize to D T D^T D T tasks with only O ~ ( D ) \widetilde{O}(D) O ( D ) demonstrations. Study evaluates LLMs like ChatGPT and GPT-4 on financial analysis tasks.
problem Assessing financial reasoning capabilities of large language models.
method Mock CFA exam questions, Zero-Shot, Chain-of-Thought, Few-Shot scenarios.
result LLMs perform well on financial analysis tasks, but have limitations.
CHOOSE enhances shallow Transformers for wireless symbol detection.
problem Improving wireless symbol detection with shallow Transformers.
method Introducing autoregressive latent reasoning steps within hidden space.
result Lightweight Transformers achieve comparable performance to deep models.
New system resists meme coin copy trading bots.
problem Manipulative bots exploit copy trading in illiquid meme coins.
method Multi-agent architecture with LLM and CoT reasoning.
result System outperforms other methods in prediction and economic performance.
Paper presents LLM-enhanced contract metadata extraction.
problem Automatic detection and annotation of legal clauses in contracts.
method Integration of publicly available and proprietary datasets with advanced LLM methodologies.
result Substantial improvements in clause identification accuracy and efficiency.
Study proposes a multi-agent framework to mitigate bias in sentiment analysis.
problem Bias in sentiment analysis models.
method Integrates multiple LLMs, incorporates dialogue sessions, and uses probabilistic prediction.
result KCS+IBC reduces entropy and increases variance, suggesting improved balance between aggregation and diversity.