Forward-prediction models enhance physical reasoning, but only for specific tasks.
problem Improving physical reasoning in complex tasks involving many objects.
method Incorporated forward-prediction models into simple physical-reasoning agents and evaluated their performance on the PHYRE benchmark.
result Forward-prediction models improve physical-reasoning performance, especially on complex tasks, but generalization to new task templates is challenging.
The study compares feed-forward and attention layers in language models.
problem Understanding the role of feed-forward and attention layers in language models.
method Empirical and theoretical analysis in a synthetic setting.
result Feed-forward layers learn simple distributional associations, while attention layers focus on in-context reasoning.
Transformers learn multi-step reasoning through gradient descent.
problem Understanding how transformers solve symbolic multi-step reasoning tasks.
method Theoretical analysis of gradient descent dynamics and multi-phase training.
result Trained one-layer transformers can solve both backward and forward reasoning tasks with generalization guarantees.
Transformers improve logical reasoning on longer proofs but struggle with length.
problem Understanding systematic generalization in neural proof generation.
method Soft theorem-proving using Transformer models, evaluating logical consistency and inference accuracy.
result Transformers improve generalization with longer proofs but have difficulty with length.
New approach uses 'forward-looking' counterfactuals for treatment choice.
problem Using traditional 'retrospective' counterfactuals in treatment choice leads to counterintuitive results.
method Introduces 'counterfactual treatment choice' for forward-looking counterfactuals.
result Mismatches between interventional and forward-looking counterfactuals can lead to counterintuitive results.
KINet learns object interactions without supervision for robotic pushing.
problem Lack of supervised data for object-centric forward prediction.
method End-to-end unsupervised framework using keypoint representation and contrastive estimation.
result Automatically generalizes to unseen scenarios and accurately predicts future states.
Single-pass method estimates neural network uncertainty.
problem Uncertainty estimation in deep learning requires multiple passes.
method Probabilistic reasoning over neural network depths.
result Single forward pass for uncertainty estimation.
A new method for math reasoning that allows for iterative correction.
problem Standard reasoning models commit to each token and cannot recover from early errors.
method Generative framework with latent thought vectors for iterative self-correction.
result 30 rethinking iterations surpass baselines with 15 times more parameters.
The paper addresses human-like decision-making in multi-agent systems using bounded risk-sensitive Markov Games.
problem Modeling human-like decision-making in multi-agent systems with risk-seeking and loss-aversion behaviors.
method Forward policy design and inverse reward learning with iterative reasoning and cumulative prospect theory.
result The proposed algorithms demonstrate both risk-averse and risk-seeking behaviors in multi-agent systems.
In his stimulating article on the reasons for two puzzling observations about the behaviour of interest rates, exchange rates and the rate of inflation, Charles Engel (2016) puts forward an explanation that rests on the concept of a non-pecuniary liquidity return on assets. Albeit intriguing the analysis struggles to a…
This article provides a comprehensive study of different ways to make speed benchmarks of gradient boosted decision trees algorithm. We show main problems of several straight forward ways to make benchmarks, explain, why a speed benchmarking is a challenging task and provide a set of reasonable requirements for a bench…
Efficiently trains forward processes to minimize generative trajectories curvature.
problem High curvature of generative trajectories slows down sampling speed.
method Trains forward process to minimize curvature without ODE/SDE simulation.
result Lower curvature than previous models, decreased sampling costs.
Thinking LLMs struggle with stock prediction, especially as data complexity increases.
problem Evaluating the performance of 'thinking' LLMs in stock prediction, especially under varying levels of cross-sectional complexity.
method Rolling 48m/1m walk-forward evaluation, comparing direct LLMs, TLLMs, and classical learners on cross-sectional ranking loss, MSE, and backtests with transaction costs.
result TLLMs' ranking quality deteriorates as cross-sectional complexity grows, while direct LLMs remain stable.
uGMM-NN integrates probabilistic reasoning into neural networks.
problem Capturing multimodality and uncertainty in neural network activations.
method Parameterizes activations as univariate Gaussian mixtures with learnable parameters.
result Competitive discriminative performance with probabilistic activations.
A VAE model predicts material properties and microstructures.
problem Building forward and inverse structure-property linkages in materials science.
method Combines VAE with regression, using a two-level prior and multi-modal Gaussian mixture.
result The model achieves accurate forward and inverse predictions of material properties and microstructures.
Language grounded image understanding tasks have often been proposed as a method for evaluating progress in artificial intelligence. Ideally, these tasks should test a plethora of capabilities that integrate computer vision, reasoning, and natural language understanding. However, rather than behaving as visual Turing t…
Transformers mimic Bayesian reasoning in controlled settings, revealing geometric mechanisms.
problem Verifying if transformers perform Bayesian reasoning rigorously in natural data.
method Constructing Bayesian wind tunnels with known posteriors and proving memorization impossibility.
result Transformers achieve 10−3-10−4 bit accuracy in Bayesian posteriors, while MLPs fail. Improves predictions by integrating forward-looking views into dynamic factor models.
problem Poor forecasts from historical data when dynamics change.
method Combines historical data with forward-looking views using a dynamic factor model.
result Derives optimal portfolio strategies influenced by both myopic and intertemporal factors.
TREK uses distillation to help students solve hard problems.
problem Stalled progress on hard prompts when current policy lacks useful reasoning trajectories.
method TREK combines distillation and reinforcement learning to expand student support.
result TREK significantly improves student performance on mathematical reasoning and agentic tasks.
The paper discusses scalable learning for wireless data-driven systems.
problem Expanding data volume and model complexity limit centralized learning solutions.
method Discusses scalable architecture and local learning strategies.
result Promising research directions in scalable data-driven wireless communications.
Recent work on fairness in machine learning has focused on various statistical discrimination criteria and how they trade off. Most of these criteria are observational: They depend only on the joint distribution of predictor, protected attribute, features, and outcome. While convenient to work with, observational crite…
Study reveals limitations of fair representation learning methods and cautions against their use in performance-sensitive tasks.
problem Limitations of fair representation learning methods in performance-sensitive tasks.
method Using causal reasoning, the study defines and formalizes different sources of dataset bias and examines the performance of fair representation learning under distribution shifts.
result Fundamental limitations on fair representation learning when evaluation data is drawn from the same distribution as training data.
We explore the behavior of a standard convolutional neural net in a continual-learning setting that introduces visual classification tasks sequentially and requires the net to master new tasks while preserving mastery of previously learned tasks. This setting corresponds to that which human learners face as they acquir…
Identifying changes in the generative process of sequential data, known as changepoint detection, has become an increasingly important topic for a wide variety of fields. A recently developed approach, which we call EXact Online Bayesian Changepoint Detection (EXO), has shown reasonable results with efficient computati…
Deep learning is increasingly being used in high-stake decision making applications that affect individual lives. However, deep learning models might exhibit algorithmic discrimination behaviors with respect to protected groups, potentially posing negative impacts on individuals and society. Therefore, fairness in deep…
Endowing robots with human-like physical reasoning abilities remains challenging. We argue that existing methods often disregard spatio-temporal relations and by using Graph Neural Networks (GNNs) that incorporate a relational inductive bias, we can shift the learning process towards exploiting relations. In this work,…
Dropout is a popular regularization technique in deep learning. Yet, the reason for its success is still not fully understood. This paper provides a new interpretation of Dropout from a frame theory perspective. By drawing a connection to recent developments in analog channel coding, we suggest that for a certain famil…
Sequence models quantify uncertainty over latent concepts.
problem Quantifying uncertainty in latent environments.
method Exchangeable sequence models, equivalent to empirical Bayes and posterior inference.
result Sequence prediction loss controls uncertainty quantification.
Cascading flows improve variational inference in structured programs.
problem Challenges in variational inference for complex probabilistic programs.
method Integrates normalizing flows and ASVI to create cascading flows, which embed the forward-pass of probabilistic programs.
result Cascading flows outperform normalizing flows and ASVI in structured inference problems.
Detects corruption in agentic models during execution.
problem Inconsistent context, retrieval errors, or adversarial inputs corrupt intermediate steps of reasoning chains.
method Analyzes token graphs induced by attention and computes spectral statistics to emit accept/reject signals.
result A single threshold on the high frequency energy ratio optimally detects context inconsistency in agentic models.
The paper explores how neural networks learn logical functions and their generalization error.
problem Learning logical functions with neural networks and understanding generalization error.
method Gradient descent on neural networks, analyzing noise-stability and Boolean influence.
result Gradient descent on certain neural architectures tends to favor low-degree representations, impacting generalization error.
The paper critiques existing uncertainty concepts and proposes a new decision-theoretic approach.
problem Incoherence in existing discussions of aleatoric and epistemic uncertainty.
method Decision-theoretic perspective that relates uncertainty, predictive performance, and statistical dispersion.
result Popular information-theoretic quantities can be poor estimators but still useful for guiding data acquisition.
A dry decade in the Navajo Nation has killed vegetation, dessicated soils, and released once-stable sand into the wind. This sand now covers one-third of the Nation's land, threatening roads, gardens and hundreds of homes. Many arid regions have similar problems: global warming has increased dune movement across farmla…
Deep Bellman Hedging uses reinforcement learning to optimize financial portfolio hedging.
problem Optimizing financial portfolio hedging with derivatives and trading frictions.
method Actor-critic reinforcement learning algorithm with continuous state and action spaces.
result Trained model provides optimal hedge for any initial portfolio and market state.
The relational model is a ubiquitous representation of big-data, in part due to its extensive use in databases. In this paper, we propose the Equivariant Entity-Relationship Network (EERN), which is a Multilayer Perceptron equivariant to the symmetry transformations of the Entity-Relationship model. To this end, we ide…
Modern approaches to stock pricing in quantitative finance are typically founded on the 'Black-Scholes model' and the underlying 'random walk hypothesis'. Empirical data indicate that this hypothesis works well in stable situations but, in abrupt transitions such as during an economical crisis, the random walk model fa…
New method for dynamic valuation in markets with random endowments.
problem Dynamic valuation in markets with random endowments.
method Developed new FBSDE systems and established optimality conditions.
result Established necessary and sufficient conditions for optimality.
This paper shows how forward rate interpolations are equivalent to discount factor interpolations in yield curve construction.
problem The challenge of choosing between different interpolation methods for yield curve construction.
method Demonstrates the equivalence between forward rate interpolations and discount factor interpolations.
result Some popular interpolation methods on forward rates are equivalent to classical interpolation methods on discount factors.
The biological plausibility of the backpropagation algorithm has long been doubted by neuroscientists. Two major reasons are that neurons would need to send two different types of signal in the forward and backward phases, and that pairs of neurons would need to communicate through symmetric bidirectional connections. …
Real Estate Investment Trusts (REITs) are the only truly liquid assets related to real estate investments. We study the behavior of U.S. REITs over the past three decades and document their return characteristics. REITs have somewhat less market risk than equity; their betas against a broad market index average about .…
This paper defines less discriminatory algorithms and explores their feasibility.
problem Creating algorithms that are less discriminatory while meeting business needs.
method Formal definition of less discriminatory algorithms, evaluation of feasibility, and search for alternatives.
result Formal definitions of less discriminatory algorithms face challenges due to lack of held-out data, necessitating a reliance on reasonableness standards.
Paper explores volatility swaps in rough volatility models.
problem Understanding volatility swaps in rough volatility models.
method Examines the relationship between forward start volatility swaps and implied volatilities in rough volatility models.
result The leading term approximation error in the correlated case does not depend on the time to forward start date.
Unified Bayesian framework for uncertainty quantification in mechanics.
problem Propagation of input uncertainties and inference of unknown parameters.
method Bayesian probability theory for forward and inverse problems.
result Unified theoretical framework for both forward and inverse UQ.
HierarchicalForecast provides a Python framework for coherent hierarchical forecasting.
problem Ensuring forecasts at disaggregate levels add up to aggregate forecasts.
method Preprocessed datasets, evaluation metrics, and statistical baseline models.
result Python-based reference framework for statistical and ML forecasting.
Within the Private Equity (PE) market, the event of a private company undertaking an Initial Public Offering (IPO) is usually a very high-return one for the investors in the company. For this reason, an effective predictive model for the IPO event is considered as a valuable tool in the PE market, an endeavor in which …
Deep learning has achieved remarkable success in diverse applications; however, its use in solving partial differential equations (PDEs) has emerged only recently. Here, we present an overview of physics-informed neural networks (PINNs), which embed a PDE into the loss of the neural network using automatic differentiat…
Develops a new class of forward performance processes for investment pools.
problem Investment performance in market models with continuous semimartingale stock prices.
method Constructs a broad class of forward performance processes with power mixture initial conditions.
result Characterizes and derives properties of two-power mixture forward performance processes.
Bayesian neural networks benefit from fully marginalizing over all modes to improve generalization.
problem Bayesian neural networks suffer from multimodal posterior distributions that can lead to suboptimal generalization.
method Use appropriate Bayesian sampling tools to fully marginalize over all posterior modes.
result Training with full marginalization improves the ability of the network to reason between multiple candidate solutions.