A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Unified view on selective credit assignment for reinforcement learning.
COCOA improves credit assignment in reinforcement learning by measuring contributions to rewards.
We consider the problem of efficient credit assignment in reinforcement learning. In order to efficiently and meaningfully utilize new data, we propose to explicitly assign credit to past decisions based on the likelihood of them having led to the observed outcome. This approach uses new information in hindsight, rathe…
Eigenoptions improve credit assignment in reinforcement learning.
Credit assignment in Meta-reinforcement learning (Meta-RL) is still poorly understood. Existing methods either neglect credit assignment to pre-adaptation behavior or implement it naively. This leads to poor sample-efficiency during meta-training as well as ineffective task identification strategies. This paper provide…
In this work, we study the credit assignment problem in reward augmented maximum likelihood (RAML) learning, and establish a theoretical equivalence between the token-level counterpart of RAML and the entropy regularized reinforcement learning. Inspired by the connection, we propose two sequence prediction algorithms, …
LICA learns credit assignment for cooperative agents without explicit formulation.
The paper explores how planning with models improves credit assignment in reinforcement learning.
This paper proposes a definition of system health in the context of multiple agents optimizing a joint reward function. We use this definition as a credit assignment term in a policy gradient algorithm to distinguish the contributions of individual agents to the global reward. The health-informed credit assignment is t…
Introduces expected eligibility traces for more efficient credit assignment in reinforcement learning.
MRC improves credit assignment in multi-agent LLM systems, achieving high returns and transparency.
New framework for modular reinforcement learning reduces sample complexity.
New methods improve deep reinforcement learning by accelerating credit assignment.
This paper proposes a new algorithm for learning guidance rewards in RL.
Proposes a new neural network approach to credit assignment.
Shapley Flow interprets model predictions using a graph-based approach to feature importance.
In preference-based reinforcement learning (RL), an agent interacts with the environment while receiving preferences instead of absolute feedback. While there is increasing research activity in preference-based RL, the design of formal frameworks that admit tractable theoretical analysis remains an open challenge. Buil…
Deep learning has achieved impressive prediction accuracies in a variety of scientific and industrial domains. However, the nested non-linear feature of deep learning makes the learning highly non-transparent, i.e., it is still unknown how the learning coordinates a huge number of parameters to achieve a decision makin…
MACC learns communication protocols by adapting counterfactual reasoning.
Learning long-term dependencies in extended temporal sequences requires credit assignment to events far back in the past. The most common method for training recurrent neural networks, back-propagation through time (BPTT), requires credit information to be propagated backwards through every single step of the forward c…
We present a recurrent neural network memory that uses sparse coding to create a combinatoric encoding of sequential inputs. Using several examples, we show that the network can associate distant causes and effects in a discrete stochastic process, predict partially-observable higher-order sequences, and enable a DQN a…
Improved GFlowNets learn more efficiently with trajectory balance.
Delta-AI speeds up inference in sparse PGMs by local credit assignment.
A variety of cooperative multi-agent control problems require agents to achieve individual goals while contributing to collective success. This multi-goal multi-agent setting poses difficulties for recent algorithms, which primarily target settings with a single global reward, due to two new challenges: efficient explo…
Small Medium-sized Enterprises (SMEs) face many obstacles when they try to access credit market. These obstacles are increased if the SMEs are innovative. In this case, financial data are insufficient or even not reliable. Thus, when building a judgemental rating model, mainly based on qualitative criteria (soft inform…
Study finds implicit government guarantee improves municipal investment bond ratings.
The study develops a machine learning system for credit scoring and default prediction.
Stochastic computation graphs (SCGs) provide a formalism to represent structured optimization problems arising in artificial intelligence, including supervised, unsupervised, and reinforcement learning. Previous work has shown that an unbiased estimator of the gradient of the expected loss of SCGs can be derived from a…
Deep reinforcement learning approaches have shown impressive results in a variety of different domains, however, more complex heterogeneous architectures such as world models require the different neural components to be trained separately instead of end-to-end. While a simple genetic algorithm recently showed end-to-e…
Two local learning rules are investigated to avoid weight transport in neural networks.
Credit scoring models support loan approval decisions in the financial services industry. Lenders train these models on data from previously granted credit applications, where the borrowers' repayment behavior has been observed. This approach creates sample bias. The scoring model (i.e., classifier) is trained on accep…
The role of credit rating agencies has been under severe scrutiny after the subprime crisis. In this paper we explore the relationship between credit ratings and informational efficiency of a sample of thirty nine corporate bonds of US oil and energy companies from April 2008 to November 2012. For that purpose, we use …
Reward shaping is one of the most effective methods to tackle the crucial yet challenging problem of credit assignment in Reinforcement Learning (RL). However, designing shaping functions usually requires much expert knowledge and hand-engineering, and the difficulties are further exacerbated given multiple similar tas…
The paper formalizes and analyzes multi-agent Q-learning with value factorization.
New STDP rule for spiking neurons solves discrete action reinforcement learning tasks.
We consider the task of mapping pseudocode to long programs that are functionally correct. Given test cases as a mechanism to validate programs, we search over the space of possible translations of the pseudocode to find a program that passes the validation. However, without proper credit assignment to localize the sou…
Paper analyzes deep learning models for credit rating prediction using text and numerical data.
Recent advances in deep reinforcement learning algorithms have shown great potential and success for solving many challenging real-world problems, including Go game and robotic applications. Usually, these algorithms need a carefully designed reward function to guide training in each time step. However, in real world, …
Improves text-to-image diffusion models using GFlowNets.
Identifying customer segments in retail banking portfolios with different risk profiles can improve the accuracy of credit scoring. The Variational Autoencoder (VAE) has shown promising results in different research domains, and it has been documented the powerful information embedded in the latent space of the VAE. We…
Paper compares ML methods for credit scoring, highlighting feature selection and scaling impacts.
New method combines value function decomposition and policy gradients for cooperative multi-agent reinforcement learning.
New insights into how to inspect and learn from multi-stage processes and AI reasoning.
We propose Stochastic Neural Architecture Search (SNAS), an economical end-to-end solution to Neural Architecture Search (NAS) that trains neural operation parameters and architecture distribution parameters in same round of back-propagation, while maintaining the completeness and differentiability of the NAS pipeline.…
This paper provides a link between causal inference and machine learning techniques - specifically, Classification and Regression Trees (CART) - in observational studies where the receipt of the treatment is not randomized, but the assignment to the treatment can be assumed to be randomized (irregular assignment mechan…
Deep Reinforcement Learning (DRL) algorithms have been successfully applied to a range of challenging control tasks. However, these methods typically suffer from three core difficulties: temporal credit assignment with sparse rewards, lack of effective exploration, and brittle convergence properties that are extremely …
Procedure verifies if machine learning models assign fixed predictions that preclude access.
Meta-learning adjusts TD learning's eligibility trace parameter for more efficient reinforcement learning.