Unified view on selective credit assignment for reinforcement learning.
problem Efficient credit assignment in reinforcement learning.
method Unified temporal-difference algorithms with selective weightings.
result New algorithms for backward credit assignment and off-policy learning.
Eigenoptions improve credit assignment in reinforcement learning.
problem Improving credit assignment in reinforcement learning models.
method Investigated eigenoptions for credit assignment in model-free RL, comparing pre-specified and online discovery methods.
result Pre-specified eigenoptions aid exploration and credit assignment, while online discovery can hinder learning.
A new method backtracks through a few key past states to speed up credit assignment in long sequences.
problem Computational inefficiency of back-propagation through time for long sequences.
method Sparse attentive backtracking using learned attention mechanisms to skip connections.
result Matches or outperforms regular BPTT and truncated BPTT in tasks with long-term dependencies.
Introduces expected eligibility traces for more efficient credit assignment in reinforcement learning.
problem Efficiently assigning credit to states and actions in reinforcement learning.
method Introduces expected eligibility traces, allowing updates to counterfactual sequences.
result Substantial improvements in temporal-difference learning can be achieved with expected traces.
New framework for modular reinforcement learning reduces sample complexity.
problem Achieving independent credit assignment in reinforcement learning.
method Defining modular credit assignment as minimizing algorithmic mutual information, introducing modularity criterion for causal analysis.
result Single-step temporal difference action-value methods meet the modularity criterion, improving sample efficiency.
This paper proposes a new algorithm for learning guidance rewards in RL.
problem Long-term temporal credit assignment in sparse or delayed reward environments.
method Surrogate RL objective with trajectory-space smoothing to learn guidance rewards.
result Guidance rewards can be learned without additional neural networks and have intuitive interpretation.
Improved GFlowNets learn more efficiently with trajectory balance.
problem Inefficient credit assignment in GFlowNets leads to suboptimal learning.
method Proposed trajectory balance as a new learning objective.
result Trajectory balance leads to more efficient and robust GFlowNet learning.
New methods improve deep reinforcement learning by accelerating credit assignment.
problem Challenges in achieving fast and stable off-policy learning in deep reinforcement learning.
method Extends the generalized PBE objective to support multistep credit assignment and derives three gradient-based methods.
result Proposed methods outperform PPO and StreamQ in MuJoCo and MinAtar environments.
New algorithm learns reward signals from episodic returns for better reinforcement learning.
problem Difficulty in designing reward functions for real-world reinforcement learning tasks.
method Introduces a new algorithm that decomposes episodic returns into time-step rewards using deep neural networks.
result Learning reward signals from episodic returns improves reinforcement learning efficiency.
SLAYER improves SNN training by backpropagating spike errors.
problem Non-differentiability of spike function in SNNs.
method Introduces SLAYER for learning weights and delays, using temporal credit assignment.
result SLAYER achieves state-of-the-art performance on various datasets.
Paper tackles end-to-end training of complex neural networks using DIP method.
problem Training complex heterogeneous neural network models end-to-end.
method Deep Innovation Protection (DIP) method using multiobjective optimization.
result End-to-end training of complex heterogeneous neural network models is possible.
New algorithms assign credit to past decisions based on hindsight.
problem Efficient credit assignment in reinforcement learning.
method Explicit credit assignment using new data in hindsight.
result Value functions can be rewritten to include hindsight credit assignment.
Meta-learning adjusts TD learning's eligibility trace parameter for more efficient reinforcement learning.
problem Efficiently tuning the eligibility trace parameter for temporal difference learning.
method Meta-learning method to adjust eligibility trace parameter state-dependently.
result Improves overall quality of update targets, minimizing target error.
COCOA improves credit assignment in reinforcement learning by measuring contributions to rewards.
problem Improving sample efficiency in reinforcement learning through better credit assignment methods.
method Counterfactual Contribution Analysis (COCOA) for precise credit assignment.
result COCOA achieves lower bias and variance compared to Hindsight Credit Assignment (HCA), improving reinforcement learning performance.
ProMP improves Meta-RL by better handling credit assignment.
problem Poor credit assignment in Meta-RL leads to suboptimal performance.
method Develops a novel meta-learning algorithm controlling pre-adaptation and adapted policies.
result Significantly improves sample-efficiency and asymptotic performance.
Deep Reinforcement Learning (DRL) algorithms have been successfully applied to a range of challenging control tasks. However, these methods typically suffer from three core difficulties: temporal credit assignment with sparse rewards, lack of effective exploration, and brittle convergence properties that are extremely …
In this work, we study the credit assignment problem in reward augmented maximum likelihood (RAML) learning, and establish a theoretical equivalence between the token-level counterpart of RAML and the entropy regularized reinforcement learning. Inspired by the connection, we propose two sequence prediction algorithms, …
The paper introduces a health-informed policy gradient method for multi-agent reinforcement learning.
problem Optimizing joint reward functions in multi-agent systems with varying agent health.
method Health-informed credit assignment in a multi-agent proximal policy optimization algorithm.
result Significant improvement in learning performance compared to traditional methods.
Neural network uses local credit assignment to learn distant cause-effect relationships.
problem Learning distant cause-effect relationships in sequential data.
method Sparse coding in a recurrent neural network memory with local and immediate credit assignment.
result Network can predict partially-observable higher-order sequences and navigate mazes.
A model explains credit assignment in deep learning networks.
problem Understanding how deep learning assigns credit to parameters.
method Mean-field learning model with ensembles of sub-networks.
result Synaptic connections can be categorized into three types.
LICA learns credit assignment for cooperative agents without explicit formulation.
problem Credit assignment among cooperative agents.
method Centralized critic as hypernetwork, adaptive entropy regularization.
result Significantly outperforms previous methods on benchmarks.
The paper explores how planning with models improves credit assignment in reinforcement learning.
problem Improving credit assignment in reinforcement learning.
method Investigates the use of forward and backward models for planning in reinforcement learning.
result Establishes the relative merits and limitations of forward and backward planning mechanisms.
Survive method improves model-based RL by avoiding terminal states, reducing sample complexity.
problem High sample complexity in model-free RL methods limits real-world applications.
method Introduces 'survival' concept to model-based RL, focusing on avoiding terminal states instead of maximizing rewards.
result Survive method reduces training effort by focusing on terminal states, improving model-based RL performance.
Model improves mortgage credit risk prediction with spatio-temporal machine learning.
problem Improving accuracy of default probabilities and loan portfolio loss distributions in mortgage credit risk.
method Combines tree-boosting with a latent spatio-temporal Gaussian process model.
result Predictive models outperform conventional methods due to non-linear and spatio-temporal effects.
MRC improves credit assignment in multi-agent LLM systems, achieving high returns and transparency.
problem Lack of principled credit assignment in multi-agent LLM decision systems, vulnerability to regime shifts, and limited transparency.
method Market Regime Council (MRC) computes exact Shapley credits, uses exponentially weighted performance histories, Bayesian adaptive mixture, and regime-dependent multipliers.
result MRC achieves a Sharpe ratio of 1.51 and a cumulative return of 440.1% over 1,037 trading days, ranking first on CR, SR, and IR.
DPS uses posterior sampling for preference-based RL, achieving a first regret guarantee.
problem Formal frameworks for preference-based RL with theoretical analysis.
method Preference-based posterior sampling, Bayesian credit assignment.
result First asymptotic Bayesian no-regret rate for preference-based RL.
Meta-learning framework for credit risk assessment of SMEs, aligning financial statement dates with evaluation dates.
problem Temporal misalignment of credit scoring models leading to bias and inconsistent predictions.
method Two-step temporal decomposition: static model for annual PDs, dynamic model for monthly PDs; stacking architecture to aggregate multiple models.
result Framework effectively captures credit risk evolution over time, improving temporal consistency and predictive stability.
This project proposes using reinforcement learning to train spiking neural networks.
problem Training spiking neural networks using traditional methods is challenging due to the discrete nature of spikes.
method The project investigates two approaches: 1) treating each neuron as an RL agent, 2) applying the reparameterization trick.
result The project demonstrates that reinforcement learning can be applied to train spiking neural networks.
Credit risk analysis improved with a joint model for spatial and temporal effects.
problem Predicting borrower's time-to-event with spatial and temporal covariates.
method Spatio-Temporal Joint Model (STJM) using Bayesian hierarchical approach and INLA.
result Spatial effects improve joint model performance, but spatio-temporal interactions have less impact.
Proposes a new neural network approach to credit assignment.
problem Credit assignment problem in deep neural networks.
method Contrastive similarity matching objective function.
result Deep networks learn to match similarity between layers.
CM3 learns multi-agent cooperation by first achieving individual goals.
problem Cooperative multi-agent control with multiple goals and interactions.
method Two-stage curriculum: first learn individual goals, then cooperation; new policy gradient with credit function.
result CM3 learns faster on multi-goal multi-agent problems than existing algorithms.
New methods reduce variance in stochastic computation graph optimization.
problem High variance in gradient estimates from SCGs.
method Introducing value functions, baselines, and critics to derive lower-variance gradient estimates from partial model evaluations.
result Lower-variance gradient estimates from partial model evaluations, making optimization more efficient.
We present a new approach to understanding credit relationships between commercial banks and quoted firms, and with this approach, examine the temporal change in the structure of the Japanese credit network from 1980 to 2005. At each year, the credit network is regarded as a weighted bipartite graph where edges corresp…
Shapley Flow interprets model predictions using a graph-based approach to feature importance.
problem Existing feature importance methods ignore or hide feature dependencies.
method Shapley Flow considers the entire causal graph and assigns credit to edges.
result Shapley Flow provides a deeper, graph-based view of feature importance.
MACC learns communication protocols by adapting counterfactual reasoning.
problem Credit assignment and non-stationarity in communication environments.
method Adapts counterfactual reasoning to overcome credit assignment and uses action policy and Q-function of other agents to handle non-stationarity.
result MACC outperforms state-of-the-art baselines in four scenarios.
Meta-learning for efficient reward shaping in RL.
problem Challenges in designing effective reward shaping functions in RL.
method Meta-learning framework to automatically learn reward shaping on multiple tasks.
result Significantly improved learning efficiency and interpretable visualizations across various RL settings.
PRISM integrates diverse rewards in MORL, improving sample efficiency and Pareto coverage.
problem Heterogeneous MORL where dense objectives dominate, leading to poor sample efficiency.
method PRISM uses reflectional symmetry and ReSymNet to reconcile temporal-frequency mismatches and accelerate exploration.
result PRISM consistently outperforms sparse-reward baselines and oracles, achieving significant Pareto gains.
Deep learning improves credit risk assessment without new data.
problem Improving credit risk assessment in banking without new data.
method Sequential deep learning using temporal convolutional networks.
result Sequential deep learning outperformed tree-based models in credit risk assessment.
SPoC uses search to translate pseudocode into correct programs with error localization.
problem Mapping pseudocode to functionally correct long programs.
method Search-based approach guided by compilation errors for credit assignment.
result Search improves synthesis success rate from 25.6% to 44.7%.
This work is attached to the BRICS 2013 competition. We propose a two-stage model for dealing with the temporal degradation of credit scoring models. This methodology produced motivating results in a 1-year horizon. We anticipate that it can be extended to other applications of risk assessment with great success. Futur…
CaT-GNN improves credit card fraud detection by integrating causal reasoning into GNNs.
problem Credit card fraud detection overlooks causal structure of transactions.
method CaT-GNN combines causal invariant learning and temporal graph neural networks.
result CaT-GNN outperforms existing methods on various datasets.
Delta-AI speeds up inference in sparse PGMs by local credit assignment.
problem Efficient inference in sparse probabilistic graphical models.
method Local credit assignment in agent's policy learning objective.
result Trained sampler recovers marginals and conditional distributions.
GASIL encourages agents to imitate past good trajectories in reinforcement learning.
problem Long-term credit assignment in sparse and delayed reward environments.
method Generative Adversarial Imitation Learning (GASIL) framework.
result GASIL improves performance in reinforcement learning tasks with delayed rewards.
HED Score improves temporal evaluation of detection accuracy.
problem Temporal agnosticism in existing evaluation frameworks for non-stationary processes.
method Measure-theoretic HED Score integrating exponentially decaying kernel over posterior probability stream.
result HED Score achieves 388.8% improvement over ROC/AUC on NSL-KDD benchmark.
Small Medium-sized Enterprises (SMEs) face many obstacles when they try to access credit market. These obstacles are increased if the SMEs are innovative. In this case, financial data are insufficient or even not reliable. Thus, when building a judgemental rating model, mainly based on qualitative criteria (soft inform…
Study finds implicit government guarantee improves municipal investment bond ratings.
problem Questioning the objectivity of municipal investment bond ratings due to implicit government guarantee.
method Text mining of policy documents and PMC index model for implicit guarantee strength calculation.
result Implicit government guarantee boosts municipal investment bond ratings, especially in less developed regions.
The study develops a machine learning system for credit scoring and default prediction.
problem Developing a robust credit rating and default prediction system.
method Combines NLP, AE, GBM, DE, and SHAP/LIME for model interpretability.
result Obtained excellent out-of-sample performance in credit rating and default prediction.
CREDIT learns to master pair trading with risk-aware RL, outperforming existing methods.
problem Challenges in applying RL to pair trading due to temporal correlations and risk considerations.
method Risk-aware recurrent reinforcement learning (RL) with bidirectional GRU and temporal attention.
result CREDIT achieves significant profit in pair trading over five years of U.S. stock data.