Unified view on selective credit assignment for reinforcement learning.
problem Efficient credit assignment in reinforcement learning.
method Unified temporal-difference algorithms with selective weightings.
result New algorithms for backward credit assignment and off-policy learning.
New algorithms assign credit to past decisions based on hindsight.
problem Efficient credit assignment in reinforcement learning.
method Explicit credit assignment using new data in hindsight.
result Value functions can be rewritten to include hindsight credit assignment.
COCOA improves credit assignment in reinforcement learning by measuring contributions to rewards.
problem Improving sample efficiency in reinforcement learning through better credit assignment methods.
method Counterfactual Contribution Analysis (COCOA) for precise credit assignment.
result COCOA achieves lower bias and variance compared to Hindsight Credit Assignment (HCA), improving reinforcement learning performance.
ProMP improves Meta-RL by better handling credit assignment.
problem Poor credit assignment in Meta-RL leads to suboptimal performance.
method Develops a novel meta-learning algorithm controlling pre-adaptation and adapted policies.
result Significantly improves sample-efficiency and asymptotic performance.
Eigenoptions improve credit assignment in reinforcement learning.
problem Improving credit assignment in reinforcement learning models.
method Investigated eigenoptions for credit assignment in model-free RL, comparing pre-specified and online discovery methods.
result Pre-specified eigenoptions aid exploration and credit assignment, while online discovery can hinder learning.
In this work, we study the credit assignment problem in reward augmented maximum likelihood (RAML) learning, and establish a theoretical equivalence between the token-level counterpart of RAML and the entropy regularized reinforcement learning. Inspired by the connection, we propose two sequence prediction algorithms, …
The paper introduces a health-informed policy gradient method for multi-agent reinforcement learning.
problem Optimizing joint reward functions in multi-agent systems with varying agent health.
method Health-informed credit assignment in a multi-agent proximal policy optimization algorithm.
result Significant improvement in learning performance compared to traditional methods.
Neural network uses local credit assignment to learn distant cause-effect relationships.
problem Learning distant cause-effect relationships in sequential data.
method Sparse coding in a recurrent neural network memory with local and immediate credit assignment.
result Network can predict partially-observable higher-order sequences and navigate mazes.
A model explains credit assignment in deep learning networks.
problem Understanding how deep learning assigns credit to parameters.
method Mean-field learning model with ensembles of sub-networks.
result Synaptic connections can be categorized into three types.
LICA learns credit assignment for cooperative agents without explicit formulation.
problem Credit assignment among cooperative agents.
method Centralized critic as hypernetwork, adaptive entropy regularization.
result Significantly outperforms previous methods on benchmarks.
The paper explores how planning with models improves credit assignment in reinforcement learning.
problem Improving credit assignment in reinforcement learning.
method Investigates the use of forward and backward models for planning in reinforcement learning.
result Establishes the relative merits and limitations of forward and backward planning mechanisms.
A new method backtracks through a few key past states to speed up credit assignment in long sequences.
problem Computational inefficiency of back-propagation through time for long sequences.
method Sparse attentive backtracking using learned attention mechanisms to skip connections.
result Matches or outperforms regular BPTT and truncated BPTT in tasks with long-term dependencies.
Introduces expected eligibility traces for more efficient credit assignment in reinforcement learning.
problem Efficiently assigning credit to states and actions in reinforcement learning.
method Introduces expected eligibility traces, allowing updates to counterfactual sequences.
result Substantial improvements in temporal-difference learning can be achieved with expected traces.
MRC improves credit assignment in multi-agent LLM systems, achieving high returns and transparency.
problem Lack of principled credit assignment in multi-agent LLM decision systems, vulnerability to regime shifts, and limited transparency.
method Market Regime Council (MRC) computes exact Shapley credits, uses exponentially weighted performance histories, Bayesian adaptive mixture, and regime-dependent multipliers.
result MRC achieves a Sharpe ratio of 1.51 and a cumulative return of 440.1% over 1,037 trading days, ranking first on CR, SR, and IR.
DPS uses posterior sampling for preference-based RL, achieving a first regret guarantee.
problem Formal frameworks for preference-based RL with theoretical analysis.
method Preference-based posterior sampling, Bayesian credit assignment.
result First asymptotic Bayesian no-regret rate for preference-based RL.
New framework for modular reinforcement learning reduces sample complexity.
problem Achieving independent credit assignment in reinforcement learning.
method Defining modular credit assignment as minimizing algorithmic mutual information, introducing modularity criterion for causal analysis.
result Single-step temporal difference action-value methods meet the modularity criterion, improving sample efficiency.
New methods improve deep reinforcement learning by accelerating credit assignment.
problem Challenges in achieving fast and stable off-policy learning in deep reinforcement learning.
method Extends the generalized PBE objective to support multistep credit assignment and derives three gradient-based methods.
result Proposed methods outperform PPO and StreamQ in MuJoCo and MinAtar environments.
This paper proposes a new algorithm for learning guidance rewards in RL.
problem Long-term temporal credit assignment in sparse or delayed reward environments.
method Surrogate RL objective with trajectory-space smoothing to learn guidance rewards.
result Guidance rewards can be learned without additional neural networks and have intuitive interpretation.
Proposes a new neural network approach to credit assignment.
problem Credit assignment problem in deep neural networks.
method Contrastive similarity matching objective function.
result Deep networks learn to match similarity between layers.
CM3 learns multi-agent cooperation by first achieving individual goals.
problem Cooperative multi-agent control with multiple goals and interactions.
method Two-stage curriculum: first learn individual goals, then cooperation; new policy gradient with credit function.
result CM3 learns faster on multi-goal multi-agent problems than existing algorithms.
New methods reduce variance in stochastic computation graph optimization.
problem High variance in gradient estimates from SCGs.
method Introducing value functions, baselines, and critics to derive lower-variance gradient estimates from partial model evaluations.
result Lower-variance gradient estimates from partial model evaluations, making optimization more efficient.
Shapley Flow interprets model predictions using a graph-based approach to feature importance.
problem Existing feature importance methods ignore or hide feature dependencies.
method Shapley Flow considers the entire causal graph and assigns credit to edges.
result Shapley Flow provides a deeper, graph-based view of feature importance.
Paper tackles end-to-end training of complex neural networks using DIP method.
problem Training complex heterogeneous neural network models end-to-end.
method Deep Innovation Protection (DIP) method using multiobjective optimization.
result End-to-end training of complex heterogeneous neural network models is possible.
MACC learns communication protocols by adapting counterfactual reasoning.
problem Credit assignment and non-stationarity in communication environments.
method Adapts counterfactual reasoning to overcome credit assignment and uses action policy and Q-function of other agents to handle non-stationarity.
result MACC outperforms state-of-the-art baselines in four scenarios.
Meta-learning for efficient reward shaping in RL.
problem Challenges in designing effective reward shaping functions in RL.
method Meta-learning framework to automatically learn reward shaping on multiple tasks.
result Significantly improved learning efficiency and interpretable visualizations across various RL settings.
SPoC uses search to translate pseudocode into correct programs with error localization.
problem Mapping pseudocode to functionally correct long programs.
method Search-based approach guided by compilation errors for credit assignment.
result Search improves synthesis success rate from 25.6% to 44.7%.
New algorithm learns reward signals from episodic returns for better reinforcement learning.
problem Difficulty in designing reward functions for real-world reinforcement learning tasks.
method Introduces a new algorithm that decomposes episodic returns into time-step rewards using deep neural networks.
result Learning reward signals from episodic returns improves reinforcement learning efficiency.
Improved GFlowNets learn more efficiently with trajectory balance.
problem Inefficient credit assignment in GFlowNets leads to suboptimal learning.
method Proposed trajectory balance as a new learning objective.
result Trajectory balance leads to more efficient and robust GFlowNet learning.
Delta-AI speeds up inference in sparse PGMs by local credit assignment.
problem Efficient inference in sparse probabilistic graphical models.
method Local credit assignment in agent's policy learning objective.
result Trained sampler recovers marginals and conditional distributions.
GASIL encourages agents to imitate past good trajectories in reinforcement learning.
problem Long-term credit assignment in sparse and delayed reward environments.
method Generative Adversarial Imitation Learning (GASIL) framework.
result GASIL improves performance in reinforcement learning tasks with delayed rewards.
Small Medium-sized Enterprises (SMEs) face many obstacles when they try to access credit market. These obstacles are increased if the SMEs are innovative. In this case, financial data are insufficient or even not reliable. Thus, when building a judgemental rating model, mainly based on qualitative criteria (soft inform…
Study finds implicit government guarantee improves municipal investment bond ratings.
problem Questioning the objectivity of municipal investment bond ratings due to implicit government guarantee.
method Text mining of policy documents and PMC index model for implicit guarantee strength calculation.
result Implicit government guarantee boosts municipal investment bond ratings, especially in less developed regions.
The study develops a machine learning system for credit scoring and default prediction.
problem Developing a robust credit rating and default prediction system.
method Combines NLP, AE, GBM, DE, and SHAP/LIME for model interpretability.
result Obtained excellent out-of-sample performance in credit rating and default prediction.
Paper proposes a self-learning framework for reject inference in credit scoring.
problem Sample bias in credit scoring models due to training on accepted cases only.
method Develops a self-learning framework considering distinct training regimes for iterative labeling and model training, introduces a new evaluation measure.
result Demonstrates the superiority of the adjusted self-learning framework over regular self-learning and previous reject inference strategies.
Two local learning rules are investigated to avoid weight transport in neural networks.
problem Local learning rules that avoid weight transport are unstable and require tuning.
method Investigated two non-local learning rules and a more robust local rule.
result Non-local learning rules match state-of-the-art performance and operate effectively in noisy updates.
The role of credit rating agencies has been under severe scrutiny after the subprime crisis. In this paper we explore the relationship between credit ratings and informational efficiency of a sample of thirty nine corporate bonds of US oil and energy companies from April 2008 to November 2012. For that purpose, we use …
New hierarchical RL agent learns to generalize in complex multi-agent games.
problem Current RL methods struggle to generalize to unseen opponents in multi-agent games.
method Proposed a hierarchical RL architecture grounded in game-theoretic structure.
result Hierarchical agent generalizes to unseen opponents, while baselines fail.
The paper formalizes and analyzes multi-agent Q-learning with value factorization.
problem Understanding and improving the convergence of multi-agent Q-learning with value factorization.
method Formalized a multi-agent fitted Q-iteration framework for analyzing factorized multi-agent Q-learning.
result Multi-agent Q-learning with linear value factorization can converge under certain conditions.
New STDP rule for spiking neurons solves discrete action reinforcement learning tasks.
problem Applying standard STDP to discrete action reinforcement learning tasks.
method Feedback-modulated TD-STDP learning rule for spiking neuron networks.
result Feedback modulation improves credit assignment in reinforcement learning.
This work tackles ML accountability by quantifying contributions of algorithms and datasets.
problem Determining responsibility for poor performance on subpopulations at test time.
method Proposes Extended Shapley as a principled framework for joint credit assignment.
result Demonstrates Extended Shapley can be used to address ML accountability questions.
Paper analyzes deep learning models for credit rating prediction using text and numerical data.
problem Improving credit rating prediction using multi-modal deep learning.
method Testing different deep learning models and fusion strategies for structured and unstructured datasets.
result CNN-based multi-modal model with two fusion strategies outperformed other models.
Improves text-to-image diffusion models using GFlowNets.
problem Aligning diffusion models with text descriptions.
method Post-training diffusion models with GFlowNets to generate high-reward images.
result Effective alignment of large-scale text-to-image diffusion models with reward information.
Identifying customer segments in retail banking portfolios with different risk profiles can improve the accuracy of credit scoring. The Variational Autoencoder (VAE) has shown promising results in different research domains, and it has been documented the powerful information embedded in the latent space of the VAE. We…
Paper compares ML methods for credit scoring, highlighting feature selection and scaling impacts.
problem Determining default risk in credit scoring models.
method Eight ML methods (SVM, Naive Bayes, DT, RF, XGBoost, KNN, MLP, LR) with feature selection and scaling.
result Feature selection and scaling improve model performance in credit scoring.
New method combines value function decomposition and policy gradients for cooperative multi-agent reinforcement learning.
problem Challenges in cooperative multi-agent reinforcement learning, especially credit assignment and large action spaces.
method Decomposed Soft Actor-Critic (mSAC) method with Q network architecture, discrete probabilistic policy, and counterfactual advantage function.
result Significantly outperforms policy-based approach COMA and achieves competitive results with SOTA value-based approach Qmix.
New insights into how to inspect and learn from multi-stage processes and AI reasoning.
problem Understanding how to attribute outcomes to early stages in multi-stage operations and AI reasoning.
method Information-theoretic analysis and mathematical proofs of four key results.
result Uniform checkpoint spacing is minimax-optimal for inspection design under homogeneous signal attenuation.
This paper provides a link between causal inference and machine learning techniques - specifically, Classification and Regression Trees (CART) - in observational studies where the receipt of the treatment is not randomized, but the assignment to the treatment can be assumed to be randomized (irregular assignment mechan…
Deep Reinforcement Learning (DRL) algorithms have been successfully applied to a range of challenging control tasks. However, these methods typically suffer from three core difficulties: temporal credit assignment with sparse rewards, lack of effective exploration, and brittle convergence properties that are extremely …