FinFlowRL combines imitation and reinforcement learning for better financial control.
problem Traditional stochastic control methods fail in real-world finance due to changing market conditions.
method FinFlowRL uses imitation learning to pretrain an adaptive meta policy, then finetunes it with reinforcement learning.
result FinFlowRL consistently outperforms individual strategies across various market conditions.
Paper introduces a novel reward function for noisy financial markets using imitation learning.
problem Noisy reward function in financial markets hinders RL agent performance.
method Integrates imitation learning feedback with reinforcement learning to improve reward function design.
result Improves financial performance metrics compared to traditional benchmarks and RL agents.
IMM uses imitation learning and predictive representation learning to improve market making strategies.
problem Challenges in training RL agents for multi-price level market making strategies.
method IMM combines RL and imitation learning, introducing effective state and action representations and a representation learning unit.
result IMM outperforms existing RL-based market making strategies in financial criteria.
Model shows how imitation and randomness stabilize financial markets.
problem Stabilizing financial markets with self-organized criticality.
method Simple order book mechanism with self-organized criticality dynamics.
result Imitation and randomness stabilize market fluctuations.
Generative tools mimic stock market traders using synthetic data.
problem Imitating trading behavior of stock market participants.
method Modified state-space model applied to limit order book data, trained on synthetic data generated from a heterogeneous agent-based model.
result Model's predicted distribution matches ground truths from the agent-based model.
FlowOE learns from experts to optimize financial trades.
problem Optimal execution in dynamic financial markets using static models.
method Imitation learning with flow matching models, incorporating refining loss function.
result Significantly outperforms expert models and traditional benchmarks.
RL framework optimizes trading costs in noisy markets.
problem Optimal execution and placement in noisy markets.
method Dual-window Denoise PPO RL network, imitation learning, comprehensive market features, flexible action formulation.
result RL agents outperformed TWAP strategy in execution cost.
The study classifies and imitates trading agents in financial markets.
problem Classifying and imitating trading agents in continuous double auctions.
method Developed an agent-based model for trading, applied opponent modeling for classification, and used behavioral cloning for imitation.
result Techniques for classification and imitation were experimentally compared and evaluated.
Lab experiment reveals market imitation and win-stay lose-shift patterns in financial decision-making.
problem Understanding how people make decisions in financial markets.
method Lab-in-the-field experiment with financial information, statistical analysis, and cohort analysis.
result Market imitation and win-stay lose-shift strategies emerge as dominant behaviors in financial decision-making.
Study optimal investment under imitation of decision-changing rates.
problem Optimal investment under imitation of decision-changing rates.
method Proposed integral disparity to quantify imitation, derived general solution using variational method, analyzed asymptotic properties, validated with real data.
result Investor's optimal decisions under imitation of decision-changing rates.
FlowHFT learns adaptive trading strategies from multiple models for diverse market conditions.
problem Traditional HFT models are limited by specific market conditions and cannot adapt to dynamic markets.
method FlowHFT uses flow matching policy to learn from multiple expert models and adapt to various market scenarios.
result FlowHFT consistently outperforms individual expert models in multiple market conditions.
The \$-Game was recently introduced as an extension of the Minority Game. In this paper we compare this model with the well know Minority Game and the Majority Game models. Due to the inter-temporal nature of the market payoff, we introduce a two step transaction with single and mixed group of interacting traders. When…
We present a financial market model, characterized by self-organized criticality, that is able to generate endogenously a realistic price dynamics and to reproduce well-known stylized facts. We consider a community of heterogeneous traders, composed by chartists and fundamentalists, and focus on the role of informative…
MetaTrader combines diverse expert strategies to optimize portfolio performance.
problem Optimizing portfolio performance in changing financial markets.
method Two-stage RL approach: imitation learning followed by a meta-policy.
result MetaTrader significantly outperforms state-of-the-art baselines in balancing profits and risks.
The paper uses a novel framework to learn option prices by imitating principal investor behavior.
problem Challenges in modeling stock price changes and decision making in equity markets.
method Non-deterministic Markov decision process, Bayesian deep neural network, reinforcement learning.
result Optimal option prices learned through imitation of principal investor behavior.
Improved power arbitrage through domain-adapted reinforcement learning.
problem Optimizing profit in the Dutch power market through arbitrage opportunities.
method Dual-agent reinforcement learning with imitation of power traders' behaviors.
result Significant improvement in cumulative profit and loss (P&L) with a three-fold increase.
QTNet uses deep reinforcement learning to automate trading strategies.
problem Handling noisy and high-frequency financial data, balancing exploration and exploitation.
method QTNet employs deep reinforcement learning (DRL) with imitative learning to autonomously formulate trading strategies.
result QTNet demonstrates proficiency in extracting robust market features and adaptability to diverse conditions.
Study on how technical analysis affects wealth distribution in agent-based model of investors.
problem Understanding wealth distribution and return rates in financial markets.
method Agent-based model with psychological profiles and decision-making strategies.
result Anti-imitation strategy becomes most profitable when applied to investors' hub.
Constructs portfolios based on Hellinger distance to normal, finding market invariance.
problem Finding a market invariant for portfolio construction.
method Uses Hellinger distance to normal distribution for portfolio construction and analysis.
result Minimum Hellinger distance varies drastically between markets, suggesting market invariance.
The paper proposes a test for trading intelligence similar to the Turing Test.
problem Determining if trading programs exhibit intelligent behavior.
method Develops a test based on the Turing Test for financial trading programs.
result Introduces a method to assess trading intelligence in financial markets.
We present a simple model of a stock market where a random communication structure between agents gives rise to a heavy tails in the distribution of stock price variations in the form of an exponentially truncated power-law, similar to distributions observed in recent empirical studies of high frequency market data. Ou…
Deep learning predicts stock trends from chaotic online news.
problem Predicting stock trends from volatile and non-stationary stock market data.
method Hybrid Attention Networks and self-paced learning mechanism.
result Demonstrated effectiveness in predicting stock trends from online news.
Establishing unambiguously the existence of speculative bubbles is an on-going controversy complicated by the need of defining a model of fundamental prices. Here, we present a novel empirical method which bypasses all the difficulties of the previous approaches by monitoring external indicators of an anomalously growi…
New method aligns states for better imitation learning.
problem Different dynamics models between imitator and expert.
method State alignment-based reinforcement learning.
result Superior performance on various imitation learning settings.
We propose a generic model for multiple choice situations in the presence of herding and compare it with recent empirical results from a Web-based music market experiment. The model predicts a phase transition between a weak imitation phase and a strong imitation, `fashion' phase, where choices are driven by peer press…
ADVISOR dynamically balances imitation and reinforcement learning to overcome the imitation gap.
problem The gap between imitation learning and reinforcement learning when teaching agents have privileged information.
method Adaptive Insubordination (ADVISOR) dynamically weights imitation and reward-based reinforcement learning losses.
result On-the-fly switching with ADVISOR outperforms pure imitation, pure reinforcement learning, and their combinations.
Imitative and contrarian behaviors are the two typical opposite attitudes of investors in stock markets. We introduce a simple model to investigate their interplay in a stock market where agents can take only two states, bullish or bearish. Each bullish (bearish) agent polls m "friends'' and changes her opinion to bear…
HGAIL learns policies without expert demonstrations.
problem Lack of expert demonstrations in imitation learning.
method Combines hindsight and GAIL to learn policies.
result Comparable performance to current methods, with curriculum learning.
FinFlowRL learns from experts to optimize financial control in changing markets.
problem Traditional finance control methods fail in real-world, non-stationary markets.
method Imitation-Reinforcement Learning framework that pretrains on expert strategies and finetunes in noise space.
result Consistently outperforms individually optimized experts across diverse market conditions.
New algorithm reduces sample complexity for imitation learning.
problem High sample complexity limits deployment of adversarial imitation algorithms.
method Trajectory-centric reinforcement learning ideas.
result Improved learning rate and efficiency in imitation tasks.
Reinforcement learning mimics expert behavior.
problem Learning from expert demonstrations in reinforcement learning.
method Reduction to reinforcement learning with a stationary reward.
result Expert reward can be recovered and imitation learning is bounded.
Guarantees for third-person imitation learning from offline data.
problem Improving generalizability in imitation learning.
method Problem-dependent statistical learning guarantees for third-person imitation from offline observation.
result Strong performance guarantees for transferred policies in the offline setting.
TRAIL improves robot imitation learning by focusing on task-relevant features.
problem Discriminator networks learn spurious associations, providing poor reward signals.
method Constrained discriminator optimization to learn task-relevant rewards.
result TRAIL outperforms GAIL and behaviour cloning in robotic manipulation tasks.
GWIL uses Gromov-Wasserstein distance to align expert and imitation agent states.
problem Cross-domain imitation learning challenges due to different system dimensions and stationary distributions.
method Gromov-Wasserstein Imitation Learning (GWIL) using Gromov-Wasserstein distance.
result GWIL effectively aligns expert and imitation agent states in various continuous control domains.
New metric solves correspondence problem for robotic arm imitation learning.
problem Establishing corresponding states and actions between different robotic arms.
method Introducing a distance measure between dissimilar robotic arms and using it as a loss function.
result The distance measure effectively learns imitation policies by minimizing distance between robotic arms.
Zero-shot imitation learns to imitate tasks without expert demonstrations.
problem Learning to imitate tasks without expert supervision.
method Explore world without supervision, distill experience into goal-conditioned policy, use forward consistency loss.
result Zero-shot imitation policy can mimic expert tasks after seeing just goal demonstrations.
Study risk-sensitive imitation learning using GAIL and Wasserstein distance.
problem Improve imitation learning performance by considering risk profiles.
method Formulate risk-sensitive imitation learning, derive optimization problems for JS divergence and Wasserstein distance, develop algorithms.
result RS-GAIL algorithms outperform GAIL and RAIL in MuJoCo and OpenAI tasks.
New framework integrates imitation and reinforcement learning for better robot performance.
problem Combining reinforcement and imitation learning for intelligent robotics.
method Extends probabilistic generative model framework for reinforcement learning and develops pMDP-MO for Markov decision processes.
result Significantly better performance than reinforcement or imitation learning alone.
Proof shows imitation of expert's reward and solutions in multi-objective optimization.
problem Multi-objective optimization with reward and solution imitation.
method Wasserstein inverse reinforcement learning.
result Wasserstein inverse reinforcement learning enables imitation of expert's reward and solutions in multi-objective optimization.
Researchers improved Minecraft game performance using imitation learning.
problem Achieving state-of-the-art performance in immersive environments like Minecraft.
method Applied imitation learning to Minecraft, optimizing network architecture, loss function, and data augmentation.
result Reported stronger results than previous experiments, reaching second place in a competition.
Paper tackles robust imitation learning from noisy demonstrations.
problem Learning from noisy demonstrations is challenging.
method Optimizes a classification risk with a symmetric loss, combining pseudo-labeling and co-training.
result Our method is more robust than state-of-the-art methods.
We find empirically a characteristic sharp peak-flat trough pattern in a large set of commodity prices. We argue that the sharp peak structure reflects an endogenous inter-market organization, and that peaks may be seen as local ``singularities'' resulting from imitation and herding. These findings impose a novel strin…
Extends InfoGAIL to improve imitation learning over extended periods.
problem Stable imitation of extended expert behavior from multi-modal data.
method Incorporates 'burn-in demonstrations' to condition policies at test time.
result Improves mutual information and long-term imitation of expert behavior.
New framework for multi-agent imitation learning in complex environments.
problem Multi-agent settings with multiple Nash equilibria and non-stationary environments.
method Generalized inverse reinforcement learning and actor-critic algorithm.
result Practical multi-agent imitation learning with good empirical performance.
Paper uses Sinkhorn distances to improve imitation learning effectiveness.
problem Improving imitation learning algorithms by comparing occupancy measures.
method Formulates imitation learning as Sinkhorn distance minimization, combining optimal transport and cosine distances.
result Proposes a new critic network and transport plan that guide imitation learning.
New method uses bi-level optimization to learn useful representations for imitation learning.
problem Learning useful representations for multiple tasks in imitation learning settings.
method Formulates representation learning as a bi-level optimization problem.
result Bi-level optimization framework provides sample complexity benefits for imitation learning.
New self-imitation learning method improves performance in continuous control tasks.
problem Improving off-policy learning in continuous control tasks.
method Proposes a n-step lower bound to generalize lower-bound Q-learning and introduces a new family of self-imitation learning algorithms.
result n-step lower bound Q-learning achieves a better trade-off between bias and contraction rate, leading to improved performance.
GASIL encourages agents to imitate past good trajectories in reinforcement learning.
problem Long-term credit assignment in sparse and delayed reward environments.
method Generative Adversarial Imitation Learning (GASIL) framework.
result GASIL improves performance in reinforcement learning tasks with delayed rewards.