A new IL framework estimates invariant predictors with single domain data.
problem Deep networks inherit spurious correlations and fail on unseen domains.
method Assumes multiple labeled domains for higher-level tasks, uses single domain for target task, employs cross-validation for hyperparameter selection.
result Empirically demonstrates effectiveness and correctness of hyperparameter selection.
This work uses PAC-Bayes for structured prediction with ILE, yielding insights and algorithms.
problem Structured prediction with interdependent outputs and implicit loss embeddings.
method PAC-Bayes perspective applied to ILE framework, deriving generalization bounds and learning algorithms.
result Two learning algorithms derived from PAC-Bayes bounds, analyzed and implemented.
New method detects information leakage using approximate Bayes predictor.
problem Unintentional exposure of sensitive information via observable data.
method Statistical learning theory and information theory framework, approximating Bayes predictor's log-loss and accuracy.
result MI can be accurately estimated to detect ILs, outperforming state-of-the-art baselines.
Imitation learning (IL) consists of a set of tools that leverage expert demonstrations to quickly learn policies. However, if the expert is suboptimal, IL can yield policies with inferior performance compared to reinforcement learning (RL). In this paper, we aim to provide an algorithm that combines the best aspects of…
iLED framework offers interpretable dynamics for multiscale systems.
problem Modeling high-dimensional multiscale systems is challenging.
method Interpretable Learning Effective Dynamics (iLED) framework based on Mori-Zwanzig and Koopman operator theory.
result Comparable accuracy to state-of-the-art approaches with added interpretability.
This work explains why online imitation learning improves faster than theory predicts.
problem Online imitation learning's empirical policy improvement speed exceeds theoretical predictions.
method The authors analyze online imitation learning with a convex, smooth, and non-negative loss function, proving policy improvement in expectation and high probability.
result Adopting a sufficiently expressive policy class in online IL increases both policy improvement speed and performance bias.
Paper tackles covariate shift in offline IL using less proficient behavior data.
problem Mitigating covariate shift in Imitation Learning with limited data coverage.
method Model-based IL from Offline Data (MILO) framework.
result MILO can combat covariate shift even with sub-optimal behavior policy data.
Imitation learning (IL) aims to learn an optimal policy from demonstrations. However, such demonstrations are often imperfect since collecting optimal ones is costly. To effectively learn from imperfect demonstrations, we propose a novel approach that utilizes confidence scores, which describe the quality of demonstrat…
MobILE learns from expert demonstrations without access to actions, achieving strong performance guarantees.
problem Imitation Learning from Observations alone (ILFO) where actions are not observed.
method MobILE integrates optimism in the face of uncertainty into distribution matching IL framework, trading off exploration and imitation.
result MobILE provides strong performance guarantees for certain MDP dynamics and demonstrates efficacy on OpenAI Gym tasks.
We study how to effectively leverage expert feedback to learn sequential decision-making policies. We focus on problems with sparse rewards and long time horizons, which typically pose significant challenges in reinforcement learning. We propose an algorithmic framework, called hierarchical guidance, that leverages the…
Unified methods for hedging impermanent loss in decentralized exchanges.
problem Hedging impermanent loss in liquidity provision at decentralized exchanges.
method Static and dynamic approaches using arbitrage-based methods for valuation and risk management.
result Unified valuation and hedging formulas for IL protection claims.
NDI learns from expert demonstrations by estimating occupancy measures.
problem Imitation Learning (IL) for complex systems.
method Density estimation of expert's occupancy measure followed by RL.
result NDI achieves state-of-the-art performance on control tasks.
Behavior cloning can achieve horizon-independent sample complexity in offline imitation learning.
problem Sample complexity in imitation learning increases with problem horizon.
method New analysis of behavior cloning with logarithmic loss.
result Behavior cloning can achieve linear dependence on horizon in offline IL under dense rewards.
Paper evaluates deadline-ILS on insider trading contracts, finding it distinguishes signals from noise.
problem Deadlines in insider trading contracts and information leakage detection.
method Empirical evaluation using FFIC dataset, hazard-rate estimation, cross-market wallet analysis.
result Deadline-ILS distinguishes signal from proxy artefact, with a significant shift in magnitude.
New state-only IL algorithm tackles MDP transition mismatch.
problem Transition dynamics mismatch between expert and imitator MDPs.
method Adversarial state-only IL with two subproblems solved iteratively.
result Effective performance improvement in transition dynamics mismatch scenarios.
Study evaluates information leakage in Polymarket markets, finding limited applicability and resolution ambiguity.
problem Limited applicability of ILS-dl framework across Polymarket markets.
method Scaling from single-case to population-scale evaluation using ILS-dl framework.
result Only 0.7% of candidate markets yield computable ILS-dl values, and resolution semantics are the main obstacle.
EBIL simplifies IL by estimating expert energy as reward, achieving effective performance.
problem Recovering optimal policy from expert demonstrations without reward signals.
method EBIL uses a two-stage solution: first estimating expert energy as reward, then learning policy.
result EBIL achieves effective performance and interpretable reward signals.
The paper explores IL and LVR in AMMs, identifying three regimes and the effect of fees.
problem The relationship between impermanent loss and loss-versus-rebalancing in AMMs.
method Statistical analysis, focus on fees, block times, and continuous time limit.
result Three regimes identified: identical, distinct distribution functions, and distinct averages.
Interactive IL algorithm queries noisy expert for feedback, reducing sample size.
problem Imitation learning with noisy expert feedback requires many samples.
method Selective sampling algorithm with online regression oracle.
result Interactive IL algorithm reduces query complexity and improves performance.
EGFs use ergodicity to simplify generative flows for easier training and imitation learning.
problem Challenges in training generative flows, especially in continuous settings and for imitation learning.
method EGFs leverage ergodicity to build simple flows with universality guarantees and tractable FM loss. They introduce a KL-weakFM loss for IL training without a separate reward model.
result EGFs simplify generative flow training and enable effective imitation learning.
The goal of imitation learning (IL) is to learn a good policy from high-quality demonstrations. However, the quality of demonstrations in reality can be diverse, since it is easier and cheaper to collect demonstrations from a mix of experts and amateurs. IL in such situations can be challenging, especially when the lev…
Imitation Learning (IL) is an appealing approach to learn desirable autonomous behavior. However, directing IL to achieve arbitrary goals is difficult. In contrast, planning-based algorithms use dynamics models and reward functions to achieve goals. Yet, reward functions that evoke desirable behavior are often difficul…
H-ReIL learns to drive safely in near-accident scenarios.
problem Driving safely in high-risk near-accident situations.
method Hierarchical RL and IL approach.
result High-level policy switches between low-level policies for safe driving.
Learning to solve complex goal-oriented tasks with sparse terminal-only rewards often requires an enormous number of samples. In such cases, using a set of expert trajectories could help to learn faster. However, Imitation Learning (IL) via supervised pre-training with these trajectories may not perform as well and gen…
The paper develops a new framework for pricing and hedging liquidity in crypto markets.
problem Arbitrage and risk management in crypto market making.
method Developed a new mathematical framework using a coordinate system defined by price and intrinsic liquidity.
result Established a linear dependence of asset reserves and value functions on intrinsic liquidity, facilitating arbitrage-free pricing and delta hedging.
Predictive coding networks use inference learning for efficient AI modeling.
problem Traditional AI methods struggle with complex neural patterns.
method Inference learning for hierarchical Bayesian inference models.
result PCNs outperform traditional BP methods in efficiency and flexibility.
Survey of methods for learning from observation without requiring expert actions.
problem Lack of access to expert actions in imitation learning.
method Survey and classification of state-only imitation learning (SOIL) methods.
result Identification of open problems and future research directions.
Paper proposes a new RL approach combining IL and RL methods to improve decision-making.
problem Challenges in RL with large state and action spaces, and difficulty in reward determination.
method Combines Imitation Learning and RL methods (SARSA and A3C) to learn sequential decision-making policies.
result Significantly decreases human effort and exploration time in learning decision-making policies.
The paper examines statistical properties of IL and LVR in automated market makers.
problem Assessing the performance of automated market makers and their profitability.
method Analysis of random walk properties and statistical integral combined with CFMM mechanics.
result IL and LVR have identical expectation values but different distribution functions for Brownian motion.
The study models and values CAT bonds across multiple regions.
problem Valuation of CAT bonds with dependencies across different regions.
method Developed models for independent, proportional, and arbitrary two-dimensional distribution cases of catastrophe losses in different areas. Applied normal approximation and Wang's transform for pricing.
result Illustrated differences in scenarios and performance of the approximation on real data.
Paper tackles class-incremental learning by combining self-supervised learning to mitigate prior information loss.
problem Catastrophic forgetting and prior information loss in class-incremental learning.
method Combining self-supervised learning with class-incremental learning to mitigate prior information loss.
result Proposed method outperforms state-of-the-art methods.
ForesightFlow detects informed trading on prediction markets using an information leakage score.
problem Detecting informed trading on decentralized prediction markets.
method Developed an Information Leakage Score (ILS) framework to quantify the fraction of terminal information move priced in before public news events.
result The score connects label generation to proper-scoring-rule literature and reveals systematic biases in insider trading documentation.
In many settings, it is desirable to learn decision-making and control policies through learning or bootstrapping from expert demonstrations. The most common approaches under this Imitation Learning (IL) framework are Behavioural Cloning (BC), and Inverse Reinforcement Learning (IRL). Recent methods for IRL have demons…
This paper surveys imitation learning methods and challenges.
problem Difficulty in programming AI systems in complex environments.
method Learning from expert demonstrations to adapt behaviors.
result Overview of recent advances and emerging areas in IL.
Scaling up model and data size improves imitation learning in single-agent games.
problem Limited recovery of expert behavior in single-agent games using imitation learning.
method Investigate the effect of scaling model and data size on imitation learning performance.
result IL loss and mean return scale with compute budget, resulting in power laws.
PWIL learns agent behavior from expert using Wasserstein distance.
problem Learning agent behavior from expert demonstrations efficiently.
method Primal Wasserstein Imitation Learning (PWIL) with offline reward function.
result PWIL achieves expert behavior on MuJoCo tasks with minimal interactions.
Every 4-dimensional infrasolvmanifold M with β1(M;Q)>0 or which is flat or has one of the geometries Nil4, Solm,n4, or Sol04 bounds. However there are non-orientable Sol14-manifolds which do not bound. The question remains open for $\mathbb{N}il^3\times…
In this article, we develop a general framework to study optimal execution and to price block trades. We prove existence of optimal liquidation strategies and we provide regularity results for optimal strategies under very general hypotheses. We exhibit a Hamiltonian characterization for the optimal strategy that can b…
MAMBA learns policies competitive with multiple conflicting oracles.
problem Learning policies from multiple conflicting oracles in reinforcement learning.
method MAMBA uses a gradient estimator in the style of GAE to optimize policies, leveraging demonstrations from multiple weak oracles.
result MAMBA outperforms the state-of-the-art in learning policies competitive with multiple conflicting oracles.
We show that if M is an orientable 4-dimensional infrasolvmanifold and either β=β1(M;Q)≥2 or M is a Sol04- or a Solm,n4-manifold (with m=n) then M is parallelizable. There are non-parallelizable examples with β=1 for each of the other solvable Lie geometries $\ma…
We show that for any simple non-oriented graph G with at least thirteen vertices either G or its complement is intrinsically linked.
Causal Imitation Learning handles noisy measurements and distribution shifts.
problem Learning from noisy state observations and distributional shifts.
method Causal inference framework and adversarial RKHS learning.
result Improved robustness to distribution shifts compared to standard methods.
New method improves RL/IL agents' adaptability to unseen environments.
problem Current RL/IL techniques struggle with generalizing to unseen environments.
method Zero-shot compositional policy learning with multi-modal fusion and attention mechanism.
result Language grounding enhances generalization across varied environments.
WDAIL uses Wasserstein distance for more effective reward shaping in IL.
problem Fixed reward functions in GAIL limit performance on complex tasks.
method Introduces Wasserstein distance and PPO for improved reward shaping and stability.
result Significant performance improvement in complex MuJoCo tasks.
RPI combines imitation and reinforcement learning to improve policies efficiently.
problem High sample complexity in reinforcement learning.
method Active interleaving between imitation and reinforcement learning, using oracle queries for exploration.
result RPI outperforms existing methods across various domains.
Interactive IL beats BC by state-wise annotation cost.
problem Behavior Cloning struggles with annotation cost in sequential decision making.
method Proved Stagger and Warm Stagger algorithms to outperform BC.
result Interactive and hybrid IL methods outperform BC with state-wise annotation.
Framework improves risk neutral density estimation in illiquid markets.
problem Challenges in estimating Risk Neutral Density in illiquid markets.
method Introduces Deep Log-Sum-Exp Neural Network leveraging Deep and Transfer learning.
result Framework recovers Risk Neutral Density with few option quotes in severe illiquidity.
Uniswap v3 LPs suffer significant Impermanent Loss despite higher fees.
problem Impermanent Loss in leveraged liquidity provision on Uniswap v3.
method Analysis of 17 pools covering 43% of TVL, calculating fees and IL.
result LPs would have been better off by $60.8m had they HODLd.