This paper investigates how to efficiently transition and update policies, trained initially with demonstrations, using off-policy actor-critic reinforcement learning. It is well-known that techniques based on Learning from Demonstrations, for example behavior cloning, can lead to proficient policies given limited data…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Cloning functionality”
Paper introduces DNTs to clone black-box models efficiently.
problem Cloning functionality of black-box models.
method Deep Neural Trees (DNTs) trained with active learning.
result Trained DNT can clone task-specific behavior of black-box models.
We address the problem of imitation learning with multi-modal demonstrations. Instead of attempting to learn all modes, we argue that in many tasks it is sufficient to imitate any one of them. We show that the state-of-the-art methods such as GAIL and behavior cloning, due to their choice of loss function, often incorr…
MAZE attacks a model without data, using synthetic inputs.
problem Model Stealing attacks that require data to replicate a target model.
method MAZE uses zeroth-order gradient estimation to create synthetic inputs.
result MAZE achieves high clone accuracy without data.
The majority of cancer treatments end in failure due to Intra-Tumor Heterogeneity (ITH). ITH in cancer is represented by clonal evolution where different sub-clones compete with each other for resources under conditions of Darwinian natural selection. Predicting the growth of these sub-clones within a tumour is among t…
This paper provides theoretical foundations for using quantized actions in behavior cloning.
problem Applying autoregressive models to continuous control requires discretizing actions through quantization, which is poorly understood.
method The paper analyzes quantization error propagation and statistical sample complexity, and proposes model-based augmentation.
result Behavior cloning with quantized actions achieves optimal sample complexity, matching existing lower bounds.
The paper provides theoretical guarantees for behavior cloning using generative models.
problem Behavior cloning of complex expert demonstrations using generative models.
method The paper proposes a theoretical framework invoking low-level controllers to stabilize imitation around expert demonstrations. It shows that with suitable low-level stability guarantees and powerful generative models, pure supervised behavior cloning can match expert trajectories.
result The paper proves that with a suitable low-level stability guarantee and a powerful enough generative model, pure supervised behavior cloning can generate trajectories matching the per-time step distribution of essentially arbitrary expert trajectories in an optimal transport cost.
Deep Neural Networks (DNNs) are susceptible to model stealing attacks, which allows a data-limited adversary with no knowledge of the training dataset to clone the functionality of a target model, just by using black-box query access. Such attacks are typically carried out by querying the target model using inputs that…
Compressed imitation learning uses simplicity priors for efficient expert behavior copying.
problem Efficiently learn expert behaviors with minimal data.
method Utilizes policy simplicity as a prior for sample-efficient imitation learning.
result Significantly higher scores achieved with limited expert demonstrations.
Behavior cloning training instabilities amplified by SGD noise over long horizons.
problem Training instabilities in behavior cloning with deep neural networks.
method Empirical dissection of minibatch SGD updates and their effects on long-horizon rewards.
result Exponential moving average (EMA) of iterates effectively mitigates gradient variance amplification (GVA).
The susceptibility of deep learning to adversarial attack can be understood in the framework of the Renormalisation Group (RG) and the vulnerability of a specific network may be diagnosed provided the weights in each layer are known. An adversary with access to the inputs and outputs could train a second network to clo…
Improves RL from historical data by stitching trajectories.
problem Lack of high-quality data for offline RL.
method Trajectory Stitching (TS) to augment historical data with synthetic actions.
result Improves RL policy performance over baseline.
ORIL learns a reward function from unlabeled data to improve robot learning.
problem Leveraging unlabeled data for robot learning.
method ORIL learns a reward function from demonstrator and unlabeled trajectories, annotates data, and trains an agent via offline reinforcement learning.
result ORIL consistently outperforms BC agents on various robotic tasks.
New technique improves imitation learning by preventing local minima and exploring states.
problem Behavioral cloning gets stuck in local minima and lacks effective exploration.
method Two-phase model with sampling mechanisms and self-attention modules.
result Significantly outperforms previous state-of-the-art in various environments.
It is difficult to be able to imitate well in unknown states from a small amount of expert data and sampling data. Supervised learning methods such as Behavioral Cloning do not require sampling data, but usually suffer from distribution shift. The methods based on reinforcement learning, such as inverse reinforcement l…
Behavior cloning can achieve horizon-independent sample complexity in offline imitation learning.
problem Sample complexity in imitation learning increases with problem horizon.
method New analysis of behavior cloning with logarithmic loss.
result Behavior cloning can achieve linear dependence on horizon in offline IL under dense rewards.
AI agent plays CSGO deathmatch with human-like style.
problem Lack of API for CSGO limits data for reinforcement learning.
method Behavioural cloning on large noisy and expert datasets.
result Matches human difficulty level in deathmatch mode.
BCO* improves BCO by concurrently training inverse dynamics and expert policy.
problem Efficiently learn from unlabeled demonstrations without requiring many initial interactions.
method Introduce BCO* that concurrently trains an inverse dynamics model and expert policy.
result BCO* eliminates the need for initial interactions and improves sample complexity.
Improves BC policies by generating new plausible trajectories.
problem Sub-optimal data quality in BC leads to poor policy performance.
method Trajectory Stitching (TS) generates new plausible transitions.
result TS significantly improves behavioural policies over original data.
Imitation learning trains a policy from expert demonstrations. Imitation learning approaches have been designed from various principles, such as behavioral cloning via supervised learning, apprenticeship learning via inverse reinforcement learning, and GAIL via generative adversarial learning. In this paper, we propose…
We describe a procedure for constructing a generalized Thompson group out of a family of groups that is equipped with what we call a cloning system. The previously known Thompson groups F, V, Vbr and Fbr arise from this procedure using, respectively, the systems of trivial groups, symmetric groups, braid groups and pur…
Action chunking and data exploration improve behavior cloning in robotics.
problem Exponential errors in learning from demonstrations for continuous control tasks.
method Action chunking and exploratory data collection.
result Control-theoretic stability is key to improving imitation learning.
Study uses machine learning to predict potato clones suitable for processing.
problem Efficiently identifying high-yield, disease-resistant potato varieties.
method Leveraged machine learning algorithms on Russet potato clones data from Oregon.
result Non-linear models like SVM and HGBC outperform traditional linear models in agricultural trials.
NewtonianVAE: Proportional Control and Goal Identification from Pixels via Physical Latent Spacescs.LG
Framework simplifies vision-based control and goal discovery.
problem Learning proportional control from visual data.
method Introduces NewtonianVAE for proportional control and goal discovery.
result Dramatic simplification and acceleration of vision-based controllers.
New method recovers diverse policies from expert data using state-action pair weighting.
problem Recovering diverse policies from expert trajectories.
method Pointwise mutual information weighted behavioral cloning.
result Effective in focusing on state-action pairs most representative of the style.
New RL method learns value function for many policies using few key states.
problem Evaluate and improve policies in continuous control problems.
method Combines actor-critic architecture and policy embedding to learn a single value function for many policies.
result Value function minimizes prediction error by learning a small set of 'probing states' and their impact on policies' returns.
The study classifies and imitates trading agents in financial markets.
problem Classifying and imitating trading agents in continuous double auctions.
method Developed an agent-based model for trading, applied opponent modeling for classification, and used behavioral cloning for imitation.
result Techniques for classification and imitation were experimentally compared and evaluated.
FedCD improves non-IID federated learning performance.
problem Learning from non-IID data in federated learning.
method FedCD dynamically groups devices with similar data.
result FedCD achieves higher accuracy and faster convergence.
New groups constructed from tree automorphisms, proving finiteness.
problem Finiteness properties of self-similar and Röver groups.
method Braiding of self-similar groups, use of -ary cloning systems, analysis of embedded disks.
result Construction of a new group (braided Röver group) of type .
Interactive IL beats BC by state-wise annotation cost.
problem Behavior Cloning struggles with annotation cost in sequential decision making.
method Proved Stagger and Warm Stagger algorithms to outperform BC.
result Interactive and hybrid IL methods outperform BC with state-wise annotation.
Comma.ai's approach to Artificial Intelligence for self-driving cars is based on an agent that learns to clone driver behaviors and plans maneuvers by simulating future events in the road. This paper illustrates one of our research approaches for driving simulation. One where we learn to simulate. Here we investigate v…
Demonstration-Regularized RLstat.ML
Demonstration-regularized RL reduces sample complexity for policy identification.
problem Improving reinforcement learning's sample efficiency with expert demonstrations.
method KL-regularization of behavior cloning using expert demonstrations.
result Demonstration-regularized RL achieves optimal policy identification with reduced sample complexity.
Learning Self-Correctable Policies and Value Functions from Demonstrations with Negative Samplingcs.LG
Imitation learning, followed by reinforcement learning algorithms, is a promising paradigm to solve complex control tasks sample-efficiently. However, learning from demonstrations often suffers from the covariate shift problem, which results in cascading errors of the learned policy. We introduce a notion of conservati…
New method uses bi-level optimization to learn useful representations for imitation learning.
problem Learning useful representations for multiple tasks in imitation learning settings.
method Formulates representation learning as a bi-level optimization problem.
result Bi-level optimization framework provides sample complexity benefits for imitation learning.
New algorithm robustly learns from corrupted demonstrations, even with constant fraction of noise.
problem Learning from corrupted demonstrations where a fraction of data is noise or outliers.
method Proposes a novel robust algorithm using a Median-of-Means (MOM) objective.
result Guarantees accurate policy estimation even with constant fraction of outliers, similar to classical methods in expert demonstration settings.
Diffusion models mimic human actions in sequential tasks.
problem Cloning human behavior in dynamic environments is challenging.
method Adapting diffusion models to handle stochastic, multimodal, and correlated actions.
result Diffusion models closely replicate human behavior in robotic and gaming tasks.
Imitation learning is the problem of recovering an expert policy without access to a reward signal. Behavior cloning and GAIL are two widely used methods for performing imitation learning. Behavior cloning converges in a few iterations but doesn't achieve peak performance due to its inherent iid assumption about the st…
New method learns robot skills from data, matching or outperforming existing methods.
problem Learning robot skills from fixed datasets.
method Offline Reinforcement Learning via Supervised Learning using implicit models.
result Implicit models can match or outperform explicit models in acquiring robotic skills.
MISIM improves code similarity systems with neural learning.
problem Improving accuracy of code semantics similarity systems.
method Context-aware semantics structure and neural code similarity scoring algorithm.
result MISIM achieves 8.08% better accuracy than state-of-the-art systems.
Unified framework for policy learning using weak supervision.
problem High-quality supervision is often infeasible or expensive in practice.
method Treat weak supervision as imperfect peer information and evaluate policies based on correlated agreement.
result Substantial performance improvements, especially in complex or noisy environments.
A new policy switching technique improves offline RL performance.
problem Challenges in adapting off-policy algorithms to different datasets and tasks.
method Combines off-policy RL and BC, using epistemic uncertainty for policy switching.
result Outperforms individual algorithms and state-of-the-art methods on benchmarks.
Imitation learning can reproduce policies by observing experts, which poses a problem regarding policy privacy. Policies, such as human, or policies on deployed robots, can all be cloned without consent from the owners. How can we protect against external observers cloning our proprietary policies? To answer this quest…
A simple approach to offline RL without additional complexity.
problem Learning from a fixed dataset of actions with value estimation errors.
method Adding a behavior cloning term to the policy update of an online RL algorithm and normalizing the data.
result Matches the performance of state-of-the-art offline RL algorithms with minimal changes.
GDT improves reinforcement learning by matching future state information efficiently.
problem Efficient learning of multi-task policies from trajectory data.
method Generalized Decision Transformer (GDT) for offline hindsight information matching.
result GDT enables effective offline multi-task state-marginal matching and imitation learning.
Few quantum measurements limit learning entanglement.
problem Challenges in learning entanglement from limited data.
method Hybrid quantum-classical learning techniques.
result Generalization errors increase with few measurements.
Contrastive Code Representation Learning improves code summarization and type inference.
problem Code representations are sensitive to edits, hindering downstream semantic understanding tasks.
method ContraCode: a contrastive pre-training task that learns code functionality.
result Contrastive pre-training improves code summarization and type inference accuracy.
Variable order sequence modeling is an important problem in artificial and natural intelligence. While overcomplete Hidden Markov Models (HMMs), in theory, have the capacity to represent long-term temporal structure, they often fail to learn and converge to local minima. We show that by constraining HMMs with a simple …
Paper tackles covariate shift in offline IL using less proficient behavior data.
problem Mitigating covariate shift in Imitation Learning with limited data coverage.
method Model-based IL from Offline Data (MILO) framework.
result MILO can combat covariate shift even with sub-optimal behavior policy data.