State-only imitation learning improves dexterous manipulation learning from videos.
problem High sample complexity in complex domains like dexterous manipulation.
method Train an inverse dynamics model to predict actions from states and train the policy jointly.
result Performs on par with state-action approaches and outperforms RL alone.
We use reinforcement learning (RL) to learn dexterous in-hand manipulation policies which can perform vision-based object reorientation on a physical Shadow Dexterous Hand. The training is performed in a simulated environment in which we randomize many of the physical properties of the system like friction coefficients…
This work tackles real-world robotic reinforcement learning challenges.
problem Limited success of reinforcement learning in real-world robotics.
method Proposes a system for autonomous real-world learning without instrumentation.
result Demonstrates a complete system that learns without human intervention.
SAVO actor improves reinforcement learning by avoiding local optima in complex Q-functions.
problem Gradient ascent in complex Q-functions leads to suboptimal solutions.
method SAVO actor generates multiple action proposals and truncates poor local optima.
result SAVO actor finds optimal actions more frequently and outperforms other architectures.
This work improves RL for complex robotic tasks by guiding exploration with task-specific goal distributions.
problem Solving long-horizon, complex sequential tasks in robotics with sparse rewards.
method Extends hindsight relabelling to task-specific goal distributions using a small set of demonstrations.
result Significantly higher overall performance on complex robotic manipulation tasks.
Paper tackles multi-object reinforcement learning, improving skill extrapolation.
problem Learning robust manipulation tasks in multi-object settings.
method Introduces a linear relation network module to enhance skill generalization.
result Agents can extrapolate and generalize to any new object number, scaling linearly.
We adapt the ideas underlying the success of Deep Q-Learning to the continuous action domain. We present an actor-critic, model-free algorithm based on the deterministic policy gradient that can operate over continuous action spaces. Using the same learning algorithm, network architecture and hyper-parameters, our algo…
AWAC combines offline and online data to accelerate RL learning.
problem Challenges in applying RL to real-world robotic control due to exploration and sample complexity.
method Combines sample-efficient dynamic programming with maximum likelihood policy updates.
result AWAC enables rapid learning of robotic skills with prior data and online experience.
The study of dexterous manipulation has provided important insights in humans sensorimotor control as well as inspiration for manipulation strategies in robotic hands. Previous work focused on experimental environment with restrictions. Here we describe a method using the deformation and color distribution of the finge…
KL-regularized RL from expert demos can lead to slow, unstable learning.
problem Pathological training dynamics in KL-regularized RL from expert demonstrations.
method Empirical analysis and non-parametric behavioral reference policies.
result KL-regularized RL can be significantly improved by using non-parametric behavioral policies.
We propose a plan online and learn offline (POLO) framework for the setting where an agent, with an internal model, needs to continually act and learn in the world. Our work builds on the synergistic relationship between local model-based control, global value function learning, and exploration. We study how local traj…
Paper proposes a sequential statistical test for comparing imitation learning policies with near-optimal stopping.
problem Challenges in rigorously comparing imitation learning policies due to small sample sizes and potential p-hacking.
method Sequential statistical test that adapts the number of trials based on intermediate results, achieving near-optimal stopping.
result Reduces the number of evaluation trials by up to 32% compared to state-of-the-art baselines, saving significant time and effort.
ROBEL is an open-source platform of cost-effective robots designed for reinforcement learning in the real world. ROBEL introduces two robots, each aimed to accelerate reinforcement learning research in different task domains: D'Claw is a three-fingered hand robot that facilitates learning dexterous manipulation tasks, …
Non-invasive myoelectric prostheses require a long training time to obtain satisfactory control dexterity. These training times could possibly be reduced by leveraging over training efforts by previous subjects. So-called domain adaptation algorithms formalize this strategy and have indeed been shown to significantly r…
Model-free deep reinforcement learning (RL) algorithms have been successfully applied to a range of challenging sequential decision making and control tasks. However, these methods typically suffer from two major challenges: high sample complexity and brittleness to hyperparameters. Both of these challenges limit the a…
Paper teaches robots to play piano with touch and learning.
problem Teaching robots to play piano with touch and emotion.
method Reinforcement learning from scratch with touch-augmented reward and curriculum.
result Robots can play piano with correct key positions and various requirements.
A game-theoretic approach simplifies MBRL design and improves sample efficiency.
problem Designing stable and efficient MBRL algorithms using rich function approximators.
method Develops a game-theoretic framework where MBRL is modeled as a Stackelberg game between policy and model players.
result Proposed algorithms are highly sample efficient and match asymptotic performance of model-free policy gradient.
Deep learning is an established framework for learning hierarchical data representations. While compute power is in abundance, one of the main challenges in applying this framework to robotic grasping has been obtaining the amount of data needed to learn these representations, and structuring the data to the task at ha…
AI learns market manipulation through simulation, suggesting regulation.
problem Regulating AI to prevent market manipulation.
method Used a genetic algorithm in an artificial market simulation.
result AI discovered market manipulation as an optimal strategy.
The paper analyzes how leverage affects manipulation in event-linked markets, offering new insights into regulation.
problem Manipulation and insider information in leveraged event-linked markets.
method Develops a two-axis manipulation taxonomy and analyzes leverage's effects on market-price and outcome manipulation.
result Leverage scales market-price manipulation linearly but shifts the cost-benefit threshold for outcome manipulation.
New model improves neural network robustness against input manipulations.
problem Improving neural network robustness against input manipulations.
method Causal view and deep causal manipulation augmented model (deep CAMA) with data augmentation and test-time fine-tuning.
result Deep CAMA shows superior robustness against unseen manipulations compared to traditional models.
New method learns various data manipulation schemes for model training.
problem Improving model training with data manipulation.
method Adapts RL reward learning algorithm for data manipulation learning.
result Significant improvement in classification performance.
This paper studies poisoning attacks in episodic RL and discovers their effectiveness depends on reward bounds.
problem Understanding security threats to RL algorithms through poisoning attacks.
method Examined two types of poisoning attacks: reward and action manipulation, in bounded and unbounded reward settings.
result The effectiveness of poisoning attacks depends on reward bounds, with different attack costs and success rates.
Manipulating video content is easier than ever. Due to the misuse potential of manipulated content, multiple detection techniques that analyze the pixel data from the videos have been proposed. However, clever manipulators should also carefully forge the metadata and auxiliary header information, which is harder to do …
Donors who defer their donations volunteer less in the future.
problem Volunteer labor can be less beneficial to charities than its costs.
method Regression discontinuity design with a procedure to handle manipulation.
result Donor manipulation invalidates standard regression discontinuity design, but a new method provides partial identification bounds.
In this work we propose a model that can manipulate individual visual attributes of objects in a real scene using examples of how respective attribute manipulations affect the output of a simulation. As an example, we train our model to manipulate the expression of a human face using nonphotorealistic 3D renders of a f…
Prediction markets can be manipulated by traders who can move contract settlements, harming price discovery.
problem Manipulation of settlement times in prediction markets leads to unfair wealth transfer and harms price discovery.
method Developed a model showing how settlement manipulation transfers wealth and harms price discovery, and observed real-world effects on Polymarket's Bitcoin contract.
result Manipulators capture significant profits from retail traders, especially when settlement times are short.
DIGIT is a low-cost tactile sensor for in-hand manipulation.
problem Difficulty in sensing contact forces limits robotic manipulation.
method DIGIT miniaturizes and improves a vision-based tactile sensor.
result DIGIT enables better control of interactions with the environment.
Paper presents a new port-Hamiltonian model for vehicle manipulators.
problem Complex mechanical systems' energy flow and conservation.
method Derives port-Hamiltonian dynamics from Hamiltonian reduction theory.
result Establishes mathematical equivalence with existing formulations.
Optimal benchmark design varies based on costs in financial manipulation.
problem Manipulation of price benchmarks in finance.
method Analyzes empirical pattern and cost structures to determine optimal benchmark design.
result The optimal benchmark depends on the relative sizes of fixed and variable costs.
Study examines reasons for Nutek India's share price drop.
problem Deep fall in Nutek India Limited's share price.
method Analyzed transactions and market forces to identify manipulation.
result Identified intentional interference in supply and demand.
The paper discusses decades of stock market manipulation and its benefits.
problem Manipulating public equity markets over 30 years.
method Quantitative analysis of market impact and price movement.
result Price manipulation is a valuable and profitable tool.
New method uses statistical physics to detect financial market manipulation.
problem Detecting financial market manipulation activities like spoofing and layering.
method Modeling order book dynamics as particle motion and using momentum measure.
result Method outperforms conventional Z-score-based anomaly detection.
Market manipulation is a strategy used by traders to alter the price of financial securities. One type of manipulation is based on the process of buying or selling assets by using several trading strategies, among them spoofing is a popular strategy and is considered illegal by market regulators. Some promising tools h…
New foundation for Shapley value immune to coalitional manipulations.
problem Coalitional manipulations in game theory.
method Revised Shapley value criteria for coalitional games.
result Shapley value is immune to coalitional manipulations under new criteria.
Framework detects covert financial market manipulation using LOB representations.
problem Detecting covert financial market manipulation (spoofing) from complex anomaly patterns in multilevel prices.
method Cascaded contrastive representation learning of LOB data.
result Transformer-based architectures achieve state-of-the-art results in detection performance.
Study reveals widespread manipulation of meme coins, leading to significant economic losses.
problem Widespread manipulation of meme coins leading to economic losses.
method Cross-chain analysis of 34,988 tokens across Ethereum, BNB Smart Chain, Solana, and Base.
result 82.8% of high-return tokens show evidence of artificial growth strategies.
This paper focuses on an extension of the Limit Order Book (LOB) model with general shape introduced by Alfonsi, Fruth and Schied. Here, the additional feature allows a time-varying LOB depth. We solve the optimal execution problem in this framework for both discrete and continuous time strategies. This gives in partic…
A novel RL approach learns robotic manipulation without human demonstrations.
problem Learning robotic manipulation policies efficiently and effectively.
method Introducing simulated locomotion demonstration rewards (SLDRs) to enable RL learning.
result The approach achieves higher success rates and faster learning compared to alternatives.
A new method for robot manipulation tasks using imagined object goals.
problem Learning robot manipulation in sparse reward environments.
method Train objects to reach target positions, then use predictions to create a curriculum of tasks.
result Higher success rates in challenging learning scenarios compared to alternatives.
This work tackles force control for contact-rich manipulation tasks with rigid robots using RL.
problem Challenges in working with real robotic hardware, especially position-controlled robots.
method Combines RL with traditional force control techniques, implementing parallel position/force control and admittance control.
result Validated methods on both simulation and real robot (UR3 e-series) for force control.
Manipulation is an important issue for both developed and emerging stock markets. For the study of manipulation, it is critical to analyze investor behavior in the stock market. In this paper, an analysis of the full transaction records of over a hundred stocks in a one-year period is conducted. For each stock, a tradi…
Study on costs of manipulating AMM-based price oracles.
problem Cost of manipulation in AMM-based on-chain price oracles.
method Analyzes the robustness of AMM-based oracles to strategic manipulation, considering different aggregation methods and market conditions.
result Manipulation costs depend on the total quote depth and can be minimized by optimal liquidity weights.
Many problems in image processing and computer vision (e.g. colorization, style transfer) can be posed as 'manipulating' an input image into a corresponding output image given a user-specified guiding signal. A holy-grail solution towards generic image manipulation should be able to efficiently alter an input image wit…
Investigates optimal execution under time-varying liquidity, preventing price manipulation.
problem Optimal execution with time-varying liquidity impacts and price manipulation prevention.
method Almgren-Chriss framework, deterministic time variation, well-posedness, second-order conditions, price manipulation prevention.
result Sufficient conditions for a unique solution and prevention of price manipulation.
Tool manipulation is vital for facilitating robots to complete challenging task goals. It requires reasoning about the desired effect of the task and thus properly grasping and manipulating the tool to achieve the task. Task-agnostic grasping optimizes for grasp robustness while ignoring crucial task-specific constrain…
New attack manipulates UCB algorithm, new defense algorithm reduces pseudo-regret.
problem Adversarial attacks on stochastic bandit algorithms.
method Introducing action-manipulation attacks and proposing a robust defense algorithm.
result Proposed defense algorithm reduces pseudo-regret to O(max{log T, A}).
A new reward shaping method balances learning efficiency and effectiveness for robot manipulation.
problem Efficient and effective learning in robot manipulations with system uncertainty.
method Dense2Sparse reward shaping method combining dense and sparse rewards.
result Dense2Sparse method achieves higher expected reward and better system uncertainty tolerance.