Survey examines challenges and solutions in sim-to-real transfer for robotics.
problem Challenges in transferring robotic systems from simulation to real-world environments.
method Leveraging techniques like domain randomization, real-to-sim transfer, state and action abstractions, and sim-real co-training.
result Promising results in closing the reality gap across various robotic domains.
Bayesian optimization adapts domain parameters for more robust robot policies.
problem Learning policies for robot control from simulation data often fails in the real world due to the 'reality gap'.
method Bayesian Domain Randomization (BayRn) uses Bayesian optimization to adapt domain parameter distributions during training.
result BayRn achieves better sim-to-real transfer compared to fixed distribution methods.
New RL method tackles sim-to-real gap using interactive data collection.
problem Sim-to-real gap in reinforcement learning.
method Distributionally robust reinforcement learning with interactive data collection.
result Proves sample-efficient learning is impossible without additional assumptions.
Paper tackles sim-to-real transfer in continuous domains with partial observations.
problem Lack of theoretical foundation for sim-to-real transfer in continuous domains with partial observations.
method Developed a new algorithm for infinite-horizon average-cost LQGs and established a regret bound.
result A popular robust adversarial training algorithm can learn competitive policies from simulation to real-world environments.
This work analyzes how multi-agent reinforcement learning can bridge the gap to reality in distributed multi-robot systems.
problem Collaborative learning in distributed multi-robot systems with varying sensors and actuators.
method Simulation-based analysis using PPO and Bullet physics engine, considering different types of perturbations.
result PPO's robustness is affected by the presence of different types of perturbations and the number of agents experiencing them.
Mobile network that millions of people use every day is one of the most complex systems in the world. Optimization of mobile network to meet exploding customer demand and reduce capital/operation expenditures poses great challenges. Despite recent progress, application of deep reinforcement learning (DRL) to complex re…
RL controls small soccer robots in a real league, beating human-designed policies.
problem Training robots to play complex, real-world sports.
method Sim-to-Real RL approach, training in simulated environment, applying to real-world robots.
result Robots learned policies to compete effectively, beating human-designed strategies.
This work studies reinforcement learning in the Sim-to-Real setting, in which an agent is first trained on a number of simulators before being deployed in the real world, with the aim of decreasing the real-world sample complexity requirement. Using a dynamic model known as a rich observation Markov decision process (R…
This work shows how to use simulators to learn efficient exploration in real-world RL.
problem Sample complexity of real-world reinforcement learning.
method Coupling exploratory policies learned in simulators with practical approaches.
result Polynomial sample complexity in real world, exponential improvement over direct sim2real transfer.
Particle physics or High Energy Physics (HEP) studies the elementary constituents of matter and their interactions with each other. Machine Learning (ML) has played an important role in HEP analysis and has proven extremely successful in this area. Usually, the ML algorithms are trained on numerical simulations of the …
Method captures fabric mechanics from depth images without expensive setups.
problem Estimating mechanical parameters of fabrics accurately and efficiently.
method Sim-to-real strategy using learning-based framework trained on synthetic data.
result Metric correlates with human judgments of fabric drape similarity.
Simulation-to-real transfer is an important strategy for making reinforcement learning practical with real robots. Successful sim-to-real transfer systems have difficulty producing policies which generalize across tasks, despite training for thousands of hours equivalent real robot time. To address this shortcoming, we…
New algorithm breaks multiagency gap in robust MARL.
problem Vulnerability of MARL to sim-to-real gaps.
method Distributionally robust Markov games (RMGs) with a new uncertainty set formulation.
result First algorithm to break the curse of multiagency for RMGs.
Physics-informed learning framework for pH systems and EB-PBC control.
problem Control of port-Hamiltonian systems from trajectory data.
method Co-learning of pH system model and EB-PBC through alternating optimization.
result Proven stability and robustness of the learned controller.
New algorithm reduces sample size for robust reinforcement learning.
problem Creating robust policies for multi-agent reinforcement learning.
method Model-based algorithm RTZ-VI-LCB for tabular robust two-player zero-sum games.
result Establishes near-optimal sample complexity guarantees for offline robust reinforcement learning.
This work tackles robust RL in multi-agent settings, improving sample efficiency.
problem Overcoming environmental uncertainties in multi-agent reinforcement learning.
method Proposes DRNVI, a sample-efficient algorithm for learning robust equilibria in RMGs.
result Establishes near-optimal sample complexity for solving RMGs.
CausalWorld benchmarks robotic manipulation tasks with causal structure for transfer learning.
problem Challenges in transferring learned skills to new robotic manipulation environments.
method Proposes a simulation-based benchmark with a combinatorial family of tasks.
result Demonstrates the feasibility of tasks in the benchmark and provides baseline results.
Algorithm learns robust equilibrium in online Markov games with interactive data.
problem Sim-to-real gap in reinforcement learning.
method Distributionally robust RL with minimum value assumption, least square value iteration.
result Sample-efficient algorithm for robust equilibrium in online Markov games.
The paper proposes a method to learn from both simulation and real-world data.
problem Training autonomous systems in simulation and applying them to real-world environments.
method Balancing samples from simulation and real-world data using a replay buffer.
result The method achieves better performance in real-world tasks compared to training only in simulation.
Current end-to-end deep Reinforcement Learning (RL) approaches require jointly learning perception, decision-making and low-level control from very sparse reward signals and high-dimensional inputs, with little capability of incorporating prior knowledge. This results in prohibitively long training times for use on rea…
Robust reinforcement learning agents generalize well to out-of-distribution settings using pretrained representations.
problem Achieving sample-efficient reinforcement learning agents that generalize to real-world settings.
method Trained 240 representations and 10,000 RL policies on a simulated robotic setup, evaluating different pretrained VAE-based representations' effects on OOD generalization.
result Many reinforcement learning agents are surprisingly robust to realistic distribution shifts, including sim-to-real cases.
High-throughput 3D control training system achieves 100,000 FPS.
problem Lack of efficient, single-machine reinforcement learning systems.
method Sample Factory combines asynchronous sampling and off-policy correction.
result Achieves 100,000 FPS on 3D control problems without sacrificing sample efficiency.
Recently, reinforcement learning (RL) algorithms have demonstrated remarkable success in learning complicated behaviors from minimally processed input. However, most of this success is limited to simulation. While there are promising successes in applying RL algorithms directly on real systems, their performance on mor…
Learning robot tasks or controllers using deep reinforcement learning has been proven effective in simulations. Learning in simulation has several advantages. For example, one can fully control the simulated environment, including halting motions while performing computations. Another advantage when robots are involved…
Paper presents a world model that learns invariant causal features using contrastive unsupervised learning.
problem Learning invariant causal features in unsupervised settings.
method Contrastive unsupervised learning with intervention invariant auxiliary task.
result Significantly outperforms state-of-the-art methods on out-of-distribution point navigation tasks.
The paper explores gaps in curvature-related metrics and rigidity.
problem Understanding gaps in curvature-related metrics and rigidity.
method Analyzes three types of gaps: spectral, metric-rigidity, and topological-rigidity.
result Proposes open problems in the field.
Study gap-dependent regret bounds for risk-sensitive RL.
problem Risk-sensitive reinforcement learning with entropic risk measure.
method Propose cascaded gaps to adapt to problem structures, derive regret bounds.
result Exponential improvement over existing bounds in appropriate settings.
Paper analyzes origami slope gaps and their distribution, finding a unique pattern.
problem Analyzing slope gaps in origami surfaces.
method Derived slope gap distribution of a specific origami by considering return times under the horocycle flow.
result Found a unique distribution of origami slope gaps, not a sum of scaled Hall distributions.
Random hyperbolic surfaces have nearly optimal spectral gaps.
problem Proving the nearly optimal spectral gap conjecture for random Belyi surfaces.
method Using the Brooks-Makover model, the authors show a spectral gap greater than 1/4 - c/log(n).
result A random hyperbolic surface in the Brooks-Makover model has a spectral gap greater than 1/4 - c/log(n).
Paper improves volume gap between minimal submanifolds and unit spheres.
problem Volume gap between minimal submanifolds and unit spheres.
method Modified Cheng-Li-Yau coefficients and applied Cheng-Yang eigenvalue estimate for Laplacian.
result Enhanced volume gap between minimal submanifolds and unit spheres.
The article proves a conjecture about the fundamental gap for horoconvex domains in hyperbolic space.
problem Proving a conjecture about the fundamental gap for horoconvex domains in hyperbolic space.
method Establishing conformal log-concavity estimates for the first eigenfunction.
result Proves a conjecture about the fundamental gap for horoconvex domains in hyperbolic space.
Kahler-Einstein metrics linked to eigenvalue gaps on Fano manifolds.
problem Existence of Kahler-Einstein metrics on Fano manifolds.
method Characterization via eigenvalue gaps of Cauchy-Riemann and Hamiltonian vector fields.
result Existence of Kahler-Einstein metrics linked to eigenvalue gaps.
Local gaps in Ricci shrinkers depend only on dimension.
problem Understanding local properties of Ricci shrinkers.
method Proved local versions of Ricci curvature and entropy gap theorems.
result Local gaps depend only on dimension, not global entropy.
Study shows gaps in Bitcoin order book are linked to returns but only in the short term.
problem Understanding the relationship between gaps and returns in Bitcoin order books.
method Examined the dynamics of gaps and returns in a Bitcoin order book without considering long-term causation.
result The causal relationship between gaps and returns is limited to instantaneous causation.
Researchers compute gap distributions for saddle connection directions on specific translation surfaces.
problem Computing gap distributions for saddle connection directions on translation surfaces.
method Translation to dynamical question of return times to a transversal under the horocycle flow.
result Gap distributions have support at 0 and quadratic tail decay.
Proves gap rigidity theorem for Hermitian symmetric spaces.
problem Gap rigidity problems in compact Hermitian symmetric spaces.
method Dual analogy to Mok's noncompact case theorem, theorem on higher dimensional submanifolds.
result Proves gap rigidity theorem for diagonal curves in tube type spaces.
The paper introduces gapped scale-sensitive dimensions to improve learning rate bounds.
problem Improving lower bounds on rates of convergence in statistical and online learning.
method Introducing and analyzing gapped scale-sensitive dimensions for function classes.
result Gapped dimensions lead to stronger lower bounds on offset Rademacher averages.
The article explores the fundamental gap in Bakry-Emery geometry.
problem The fundamental gap in Bakry-Emery geometry.
method Recalled Bakry-Emery geometry and connected eigenvalues with boundary conditions. Showed a connection between fundamental gap and Bakry-Emery geometry.
result Presented key ideas in Andrews's and Clutterbuck's proof of the fundamental gap conjecture.
The paper calculates gap distributions for translation surfaces, focusing on the double heptagon.
problem Calculating gap distributions for translation surfaces.
method Describes a procedure to find winning holonomy vectors and applies it to the double heptagon.
result Explicitly computed gap distribution for the regular double heptagon translation surface.
Improved gap-dependent bounds for reinforcement learning with linear approximations.
problem Achieving nearly minimax-optimal performance with linear function approximation.
method Developed and analyzed the LSVI-UCB++ algorithm and its concurrent variant.
result First gap-dependent regret bound for nearly minimax-optimal algorithm LSVI-UCB++.
We present a data-driven framework called generative adversarial privacy (GAP). Inspired by recent advancements in generative adversarial networks (GANs), GAP allows the data holder to learn the privatization mechanism directly from the data. Under GAP, finding the optimal privacy mechanism is formulated as a constrain…
Computing unlinking number is usually very difficult and complex problem, therefore we define BJ-unlinking number and recall Bernhard-Jablan conjecture stating that the classical unknotting/unlinking number is equal to the BJ-unlinking number. We compute BJ-unlinking number for various families of knots and links for w…
ReCoRe learns invariant features for world navigation using contrastive learning and regularizers.
problem Limited sample efficiency and overfitting to training scenarios in RL for visual navigation.
method Contrastive unsupervised learning and intervention-invariant regularizer.
result Significantly improves sample efficiency and generalization in out-of-distribution point navigation tasks.
New methods reduce bias in estimating optimality gaps for risk-averse stochastic programs.
problem Optimality gap estimation bias in risk-averse stochastic programs.
method Two independent samples, each estimating a different component of the optimality gap.
result Our method reduces bias in estimating optimality gaps for risk-averse problems.
New conditions prevent gaps in optimal control problems.
problem Preventing gaps in optimal control problems with state constraints.
method Developed new sufficient conditions not relying on convexity.
result Derived bounds for the size of the relaxation gap.
Federated learning studies separate client data and distribution gaps.
problem Understanding performance differences in federated learning across different datasets.
method Proposed a framework to disentangle out-of-sample and participation gaps.
result Dataset synthesis strategy is crucial for realistic simulations of federated learning generalization.
The paper proves gap theorems for Yang-Mills on manifolds with positive Yamabe.
problem Yang-Mills theory on manifolds with positive Yamabe constant.
method Extending Gursky-Kelleher-Streets results to complete manifolds.
result Equality in gap theorem described in terms of basic instanton.
Study simplicial volume for fixed fundamental groups, finding gaps.
problem Understanding simplicial volume for manifolds with fixed fundamental group.
method Relate gap problem to rationality questions in bounded (co)homology.
result Show existence of gaps in simplicial volume spectrum at zero.