Paper tackles sim-to-real transfer in continuous domains with partial observations.
problem Lack of theoretical foundation for sim-to-real transfer in continuous domains with partial observations.
method Developed a new algorithm for infinite-horizon average-cost LQGs and established a regret bound.
result A popular robust adversarial training algorithm can learn competitive policies from simulation to real-world environments.
Survey examines challenges and solutions in sim-to-real transfer for robotics.
problem Challenges in transferring robotic systems from simulation to real-world environments.
method Leveraging techniques like domain randomization, real-to-sim transfer, state and action abstractions, and sim-real co-training.
result Promising results in closing the reality gap across various robotic domains.
Bayesian optimization adapts domain parameters for more robust robot policies.
problem Learning policies for robot control from simulation data often fails in the real world due to the 'reality gap'.
method Bayesian Domain Randomization (BayRn) uses Bayesian optimization to adapt domain parameter distributions during training.
result BayRn achieves better sim-to-real transfer compared to fixed distribution methods.
Simulation-to-real transfer is an important strategy for making reinforcement learning practical with real robots. Successful sim-to-real transfer systems have difficulty producing policies which generalize across tasks, despite training for thousands of hours equivalent real robot time. To address this shortcoming, we…
This work shows how to use simulators to learn efficient exploration in real-world RL.
problem Sample complexity of real-world reinforcement learning.
method Coupling exploratory policies learned in simulators with practical approaches.
result Polynomial sample complexity in real world, exponential improvement over direct sim2real transfer.
Particle physics or High Energy Physics (HEP) studies the elementary constituents of matter and their interactions with each other. Machine Learning (ML) has played an important role in HEP analysis and has proven extremely successful in this area. Usually, the ML algorithms are trained on numerical simulations of the …
CausalWorld benchmarks robotic manipulation tasks with causal structure for transfer learning.
problem Challenges in transferring learned skills to new robotic manipulation environments.
method Proposes a simulation-based benchmark with a combinatorial family of tasks.
result Demonstrates the feasibility of tasks in the benchmark and provides baseline results.
Method captures fabric mechanics from depth images without expensive setups.
problem Estimating mechanical parameters of fabrics accurately and efficiently.
method Sim-to-real strategy using learning-based framework trained on synthetic data.
result Metric correlates with human judgments of fabric drape similarity.
Mobile network that millions of people use every day is one of the most complex systems in the world. Optimization of mobile network to meet exploding customer demand and reduce capital/operation expenditures poses great challenges. Despite recent progress, application of deep reinforcement learning (DRL) to complex re…
Learning robot tasks or controllers using deep reinforcement learning has been proven effective in simulations. Learning in simulation has several advantages. For example, one can fully control the simulated environment, including halting motions while performing computations. Another advantage when robots are involved…
RL controls small soccer robots in a real league, beating human-designed policies.
problem Training robots to play complex, real-world sports.
method Sim-to-Real RL approach, training in simulated environment, applying to real-world robots.
result Robots learned policies to compete effectively, beating human-designed strategies.
This work studies reinforcement learning in the Sim-to-Real setting, in which an agent is first trained on a number of simulators before being deployed in the real world, with the aim of decreasing the real-world sample complexity requirement. Using a dynamic model known as a rich observation Markov decision process (R…
Current end-to-end deep Reinforcement Learning (RL) approaches require jointly learning perception, decision-making and low-level control from very sparse reward signals and high-dimensional inputs, with little capability of incorporating prior knowledge. This results in prohibitively long training times for use on rea…
New RL method tackles sim-to-real gap using interactive data collection.
problem Sim-to-real gap in reinforcement learning.
method Distributionally robust reinforcement learning with interactive data collection.
result Proves sample-efficient learning is impossible without additional assumptions.
High-throughput 3D control training system achieves 100,000 FPS.
problem Lack of efficient, single-machine reinforcement learning systems.
method Sample Factory combines asynchronous sampling and off-policy correction.
result Achieves 100,000 FPS on 3D control problems without sacrificing sample efficiency.
Recently, reinforcement learning (RL) algorithms have demonstrated remarkable success in learning complicated behaviors from minimally processed input. However, most of this success is limited to simulation. While there are promising successes in applying RL algorithms directly on real systems, their performance on mor…
Paper presents a world model that learns invariant causal features using contrastive unsupervised learning.
problem Learning invariant causal features in unsupervised settings.
method Contrastive unsupervised learning with intervention invariant auxiliary task.
result Significantly outperforms state-of-the-art methods on out-of-distribution point navigation tasks.
ReCoRe learns invariant features for world navigation using contrastive learning and regularizers.
problem Limited sample efficiency and overfitting to training scenarios in RL for visual navigation.
method Contrastive unsupervised learning and intervention-invariant regularizer.
result Significantly improves sample efficiency and generalization in out-of-distribution point navigation tasks.
This work analyzes how multi-agent reinforcement learning can bridge the gap to reality in distributed multi-robot systems.
problem Collaborative learning in distributed multi-robot systems with varying sensors and actuators.
method Simulation-based analysis using PPO and Bullet physics engine, considering different types of perturbations.
result PPO's robustness is affected by the presence of different types of perturbations and the number of agents experiencing them.
The paper proposes a method to learn from both simulation and real-world data.
problem Training autonomous systems in simulation and applying them to real-world environments.
method Balancing samples from simulation and real-world data using a replay buffer.
result The method achieves better performance in real-world tasks compared to training only in simulation.
Physics-informed learning framework for pH systems and EB-PBC control.
problem Control of port-Hamiltonian systems from trajectory data.
method Co-learning of pH system model and EB-PBC through alternating optimization.
result Proven stability and robustness of the learned controller.
Transfer learning improves portfolio optimization by identifying transfer risk.
problem Financial portfolio optimization problem.
method Introduces transfer risk concept within transfer learning framework.
result Transfer risk is a significant indicator of transferability and enhances portfolio management efficiency.
Robust reinforcement learning agents generalize well to out-of-distribution settings using pretrained representations.
problem Achieving sample-efficient reinforcement learning agents that generalize to real-world settings.
method Trained 240 representations and 10,000 RL policies on a simulated robotic setup, evaluating different pretrained VAE-based representations' effects on OOD generalization.
result Many reinforcement learning agents are surprisingly robust to realistic distribution shifts, including sim-to-real cases.
Paper analyzes transfer risk in transfer learning for finance.
problem Evaluate transferability of transfer learning in finance.
method Proposes transfer risk concept and applies to stock return prediction and portfolio optimization.
result Transfer risk correlates with transfer learning performance and identifies appropriate source tasks.
This paper explores the connection between adversarial and knowledge transferability.
problem Understanding the factors affecting knowledge transferability.
method Theoretical analysis and practical metrics for adversarial transferability.
result Adversarial transferability and knowledge transferability are closely related.
Mathematical framework for transfer learning feasibility and transfer risk.
problem Theoretical analysis of transfer learning.
method Reformulated transfer learning as an optimization problem, introduced transfer risk concept.
result Demonstrated the potential and benefits of incorporating transfer risk in transfer learning evaluation.
New algorithm reduces sample size for robust reinforcement learning.
problem Creating robust policies for multi-agent reinforcement learning.
method Model-based algorithm RTZ-VI-LCB for tabular robust two-player zero-sum games.
result Establishes near-optimal sample complexity guarantees for offline robust reinforcement learning.
Transfer learning borrows knowledge from a source domain to facilitate learning in a target domain. Two primary issues to be addressed in transfer learning are what and how to transfer. For a pair of domains, adopting different transfer learning algorithms results in different knowledge transferred between them. To dis…
Study measures impact of data and neural net similarity on transferability in restaurant sales data.
problem Identify indicators for successful transferability of neural nets across different data sets.
method Empirical study on sales data from six restaurants, calculating indicators based on data and neural net similarities.
result Negative correlations between transferability and indicators, allowing better model performance and fewer transfers.
The paper analyzes phase transitions in transfer learning for perceptrons.
problem Understanding when transfer learning from a source task to a target task is beneficial.
method Theoretical analysis of a pair of related perceptron learning tasks.
result Reveals a phase transition from negative to positive transfer as task similarity changes.
Adaptive source selection for positive transfer in linear models improves target dataset performance.
problem Limited task-specific labeled data in business settings.
method Greedily decides from which sources and how many samples to incorporate into the target dataset using an accept/reject rule based on a data-dependent estimate of the transfer gain.
result Consistent gains over classical and recent strong baselines while avoiding negative transfer.
New research on limits of transfer learning, proving key selection and dependence requirements.
problem Insufficient theoretical foundation for transfer learning.
method Proved novel results on transfer learning, emphasizing selection of information and dependence between domains.
result Upper bound on improvement possible with transfer learning, highlighting the need for careful selection.
Transfer learning aims at improving the performance of target learners on target domains by transferring the knowledge contained in different but related source domains. In this way, the dependence on a large number of target domain data can be reduced for constructing target learners. Due to the wide application prosp…
Proposes a transfer learning method for high-dimensional quantile regression.
problem Inadequate handling of heterogeneity and heavy tails in transfer learning.
method High-dimensional quantile regression framework with double transfer learning estimator.
result Established error bounds and valid confidence intervals for high-dimensional quantile regression coefficients.
Localized transfer learning improves nonparametric regression performance.
problem Improving nonparametric regression performance on target tasks.
method Localized transfer learning framework that models heterogeneity and partition covariate space into cells.
result Sharp minimax rates show local transfer mitigates the curse of dimensionality.
Transfer learning aims at building robust prediction models by transferring knowledge gained from one problem to another. In the semantic Web, learning tasks are enhanced with semantic representations. We exploit their semantics to augment transfer learning by dealing with when to transfer with semantic measurements an…
Investigates transfer learning in spatial statistics.
problem Applying transfer learning to spatial statistics.
method Simple MLP models for spatial data.
result Potential of transfer learning in spatial statistics.
This work transfers causal knowledge between tasks for Individual Treatment Effect estimation.
problem Estimating Individual Treatment Effects (ITE) requires a large amount of data, making it challenging.
method The authors introduce a practical framework for efficient transfer of causal knowledge between tasks, using a Causal Inference Task Affinity (CITA) measure.
result ITE knowledge transfer can significantly reduce the amount of data needed for ITE estimation.
With the help of transfer entropy, we analyze information flows between communities of complex networks. We show that the transfer entropy provides a coherent description of interactions between communities, including non-linear interactions. To put some flesh on the bare bones, we analyze transfer entropies between co…
Simple methods improve regression transferability estimation.
problem Estimating how well regression models transfer between tasks.
method Two simple, computationally efficient approaches based on negative regularized mean squared error.
result Significantly outperform existing methods in accuracy and efficiency.
This paper defines and quantifies transferability in domain generalization.
problem Understanding and quantifying transferability between domains.
method Formal definition and estimation of transferability, upper bound for target error.
result Many algorithms do not learn transferable features, proposing a new algorithm.
Training a source model optimally for its own task is suboptimal for downstream transfer.
problem The optimality of a source model for its own task hinders downstream transfer performance.
method Analyzes L2-SP ridge regression, characterizes transfer-optimal source penalty, and identifies alignment-dependent effects.
result Transfer benefits from stronger source regularization when aligned imperfectly, and from weaker regularization when aligned perfectly.
AdaTrans adapts to feature and sample transfer in high-dimensional regression.
problem High-dimensional linear regression with more features than samples.
method F-AdaTrans and S-AdaTrans methods using fused-penalties and adaptive weights.
result AdaTrans achieves convergence rates close to oracle estimators and near-minimax optimal rates.
When labeled data is scarce for a specific target task, transfer learning often offers an effective solution by utilizing data from a related source task. However, when transferring knowledge from a less related source, it may inversely hurt the target performance, a phenomenon known as negative transfer. Despite its p…
Adversarial perturbations fool wearable sensor systems, showing transferability across different systems.
problem Adversarial examples fool wearable sensor systems, showing transferability across different systems.
method Study of adversarial transferability in wearable sensor systems from four perspectives: systems, subjects, sensor body locations, and datasets.
result Strong untargeted transferability in most cases, targeted attacks less successful.
We observe standard transfer learning can improve prediction accuracies of target tasks at the cost of lowering their prediction fairness -- a phenomenon we named discriminatory transfer. We examine prediction fairness of a standard hypothesis transfer algorithm and a standard multi-task learning algorithm, and show th…
Paper tackles continuous transfer learning with evolving target domains.
problem Challenges of negative transfer in evolving target domains.
method Proposes label-informed C-divergence for measuring distribution shift and negative transfer.
result Demonstrates effectiveness of TransLATE framework in minimizing classification error and C-divergence.
Paper introduces a new method for improving reinforcement learning performance using transfer learning.
problem Improving reinforcement learning performance with limited sample sizes in dynamic decision-making scenarios.
method Developed a novel ``re-weighted targeting procedure'' and ``transfer deep Q∗-learning'' approach. result Demonstrated improved reinforcement learning performance through strategic sample construction.