A new framework for adaptive behavior using reusable value profiles.
problem Adaptive behavior in changing environments requires switching among value-control regimes, but maintaining separate parameters for each situation is impractical.
method Introduces value profiles: reusable bundles of parameters assigned to hidden states, allowing for state-conditional strategy recruitment without independent parameters for each context.
result Profile-based models outperform simpler alternatives in probabilistic reversal learning, suggesting belief-dependent control of adaptive behavior.
CompILE learns reusable segments from demonstrations for hierarchical task execution.
problem Learning reusable, variable-length segments of hierarchical behavior from demonstrations.
method Unsupervised, fully-differentiable sequence segmentation module for latent encoding and re-composition.
result Model generalizes to longer sequences and unseen environments, learns task boundaries and event encodings.
MCP learns reusable skills for complex tasks by combining simple ones.
problem Learning complex tasks with many skills requires impractical amounts of data.
method Factorizes skills into primitives that can be combined multiplicatively.
result MCP can learn and reuse skills for novel tasks from pre-training.
ReSkill reconciles RL skill creation with policy optimization.
problem RL policies lack reusable strategies across tasks.
method Integrates skill creation into RL loop with three mechanisms.
result Consistently outperforms existing methods, especially on unseen tasks.
Method learns diverse robot skills for complex tasks.
problem Simulation-to-real transfer for robot skills.
method Decomposes skills, learns parameterized embeddings, and composes them.
result Transferable high-level policies using low-level skills.
This work learns a default policy to speed up RL learning by restricting information it receives.
problem Information asymmetry in reinforcement learning.
method KL regularized expected reward objective with a learned default policy.
result Learning a default policy can significantly speed up and improve RL learning.
Successor Options discovers reusable skills using landmark states.
problem Discovering reusable skills in reinforcement learning.
method Leverages Successor Representations to build a state space model and learns intra-option policies using a novel pseudo-reward.
result Demonstrates the approach's efficacy on grid-worlds and high-dimensional robotic control environments.
The paper proposes reusable network components by making them compatible across tasks.
problem Training networks for different tasks independently leads to incompatible components.
method The paper splits a network into a features extractor and a target task head, and proposes various approaches to make them compatible.
result The proposed methods produce components that are directly compatible without compromising accuracy on original tasks.
Study uses PPLs to model and forecast COVID-19 spread and policy interventions.
problem Forecast and control the spread of COVID-19 using policy interventions.
method Compartmental models, probabilistic programming, inference, greedy algorithm.
result Optimal series of policy interventions can control the infected population.
MSPM uses modular agents to manage financial portfolios efficiently.
problem Scalability and reusability issues in RL-based financial portfolio management.
method Modular design with Evolving Agent Module (EAM) and Strategic Agent Module (SAM).
result MSPM improves profit accumulation by at least 186.5% compared to CRP.
This work explores how incorporating prior knowledge into reinforcement learning can lead to faster learning and transfer.
problem Challenging and diverse reinforcement learning tasks require better integration of prior knowledge.
method Exploits hierarchical structures in KL-regularized reinforcement learning to incorporate prior knowledge and facilitate transfer.
result Hierarchical structures can lead to faster learning and transfer in reinforcement learning tasks.
ReOPD uses pre-collected teacher trajectories to distill knowledge from multi-turn interactions.
problem The cost of fully online on-policy distillation for multi-turn interactions.
method ReOPD, an off-environment alternative that reuses pre-collected teacher trajectories as replayed prefixes, addressing the prefix trap and distribution shift.
result ReOPD preserves or improves OPD-level accuracy, uses zero tool calls, and is at least 4imes faster per training step. Signed compression progress on a sealed audit is goodhart-resistant.
problem Intrinsic motivation for agents to improve their world models by compressing experience.
method Rewarding agents for the signed decrease of a fixed sealed-audit loss.
result Cumulative reward telescopes exactly to endpoint audit improvement, preventing infinite reward push while true audit performance stagnates.
New method learns temporal abstractions by defining interest functions.
problem Learning temporal abstractions with limited, variable durations.
method Introduced interest functions to define initiation sets, enabling gradient-based learning.
result Demonstrated effectiveness in discrete and continuous environments.
Integrates skills and world models for efficient task solving and transfer.
problem Quickly solve new tasks in complex environments using reusable knowledge.
method Leverages partial amortization for fast adaptation and online skill planning.
result Improved sample efficiency in single tasks and transfer between tasks.
Generative model prices basket options efficiently.
problem Real-time pricing of basket options with varying market inputs.
method Truncated path signatures and Mixture Density Networks (MDN) for learning the terminal density.
result The model produces small pricing errors and matches Monte Carlo simulations closely.
CK simplifies ML model deployment and reproducibility with open APIs and DevOps.
problem Making ML models reproducible and deployable across different environments.
method Decompose complex systems into reusable sub-components with unified APIs and DevOps principles.
result Automatically co-design and optimize ML models for speed, accuracy, energy, and size.
The FAIRnets Ontology makes neural networks findable, accessible, interoperable, and reusable.
problem The resource-intensive training of neural networks and the lack of training data availability.
method Development of FAIRnets Ontology to model neural networks on a meta-level and creation of a knowledge graph (FAIRnets) of over 18,400 neural networks.
result The FAIRnets Ontology and knowledge graph enable the reuse and recommendation of neural networks to data scientists.
Paper uses deep reinforcement learning for better control of rocket engines during start-up phases.
problem Lack of optimal control during transient phases of liquid rocket engines.
method Deep reinforcement learning approach for optimal control of a gas-generator engine's continuous start-up phase.
result Deep reinforcement learning controller achieves highest performance and minimal computational effort.
The design of gaits for robot locomotion can be a daunting process which requires significant expert knowledge and engineering. This process is even more challenging for robots that do not have an accurate physical model, such as compliant or micro-scale robots. Data-driven gait optimization provides an automated alter…
Probabilistic inference procedures are usually coded painstakingly from scratch, for each target model and each inference algorithm. We reduce this effort by generating inference procedures from models automatically. We make this code generation modular by decomposing inference algorithms into reusable program-to-progr…
We develop a technique for generalising from data in which models are samplers represented as program text. We establish encouraging empirical results that suggest that Markov chain Monte Carlo probabilistic programming inference techniques coupled with higher-order probabilistic programming languages are now sufficien…
New model shows hierarchical proof structure helps theorem provers.
problem How to efficiently learn proofs from teacher traces.
method Model proof search as MDP, analyze imitation learning from traces.
result Hierarchical proof structure leads to more efficient learning.
Modular deep learning framework using pairwise labels without backpropagation.
problem Efficiently training deep neural networks with limited supervision.
method Stacked linear models in feature spaces, provably optimal modular learning framework.
result High accuracy (94.88%) achieved with minimal labeled examples (1 per class).
SMC analysis reveals key transient effects in macroeconomic ABM.
problem Analysis of complex ABMs is challenging and often relies on ad hoc methods.
method Statistical model checking (SMC) implemented through MultiVeStA.
result Clear contrast across parameter families in macro-financial and structural sweeps.
New method reduces fine-tuning cost for reused models.
problem Repeating fine-tuning costs with outdated foundation models.
method Portable Reward Tuning (PRT) trains a reward model to maximize the same loss function as fine-tuning.
result PRT achieves comparable accuracy to inference-time tuning with less inference cost.
Auto-encoding is an important task which is typically realized by deep neural networks (DNNs) such as convolutional neural networks (CNN). In this paper, we propose EncoderForest (abbrv. eForest), the first tree ensemble based auto-encoder. We present a procedure for enabling forests to do backward reconstruction by ut…
In many real-world applications, data are often collected in the form of stream, and thus the distribution usually changes in nature, which is referred as concept drift in literature. We propose a novel and effective approach to handle concept drift via model reuse, leveraging previous knowledge by reusing models. Each…
The paper develops a learning framework for diverse legged robots.
problem General and autonomous learning of core skills in locomotion.
method Data-efficient, off-policy multi-task RL algorithm with semantically identical reward functions.
result The same algorithm can learn diverse and reusable locomotion skills across different legged robots.
Method learns reusable actions from human demonstrations.
problem Learning reusable actions from human demonstrations.
method Skill segmentation, inverse reinforcement learning, one-class support vector machine.
result Inferred options improve learning and planning.
Contextualized ML learns context-dependent effects using deep learning.
problem Learning heterogeneous and context-dependent effects in data.
method Applying deep learning to the meta-relationship between contextual information and context-specific parametric models.
result Unified framework for cluster analysis and cohort modeling.
A neural network learns relational representations from raw data.
problem Learning reusable representations from raw pixel data.
method Explicitly relational neural network architecture trained on visual relational tasks.
result The architecture outperforms baselines on unseen tasks.
Baconian simplifies MBRL experiments by providing a flexible framework.
problem Lack of reusable open-source frameworks for MBRL research.
method Developed a flexible and modularized framework, Baconian.
result Facilitates MBRL experiments by allowing customization and reuse of algorithms.
New algorithm corrects bias in LDP-released data for better analysis.
problem Bias in data released under Local Differential Privacy (LDP).
method Inverse Weierstrass Private Stochastic Gradient Descent (IWP-SGD).
result Converges to true population risk minimizer at O(1/n) rate. Automated discovery of adaptive attacks improves adversarial defense evaluation.
problem Challenges in reliably evaluating adversarial defenses.
method Formalizes adaptive attacks as reusable building blocks in a search space for automatic discovery.
result Our tool discovers significantly stronger attacks than AutoAttack, improving adversarial defense evaluation.
ModSSC unifies semi-supervised classification for various data types.
problem Fragmented support for semi-supervised classification across different methods, settings, and data types.
method ModSSC is a modular Python framework that supports reproducible and controlled experimentation for semi-supervised classification on heterogeneous data.
result ModSSC enables systematic comparison of semi-supervised learning across various datasets and model backbones.
Several methods were recently proposed for the task of translating images between domains without prior knowledge in the form of correspondences. The existing methods apply adversarial learning to ensure that the distribution of the mapped source domain is indistinguishable from the target domain, which suffers from kn…
VSML unifies meta learning concepts and enables simple backpropagation.
problem Improving and unifying meta learning concepts for neural networks.
method Unified approach using variable shared meta learning and simple weight-sharing.
result Simple backpropagation can be implemented and meta learned without gradient calculation.
This paper analyzes and improves active learning techniques for real-world projects.
problem Reducing labelling effort in machine learning models with real-world constraints.
method Systematic study of active learning issues, proposing techniques to address model convergence, annotation error, and dataset imbalance.
result Presentation of two techniques to speed up active learning: partial uncertainty sampling and larger query size.
Framework reconstructs missing spatio-temporal data for extreme value prediction.
problem Predicting extreme values from incomplete spatio-temporal data.
method Convolutional deep neural networks and autoencoder-like models for conditional sampling.
result Framework produces accurate reconstructions of missing data for extremal values.
Inference for GP models with non-Gaussian noises is computationally expensive when dealing with large datasets. Many recent inference methods approximate the posterior distribution with a simpler distribution defined on a small number of inducing points. The inference is accurate only when data points have strong corre…
The recent proliferation of richly structured probabilistic models raises the question of how to automatically determine an appropriate model for a dataset. We investigate this question for a space of matrix decomposition models which can express a variety of widely used models from unsupervised learning. To enable mod…
DeepSwarm uses swarm intelligence to optimize CNN architectures.
problem Optimizing Convolutional Neural Networks (CNNs).
method Ant Colony Optimization (ACO) for neural architecture search.
result DeepSwarm outperforms existing methods on MNIST, Fashion-MNIST, and CIFAR-10 datasets.
The paper examines how updates to probabilistic models influence behavior based on evidence.
problem Understanding how updates to probabilistic models influence behavior based on evidence.
method Study of KL-regularized soft updates as Bayesian posterior updates within a single probabilistic model.
result Posterior updates determine relative incentives but not absolute rewards, which are ambiguous up to context-specific baselines.
cuSLINK clusters data faster on GPUs, saving space and time.
problem Efficiently clustering large datasets on GPUs.
method Reformulated SLINK algorithm on GPU with optimized building blocks.
result cuSLINK uses O(Nk) space and trades off space and time with parameter k. ANN predicts methane heat transfer in rocket engines efficiently.
problem Accurate heat transfer prediction for supercritical methane in rocket engine cooling channels.
method Artificial neural networks trained on CFD simulation data.
result ANN model predicts maximum wall temperature with convincing precision.
Deep tabular models outperform GBDT in medical diagnosis tasks.
problem Transfer learning for tabular data in medical diagnosis.
method Proposes a pseudo-feature method for transfer learning between different feature sets.
result Tabular neural networks outperform GBDT in medical diagnosis tasks.
Framework for lifelong learning of compositional structures.
problem Learning to reuse self-contained chunks of knowledge for novel problems.
method Separates learning into combining existing components and adapting them.
result Framework handles trade-off between stability and flexibility.