A simple approach improves performance on both past and future tasks in lifelong learning.
problem Forgetting in lifelong learning, where performance on past tasks degrades when learning new tasks.
method Representation ensembling to improve performance on both future and past tasks.
result Representation ensembling demonstrates both forward and backward transfer across various datasets.
Framework for lifelong learning using eigentasks to avoid forgetting and transfer knowledge.
problem Avoiding forgetting and transferring knowledge in lifelong learning.
method Eigentask framework: skills paired with generative models, wake-sleep cycle for learning and consolidation.
result Improved performance in supervised continual learning, evidence of forward knowledge transfer.
Develops hyperparameter transfer methods for Dense Associative Memories.
problem Challenges in transferring hyperparameters for DenseAMs due to unique architecture and activation functions.
method Derives explicit prescriptions for hyperparameter transfer from small to large models.
result Excellent agreement between theoretical and empirical results.
Enhances PCE surrogates using transfer learning for expensive simulations.
problem Over-sampling in PCE for expensive forward models.
method Transfer learning from similar tasks to a new task with limited training data.
result Improves scalability and accuracy of PCE surrogates.
Current reinforcement learning (RL) methods can successfully learn single tasks but often generalize poorly to modest perturbations in task domain or training procedure. In this work, we present a decoupled learning strategy for RL that creates a shared representation space where knowledge can be robustly transferred. …
End-to-end optimization has achieved state-of-the-art performance on many specific problems, but there is no straight-forward way to combine pretrained models for new problems. Here, we explore improving modularity by learning a post-hoc interface between two existing models to solve a new task. Specifically, we take i…
Enhances investment performance by leveraging cross-market information.
problem Maximizing portfolio performance in asset markets with shared characteristics.
method Transfer learning applied to portfolio optimization.
result Achieves maximum Sharpe ratio asymptotically.
Transferring knowledge across many streaming processes remains an uncharted territory in the existing literature and features unique characteristics: no labelled instance of the target domain, covariate shift of source and target domain, different period of drifts in the source and target domains. Autonomous transfer l…
Forward hedging reshapes incentive provision in firms.
problem How does forward hedging affect incentive provision in firms?
method We consider a CARA framework to jointly characterize optimal production, compensation, and static hedging in equilibrium.
result Delegation and external hedging are partial substitutes, and delegation can increase firm value even when the agent is more risk averse.
Improves model accuracy in medical imaging with small datasets using transfer learning.
problem Challenges in training neural networks on small medical imaging datasets.
method Comparison of current techniques, proposing one cycle training, discriminative learning rates, gradual freezing, and parameter modification.
result Transfer learning is crucial for small datasets, especially when images from the same part of the body are available.
A new method estimates generative model mappings using kernel transfer operators, reducing costs and improving performance.
problem Efficiently estimating mappings between known and unknown distributions in generative models.
method Adapting kernel transfer operators to estimate mappings, reducing computational costs.
result Significant runtime savings and good empirical performance compared to existing methods.
New method creates powerful, transferable adversarial attacks for deep learning networks.
problem Vulnerability of deep learning networks to adversarial attacks.
method Developed a method to generate strong, transferable adversarial attacks.
result Demonstrated that the proposed method can easily transfer strong adversarial attacks between different models.
Unified framework supports lifelong learning properties.
problem Capturing desirable properties of human learning in machine learning.
method Unified framework with one central mechanism.
result Supports almost all desirable properties of human learning.
Extended flatness approach for discrete-time systems considers forward and backward shifts.
problem Defining flatness for discrete-time systems with forward-shifts.
method Introducing backward-shifts to extend flatness definition.
result Extended flat systems maintain key properties like reachability and controllability.
The introduction of deep learning and transfer learning techniques in fields such as computer vision allowed a leap forward in the accuracy of image classification tasks. Currently there is only limited use of such techniques in neuroscience. The challenge of using deep learning methods to successfully train models in …
Recently, there has been great interest in the field of audio style transfer, where a stylized audio is generated by imposing the style of a reference audio on the content of a target audio. We improve on the current approaches which use neural networks to extract the content and the style of the audio signal and propo…
Transfer learning methods for reinforcement learning (RL) domains facilitate the acquisition of new skills using previously acquired knowledge. The vast majority of existing approaches assume that the agents have the same design, e.g. same shape and action spaces. In this paper we address the problem of transferring pr…
Injecting human knowledge is an effective way to accelerate reinforcement learning (RL). However, these methods are underexplored. This paper presents our discovery that an abstract forward model (thought-game (TG)) combined with transfer learning (TL) is an effective way. We take StarCraft II as our study environment.…
Training generative adversarial networks (GANs) on high quality (HQ) images involves important computing resources. This requirement represents a bottleneck for the development of applications of GANs. We propose a transfer learning technique for GANs that significantly reduces training time. Our approach consists of f…
Geotechnics adopts data-driven methods from materials informatics.
problem Soil complexity and lack of comprehensive data.
method Leveraging deep learning and transfer learning for feature extraction.
result Revolutionary potential of advanced computational tools in geotechnics.
Deep learning benchmarks ECG analysis with strong performance.
problem Lack of appropriate datasets and evaluation procedures for ECG interpretation.
method Benchmarking on PTB-XL and ICBEB2018 datasets using convolutional neural networks.
result Convolutional neural networks, especially resnet- and inception-based architectures, outperform feature-based algorithms.
Paper introduces a new value function for state transitions and optimal policy learning.
problem Learning optimal policies from state transitions and actions.
method Develops a forward dynamics model to maximize a novel value function Q(s,s′). result Demonstrates benefits in value function transfer, redundant action spaces, and off-policy learning.
Unified framework predicts S&P500 index direction using transfer learning and causal graph.
problem Predicting the movement of financial indices like S&P500.
method Transfer learning, causal graph, multidisciplinary knowledge, VAE network.
result 74.3% accuracy, 67% F1-score, 0.42 Matthew correlation on 12 years test period.
Proposes a new method to learn representations directly optimized for a task.
problem Learning representations optimized for unrelated tasks.
method Jointly learns representation and prediction function for a specific task.
result Learned representations outperform pre-trained ones and are more sample-efficient.
This paper presents a novel Inter Catchment Wastewater Transfer (ICWT) method for mitigating sewer overflow. The ICWT aims at balancing the spatial mismatch of sewer flow and treatment capacity of Wastewater Treatment Plant (WWTP), through collaborative operation of sewer system facilities. Using a hydraulic model, the…
Generative AI improves surrogate models by blending LF and HF data.
problem Data scarcity between high-fidelity and low-fidelity simulations.
method Probabilistic multi-fidelity surrogate framework using generative transfer learning.
result The model achieves HF accuracy with fewer HF evaluations.
New method discovers concepts in hidden feature layers using sparse subspace clustering.
problem Local attribution methods fail to identify coherent model behavior across samples.
method Sparse Subspace Clustering (SSCC) for concept discovery.
result Empirically validated method for various image classification tasks.
Meta-learning has been proposed as a framework to address the challenging few-shot learning setting. The key idea is to leverage a large number of similar few-shot tasks in order to learn how to adapt a base-learner to a new task for which only a few labeled samples are available. As deep neural networks (DNNs) tend to…
We investigate the existence of affine realizations for Lévy driven interest rate term structure models under the real-world probability measure, which so far has only been studied under an assumed risk-neutral probability measure. For models driven by Wiener processes, all results obtained under the risk-neutral appro…
A hybrid physics-ML model predicts FO water flux with high accuracy and uncertainty quantification.
problem Challenges in accurately modeling Forward Osmosis water flux due to complex internal mass transfer phenomena.
method Robust Hybrid Physics-ML framework using Gaussian Process Regression (GPR) for uncertainty-aware Jw prediction.
result Achieved a state-of-the-art MAPE of 0.26% and R2 of 0.999 on independent test data.
New optimizers control network width scaling, improving stability and transfer across different model sizes.
problem Designing stable optimizers for networks of varying widths.
method Interpreting optimizers as steepest descent under mean-normalized operator norms, enabling layerwise composability and width-independent bounds.
result New optimizers like row normalization and column normalization provide stable learning-rate transfer across different model widths.
Improves memory efficiency for meta-learning with large images.
problem High memory usage in meta-learning for few-shot classification.
method LITE: episodic training scheme that decomposes task gradients and back-propagates only a random subset of images.
result Achieves state-of-the-art accuracy on real-world and challenging benchmarks.
Artificial neural networks suffer from catastrophic forgetting when they are sequentially trained on multiple tasks. To overcome this problem, we present a novel approach based on task-conditioned hypernetworks, i.e., networks that generate the weights of a target model based on task identity. Continual learning (CL) i…
We consider the problem of building a state representation model in a continual fashion. As the environment changes, the aim is to efficiently compress the sensory state's information without losing past knowledge. The learned features are then fed to a Reinforcement Learning algorithm to learn a policy. We propose to …
In the continual learning setting, tasks are encountered sequentially. The goal is to learn whilst i) avoiding catastrophic forgetting, ii) efficiently using model capacity, and iii) employing forward and backward transfer learning. In this paper, we explore how the Variational Continual Learning (VCL) framework achiev…
A neural network learns to improve verification of neural networks.
problem Efficient verification of neural networks for safety-critical applications.
method A graph neural network (GNN) learns to imitate strong branching heuristics for effective branching in the Branch and Bound (BaB) formulation.
result Reduces the number of branches and verification time by roughly 50% compared to hand-designed strategies.
A novel ABC method for high-dimensional inverse problems using generative modeling and subset simulation.
problem Solving inverse-problems with high-dimensional inputs and expensive forward mappings.
method Joint deep generative modeling, Approximate Bayesian Computation (ABC) with Subset Simulation, and likelihood-free inference.
result Our method delivers promising performance without prior knowledge of the forward or noise distributions.
Survey of methods to train deep architectures without E2EBP.
problem Training deep architectures without end-to-end backpropagation.
method Modular training, weakly modular hybrids.
result Provably optimal alternatives to E2EBP can match or surpass E2EBP performance.
In this paper we apply change of numeraire techniques to the optimal transport approach for computing model-free prices of derivatives in a two periods model. In particular, we consider the optimal transport plan constructed in \cite{HobsonKlimmek2013} as well as the one introduced in \cite{BeiglJuil} and further studi…
Discrete-time systems can be characterized by simple flat coordinates and their shifts.
problem Characterizing flatness of discrete-time systems.
method Developed a map from flat coordinates and their shifts to system state and input, fulfilling system equations identically.
result Derived necessary conditions for a system to be flat, without requiring differential geometry methods.
Publicly pretraining models on Web data may undermine differential privacy.
problem The use of large Web-scraped datasets in differential privacy models.
method Critical review of leveraging pretrained models on public datasets for differential privacy.
result Publicizing pretrained models as 'private' could harm trust and generalize poorly.
Survey on reducing deep neural network training complexity.
problem Efficient training of large deep neural networks.
method Pruning and freezing parts of the network during training.
result Dimensionality reduction improves training efficiency.
Deep neural network predicts multiphase flow in heterogeneous domains.
problem Predicting multiphase flow in complex, heterogeneous systems.
method Deep neural network model for handling permeability heterogeneity and learning interplay of forces.
result Highly accurate predictions of CO2 saturation distribution with computational efficiency.
Transfer learning improves portfolio optimization by identifying transfer risk.
problem Financial portfolio optimization problem.
method Introduces transfer risk concept within transfer learning framework.
result Transfer risk is a significant indicator of transferability and enhances portfolio management efficiency.
Paper analyzes transfer risk in transfer learning for finance.
problem Evaluate transferability of transfer learning in finance.
method Proposes transfer risk concept and applies to stock return prediction and portfolio optimization.
result Transfer risk correlates with transfer learning performance and identifies appropriate source tasks.
This paper explores the connection between adversarial and knowledge transferability.
problem Understanding the factors affecting knowledge transferability.
method Theoretical analysis and practical metrics for adversarial transferability.
result Adversarial transferability and knowledge transferability are closely related.
Mathematical framework for transfer learning feasibility and transfer risk.
problem Theoretical analysis of transfer learning.
method Reformulated transfer learning as an optimization problem, introduced transfer risk concept.
result Demonstrated the potential and benefits of incorporating transfer risk in transfer learning evaluation.
It has been suggested that adversarial examples cause deep learning models to make incorrect predictions with high confidence. In this work, we take the opposite stance: an overly confident model is more likely to be vulnerable to adversarial examples. This work is one of the most proactive approaches taken to date, as…