Graph-based rehearsal improves continual learning performance.
problem Catastrophic forgetting in continual learning models.
method Augmenting a memory array with a learnable random graph to capture pairwise similarities.
result Our model consistently outperforms baselines on task-free continual learning benchmarks.
Pseudo rehearsal uses non-photo-realistic images to save resources without sacrificing performance.
problem Catastrophic forgetting in neural networks when learning new tasks.
method Synthetically generate non-photo-realistic images to rehearse previous tasks.
result Non-photo-realistic images can be used for rehearsal without sacrificing performance and significantly reduce resource consumption.
A new method combines regularization and generative rehearsal for continual learning.
problem Catastrophic forgetting in neural networks over past tasks.
method Uses a normalizing flow to conditionally store past task data and regularize network embeddings.
result Performs favorably compared to state-of-the-art approaches with constant memory overhead.
In supervised machine learning, an agent is typically trained once and then deployed. While this works well for static settings, robots often operate in changing environments and must quickly learn new things from data streams. In this paradigm, known as streaming learning, a learner is trained online, in a single pass…
In general, neural networks are not currently capable of learning tasks in a sequential fashion. When a novel, unrelated task is learnt by a neural network, it substantially forgets how to solve previously learnt tasks. One of the original solutions to this problem is pseudo-rehearsal, which involves learning the new t…
Pseudo-rehearsal allows neural networks to learn a sequence of tasks without forgetting how to perform in earlier tasks. Preventing forgetting is achieved by introducing a generative network which can produce data from previously seen tasks so that it can be rehearsed along side learning the new task. This has been fou…
SPARC improves continual learning with minimal memory and computational overhead.
problem Efficient continual learning for deep neural networks.
method Combines task-specific working memories and task-agnostic semantic memory.
result Significantly reduces parameter usage (6% of full-model surrogates) while maintaining performance.
Robotic vision is a field where continual learning can play a significant role. An embodied agent operating in a complex environment subject to frequent and unpredictable changes is required to learn and adapt continuously. In the context of object recognition, for example, a robot should be able to learn (without forg…
Dark Experience improves continual learning with a simple, strong baseline.
problem General Continual Learning in scenarios where tasks are not sequential and offline training is not possible.
method Mixing rehearsal with knowledge distillation and regularization.
result Dark Experience outperforms consolidated approaches and leverages limited resources.
Catastrophic forgetting of connectionist neural networks is caused by the global sharing of parameters among all training examples. In this study, we analyze parameter sharing under the conditional computation framework where the parameters of a neural network are conditioned on each input example. At one extreme, if e…
A simple technique improves continual learning by 50% on image datasets.
problem Challenges in training neural networks on a stream of shifting data.
method Experience Replay (ER) with five tricks to mitigate its shortcomings.
result ER, enhanced with tricks, achieves significant accuracy gains.
Dynamic memory prevents forgetting in continuous learning of medical images.
problem Catastrophic forgetting in machine learning models over time due to domain shifts.
method Dynamic memory to store and replay diverse training data subsets.
result Dynamic memory mitigates forgetting without knowing when shifts occur.
The utility of learning a dynamics/world model of the environment in reinforcement learning has been shown in a many ways. When using neural networks, however, these models suffer catastrophic forgetting when learned in a lifelong or continual fashion. Current solutions to the continual learning problem require experie…
PEC improves class-incremental learning by measuring prediction error.
problem Challenges in class-incremental learning, particularly forgetting and class imbalance.
method Prediction Error-based Classification (PEC) measures prediction error of a model trained on data from a class.
result PEC outperforms other methods in class-incremental learning across multiple benchmarks.
A new method combines generative models and regularization to prevent forgetting in continual learning.
problem Catastrophic forgetting in neural networks when learning new tasks.
method Uses a normalizing flow as a generative model to keep past data embeddings and regularize them.
result Performs favorably compared to existing methods with constant memory overhead.
New algorithm improves graph-based active learning by identifying unexplored regions.
problem Improving graph-based active learning by identifying unexplored regions.
method Poisson Reweighted Laplacian Uncertainty Sampling (PWLL) with a diagonal perturbation.
result PWLL effectively identifies unexplored regions in graph-based data.
Bayesian analysis shows unlabeled data improve graph-based semi-supervised learning.
problem Improving semi-supervised learning with limited labeled data.
method Bayesian nonparametric approach using unlabeled data for graph-based learning.
result Posterior contracts optimally around the truth with sufficient unlabeled data.
Our research is focused on understanding and applying biological memory transfers to new AI systems that can fundamentally improve their performance, throughout their fielded lifetime experience. We leverage current understanding of biological memory transfer to arrive at AI algorithms for memory consolidation and repl…
Graph-based methods for anomaly detection and semi-supervised learning.
problem Detecting unusual clinical actions and anomalies in hospital data.
method Label propagation, harmonic solution, regularization, graph connectivity analysis.
result Effective anomaly detection and semi-supervised learning methods for healthcare data.
Improves graph-based active learning for non-Gaussian models.
problem Efficiently selecting data points for labeling in graph-based semi-supervised learning.
method Approximates non-Gaussian distributions, introduces rank-one update and model change acquisition function.
result Enhanced active learning for graph-based SSL under non-Gaussian models.
Galerkin method outperforms graph-based methods in spectral decompositions.
problem Improving spectral decomposition methods in machine learning.
method Restricting study to a small set of test functions using the Galerkin method.
result Statistical and computational superiority of Galerkin method over graph-based approaches.
Survey of methods to incorporate external knowledge into stock price prediction.
problem Challenges in predicting stock prices due to market volatility and non-linearity.
method Survey of methods for acquiring and incorporating external knowledge into stock price prediction models.
result Systematic synthesis of previous studies on external knowledge types and their application in stock price prediction.
This paper introduces a novel graph signal processing framework for building graph-based models from classes of filtered signals. In our framework, graph-based modeling is formulated as a graph system identification problem, where the goal is to learn a weighted graph (a graph Laplacian matrix) and a graph-based filter…
Graph-based weather prediction adapted for local models.
problem Applying neural weather prediction to limited area modeling.
method Adapting graph-based Neural Weather Prediction approach to local models.
result Validation of multi-scale hierarchical model extension for Nordic region.
Active graph-based semi-supervised learning (AG-SSL) aims to select a small set of labeled examples and utilize their graph-based relation to other unlabeled examples to aid in machine learning tasks. It is also closely related to the sampling theory in graph signal processing. In this paper, we revisit the original fo…
The paper extends graph-based semi-supervised learning to infinite-dimensional Wasserstein space.
problem Graph-based semi-supervised learning in high-dimensional data.
method Laplace Learning in the Wasserstein space, proving variational convergence and characterizing the Laplace-Beltrami operator.
result Consistent classification performance in high-dimensional settings.
Shapley Flow interprets model predictions using a graph-based approach to feature importance.
problem Existing feature importance methods ignore or hide feature dependencies.
method Shapley Flow considers the entire causal graph and assigns credit to edges.
result Shapley Flow provides a deeper, graph-based view of feature importance.
This paper detects function-level obfuscation in binary code using graph-based methods.
problem Detecting and characterizing function-level obfuscation in binary code.
method Graph-based approaches, including GNNs, are compared on various datasets.
result GNNs outperform baselines in function-level obfuscation detection, especially in a 11-class classification task.
GATES improves neural architecture search by modeling operations as information transformation.
problem Improving predictor-based neural architecture search efficiency.
method GATES models operations as information transformation, covering both node and edge cell search spaces.
result GATES boosts sample efficiency and improves predictor performance.
Improved graph-based semi-supervised learning with model change active learning.
problem Identifying which unlabelled data points to label to best improve classifier performance.
method Pairing model change active learning with graph-based semi-supervised learning methods.
result Improved multiclass classification performance over prior methods.
Proposes a new algorithm for graph-based semi-parametric contextual bandits.
problem Non-stationarity in human behavior and social interaction.
method SemiGraphTS algorithm for graph-based semi-parametric reward model.
result Derives an upper bound of cumulative regret for graph-based semi-parametric model.
Poisson learning improves graph-based semi-supervised learning at very low label rates.
problem Degeneracy of Laplacian semi-supervised learning at low label rates.
method Replaces label assignment with source and sink placement, solving Poisson equation.
result Provably more stable and informative predictions than Laplacian learning.
The quality of a graph is determined jointly by three key factors of the graph: nodes, edges and similarity measure (or edge weights), and is very crucial to the success of graph-based semi-supervised learning (SSL) approaches. Recently, dynamic graph, which means part/all its factors are dynamically updated during the…
Proposes CI-GMVC to improve graph-based multi-view clustering performance.
problem Inconsistency in multi-view data affects clustering performance.
method Integrates consistent and inconsistent parts of multiple views using a unified matrix.
result Demonstrates improved clustering performance on real-world datasets.
Recommender system is an important component of many web services to help users locate items that match their interests. Several studies showed that recommender systems are vulnerable to poisoning attacks, in which an attacker injects fake data to a given system such that the system makes recommendations as the attacke…
We consider a family of problems that are concerned about making predictions for the majority of unlabeled, graph-structured data samples based on a small proportion of labeled samples. Relational information among the data samples, often encoded in the graph/network structure, is shown to be helpful for these semi-sup…
Graph-based Bayesian SSL uses graph theory to propagate labels from a few to many unlabeled features.
problem Efficiently propagating labels from a small set of labeled data to a large set of unlabeled data.
method Probabilistic framework using graph theory and Bayesian statistics.
result Mathematical foundations for improving the accuracy and efficiency of label propagation.
A new graph-based clustering method for moderate-dimensional data.
problem Performance degradation of existing graph-based clustering methods in high dimensions.
method Introduces UN-CCDs using NND-based MC-SRT for covering radii determination.
result UN-CCDs provide stable and competitive performance in moderate-sized datasets.
Flexible per-class regularization improves binary classifiers.
problem Improving binary classifiers by addressing outliers and class imbalance.
method Graph-based adaptive regularization with flexible per-class thresholds.
result Flexible thresholds improve classifier performance and address class imbalance.
It was recently shown that architectural, regularization and rehearsal strategies can be used to train deep models sequentially on a number of disjoint tasks without forgetting previously acquired knowledge. However, these strategies are still unsatisfactory if the tasks are not disjoint but constitute a single increme…
Graph-based ML improves defect prediction in software development.
problem Challenges in predicting defect-prone changes in complex software development.
method Building contribution graphs from developers and source files, using graph-based ML for defect prediction.
result Graph-based ML leads to significantly better defect prediction (F1 score up to 77.55%, MCC up to 53.16%).
Repelling random walks improve graph-based sampling efficiency.
problem Efficient graph-based sampling and statistical estimation.
method Induces correlations between trajectories of an ensemble of walkers on a graph, maintaining unbiasedness.
result Improves concentration of statistical estimators on graphs.
GraphFL tackles semi-supervised node classification on graphs using federated learning.
problem Real-world graph-based problems often require collecting the entire graph and labeling a reasonable number of labels, which is impractical and costly.
method GraphFL is a federated learning framework that addresses non-IID data, new label domains, and unlabeled data issues in graph-based semi-supervised node classification.
result GraphFL significantly outperforms compared FL baselines and self-training methods.
AUC-spec optimizes graph-based SSL for complex label distributions.
problem Training accurate models with scarce labeled data and abundant unlabeled data.
method Computes a low-dimensional representation that maximizes class separation via AUC optimization.
result AUC-spec achieves competitive results on synthetic and real-world datasets.
Study examines unsupervised and graph-based methods for anomaly detection in IoBT, outperformed by supervised stacking ensemble.
problem Anomaly detection in adversarial environments of IoBT.
method Unsupervised learning, graph-based methods, ensemble supervised learning, adversarial training.
result Supervised stacking ensemble method outperforms unsupervised and graph-based methods in detecting anomalies.
Paper detects changes in graph-based data streams using likelihood-ratios.
problem Detecting changes in synchronized graph-based data streams.
method Kernel-based likelihood-ratio estimation over graph nodes.
result Effective detection and localization of change-points.
We describe a graph-based semi-supervised learning framework in the context of deep neural networks that uses a graph-based entropic regularizer to favor smooth solutions over a graph induced by the data. The main contribution of this work is a computationally efficient, stochastic graph-regularization technique that u…
Proposes a CL technique to improve accuracy and reduce forgetting.
problem Sequential task learners struggle with forgetting information from previous tasks.
method Extracts modular parts of neural networks and estimates task relatedness.
result Remarkable performance gain in robustness to forgetting for EWC and GEM methods.