New method detects when models influence their own drift in real-time data streams.
problem Models can induce concept drift in real-time data streams.
method CheckerBoard Performative Drift Detection (CB-PDD)
result CB-PDD effectively detects performative drift in real-time data streams.
Enhash detects concept drift in data streams quickly and efficiently.
problem Detecting abrupt, gradual, virtual, or recurring events in data streams.
method Uses projection hash to insert incoming samples and detects concept drift.
result Enhash has competitive performance and moderate resource requirements compared to existing ensemble learners.
Detects drifts in data for classification tasks using constrained embeddings.
problem Drifts in data affect model performance; unsupervised methods ignore label information.
method Task-sensitive semi-supervised drift detection with constrained low-dimensional embedding.
result Successfully detects real drifts affecting classification performance.
Gradient-based methods for games suffer from discrete update steps that cause drift, affecting performance.
problem Gradient-based methods for two-player games suffer from drift due to discrete update steps.
method Derived modified continuous dynamical systems to closely follow the discrete dynamics of games.
result Identified distinct components of discretization drift that can alter or destabilize game performance.
Adversarial validation detects concept drift in user targeting systems.
problem Concept drift in user targeting automation systems deteriorates model performance over time.
method Adversarial validation approach to detect and adapt to concept drift.
result Adversarial validation effectively addresses concept drift in user targeting systems.
PDD detects concept drift using explainable AI, improving model performance in dynamic environments.
problem Detecting and adapting to concept drift in predictive models.
method Profile Drift Detection (PDD) using Partial Dependence Profiles (PDPs).
result PDD outperforms existing methods in detecting concept drift and maintaining high predictive performance.
Detects data drift and outliers affecting ML model performance over time.
problem Detecting distribution changes between training and deployment datasets for machine learning models.
method Nonparametrically tests model prediction confidence distributions for changes using Change Point Models (CPMs). Also uses nonparametric outlier methods.
result Demonstrates robustness of the method under various levels of drift class contamination.
A drift detection method for large datasets without labels.
problem Early detection of concept drift in large, unlabeled datasets.
method Classical statistical process control in a label-less setting.
result Better statistical power than previous methods under computational constraints.
Adaptive sampling detects local concept drift with limited labels.
problem Detecting local concept drift in dynamic environments with scarce labels.
method Combines residual-based exploration and exploitation with EWMA monitoring.
result Superior performance in label efficiency and drift detection accuracy.
New method detects drift in high-dimensional data.
problem Understanding and localizing concept drift in learning systems.
method Conformal predictions for drift localization.
result Our approach outperforms existing methods on image datasets.
A framework for evaluating and benchmarking concept drift detection methods
problem Data stream mining challenged by concept drift
method A novel benchmarking framework
result Fair comparisons of drift detection methods
Framework monitors insurance pricing models for drift and recalibration.
problem Maintaining predictive performance of pricing models in evolving insurance portfolios.
method Formalizes deviance loss and Murphy's score, studies Gini score, develops monitoring framework.
result Framework guides decisions on refitting or recalibrating pricing models.
This review covers learning under concept drift, including detection, understanding, and adaptation.
problem Unforeseeable changes in data distribution over time impact machine learning performance.
method Reviews and analyzes methodologies and techniques for concept drift detection, understanding, and adaptation.
result Establishes a framework for learning under concept drift with three main components.
AMUSE uses reinforcement learning to predict optimal model updates.
problem Concept drift weakens model performance over time.
method Reinforcement learning in a simulated environment.
result AMUSE proactively recommends updates based on performance improvements.
New budget quantifies drift in closed-loop learning, improving reproducibility.
problem Characterizing statistical learning under distributional drift in closed-loop settings.
method Introduces an intrinsic drift budget CT quantifying cumulative information-geometric motion of the data distribution. result Proves a drift-feedback bound of order T−1/2+CT/T for prequential reproducibility, up to controlled second-order remainder terms. A new drift detection method based on autoregressive models.
problem Concept drift in real-world data leads to decreased model performance.
method Autoregressive based drift detection method (ADDM).
result ADDM outperforms state-of-the-art drift detection methods.
Classifiers deployed in the real world operate in a dynamic environment, where the data distribution can change over time. These changes, referred to as concept drift, can cause the predictive performance of the classifier to drop over time, thereby making it obsolete. To be of any real use, these classifiers need to d…
Concept drift is formally defined as the change in joint distribution of a set of input variables X and a target variable y. The two types of drift that are extensively studied are real drift and virtual drift where the former is the change in posterior probabilities p(y|X) while the latter is the change in distributio…
With today's abundant streams of data, the only constant we can rely on is change. For stream classification algorithms, it is necessary to adapt to concept drift. This can be achieved by monitoring the model error, and triggering counter measures as changes occur. In this paper, we propose a drift detection mechanism …
CURIE uses cellular automata to detect concept drift in data streams.
problem Detecting changes in data distribution (concept drift) in data streams.
method CURIE represents data stream distribution in a cellular automata grid and uses its neighborhood rule to detect changes.
result CURIE, when hybridized with base learners, performs competitively in detection metrics and classification accuracy.
A fundamental issue for statistical classification models in a streaming environment is that the joint distribution between predictor and response variables changes over time (a phenomenon also known as concept drifts), such that their classification performance deteriorates dramatically. In this paper, we first presen…
In data stream mining, predictive models typically suffer drops in predictive performance due to concept drift. As enough data representing the new concept must be collected for the new concept to be well learnt, the predictive performance of existing models usually takes some time to recover from concept drift. To spe…
Automatic Machine Learning (Auto-ML) has attracted more and more attention in recent years, our work is to solve the problem of data drift, which means that the distribution of data will gradually change with the acquisition process, resulting in a worse performance of the auto-ML model. We construct our model based on…
Proceed adapts models proactively against concept drift in online time series forecasting.
problem Concept drift causes forecast models to adapt to outdated concepts, reducing performance.
method Proceed estimates and translates concept drift into parameter adjustments, enhancing model resilience.
result Proceed brings more performance improvements than state-of-the-art online learning methods.
Paper presents adaptive minimax risk classifiers for multidimensional concept drift.
problem Multidimensional concept drift in supervised classification.
method Adaptive minimax risk classifiers (AMRCs) tracking multivariate and high-order distribution changes.
result AMRCs provide computable tight performance guarantees and improve classification.
A new method detects concept drift without true labels.
problem Detecting concept drift in unsupervised settings.
method Student-teacher learning paradigm for drift detection.
result The method outperforms state-of-the-art approaches in experiments.
Paper proposes a framework to detect adversarial concept drifts under poisoning attacks.
problem Adversarial concept drift in data streams.
method Augmented Restricted Boltzmann Machine with improved gradient computation and energy function.
result High robustness and efficacy of the proposed drift detection framework in adversarial scenarios.
Learning from data streams is an increasingly important topic in data mining, machine learning, and artificial intelligence in general. A major focus in the data stream literature is on designing methods that can deal with concept drift, a challenge where the generating distribution changes over time. A general assumpt…
The paper develops a neural network-based classifier for diffusion process drifts.
problem Classifying diffusion processes with distinct drift functions from discrete observations.
method Derives a Bayes rule and constructs a plug-in classifier using neural networks to estimate drifts.
result Establishes convergence rates for misclassification risk, highlighting benefits of diffusion structure.
Paper tackles concept drift in Federated Learning, improving model performance.
problem Concept drift in real-world data makes existing Federated Learning methods ineffective.
method Introduces a multiscale algorithm combining extit{FedAvg} and extit{FedOMD} with non-stationary detection and adaptation.
result Achieves dynamic regret of $\Tilde{\mathcal{O}} ( \min \{ \sqrt{LT} , Δ^{\frac{1}{3}}T^{\frac{2}{3}} + \sqrt{T} \})$ for T rounds. One important assumption underlying common classification models is the stationarity of the data. However, in real-world streaming applications, the data concept indicated by the joint distribution of feature and label is not stationary but drifting over time. Concept drift detection aims to detect such drifts and adap…
DRIFT uses RL to automate functional software testing efficiently.
problem Efficient and reliable automated software testing.
method DRIFT employs Q-learning with Graph Neural Networks on symbolic UI representations.
result DRIFT can robustly test software functionalities in a fully automated manner.
Machine learning monitors detect motor overheating, adapting to concept drift.
problem Early detection of motor overheating in ships' propulsion systems.
method Machine learning and statistical methods using historical data to adapt to concept drift.
result The proposed monitors provide early detection of overheating during and after concept drifts.
Common statistical prediction models often require and assume stationarity in the data. However, in many practical applications, changes in the relationship of the response and predictor variables are regularly observed over time, resulting in the deterioration of the predictive performance of these models. This paper …
New framework tackles DG under posterior drift, where optimal classifier varies by domain.
problem Generalizing from multiple domains with varying optimal classifiers.
method Decision-theoretic framework for DG under posterior drift.
result Optimal classifier can vary significantly across domains, challenging existing DG approaches.
KT models struggle with student concept drift, but BKT remains the most stable.
problem Impact of student concept drift on KT models.
method Applied four KT models to five academic years of data.
result KT models generally degrade in performance with concept drift, BKT remains stable.
Concept drift in learning and classification occurs when the statistical properties of either the data features or target change over time; evidence of drift has appeared in search data, medical research, malware, web data, and video. Drift adaptation has not yet been addressed in high dimensional, noisy, low-context d…
Study improves survival analysis for credit risk by accounting for data drift.
problem Survival analysis in credit risk assumes a stationary data-generating process, but real-world data drift affects model performance.
method Proposes a dynamic joint modelling framework integrating longitudinal behavioural markers and hazard formulations, combined with drift-adaptive techniques.
result Proposed model outperforms classical survival models and drift-adaptive learners in various data drift scenarios.
Machine learning models fail due to concept and data drift during pandemic.
problem Machine learning models trained before the pandemic are unreliable during the pandemic.
method Detect and diagnose concept and data drift in models.
result Model resilience and robustness are crucial for future predictions.
New method detects concept drift in data streams with missing values.
problem Uncertainty introduced by missing values in concept drift detection.
method Fuzzy distance estimation and histogram bin allocation.
result Fuzzy set theory improves drift detection in data with missing values.
Optimizes quickest detection of drift in Brownian motion with false negatives.
problem Quickest detection of drift in Brownian motion with false negatives.
method Formulated as an optimal multiple stopping problem, then equivalent to a recursive optimal stopping problem, solved using free boundary methods.
result Explicit formulae for expected cost and optimal strategy found.
Framework models supervised learning in non-stationary data.
problem Non-stationary data in supervised learning.
method Statistical physics methods applied to LVQ and neural networks.
result LVQ and ReLU have different sensitivity to concept drift.
Bayesian non-parametric model adapts to concept drifts in streaming data.
problem Inference under concept drift phenomenon for non-stationary data streams.
method Variational inference algorithm for Dirichlet process mixture models with exponential forgetting.
result The proposed model outperforms state-of-the-art algorithms in clustering problems.
DDG-DA predicts future data distribution to adapt models for predictable concept drift.
problem Adapting models to streaming data with predictable concept drift.
method Train a predictor to forecast future data distribution, generate training samples, and train models on them.
result Significant improvement on multiple models in real-world tasks.
Study online conformal prediction for non-stationary data with optimal training-conditional regret.
problem Online prediction for non-stationary data streams with unknown distribution drift.
method Proposes split-conformal and full-conformal algorithms that adapt to drift detection and incorporate stability for online learning.
result Proves minimax-optimal regret for online full conformal algorithm under appropriate restrictions.
SHIFT framework identifies subgroups with large ML model performance decay.
problem Large model performance decay in subgroups when deployed.
method Subgroup-scanning Hierarchical Inference Framework (SHIFT) for performance drift.
result SHIFT identifies interpretable subgroups with large performance decay and suggests targeted actions to mitigate it.
MLDemon monitors ML systems post-deployment, improving reliability with real-time performance estimates and expert labels.
problem Ensuring reliability of machine learning systems post-deployment, especially when user inputs differ from training data.
method Integrates unlabeled and on-demand labeled data to monitor ML model performance in real-time, deciding when to acquire expert labels.
result Outperforms existing approaches in temporal datasets with diverse distribution drifts, providing theoretical optimality for distribution drifts.
In many real-world applications, data are often collected in the form of stream, and thus the distribution usually changes in nature, which is referred as concept drift in literature. We propose a novel and effective approach to handle concept drift via model reuse, leveraging previous knowledge by reusing models. Each…