LogAnMeta detects anomalies from log events using meta learning.
problem Poor performance of current log anomaly detection on new or unseen anomalies.
method Meta-learning-based hybrid few-shot classifier trained in an episodic manner.
result Demonstrates efficacy of LogAnMeta on detecting anomalies with few samples.
The paper proposes a SeqGAN model to generate balanced log messages for anomaly detection.
problem Imbalanced log data makes anomaly detection difficult.
method SeqGAN for generating balanced log messages, Autoencoder for feature extraction, GRU for anomaly detection.
result Oversampling and balancing data improves anomaly detection accuracy.
Dividing deep learning models for consistent anomaly detection in changing log data.
problem Anomaly detection methods fail when log data types change, leading to false negatives.
method Divide deep learning models based on log data correlation and extract correlations.
result Continues anomaly detection accuracy even when log data changes.
Logsy detects anomalies in logs using a novel classification-based approach.
problem Anomaly detection in unstructured logs is challenging due to limited model generalization.
method Logsy learns log representations by distinguishing normal and anomaly logs using a classification-based approach with an attention-based encoder and hyperspherical loss function.
result Logsy improves anomaly detection performance by 0.25 in F1 score compared to previous methods.
Graph neural networks detect anomalies in object-centric business processes.
problem Detecting anomalies in graph-like business processes.
method Graph convolutional autoencoder architecture for anomaly detection.
result Promising performance in detecting anomalies at the activity type and attributes level.
New method detects anomalies in computing centers' logs.
problem Anomaly detection in continuously changing log data for predictive maintenance.
method Evolving granular classifiers using Fuzzy-set-Based evolving Modeling and evolving Granular Neural Network.
result Classification model prioritizes maintenance based on anomaly severity.
This paper detects anomalies in cellular network traffic using hybrid methods.
problem Detecting anomalies in network traffic for security and analysis.
method Hybrid method combining GARCH, K-means, and Neural Network.
result Anomaly detection in cellular network traffic successfully achieved.
ADSAGE detects anomalies in graph edge sequences for insider threat detection.
problem Detecting insider threats in fine-grained audit logs using graph and text features.
method Anomaly detection at edge level, supporting numeric, categorical, and text attributes.
result ADSAGE detects anomalies in authentications and email communications effectively.
We propose a supervised anomaly detection method based on neural density estimators, where the negative log likelihood is used for the anomaly score. Density estimators have been widely used for unsupervised anomaly detection. By the recent advance of deep learning, the density estimation performance has been greatly i…
Anomaly detection using dimensionality reduction has been an essential technique for monitoring multidimensional data. Although deep learning-based methods have been well studied for their remarkable detection performance, their interpretability is still a problem. In this paper, we propose a novel algorithm for estima…
Log messages are now widely used in software systems. They are important for classification as millions of logs are generated each day. Most logs are unstructured which makes classification a challenge. In this paper, Deep Learning (DL) methods called Auto-LSTM, Auto-BLSTM and Auto-GRU are developed for anomaly detecti…
ACE explains security anomaly detection models through feature contributions.
problem Understanding which features contribute to security anomalies.
method Regression framework to locally approximate anomaly scores.
result Identifies correct contributing features in synthetic data and real data.
Correlated anomaly detection (CAD) from streaming data is a type of group anomaly detection and an essential task in useful real-time data mining applications like botnet detection, financial event detection, industrial process monitor, etc. The primary approach for this type of detection in previous researches is base…
New methods improve anomaly detection in deep networks by leveraging hierarchical likelihoods and multi-scale features.
problem Challenges in detecting anomalies in high-level features due to model bias and domain prior.
method Two methods: 1) Log likelihood ratios between in-distribution and general distribution models, 2) Multi-scale likelihood contribution.
result Strong anomaly detection performance in unsupervised setting, slightly underperforming supervised methods.
Analysis of an organization's computer network activity is a key component of early detection and mitigation of insider threat, a growing concern for many organizations. Raw system logs are a prototypical example of streaming data that can quickly scale beyond the cognitive power of a human analyst. As a prospective fi…
Deep learning has recently demonstrated state-of-the art performance on key tasks related to the maintenance of computer systems, such as intrusion detection, denial of service attack detection, hardware and software system failures, and malware detection. In these contexts, model interpretability is vital for administ…
DPGIIL clusters structural anomalies using transmissibility functions with deep learning and Dirichlet process.
problem Clustering structural anomalies in high-dimensional streaming data with optimal cluster number determination.
method Combines Dirichlet process and deep generative models for incremental learning and anomaly detection.
result DPGIIL outperforms traditional methods in anomaly detection and clustering.
Extremes play a special role in Anomaly Detection. Beyond inference and simulation purposes, probabilistic tools borrowed from Extreme Value Theory (EVT), such as the angular measure, can also be used to design novel statistical learning methods for Anomaly Detection/ranking. This paper proposes a new algorithm based o…
Gaussian graphical models (GGMs) are probabilistic tools of choice for analyzing conditional dependencies between variables in complex systems. Finding changepoints in the structural evolution of a GGM is therefore essential to detecting anomalies in the underlying system modeled by the GGM. In order to detect structur…
We leverage a streaming architecture based on ELK, Spark and Hadoop in order to collect, store, and analyse database connection logs in near real-time. The proposed system investigates outliers using unsupervised learning; widely adopted clustering and classification algorithms for log data, highlighting the subtle var…
Improves anomaly detection with contaminated unlabeled data.
problem Weakness in existing semi-supervised anomaly detection methods when unlabeled data contain anomalies.
method Integrates positive-unlabeled learning with deep anomaly detection models.
result Achieves better detection performance on various datasets.
PAC-Wrap provides provable guarantees for semi-supervised anomaly detection.
problem Ensuring reliable anomaly detection in safety-critical applications.
method PAC-Wrap wraps around existing anomaly detection methods to provide PAC guarantees.
result PAC-Wrap effectively provides rigorous guarantees for various anomaly detectors.
Study identifies high-density anomalies in normal data regions.
problem Detecting anomalies in normal data regions.
method Introduces non-parametric algorithmic frameworks for unsupervised detection.
result IPP framework yields the best detection results.
Enhanced neural networks detect thin boundaries between different types of anomalies.
problem Detecting thin boundaries between different types of anomalies in outlier detection.
method Defined new types of anomalies, enhanced Multi-Layer Perceptron Neural Network with Genetic Algorithm.
result Reduced error in anomaly detection compared to baseline methods.
We propose a supervised anomaly detection method for data with inexact anomaly labels, where each label, which is assigned to a set of instances, indicates that at least one instance in the set is anomalous. Although many anomaly detection methods have been proposed, they cannot handle inexact anomaly labels. To measur…
Survey categorizes time series anomaly detection methods.
problem Need for anomaly detection in time series data.
method Process-centric taxonomy of anomaly detection methods.
result Meta-analysis of time series anomaly detection trends.
Study examines how discretization affects anomaly detection in datasets.
problem Detecting six types of anomalies in datasets using different discretization methods.
method Conducted experiments with SECODA, a general-purpose algorithm for unsupervised anomaly detection.
result Different discretization methods favor the discovery of certain anomaly types.
Deep learning improves anomaly detection across various fields.
problem Detecting anomalies in data with advanced approaches.
method Survey of deep learning methods for anomaly detection.
result Advancements in deep anomaly detection address unique challenges.
Ensemble learning improves anomaly detection for milder symptoms.
problem Difficulty in detecting incipient anomalies due to similarity to normal conditions.
method Utilize uncertainty information from ensemble learning to identify misclassified incipient anomalies.
result Ensemble learning methods show improved performance on incipient anomaly detection.
Proposes OCGNN for detecting anomalies in graph data.
problem Detecting anomalies in graph-structured data.
method One Class Graph Neural Network (OCGNN) combining Graph Neural Networks and one-class classification.
result Significant improvements in anomaly detection compared to baselines.
Survey on LSTM-based anomaly detection for technical systems.
problem Detect anomalies in technical systems due to complex dynamics.
method Use LSTM networks and other AI techniques to detect anomalies considering temporal and contextual characteristics.
result Demonstrates the potential of LSTM networks and graph-based approaches for anomaly detection.
Survey evaluates 20 anomaly detection methods for time-series data.
problem Comparing statistical, machine learning, and deep learning methods for anomaly detection.
method Comprehensive evaluation of 20 methods from three categories on public datasets.
result Provides insights into the performance and suitability of anomaly detection methods.
Unsupervised deep learning detects image quality issues without annotations.
problem Anomaly detection in images with quality issues.
method Use of deep generative models, specifically normalizing flows, for anomaly detection.
result Accurate semantic labeling and quality trends observed in images.
Anomaly detection for high-dimensional data using large deviations principle.
problem Challenges in anomaly detection for high-dimensional data.
method Large Deviations Anomaly Detection (LAD) algorithm.
result Outperforms state-of-the-art methods on high-dimensional data sets.
Ensemble unsupervised anomaly detection using IRT for hidden ground truth.
problem Challenges in constructing an ensemble from unsupervised anomaly detection methods.
method Use Item Response Theory to compute latent traits and construct an ensemble that downplays noisy methods.
result Demonstrated effectiveness of IRT ensemble on extensive data repository.
FAMDAD detects anomalies in mixed data using kurtosis-weighted Factor Analysis.
problem Detecting anomalies in high-dimensional mixed data.
method kurtosis-weighted Factor Analysis of Mixed Data (FAMDAD).
result Anomalies are highly separable in the first and last few dimensions of the FAMDAD embedding.
System states that are anomalous from the perspective of a domain expert occur frequently in some anomaly detection problems. The performance of commonly used unsupervised anomaly detection methods may suffer in that setting, because they use frequency as a proxy for anomaly. We propose a novel concept for anomaly dete…
A novel non-supervised method detects anomalies in multivariate time series.
problem Detecting anomalies in multivariate time series data.
method Partitioning based on clustering of correlation coefficients.
result Significant improvement in anomaly detection performance.
Detects systematic anomalies in consumer complaints using NLP.
problem Detecting small, frequent anomalies in consumer complaints.
method NLP conversion of narratives, followed by anomaly detection algorithm.
result Demonstrates effectiveness of NLP for detecting systematic anomalies.
Paper introduces an unsupervised tensor-based anomaly detection method for spatiotemporal data.
problem Challenges in detecting anomalies in spatiotemporal data, especially in urban traffic monitoring and medical imaging.
method Formulates anomaly detection as a regularized robust low-rank + sparse tensor decomposition, incorporating spatiotemporal smoothness and local dependencies.
result Demonstrates improved anomaly detection performance on both synthetic and real data.
A new method assigns anomaly scores to features for better interpretation.
problem Interpreting anomaly scores from feature attributions.
method Proposes a characteristic function to attribute anomaly scores using Shapley value.
result Demonstrates the potential utility of the proposed attribution methods.
Monitoring gas turbine combustors health, in particular, early detecting abnormal behaviors and incipient faults, is critical in ensuring gas turbines operating efficiently and in preventing costly unplanned maintenance. One popular means of detecting combustor abnormalities is through continuously monitoring exhaust g…
Anomaly detection aims to detect abnormal events by a model of normality. It plays an important role in many domains such as network intrusion detection, criminal activity identity and so on. With the rapidly growing size of accessible training data and high computation capacities, deep learning based anomaly detection…
Paper proposes RAN for better anomaly detection in time series data.
problem Anomaly detection algorithms often fail to accurately detect anomalies due to incomplete reconstruction of anomaly data.
method RAN uses adversarial learning and latent vector-constrained Autoencoder to ensure consistent reconstruction of anomaly data.
result RAN outperforms other algorithms in detecting meaningful anomalies with higher AUC-ROC scores.
Proposes a fair deep anomaly detection method.
problem Ensuring fairness in deep anomaly detection.
method Adversarial network to de-correlate sensitive attributes and learned representations.
result Our approach largely removes unfairness with minimal anomaly detection performance loss.
EHBOS enhances HBOS by capturing feature interactions, improving anomaly detection.
problem Limited ability of HBOS to detect anomalies in datasets with feature interactions.
method Incorporates two-dimensional histograms to capture feature pair dependencies.
result EHBOS outperforms HBOS on datasets with critical feature interactions, achieving notable improvements in ROC AUC.
AEGR method improves anomaly detection in autoencoders without needing anomaly-free training data.
problem Challenges in anomaly detection, especially high dimensionality and noise in training sets.
method Gradient-reversal method for autoencoders, using reconstruction error and Local Outlier Factor.
result The proposed AEGR model outperforms other methods in detecting network anomalies.
Although deep learning has been applied to successfully address many data mining problems, relatively limited work has been done on deep learning for anomaly detection. Existing deep anomaly detection methods, which focus on learning new feature representations to enable downstream anomaly detection methods, perform in…