New algorithm identifies outliers in PCA without needing parameters.
problem Robust PCA with unknown outlier fraction and subspace dimension.
method Non-iterative, parameter-free method for structured and unstructured outliers.
result Analytical guarantees and performance comparison with existing methods.
Paper uses genome Markov structure for outlier detection and read classification.
problem Identifying outliers and classifying reads in genome databases.
method Applying second-order Markov models to triplet base distributions.
result Improved accuracy in outlier identification and read classification.
New structure in neural network Hessians explains outliers, improving subspace approximation.
problem Explaining outliers in the spectrum of deepnet Hessians.
method Identified a two-level structure in the Hessian, showing it's not a covariance but a second moment matrix.
result Shows the means of gradients have an additive two-way structure, leading to outliers in the spectrum.
Geometric framework detects outliers in high-dimensional data.
problem Detecting outliers in high-dimensional data.
method Geometric framework exploiting manifold structure.
result Significant improvement in outlier detection in high-dimensional data.
Identifies root causes of outliers using causal DAGs.
problem Detecting and understanding the causes of anomalies in interrelated variables.
method Formal approach using causal directed acyclic graphs (DAGs), outlier scores, and Shapley values.
result Quantifies the extent of outlier scores attributed to ancestors.
Algorithm learns data structure in real-time with outliers and change points.
problem Sequential online prediction in the presence of outliers and change points.
method INTEL algorithm using WGPs and POE model for real-time structure learning.
result Significantly better performance than benchmarks in real datasets.
Study improves data quality assessment for structural monitoring data.
problem Ensuring reliability of structural health monitoring data.
method Probabilistic data quality assessment using a conditional diffusion model.
result Significantly improves accuracy of data quality assessment.
Identifies root causes of outliers in unknown cyclic graphs.
problem Outliers in unknown cyclic graphs with linear structural equations.
method Identifies a short list of potential root causes based on strong perturbation and structural equations.
result The shortlist includes true root causes and their parents on the cycle.
Geometric approach improves functional outlier detection.
problem Detecting outliers in functional data sets.
method Developed a geometric perspective on functional manifold for outlier detection.
result Improved understanding and differentiation of outliers.
Outlier detection plays an essential role in many data-driven applications to identify isolated instances that are different from the majority. While many statistical learning and data mining techniques have been used for developing more effective outlier detection algorithms, the interpretation of detected outliers do…
New method clusters matrix-variate data with outliers.
problem Clustering matrix-variate data with outliers.
method Iterative approach using subset log-likelihoods.
result Extends OCLUST algorithm to matrix-variate normal data.
HLoOP detects outliers in hyperbolic 2-space.
problem Detecting local outliers in hyperbolic 2-space.
method Combines nearest neighbor finding and probabilistic scoring in hyperbolic space.
result Promising results on WordNet dataset.
KOD detects outliers in high-dimensional data.
problem Challenges in outlier detection in high-dimensional settings.
method Kernel transformation followed by projection pursuit approach with ensemble of directions and result combination.
result Empirical evaluations show effectiveness on various datasets.
A novel semi-supervised outlier detection model detects anomalies with few labels.
problem Efficiently detecting group anomalies with limited labeled data.
method RCC-Dual-GAN model that combines RCC and M-GAN components for semi-supervised outlier detection.
result Significantly improved accuracy in outlier detection with few labeled anomalies.
Notwithstanding the popularity of conventional clustering algorithms such as K-means and probabilistic clustering, their clustering results are sensitive to the presence of outliers in the data. Even a few outliers can compromise the ability of these algorithms to identify meaningful hidden structures rendering their o…
New PCA method handles multiple datasets and detects sparse patterns robustly.
problem Handling multi-source data with sparse and outlier-robust PCA.
method Developed a regularization problem with a penalty for structured sparsity and outlier resistance.
result The method detects global and local patterns across multiple data sources robustly.
This review examines techniques for detecting anomalies in time series data.
problem Detecting anomalies in time series data.
method Taxonomy based on main aspects of outlier detection techniques.
result Structured and comprehensive state-of-the-art on outlier detection techniques in time series.
A hierarchical clustering algorithm for data clouds without structure assumptions.
problem Exploring data clouds without making structure assumptions.
method Hierarchical topological clustering algorithm that infers persistence of outliers and clusters of arbitrary shape from data hierarchy.
result The algorithm can provide meaningful clusters in complex datasets.
Developing an explainable outlier detection method for interval-valued data using Shapley value-based approach.
problem Outlier detection in interval-valued data.
method Proposed a novel approach based on Shapley value for interval-valued data.
result Fine-grained interpretation of outliers with variable contributions.
Improved isolation forest for better outlier detection.
problem Outlier detection in multivariate data.
method Random cuts across feature space, size of feature space, and point assignment information.
result Improved results in many situations without modifying tree structure.
A new robust Wasserstein distance is proposed to handle outliers in probability distributions.
problem Outliers in probability distributions make Wasserstein distances sensitive and impractical.
method Introduces a new outlier-robust Wasserstein distance Wpε. result Achieves strong robust estimation guarantees under the Huber ε-contamination model. This paper presents a remarkably simple, yet powerful, algorithm termed Coherence Pursuit (CoP) to robust Principal Component Analysis (PCA). As inliers lie in a low dimensional subspace and are mostly correlated, an inlier is likely to have strong mutual coherence with a large number of data points. By contrast, outli…
Generative Adversarial Active Learning detects outliers by sampling and generating informative data.
problem Detecting outliers in high-dimensional data with limited labeled examples.
method SO-GAAL method using a generator and discriminator to create informative outliers.
result MO-GAAL outperforms state-of-the-art methods on various datasets.
Two-stage framework detects multi-modal outliers.
problem Detecting outliers in multi-modal data structures.
method Combines global kernel PCA and local clustering stages.
result Significantly outperforms existing methods on challenging datasets.
Surrogate Data Analysis (SDA) is a statistical hypothesis testing framework for the determination of weak chaos in time series dynamics. Existing SDA procedures do not account properly for the rich structures observed in stock return sequences, attributed to the presence of heteroscedasticity, seasonal effects and outl…
A novel outlier detection method for high-dimensional data.
problem Challenges in outlier detection in high-dimensional data.
method Principal component analysis and kernel density estimation.
result The proposed method outperforms benchmark methods in F1-score and execution time. A method uses Shapley values and Mahalanobis distances to explain multivariate outliers.
problem Explaining multivariate outlyingness in data.
method Decomposing squared Mahalanobis distance using Shapley values.
result Shapley values provide variable contributions to outlying observations.
New algorithm recovers mixture means even with many outliers.
problem Estimating mixture means when outliers overwhelm small groups.
method Proposes an algorithm for robust mixture learning with minimal list-size overhead.
result Order-optimal error guarantees for each mixture mean.
PRAE identifies outliers and reconstructs inliers in autoencoders.
problem Accurately identifying anomalies in data.
method Probabilistic Robust AutoEncoder (PRAE) approach.
result PRAE effectively removes outliers and reconstructs inliers.
Cluster analysis and outlier detection are strongly coupled tasks in data mining area. Cluster structure can be easily destroyed by few outliers; on the contrary, outliers are defined by the concept of cluster, which are recognized as the points belonging to none of the clusters. Unfortunately, most existing studies do…
A large dimensional characterization of robust M-estimators of covariance (or scatter) is provided under the assumption that the dataset comprises independent (essentially Gaussian) legitimate samples as well as arbitrary deterministic samples, referred to as outliers. Building upon recent random matrix advances in the…
Bayesian model identifies outliers and determines tensor rank in streaming data.
problem Outliers and over-fitting in streaming tensor factorization.
method Variational Bayesian Inference for robust tensor rank determination and outlier identification.
result Model accurately identifies sparse outliers and determines tensor rank.
Over the past years Robust PCA has been established as a standard tool for reliable low-rank approximation of matrices in the presence of outliers. Recently, the Robust PCA approach via nuclear norm minimization has been extended to matrices with linear structures which appear in applications such as system identificat…
Robust fuzzy clustering for EEG driver alertness with outlier detection.
problem Ambiguous state boundaries in multivariate time series data.
method RFCPCA, a robust fuzzy subspace-clustering method for MTS.
result RFCPCA improves clustering accuracy and characterizes uncertainty and outliers in MTS.
MOSAIC selects few informative exemplars from high-dimensional data with non-linear structures.
problem Representative selection from high-dimensional data with non-linear structures.
method MOSAIC uses a multi-criteria approach with a quadratic formulation to maximize global representation power, diversity, and outlier detection.
result MOSAIC maximizes data coverage in a transformed space and achieves robustness to various outlier types.
The IM effect helps detect anomalies by memorizing inliers early.
problem Challenges in fully unsupervised outlier detection.
method Theoretical study of a simple autoencoder, focusing on early training dynamics.
result Characterization of IM effect emergence, strength, and persistence.
SDCOR clusters massive datasets efficiently, detecting outliers with low memory usage.
problem Local outlier detection in large-scale datasets.
method Chunk-based density clustering with incremental updates.
result SDCOR achieves lower linear time complexity and better efficiency than traditional methods.
RobPy offers robust statistical methods in Python.
problem Lack of robust statistical methods in Python.
method Built on NumPy, SciPy, and scikit-learn, RobPy includes robust tools for various statistical tasks.
result RobPy enables more users to perform robust data analysis in Python.
EHBOS enhances HBOS by capturing feature interactions, improving anomaly detection.
problem Limited ability of HBOS to detect anomalies in datasets with feature interactions.
method Incorporates two-dimensional histograms to capture feature pair dependencies.
result EHBOS outperforms HBOS on datasets with critical feature interactions, achieving notable improvements in ROC AUC.
Modern machine learning-based recognition approaches require large-scale datasets with large number of labelled training images. However, such datasets are inherently difficult and costly to collect and annotate. Hence there is a great and growing interest in automatic dataset collection methods that can leverage the w…
A new algorithm identifies outliers in Gaussian clustering models.
problem Handling outliers in Gaussian model-based clustering.
method OCLUST algorithm removes least plausible points based on subset log-likelihoods until they adhere to a reference distribution.
result OCLUST inherently estimates the number of outliers.
Paper tackles robust Euclidean distance estimation with sparse outliers.
problem Estimating point positions from corrupted distance measurements.
method Proposes a novel algorithm using Nyström method and robust PCA.
result Achieves accurate recovery with minimal anchors and sparse outliers.
New method detects outliers in correlated data.
problem Detecting outliers in noisy, correlated datasets.
method Robust regression model that detects outliers during model fitting.
result Outlier detection method achieves better performance.
We propose an inlier-based outlier detection method capable of both identifying the outliers and explaining why they are outliers, by identifying the outlier-specific features. Specifically, we employ an inlier-based outlier detection criterion, which uses the ratio of inlier and test probability densities as a measure…
A novel approach ODAR detects outliers for clustering.
problem Outliers interfere with clustering algorithms, leading to unreliable results.
method Feature transformation to separate outliers and normal objects into distinct clusters.
result ODAR improves clustering accuracy on 7 out of 10 datasets.
New framework detects outliers in non-IID categorical data.
problem Existing outlier detection methods fail in non-IID data.
method Value-value graph-based representation and outlierness propagation.
result Significant improvement in AUC on complex data sets.
Outlier detection aims to identify unusual data instances that deviate from expected patterns. The outlier detection is particularly challenging when outliers are context dependent and when they are defined by unusual combinations of multiple outcome variable values. In this paper, we develop and study a new conditiona…
Paper proposes methods to make OT robust to outliers.
problem Optimal transport is sensitive to outliers.
method Detect outliers using adversarial training, adjust transport cost based on classifier predictions.
result Outliers are detected and do not affect transport in experiments.