Cellwise outliers challenge traditional methods in statistics and machine learning.
problem Cellwise outliers contaminate over half cases in data matrices, complicating existing methods.
method Requires techniques different from casewise methods, focusing on non-intuitive equivariance properties.
result Substantial progress in estimating location and covariance matrices, regression methods, and tensor data.
A new method preserves useful information in data rows with outlying cells.
problem Preserving useful information in data rows with outlying cells.
method Cellwise robust Minimum Covariance Determinant (cellMCD) method using observed likelihood and a penalty term on cellwise outliers.
result The cellMCD method performs well in simulations and on real data.
New robust MPCA method handles casewise and cellwise outliers in tensor data.
problem Outliers, especially casewise and cellwise, affect the performance of standard MPCA.
method Uses a single loss function to reduce the influence of both types of outliers and missing values.
result The new method improves robustness and performance in tensor data analysis.
Paper introduces detect-then-impute conformal prediction for cellwise outliers.
problem Uncertainty in prediction intervals for models with cellwise outliers.
method Detects outliers, imputes them, and constructs exchangeable features for conformal prediction.
result JDI-CP achieves a finite sample 1−2α coverage guarantee. Robust TOT regression method handles outliers in tensor data.
problem Outliers in tensor data affect standard TOT regression.
method ROTOT method using a single loss function for outliers and robust MPCA for predictor.
result ROTOT method reduces influence of both casewise and cellwise outliers.
Real data often contain anomalous cases, also known as outliers. These may spoil the resulting analysis but they may also contain valuable information. In either case, the ability to detect such anomalies is essential. A useful tool for this purpose is robust statistics, which aims to detect the outliers by first fitti…
A method uses Shapley values and Mahalanobis distances to explain multivariate outliers.
problem Explaining multivariate outlyingness in data.
method Decomposing squared Mahalanobis distance using Shapley values.
result Shapley values provide variable contributions to outlying observations.
Developing an explainable outlier detection method for interval-valued data using Shapley value-based approach.
problem Outlier detection in interval-valued data.
method Proposed a novel approach based on Shapley value for interval-valued data.
result Fine-grained interpretation of outliers with variable contributions.
A theory of cellwise contamination for compositional data using log-ratios.
problem Contamination in compositional data analysis.
method Develops a theory combining contamination model and propagation theorem.
result Reduction in cellwise breakdown value by (D−1)/D for certain estimators. RobPy offers robust statistical methods in Python.
problem Lack of robust statistical methods in Python.
method Built on NumPy, SciPy, and scikit-learn, RobPy includes robust tools for various statistical tasks.
result RobPy enables more users to perform robust data analysis in Python.
Minimal genus surfaces solve homology problems in finite complexes.
problem Finding surface representatives of homology classes with minimal genus.
method Minimizing genus and Euler characteristic, analyzing surgeries and homotopy.
result Minimizers are homotopic to cellwise coverings, but problem is undecidable in general.
We propose an inlier-based outlier detection method capable of both identifying the outliers and explaining why they are outliers, by identifying the outlier-specific features. Specifically, we employ an inlier-based outlier detection criterion, which uses the ratio of inlier and test probability densities as a measure…
A novel approach ODAR detects outliers for clustering.
problem Outliers interfere with clustering algorithms, leading to unreliable results.
method Feature transformation to separate outliers and normal objects into distinct clusters.
result ODAR improves clustering accuracy on 7 out of 10 datasets.
Clustering, or unsupervised classification, is a task often plagued by outliers. Yet there is a paucity of work on handling outliers in clustering. Outlier identification algorithms tend to fall into three broad categories: outlier inclusion, outlier trimming, and post hoc outlier identification methods, with the forme…
New framework detects outliers in non-IID categorical data.
problem Existing outlier detection methods fail in non-IID data.
method Value-value graph-based representation and outlierness propagation.
result Significant improvement in AUC on complex data sets.
Outlier detection aims to identify unusual data instances that deviate from expected patterns. The outlier detection is particularly challenging when outliers are context dependent and when they are defined by unusual combinations of multiple outcome variable values. In this paper, we develop and study a new conditiona…
Paper proposes methods to make OT robust to outliers.
problem Optimal transport is sensitive to outliers.
method Detect outliers using adversarial training, adjust transport cost based on classifier predictions.
result Outliers are detected and do not affect transport in experiments.
Outlier detection plays an essential role in many data-driven applications to identify isolated instances that are different from the majority. While many statistical learning and data mining techniques have been used for developing more effective outlier detection algorithms, the interpretation of detected outliers do…
Advances in sensor technology have enabled the collection of large-scale datasets. Such datasets can be extremely noisy and often contain a significant amount of outliers that result from sensor malfunction or human operation faults. In order to utilize such data for real-world applications, it is critical to detect ou…
Survey compares methods for generating artificial outliers.
problem Difficulty in detecting genuine outliers.
method Generates artificial outliers to approximate genuine ones.
result Variability in quality of generation approaches.
Paper tackles outlier detection in signals modeled by generative models with theoretical guarantees.
problem Recovering signals from linear measurements with sparse outliers.
method Proposes an iterative ADMM algorithm and gradient descent algorithm for outlier detection using ℓ1 and squared ℓ1 norm minimization. result Establishes theoretical recovery guarantees for signal reconstruction under sparse outliers.
Transforms distance-based outlier scores into interpretable probabilistic estimates.
problem Difficult interpretation of distance-based outlier scores.
method Generic transformation of scores into probabilistic estimates using distance probability distributions.
result Probabilistic transformation improves interpretability without impacting detection performance.
Generates synthetic data for benchmarking unsupervised outlier detection.
problem Difficulty in benchmarking unsupervised outlier detection due to rare and varied outliers in real data.
method Proposes a generic process to generate synthetic data with insightful characteristics.
result Demonstrates practicality of the generic process through a benchmark with state-of-the-art detection methods.
A novel unsupervised outlier detection method using Randomized PCA Forest.
problem Unsupervised outlier detection in datasets.
method Randomized Principal Component Analysis (RPCA) Forest for deriving an outlier score.
result Superior performance compared to classical and state-of-the-art methods.
HLoOP detects outliers in hyperbolic 2-space.
problem Detecting local outliers in hyperbolic 2-space.
method Combines nearest neighbor finding and probabilistic scoring in hyperbolic space.
result Promising results on WordNet dataset.
Robust PCA, the problem of PCA in the presence of outliers has been extensively investigated in the last few years. Here we focus on Robust PCA in the outlier model where each column of the data matrix is either an inlier or an outlier. Most of the existing methods for this model assumes either the knowledge of the dim…
New outlier detection method using graph Laplacian spectrum boosts performance.
problem Detecting outliers in large datasets efficiently.
method Boosted outlier detection based on graph Laplacian spectrum.
result Outperforms existing methods on synthetic datasets.
We evaluate how modern outlier detection methods perform in identifying outliers in e-commerce conversion rate data. Based on the limitations identified, we then present a novel method to detect outliers in e-commerce conversion rate. This unsupervised method is made more business relevant by letting it automatically a…
Enhances clustering for functional data, robust to outliers.
problem Challenges of clustering infinite-dimensional functional data and outlier sensitivity.
method Extends OCLUST algorithm to handle functional data, trimming outliers.
result Strong performance in clustering and outlier identification on simulated and real-world datasets.
A new method detects outliers using ensembles of Dirichlet process mixtures.
problem Challenges in unsupervised outlier detection using Dirichlet process mixtures.
method Ensembles of Dirichlet process Gaussian mixtures with random subspace and subsampling.
result Empirically outperforms existing approaches in unsupervised outlier detection.
Rare data in a large-scale database are called outliers that reveal significant information in the real world. The subspace-based outlier detection is regarded as a feasible approach in very high dimensional space. However, the outliers found in subspaces are only part of the true outliers in high dimensional space, in…
Robust PCA, the problem of PCA in the presence of outliers has been extensively investigated in the last few years. Here we focus on Robust PCA in the column sparse outlier model. The existing methods for column sparse outlier model assumes either the knowledge of the dimension of the lower dimensional subspace or the …
ODIM detects outliers by under-fitting generative models, outperforming other methods.
problem Identifying outliers in unlabeled data without supervision.
method Develops ODIM, a method that uses under-fitted deep generative models to detect outliers.
result ODIM outperforms other methods in detecting outliers efficiently and accurately.
A novel density-based approach QC detects outliers in data with high precision.
problem Detecting outliers in data with high precision and sensitivity.
method Quantum Clustering (QC) approach based on the density of data points.
result QC effectively finds hidden outliers and subtle outliers with parameter adjustment.
Inference in the presence of outliers is an important field of research as outliers are ubiquitous and may arise across a variety of problems and domains. Bayesian optimization is method that heavily relies on probabilistic inference. This allows outstanding sample efficiency because the probabilistic machinery provide…
Paper tackles outlier detection in multi-armed bandits, achieving high accuracy with reduced exploration costs.
problem Detecting outlier arms in multi-armed bandit settings.
method Proposes GOLD algorithm based on upper confidence bounds to identify generic outlier arms.
result Achieves 98% accuracy with 83% reduction in exploration cost compared to state-of-the-art techniques.
We focus on the problem of unsupervised cell outlier detection and repair in mixed-type tabular data. Traditional methods are concerned only with detecting which rows in the dataset are outliers. However, identifying which cells are corrupted in a specific row is an important problem in practice, and the very first ste…
PyODDS automates outlier detection for new data sources.
problem Manual outlier detection is inefficient and domain-specific.
method Automated end-to-end system with Apache Spark and database support.
result PyODDS optimizes outlier detection pipelines automatically.
This paper bridges outlier-robust estimation in robotics and computer vision with robust statistics.
problem Outlier-robust estimation for geometric perception in robotics and computer vision.
method Adapting and extending robust linear regression and list-decodable regression to non-convex domains and vector-valued measurements.
result Performance guarantees for modern estimation algorithms in the presence of outliers.
A new method detects and corrects outliers using optimal transport.
problem Outliers in data can skew estimation results, leading to inaccurate conclusions.
method Optimal transport with a concave cost function for outlier detection and correction.
result The method effectively identifies and corrects outliers, improving estimation accuracy.
Notwithstanding the popularity of conventional clustering algorithms such as K-means and probabilistic clustering, their clustering results are sensitive to the presence of outliers in the data. Even a few outliers can compromise the ability of these algorithms to identify meaningful hidden structures rendering their o…
This paper investigates differentially private analysis of distance-based outliers. The problem of outlier detection is to find a small number of instances that are apparently distant from the remaining instances. On the other hand, the objective of differential privacy is to conceal presence (or absence) of any partic…
Integrates outlier detection into neural networks for improved performance.
problem Lack of competency awareness in machine learning systems, especially in detecting outliers.
method Null Space Analysis (NuSA) of neural networks, computing and controlling null space projection.
result NuSA-trained networks maintain classification performance and detect outliers effectively.
One-Class Boundary Peeling detects outliers efficiently and robustly.
problem Unsupervised outlier detection in diverse data distributions.
method One-Class Boundary Peeling uses flexible boundaries generated by one-class SVMs and iteratively peels them.
result One-Class Boundary Peeling outperforms state-of-the-art methods in synthetic data simulations.
Adaptive algorithm for outlier detection by balancing arm exploration and threshold estimation.
problem Identifying outliers in a set of rewards where the threshold is a function of all rewards.
method Adaptively updated confidence interval for the threshold based on previous rounds' estimates, balancing exploration of individual arms and the outlier threshold.
result Efficient algorithm with reduced sample complexity for outlier detection.
New method clusters matrix-variate data with outliers.
problem Clustering matrix-variate data with outliers.
method Iterative approach using subset log-likelihoods.
result Extends OCLUST algorithm to matrix-variate normal data.
A new KF handles outliers without MSE loss.
problem Outliers degrade Kalman filter performance.
method NUV priors, EM and AM for variance estimation.
result Outlier-insensitive KF outperforms existing methods.
NLR models often perform worse than LR for outlying input data in environmental sciences.
problem NLR models often give poor predictions for input data outside the training domain.
method Screened input data for outliers, using linear extrapolation for outliers based on NLR within the non-outlier domain.
result NLROR approach reduces poor extrapolation and tends to outperform NLR and LR for outliers.