DTOR explains anomalies with rule-based explanations.
problem Need to explain anomalies in data effectively.
method Applies Decision Tree Regressor to estimate anomaly scores and generate rule-based explanations.
result DTOR produces robust and consistent rule-based explanations.
A method uses Shapley values and Mahalanobis distances to explain multivariate outliers.
problem Explaining multivariate outlyingness in data.
method Decomposing squared Mahalanobis distance using Shapley values.
result Shapley values provide variable contributions to outlying observations.
OutlierTree detects outliers using decision trees and provides explanations.
problem Detecting outliers in data while providing understandable explanations.
method Supervised decision tree splits with 1-d confidence intervals.
result Human-readable explanations for outlier detection.
New findings suggest deep generative models can misclassify outliers, requiring new evaluation methods.
problem Deep generative models often assign higher likelihood to outliers, challenging existing outlier detection methods.
method Analyzed the typical set and high-density region of DGMs, proposing a novel outlier test.
result Existing likelihood-based outlier tests may fail due to model calibration issues, not just misclassification.
The outlying property detection problem is the problem of discovering the properties distinguishing a given object, known in advance to be an outlier in a database, from the other database objects. In this paper, we analyze the problem within a context where numerical attributes are taken into account, which represents…
Outliers with opposing signals significantly affect neural network optimization.
problem Understanding and mitigating the impact of outliers with opposing signals on neural network training.
method Identifying and analyzing pairs of outliers with strong opposing signals in training data.
result Outliers with opposing signals can cause optimization to enter a narrow valley, leading to oscillatory behavior and eventual loss spikes.
Focuses on monitoring and explaining models in real-world applications.
problem Ensuring high quality machine learning services in production environments.
method Statistical techniques for model performance and data monitoring, explanations of predictions.
result Challenges and solutions for implementing monitoring and explanation in production models.
Anomaly detection algorithms are often thought to be limited because they don't facilitate the process of validating results performed by domain experts. In Contrast, deep learning algorithms for anomaly detection, such as autoencoders, point out the outliers, saving experts the time-consuming task of examining normal …
Heavy-tailed outliers are more resilient to robust estimation than adversarial ones.
problem Developing robust estimators for data with outliers.
method Analyzing the relationship between adversarial and heavy-tailed outlier models.
result Optimal estimators for heavy-tailed outliers are also optimal for adversarial settings, but not vice versa.
This paper introduces collective counterfactual explanations for groups of instances in classification models.
problem Understanding how classification models make decisions for groups of instances.
method Novel Mathematical Optimization models to find collective counterfactual explanations that minimize total perturbation cost.
result Detects critical features for entire dataset classification and handles outliers.
The initial analysis of any large data set can be divided into two phases: (1) the identification of common trends or patterns and (2) the identification of anomalies or outliers that deviate from those trends. We focus on the goal of detecting observations with novel content, which can alert us to artifacts in the dat…
New method converts LVAs into linear projections for better understanding of complex models.
problem Limited interpretability of nonlinear machine learning models.
method Animated linear projections and radial tours.
result Improved understanding of variable importance in complex models.
Method identifies root causes of anomalies in causal processes.
problem Identifying root causes of anomalies in causal processes.
method Noisy functional causal model, Bayesian learning, gradient-based attribution.
result Proposes efficient method to compute anomaly attribution scores.
RFX-Fuse combines Breiman and Cutler's Random Forest with modern ML capabilities.
problem Lack of a unified ML engine with diverse capabilities.
method Unified ML engine with native GPU/CPU support, delivering 5+ functionalities in one model.
result Native explainable similarity and imputation validation.
Counterfactual explanations can be obtained by identifying the smallest change made to a feature vector to qualitatively influence a prediction; for example, from 'loan rejected' to 'awarded' or from 'high risk of cardiovascular disease' to 'low risk'. Previous approaches often emphasized that counterfactuals should be…
SurvLIME-KS improves survival model explanations robustly.
problem Improving explanations of unreliable survival models.
method SurvLIME-KS combines Cox proportional hazards model and Kolmogorov-Smirnov bounds for robust optimization.
result SurvLIME-KS minimizes average distance and maximizes distance in approximating cumulative hazard functions.
New experiments show deep networks benefit from memorizing rare data points.
problem Understanding why neural networks perform well on training data.
method Estimating influence and memorization values of training examples.
result Deep networks benefit from memorizing rare data points for good performance.
We propose an inlier-based outlier detection method capable of both identifying the outliers and explaining why they are outliers, by identifying the outlier-specific features. Specifically, we employ an inlier-based outlier detection criterion, which uses the ratio of inlier and test probability densities as a measure…
A novel approach ODAR detects outliers for clustering.
problem Outliers interfere with clustering algorithms, leading to unreliable results.
method Feature transformation to separate outliers and normal objects into distinct clusters.
result ODAR improves clustering accuracy on 7 out of 10 datasets.
Clustering, or unsupervised classification, is a task often plagued by outliers. Yet there is a paucity of work on handling outliers in clustering. Outlier identification algorithms tend to fall into three broad categories: outlier inclusion, outlier trimming, and post hoc outlier identification methods, with the forme…
New framework detects outliers in non-IID categorical data.
problem Existing outlier detection methods fail in non-IID data.
method Value-value graph-based representation and outlierness propagation.
result Significant improvement in AUC on complex data sets.
Outlier detection aims to identify unusual data instances that deviate from expected patterns. The outlier detection is particularly challenging when outliers are context dependent and when they are defined by unusual combinations of multiple outcome variable values. In this paper, we develop and study a new conditiona…
Paper proposes methods to make OT robust to outliers.
problem Optimal transport is sensitive to outliers.
method Detect outliers using adversarial training, adjust transport cost based on classifier predictions.
result Outliers are detected and do not affect transport in experiments.
Outlier detection plays an essential role in many data-driven applications to identify isolated instances that are different from the majority. While many statistical learning and data mining techniques have been used for developing more effective outlier detection algorithms, the interpretation of detected outliers do…
Advances in sensor technology have enabled the collection of large-scale datasets. Such datasets can be extremely noisy and often contain a significant amount of outliers that result from sensor malfunction or human operation faults. In order to utilize such data for real-world applications, it is critical to detect ou…
Survey compares methods for generating artificial outliers.
problem Difficulty in detecting genuine outliers.
method Generates artificial outliers to approximate genuine ones.
result Variability in quality of generation approaches.
Paper tackles outlier detection in signals modeled by generative models with theoretical guarantees.
problem Recovering signals from linear measurements with sparse outliers.
method Proposes an iterative ADMM algorithm and gradient descent algorithm for outlier detection using ℓ1 and squared ℓ1 norm minimization. result Establishes theoretical recovery guarantees for signal reconstruction under sparse outliers.
Transforms distance-based outlier scores into interpretable probabilistic estimates.
problem Difficult interpretation of distance-based outlier scores.
method Generic transformation of scores into probabilistic estimates using distance probability distributions.
result Probabilistic transformation improves interpretability without impacting detection performance.
Generates synthetic data for benchmarking unsupervised outlier detection.
problem Difficulty in benchmarking unsupervised outlier detection due to rare and varied outliers in real data.
method Proposes a generic process to generate synthetic data with insightful characteristics.
result Demonstrates practicality of the generic process through a benchmark with state-of-the-art detection methods.
A novel unsupervised outlier detection method using Randomized PCA Forest.
problem Unsupervised outlier detection in datasets.
method Randomized Principal Component Analysis (RPCA) Forest for deriving an outlier score.
result Superior performance compared to classical and state-of-the-art methods.
HLoOP detects outliers in hyperbolic 2-space.
problem Detecting local outliers in hyperbolic 2-space.
method Combines nearest neighbor finding and probabilistic scoring in hyperbolic space.
result Promising results on WordNet dataset.
Robust PCA, the problem of PCA in the presence of outliers has been extensively investigated in the last few years. Here we focus on Robust PCA in the outlier model where each column of the data matrix is either an inlier or an outlier. Most of the existing methods for this model assumes either the knowledge of the dim…
New outlier detection method using graph Laplacian spectrum boosts performance.
problem Detecting outliers in large datasets efficiently.
method Boosted outlier detection based on graph Laplacian spectrum.
result Outperforms existing methods on synthetic datasets.
We evaluate how modern outlier detection methods perform in identifying outliers in e-commerce conversion rate data. Based on the limitations identified, we then present a novel method to detect outliers in e-commerce conversion rate. This unsupervised method is made more business relevant by letting it automatically a…
A new method centers outliers in robust PCA without manual intervention.
problem Outliers in robust PCA require manual centering, complicating the analysis.
method Introduces a 'bias trick' to automatically center non-outliers.
result First optimal RPCA algorithm with automatic centering.
Enhances clustering for functional data, robust to outliers.
problem Challenges of clustering infinite-dimensional functional data and outlier sensitivity.
method Extends OCLUST algorithm to handle functional data, trimming outliers.
result Strong performance in clustering and outlier identification on simulated and real-world datasets.
A new method detects outliers using ensembles of Dirichlet process mixtures.
problem Challenges in unsupervised outlier detection using Dirichlet process mixtures.
method Ensembles of Dirichlet process Gaussian mixtures with random subspace and subsampling.
result Empirically outperforms existing approaches in unsupervised outlier detection.
Rare data in a large-scale database are called outliers that reveal significant information in the real world. The subspace-based outlier detection is regarded as a feasible approach in very high dimensional space. However, the outliers found in subspaces are only part of the true outliers in high dimensional space, in…
Robust PCA, the problem of PCA in the presence of outliers has been extensively investigated in the last few years. Here we focus on Robust PCA in the column sparse outlier model. The existing methods for column sparse outlier model assumes either the knowledge of the dimension of the lower dimensional subspace or the …
ODIM detects outliers by under-fitting generative models, outperforming other methods.
problem Identifying outliers in unlabeled data without supervision.
method Develops ODIM, a method that uses under-fitted deep generative models to detect outliers.
result ODIM outperforms other methods in detecting outliers efficiently and accurately.
XGBOD combines unsupervised and supervised methods for better outlier detection.
problem Enhanced outlier detection from normal observations in various datasets.
method Hybrid approach using unsupervised representation learning to improve a supervised classifier.
result XGBOD outperforms competing methods across seven datasets.
A novel density-based approach QC detects outliers in data with high precision.
problem Detecting outliers in data with high precision and sensitivity.
method Quantum Clustering (QC) approach based on the density of data points.
result QC effectively finds hidden outliers and subtle outliers with parameter adjustment.
Inference in the presence of outliers is an important field of research as outliers are ubiquitous and may arise across a variety of problems and domains. Bayesian optimization is method that heavily relies on probabilistic inference. This allows outstanding sample efficiency because the probabilistic machinery provide…
Identifies root causes of outliers using causal DAGs.
problem Detecting and understanding the causes of anomalies in interrelated variables.
method Formal approach using causal directed acyclic graphs (DAGs), outlier scores, and Shapley values.
result Quantifies the extent of outlier scores attributed to ancestors.
Paper tackles outlier detection in multi-armed bandits, achieving high accuracy with reduced exploration costs.
problem Detecting outlier arms in multi-armed bandit settings.
method Proposes GOLD algorithm based on upper confidence bounds to identify generic outlier arms.
result Achieves 98% accuracy with 83% reduction in exploration cost compared to state-of-the-art techniques.
We focus on the problem of unsupervised cell outlier detection and repair in mixed-type tabular data. Traditional methods are concerned only with detecting which rows in the dataset are outliers. However, identifying which cells are corrupted in a specific row is an important problem in practice, and the very first ste…
RelatIF selects more intuitive training examples for explaining model predictions.
problem Influence functions identify outliers as explanatory examples, leading to poor explanations.
method RelatIF separates global and local influence, optimizing for local relative to global effects.
result Examples selected by RelatIF are more intuitive than those from influence functions.
PyODDS automates outlier detection for new data sources.
problem Manual outlier detection is inefficient and domain-specific.
method Automated end-to-end system with Apache Spark and database support.
result PyODDS optimizes outlier detection pipelines automatically.