Paper exposes vulnerabilities in interpreting machine learning models using adversarial attacks on PD plots.
problem Vulnerability of permutation-based interpretation methods, particularly PD plots, to adversarial attacks.
method Adversarial framework to manipulate black-box models and produce deceptive PD plots.
result It is possible to hide discriminatory behaviors in machine learning models through interpretation tools like PD plots.
SHAP explains boosted trees with additively modeled features.
problem Explaining predictions of boosted trees models with additively modeled features.
method SHAP values for additively modeled features in boosted trees models.
result SHAP dependence plot matches partial dependence plot for additively modeled features.
In practical applications of machine learning, it is necessary to look beyond standard metrics such as test accuracy in order to validate various qualitative properties of a model. Partial dependence plots (PDP), including instance-specific PDPs (i.e., ICE plots), have been widely used as a visual tool to understand or…
A new method detects interactions in machine learning models.
problem Interpreting non-linear and interaction effects in machine learning models.
method Regional effect plots with implicit interaction detection.
result The method quantifies and interprets feature effects reliably, less confounded by interactions.
Researchers formalize PD and PFI to relate them to data generating process.
problem Lack of theory linking PD and PFI to data generating process.
method Formalize PD and PFI as estimators of ground truth estimands, account for model variance with learner-PD and learner-PFI.
result PD and PFI estimates deviate from ground truth due to statistical biases, model variance, and Monte Carlo approximation errors.
We use PDPs with confidence bands to explain HPO results.
problem Lack of explainable insights into HPO results.
method Use IML techniques, specifically PDPs, with estimated confidence bands.
result Increased quality of PDPs in relevant sub-regions.
Researchers show how to manipulate Partial Dependence plots to deceive explanations of predictive models.
problem The robustness and trustworthiness of Partial Dependence (PD) explanations are compromised.
method Data poisoning using genetic and gradient algorithms to manipulate PD plots.
result PD explanations can be fooled and manipulated to mislead understanding of predictive models.
Interpreting a nonparametric regression model with many predictors is known to be a challenging problem. There has been renewed interest in this topic due to the extensive use of machine learning algorithms and the difficulty in understanding and explaining their input-output relationships. This paper develops a unifie…
Study robustness of global feature effect explanations in machine learning models.
problem Vulnerability of global feature effect explanations to data and model perturbations.
method Theoretical bounds and experimental evaluation of partial dependence plots and accumulated local effects.
result Quantifies the gap between best and worst-case scenarios of misinterpreting machine learning predictions globally.
Recent years have witnessed an increased focus on interpretability and the use of machine learning to inform policy analysis and decision making. This paper applies machine learning to examine travel behavior and, in particular, on modeling changes in travel modes when individuals are presented with a novel (on-demand)…
This text discusses several popular explanatory methods that go beyond the error measurements and plots traditionally used to assess machine learning models. Some of the explanatory methods are accepted tools of the trade while others are rigorously derived and backed by long-standing theory. The methods, decision tree…
Proposes a new method for interpreting feature importance and effects in dependent feature models.
problem Challenges in interpreting feature importance when features are dependent and interactions are present.
method Conditional Subgroup Approach
result Conditional PFI and PDP estimates based on this approach often outperform existing methods.
One of the most popular approaches to understanding feature effects of modern black box machine learning models are partial dependence plots (PDP). These plots are easy to understand but only able to visualize low order dependencies. The paper is about the question 'How much can we see?': A framework is developed to qu…
This paper reviews and advocates against the use of permute-and-predict (PaP) methods for interpreting black box functions. Methods such as the variable importance measures proposed for random forests, partial dependence plots, and individual conditional expectation plots remain popular because they are both model-agno…
New approach interprets machine learning models through feature space transformations.
problem Interpreting models with strongly dependent features in high-dimensional spaces.
method Feature space transformations, including PCA and partial orthogonalization.
result Enhances model interpretation tools for domain experts.
Model-agnostic interpretation methods can mislead if not used carefully.
problem Misinterpretation of machine learning models due to improper use of techniques.
method General pitfalls of model-agnostic interpretation methods.
result Many pitfalls exist when using global interpretation techniques for machine learning models.
AI techniques explain synthetic tabular data weaknesses.
problem Challenges in evaluating synthetic tabular data quality.
method Apply explainable AI to a binary detection classifier.
result Reveals inconsistencies, unrealistic dependencies, or missing patterns in synthetic data.
A new tree-based estimator, FastPD, efficiently estimates PD functions for machine learning models.
problem Efficiently estimating Partial Dependence functions for machine learning models.
method Proposes a new tree-based estimator, FastPD, to estimate PD functions.
result FastPD consistently estimates the desired population quantity and improves complexity from quadratic to linear.
Unified local and global explanations through functional decomposition of low dimensional structures.
problem Tackles the challenge of extracting meaningful local and global explanations from machine learning models.
method Proposes a new identification constraint to decompose the global representation into main and interaction components of arbitrary order.
result Unified local and global explanations by connecting partial dependence plots and interventional SHAP values.
We propose a novel methodology, forest floor, to visualize and interpret random forest (RF) models. RF is a popular and useful tool for non-linear multi-variate classification and regression, which yields a good trade-off between robustness (low variance) and adaptiveness (low bias). Direct interpretation of a RF model…
In recent years, a large amount of model-agnostic methods to improve the transparency, trustability and interpretability of machine learning models have been developed. We introduce local feature importance as a local version of a recent model-agnostic global feature importance method. Based on local feature importance…
New method improves HPO interpretability without sacrificing performance.
problem Difficulty in understanding HPO algorithms due to black-box nature.
method Coupling Bayesian optimization with Bayesian Algorithm Execution.
result More reliable IML explanations without compromising optimization performance.
GADGET framework decomposes global feature effects using recursive partitioning.
problem Misleading global feature effects when feature interactions are present.
method Generalized additive decomposition of global effects (GADGET) based on recursive partitioning.
result Minimizes interaction-related heterogeneity of local feature effects.
CDPs visualize causal dependencies in AI models.
problem Understanding how AI models depend on data inputs causally.
method Developed Causal Dependence Plots (CDPs) to visualize causal dependencies.
result CDPs show causal changes in predictors and outcomes.
Proposes ICE-based metric for better understanding interactions in black-box models.
problem Misleading global sensitivity metrics in black-box models due to interaction effects.
method Individual Conditional Expectation (ICE) curves to compute feature importance and interactions.
result ICE-based metric provides richer insights into feature importance and interactions.
A neural network approach for efficient conditional SHAP calculations.
problem Efficiently calculating conditional SHAP values for various models.
method Surrogate neural network approach for conditional SHAP.
result Efficiently calculates conditional SHAP values for neural networks and other regression models.
Calibration error is commonly adopted for evaluating the quality of uncertainty estimators in deep neural networks. In this paper, we argue that such a metric is highly beneficial for training predictive models, even when we do not explicitly measure the uncertainties. This is conceptually similar to heteroscedastic ne…
Study compares imputation methods' effects on IML confidence intervals.
problem Missing data impacts IML interpretation and confidence intervals.
method Compared single vs multiple imputation methods on IML confidence intervals.
result Multiple imputation provides closer coverage to nominal than single imputation.
Post-hoc model-agnostic interpretation methods such as partial dependence plots can be employed to interpret complex machine learning models. While these interpretation methods can be applied regardless of model complexity, they can produce misleading and verbose results if the model is too complex, especially w.r.t. f…
Hollow-tree Super resolves feature importance in large datasets.
problem Lack of effective scaling for large feature numbers in boosted tree models.
method Hollow-tree Super (HOTS) methodology for feature importance visualization.
result HOTS effectively resolves feature importance and directionality in high-dimensional neuroscientific data.
We propose a procedure for supervised classification that is based on potential functions. The potential of a class is defined as a kernel density estimate multiplied by the class's prior probability. The method transforms the data to a potential-potential (pot-pot) plot, where each data point is mapped to a vector of …
The paper extends explainability methods to uncertainty-aware models, revealing feature impacts on predictive entropy and likelihood.
problem Understanding the factors contributing to uncertainty in probabilistic models.
method Adapting permutation feature importance, partial dependence plots, and individual conditional expectation plots to measure feature impacts on predictive entropy and likelihood.
result Novel insights into model behaviour and feature impacts on uncertainty are obtained.
New tools explain FRF model predictions in high-dimensional ECG data.
problem Lack of interpretability in Functional Random Forests (FRF) models.
method Introduces FPDPs, FPC Probability Heatmaps, and various FPC importance metrics.
result Enhances transparency of FRF models by revealing FPC contributions.
Automate residual plot assessment with R package and Shiny application
problem Diagnosing linear models
method Computer vision model for residual plot assessment
result Predicts visual signal strength and supports model fit assessment
Modeling intraday electricity prices with a Hawkes process.
problem Capturing the dynamics of intraday electricity prices, especially microstructure noise.
method 2D marked Hawkes process with increasing baseline intensity, providing analytic moments and signature plot.
result The model fits German intraday electricity data well and converges to a Brownian motion with increasing volatility.
Study examines how balancing methods affect model behavior in imbalanced classification problems.
problem Impact of balancing methods on model behavior in imbalanced classification problems.
method Used Explainable Artificial Intelligence tools (variable importance method, partial dependence profile, accumulated local effects) to compare model behavior before and after balancing.
result Significant changes in model behavior due to balancing methods, leading to biased models.
LLMs simulate financial markets, revealing consistent trading strategies and market dynamics.
problem Testing financial theories with AI trading agents.
method Simulated stock market with LLMs using a persistent order book and varied strategies.
result LLMs can simulate different trading strategies and market dynamics.
This research optimizes Andrews plots for better visual clarity in high-dimensional data.
problem Visualizing high-dimensional datasets with clarity and aesthetics.
method Developed a method to add spectral smoothing to Andrews plots to reduce visual clutter.
result Optimal spatial-spectral smoothing leads to more aesthetically pleasing and clutter-free visualizations.
Analyzes error sources in global feature effect estimation methods.
problem Unexplored error sources in global feature effect estimation methods.
method Systematic, estimator-level analysis of bias and variance.
result Holdout data is theoretically cleanest, but estimation variance depends on sample size and model characteristics.
Many modern statistical applications ask for the estimation of a covariance (or precision) matrix in settings where the number of variables is larger than the number of observations. There exists a broad class of ridge-type estimators that employs regularization to cope with the subsequent singularity of the sample cov…
3D filament plots visualize curves in datasets, avoiding visual clutter.
problem Visualizing high-dimensional data relationships in scatterplots of points.
method Construct 3D filament plots using linear isometries and Frenet-Serret systems.
result 3D filament plots preserve Euclidean distances and avoid visual clutter.
Used to investigate the presence of distinctive recurrent behaviours in natural processes, the recurrence plots can be applied to the analysis of economic data, and, in particular, to the characterization of exchange rates of currencies too. In this paper, we will show that these plots are able to characterize the peri…
One aim of data mining is the identification of interesting structures in data. For better analytical results, the basic properties of an empirical distribution, such as skewness and eventual clipping, i.e. hard limits in value ranges, need to be assessed. Of particular interest is the question of whether the data orig…
Pareto distributions, and power laws in general, have demonstrated to be very useful models to describe very different phenomena, from physics to finance. In recent years, the econophysical literature has proposed a large amount of papers and models justifying the presence of power laws in economic data. Most of the ti…
Introduces PIT-plot for prioritizing projects based on their impact.
problem Optimizing R&D investments in project portfolios.
method Develops a new tool (PIT-plot) focusing on project impact rather than project properties.
result Identifies projects with the largest impact for risk mitigation or value-adding.
Charts are an excellent way to convey patterns and trends in data, but they do not facilitate further modeling of the data or close inspection of individual data points. We present a fully automated system for extracting the numerical values of data points from images of scatter plots. We use deep learning techniques t…
PLOT uses optimal transport to find neural site handles for causal abstraction.
problem Finding the relevant neural site for causal analysis is computationally challenging.
method PLOT employs optimal transport to localize causal variables from neural network outputs.
result PLOT efficiently finds intervention handles for causal abstraction in neural networks.
Plots show miscalibration directly as slopes of secant lines.
problem Detecting discrepancies between probabilistic predictions and actual outcomes.
method Cumulative differences between observed and expected values displayed as slopes of secant lines.
result Directly shows miscalibration without binning or kernel density estimation.