Partial dependence plot (PDP)

A partial dependence plot (PDP) visualizes the marginal effect of a feature on the outcome of a machine learning model. This effect is computed by “marginalizing out” the contribution of all other features, so that the result is a function that only depends on the feature of interest. Contributions by other features are factored into the marginal effect. The PDP is then produced by plotting the predicted outcome (y-axis) against the values of the feature of interest (x-axis). You can compute the marginal effect of more features, but when using more than two features you will not be able to visualize the result.

The computed effect is only accurate if the feature of interest does not correlate with the other features. ALE alleviates this issue because ALE plots work with a conditional a distribution instead of a marginal distribution.

A PDP is the average of the lines in an ICE plot .

Examples such as the following can be found in Christoph Molnar’s chapter on PDP:

PDP example

H2O MLI Resources

This repository by H2O.ai contains useful resources and notebooks that showcase well-known machine learning interpretability techniques. The examples use the h2o Python package with their own estimators (e.g. their own fork of XGBoost), but all code is open-source and the examples are still illustrative of the interpretability techniques. These case studies that also deal with practical coding issues and preprocessing steps, e.g. that LIME can be unstable when there are strong correlations between input variables. Read more...

InterpretML

The InterpretML toolkit, developed at Microsoft, can be decomposed in two major components: A set of interpretable “glassbox” models Techniques for explaining black box systems. W.r.t. 1, InterpretML particularly contains a new interpretable “glassbox” model that combines Generalized Additive Models (GAMs) with machine learning techniques such as gradient boosted trees, called an Explainable Boosting Machine. Other than this new interpretable model, the main utility of InterpretML is to unify existing explainability techniques under a single API. Read more...

SHAP: SHapley Additive exPlanations

The SHAP package is built on the concept of a Shapley value and can generate explanations model-agnostically. So it only requires input and output values, not model internals: SHAP (SHapley Additive exPlanations) is a game theoretic approach to explain the output of any machine learning model. It connects optimal credit allocation with local explanations using the classic Shapley values from game theory and their related extensions. (README) Additionally, this package also contains several model-specific implementations of Shapley values that are optimized for a particular machine learning model and sometimes even for a particular library. Read more...

What-If Tool

The What-If Tool (WIT) takes a pretrained model and then allows you to visualize the effect of changing e.g. classification thresholds or the data points themselves on performance, explainability and fairness metrics. Many convenient functions for gaining insight in the data set are provided, such as binning on particular features, attribution values, or inference scores, computing partial dependence plots, and typical performance indicators such as a confusion matrix or ROC curve. Read more...