SHAP Values

Perhaps the best way to explain our model is via SHAP values. These values provide both local and global feature importance scores. Thus far, we have mostly discussed global importance, that is, how a feature impacts a model on the whole. Local importance explains how a feature influences a specific prediction. SHAP values do something quite powerful: for each prediction, we can see how each feature moved the prediction away from a baseline value. I frankly think SHAP values are the best way to explain models. Therefore, I do not present some of its "competitors", such as LIME. For a thorough explanation of SHAP values, please refer to https://christophm.github.io/interpretable-ml-book/shap.html. Here, we will demonstrate how SHAP values operate via a working example.

One of the "gotchas" about SHAP is that it expects a model, not a pipeline, and preprocessed data. However, we have all our preprocessing in our pipeline. Therefore, we need to perform two tasks.

1. Extract our model from our pipeline. In explain.py, we do this on line 463.
2. We also need to preprocess our x_test dataframe using our pipeline. This part is not tricky. What can be a bit tricky is mapping the appropriate column names after the transformation. We perform this operation with the function transform_data_with_pipeline, which we call on line 464. This returns a dataframe we can pass into our SHAP explainer along with our model step from the pipeline.

Our SHAP code produces the following files.

• shap_values.csv: dataframe of SHAP values for each observation passed into the explainer
• shap_expected.csv: the expected value of our model (e.g. the positive class rate)
• shap_global.csv: global SHAP values for each feature
• shap_values_bar.png: bar plot of the global SHAP values
• shap_values_dot.png: dot plot of the global SHAP values

The dot plot is shown below. The SHAP impact is shown on the x-axis, which is the influence of moving a prediction away from some base value (e.g. the overall positive class rate in this case). Each dot represents an observation in the test set. The colors represent relative high vs. low values for that feature.

The shap library provides some additional nifty visualization options. We shall show three kinds of them, which are produced with the below script.

The waterfall plot is pretty cool as it shows how the model reached an individual prediction. For this observation, our predicted probability is 0.137, above the expected value of 0.086. We learn that a few features, in conjunction, drive the probability above the baseline.

The decision plot is simply an alternative to the dot plot displayed earlier. Each line represents an observation from our x dataframe, which in this case is our test set. As we move from bottom to top, the SHAP values are added, which shows each features' impact on the final prediction. At the top of the plot, we end at each observations' predicted value.

The interaction plot shows SHAP values for two features with notable interplay. We can say that, when the marketing_campaign is level 9 and the all_star_group is level 1, we get a greater movement away from the expected value than if only the marketing campaign is level 9.

Lastly, a couple of important nuances with SHAP values exists. First, our SHAP expected value + sum of SHAP values should equal our predicted probability. When we're explaining a CalibratedClassifierCV, using the TreeExplainer requires us to feed in each base estimator from our ensemble, produce its SHAP values, and then average all the SHAP values. This is all well and good. However, remember that a CalibratedClassifierCV includes a calibrator model in addition to the base models. Therefore, the average prediction of our base models is not necessarily our final prediction. Related, the average of our SHAP values plus the expected value may not equal our predicted probability. If we want this calculation to back out, we need to instead use the KernelExplainer. From the SHAP documentation: "Kernel SHAP is a method that uses a special weighted linear regression to compute the importance of each feature. The computed importance values are Shapley values from game theory and also coefficents from a local linear regression." The benefit of using the TreeExplainer version is that algorithm is optimized for tree-based models and it's, therefore, comparatively fast. That downside is that the expected SHAP calculation may not exactly check out. However, the SHAP values will still be very useful and informative. The KernelExplainer is more generic and slower, but the SHAP calculation will add up. We have to assess the tradeoff and determine what is best for our circumstance.

In the code, you may see check_additivity=False. This is a check in the SHAP library code that our SHAP expected value + sum of SHAP values should equal our predicted probability for an individual model (e.g. standalone XGBoost, base estimator of a CalibratedClassifierCV). Some occasional weird behavior appears to exist with Random Forest and Extra Trees models. This appears to be a known issue, evidenced by several open GitHub issues. From my experience, the error is non-deterministic: given the same exact model and the same exact data, I sometimes get the error and I sometimes don't. My guess, though I don't know for sure, is the feature perturbation is non-deterministic, and this error will occur in trees with certain characteristics. When the error occurs, SHAP claims the model output is something crazy high. This seems to be something the shap library maintainers need to fix, so I just turn off the check to prevent non-deterministic errors.

One last note on SHAP values: many gradient boosting models don't output a probability. The output is a more generic numeric value, but the interpretation will be the same relatively.