Data Science vs. Decision Science

I believe differentiation exists between data science and decision science. In contrast to data science, decision science is an analytical process aimed to aiding human understanding and interpretation. The first three steps of the "pipeline" are the same; we always need to ingest, explore, and wrangle our data. Divergence occurs at the modeling stage. Data science tends to be mostly concerned with predictive power and less focused on easy-to-understand results. In decision science, human interpretation is the goal.

A quick aside: I prefer the term "predictive power" over something like "accuracy". As we will see in a later chapter, accuracy is a misleading metric. If 95% of customers never churn, we can build a model that is 95% accurate by simply predicting "no churn" in all cases. Likewise, two models that are equally accurate may not be equally useful. For instance, if one model produces better probabilistic estimates within the defined class boundaries, it will be more useful even if it's not more "accurate" by classic terms.

OK, now back to decision science vs. data science.

A perfect example of the difference between the two topics is the use of decision trees in both domains. The decision tree is an algorithm that makes a series of conditional splits to provide a prediction. The graph below is the output from a scikit-learn decision tree. Since this is what we get natively from scikit-learn (the main machine learning library in Python), I will take the time to explain the output. What we can get out-of-the-box is admittedly a little confusing. To note, the data used is faked and a nod to my love of baseball.

In most boxes, we have five rows. First is the decision rule. Second is the gini impurity value. Third is the number of samples. Fourth is the class balance. Lastly, we have the predicted class. Our first decision is whether or not location_IL is <= 0.5. You might ask: how can the location be 0.5? scikit-learn models do not accept string values; everything must be coded as integers or floats. One way to represent categorical features (e.g. states) as integers is through dummy coding. That is, if the category is present, the value is 1. If it's not, the value is 0. Therefore, if location_IL is <= 0.5, the dummy coded value is 0 (i.e. not IL). If the value is not <= 0.5, then the dummy coded value is 1 (i.e. IL). Based on the decision split, when the location is IL, then none of the observations are Royals fans. If we are not in IL, we proceed to a split based on age. If the observation is less than 32.5 years old, we move to the left on the diagram. Here, we reach what's called a leaf node; there are no more splits. In this leaf node, all observations are Royals fans. If the observation is older than 32.5, we split to the right on our tree. We reach another split on age, a binary partition on age 42.5. If the observation is older than 42.5, they are a Royals fan. If they are under 42.5, but older than 32.5, they are not a Royals fan. In sum, we can conclude that no Royals fans are in IL. If they are not in IL, they are Royals fan if they are younger than 32.5 or older than 42.5.

A single decision tree, as shown above, explains key differences between classes based on data, meaning it is a great tool for decision scientists. A human can look at the above graph and understand what is happening. By contrast, a data scientist would (hopefully) never use a single decision tree as a predictive model. Standalone decision trees are known to overfit; that is, they capture both signal and noise and do not generalize well to new observations. Rather, data scientists use ensembles of trees that sample both columns and rows to develop a predictive model (i.e. either random forest or extra trees, depending on the exact implementation). Essentially, data scientists trade the ability to show a graphic of a single tree for improved predictive capabilities. That said, robust ways exist for explaining the decisions of ensembles of trees. The tradeoff between interpretability and predictive power is (thankfully) shrinking.

For those interested, the above decision tree was generated with the following code:

A related difference is how models are used. In decision science, we will typically use a model to help explain the past. In data science, we will generally leverage a model to predict the future. You might think, "if we can explain the past, can't we predict the future?" In general, yes, but importance nuance exists. In decision science, we don't necessarily have to contend with the data we will have at prediction time. For example, let's say we are trying to build a model to predict if a customer will buy a gadget after responding to a piece of mail. A potential feature is the sales agent with whom the customer spoke. In the world of decision science, this might be a completely valid feature. We might want to control for the agent, or we might be specifically interested in their influence. However, in the realm of data science, this feature is problematic. We would ideally want to use our model to create an audience of prospects most likely to buy from a piece of mail. At the time we are making the prediction to construct the audience, we have no idea to what agent a customer might talk. This feature becomes unusable in a production setting and should not be included in the model. In data science, the data we use is bound to what we have available "in the wild" when the model is live. Our reality is tethered to whatever data we have at the time of making predictions. We cannot "leak" information from the future. Our goal is to predict effectively, which means we might have to sacrifice features that would be good only for explanation. This seems obvious, but the application can be tricky in practice. For instance, let's say we wanted to include each customers' lifetime value (LTV) as a feature. Good idea. However, we can't just slap on their LTV as it stands today. We have to recreate it historically so as not to leak information from the future. If we sent the mailer on 1/1/2020, then the LTV used in the model must be what it was as of that date. Again, our reality is our data at the time of prediction. Using the current LTV would not be great for a decision science project, but it could potentially be fatal for a data science project.

Additionally, in decision science, the model deployment step is often replaced with “visualization” or something like “report presentation”. Likewise, decision science may not include model deployment component. That said, deploying a dashboard or model UI would certainly fall under this category, so notable overlap can exist with data science on this element.

As the title indicates, this book will focus on data science rather than decision science. That's not to say decision science is not important - the opposite is true. Proper decision science can be extremely powerful. In fact, it can be so impactful that a data scientist is often wise to spend some of their time on decision science.