Make a Confusion Matrix with Python

Metric Overview

Dataset

Let’s assume we have a dataset and we apply both Naive Bayes and Logistic Regression.

We get 2 Confusion Matrix:

We can see that Logistic Regression is better at predicting people without Heart Disease and Naive Bayes is better at predicting people with it. Now we can pick the model based on whether we want to identify someone with or without Heart Disease. BAM!

When there are only 2 possible outcomes, like Yes and No, then the corresponding Confusion Matrix has 2 rows and 2 columns: one for each Yes and No. In general, the size of the matrix corresponds to the number of classifications we want to predict. For 3 possible outcomes it would look like this:

Unfortunately there is no standard for the orientation:

In the first example both matrices contained the same number of False Negative (each 22) and False Positives (29 each). All we need to to was to compare the True Positives (142 vs 137). to quantify how much better Naive Bayes was at predicting people with Heart Disease. However, what if we had ended up with this:

• We can still see that Naive Bayes it better at predicting people with Heart Disease, but quantifying is a bit more complicated, because now we have to compare True Positives (142 vs. 139) and False Negatives (29 vs. 32)
• We still see that Logistic Regression is better in predicting people without Heart Disease, but quantifying how much better has to take True Negatives (110 vs 112) and False Positives (22 vs. 20) into account.

→ We can use Sensitivity and Specificity. (BAM)

Sensitivity and Specificity