Misclassifications Part 2 - Statistical Model

Welcome back to post two of the misclassification series! I am glad you made it here! In the last post, we presented how misclassifications cause statistics such as class totals to become biased. In this post, we will model the problem mathematically and present an algorithm that will yield unbiased estimates. As a heads up, this post is quite mathematically heavy and I can’t describe every step for the sake of conciseness. I will, however, try to put some links for background reading when relevant. Other than that, an intuitive understanding of the problem from the last post will help understand the modeling choices!

Troubling Misclassifications - What is it about?

Welcome to this two-part series on misclassifications of categorical variables! Sounds exciting, does it not? Well, with humans putting labels on everything from food and spices to entire economies and cultures, we find classes everywhere. Classifications can help us make sense of the world but also carry plenty of examples where they lead to oversimplification or are used for discriminatory purposes. Taking the position, however, that we are doing something good with classifications, it is important that the classifications we have collected are the correct ones! While it is obviously desirable to have error-free data, we will in this post examine what the consequences might be if we are not so fortunate. This is followed by a second post where an alternative statistical method is presented that sometimes can overcome the problems induced by the misclassifications!