Calmcode - statistical disservice: general

Analytics is Troublesome

1 2 3 4

There is a general reason why analytics is troublesome and why statistics sometimes provides a disservice; they ignore the translation error.

analyticalproblemactual problemdata10% boostsolved?translateoptimiseusetranslation error?

There's always a translation step from an actual problem to an analytical one. Once the analytical variant demonstrates a boost, we cannot assume that it will adress the original problem. Understanding this upfront typically can help prevent a lot of bad things.