Ekaterina_Minaeva / Shutterstock.com
Some tests are easy to analyze. Is Superman stronger than Charlie Brown? Maybe we know the answer going into it and just need to prove we’re right, or maybe the data is heavily skewed in one direction.
But some tests are harder to analyze. Who is stronger — Superman or General Zod?
When the data don’t show a clear overwhelming winner, researchers and analysts leverage statistical significance calculators to determine if their findings are valid. But when we use the words “statistically significant” what do we really mean?
The academic definition of statistical significance focuses on the reliability of a statistic. The most common representation of reliability comes in the form of a metric called a “confidence interval”. A 95% confidence interval means that if we repeated a test, the observed result would hold true 95% of the time.
Depending on the application and purpose of the analysis, analysts can be comfortable with a wide range of confidence intervals. But in business, when was the last time you were 95% confident that something was true?
What’s Your Risk Tolerance In Marketing?
Let’s put this in perspective and test your tolerance for risk:
If you’re 95% confident that redesigning your email template will yield more secondary purchases, would you make the change?
If you’re 75% confident that using a certain ad will yield an improvement in associated transactions, would you use that ad instead?
If you’re 60% confident that making a change to your homepage will generate more leads, would you make the change?
Chances are you said yes to at least the first two questions and most folks probably said yes to all three. Does that mean you’re a maverick and don’t care about making responsible decisions?
Quite the contrary.
The Differing Worlds of Business & Academia
A few years ago, I got into a conversation with an old professor about business statistics. I asked him “what’s my target confidence interval for making a good business decision based on statistics?” His response has stuck with me for years as a reminder that business isn’t academia because there’s money to be made on each decision.
He said, “In academics we strive for high confidence intervals because our success hinges on proving a point. In business, if you tell me I’ll have a 51% likelihood of achieving more success with option A over option B then I’m likely to prefer option A.”
While my professor’s response was an exaggerated example, his point was that understanding probabilities and statistics around the likelihood of success enables us to make calculated risks.
If you have a 75% chance to make a million bucks and 25% chance to lose $1,000 then odds are you’re up for the bet. But if you have a 55% chance of making a million and a 45% chance of losing a million then chances are you’ll pass on the opportunity.
My point is that while confidence intervals and statistics are highly valuable tools to improve business decision making processes, your minimum acceptable confidence interval — and the associated tolerance for risk — should reflect your position.
If the proposed opportunity is high risk and low reward then you should be more calculated. But if it’s low risk and high reward then maybe it’s worth taking a chance.
Comments