Back in January 2020, we published an article discussing Margin for Error. We realized, though, that we could definitely do a better job explaining it. Buckle up everyone, we're getting nerdy.
To start us off, we have to get down in the trenches of statistics. We promise to make this as painless as possible:
- Just about every dataset will have a margin for error. In fact, it's impossible to not have a margin for error unless you survey every member of a certain population of people, which, frankly, would be very expensive and time consuming in most cases.
- Margin for Error is defined as a statistic describing the amount of random sampling error in a survey. This definition works just fine if you have a PhD in statistics, but not so much for us. What it comes down to is, there is just no way to capture the opinions of a target audience with 100% accuracy if you haven't collected data from 100% of the population.
- Confidence Level is the less fun counterpart to margin for error. It's defined as the probability with which an estimated interval will contain the true value of the parameter. Yikes. What this means for our purposes though, is confidence level is the level of faith you have that the survey results would be exactly the same if you conducted the survey again.
- For example, a 0% confidence level means you have no faith the results would be the same, while a 100% confidence level means you're certain the results would be the same.
To illustrate this concept, we typically compare it to drawing blood, but let's use something more fun... wine.
Imagine you tested a barrel of wine for its acidity level, also known as its pH level. For this particular wine, it's pH level is 3.5 with +/-5% margin for error at a 95% confidence level. Well that sounds just fine but what does this actually mean?
- Imagine you now are about to test 100 barrels of that exact same wine, performing the exact same pH test.
- Based on the first barrel, you should reasonably expect that 95 of those barrels will have a pH level that is within 10% (remember, it's + OR - 5%) of the pH level of the first barrel (3.5).
And that's all there is to it. Now, we've definitely ignored some of the subtle nuances of winemaking in this example, but that's for more experienced sommeliers to explain.