In statistical practice, there are many situations where best practices are clear. There are many, though, where they aren’t. The granddaddy of these practices is adjusting p-values when you make multiple comparisons. There are good reasons to do it and good reasons not to. It depends on the situation.
At the heart of the issue is a concept called Family-wise Error Rate (FWER). FWER is the probability that
After all, with the typical Type I error rate of 5% used in most tests, we are allowing ourselves to “get lucky” 1 in 20 times for each test. When you figure out the probability of Type I error across all the tests, that probability skyrockets.
(more…)
Whenever you run multiple statistical tests on the same set of data, you run into the problem of the Familywise Error Rate. What this means is that the true probability
of a type 1 error somewhere in the family of tests you’re running is actually higher than the alpha=.05 you’re using for any given test.
This is a complicated and controversial issue in statistics — even statisticians argue about whether it’s a problem, when it’s a problem, and what to do about it.
In this webinar, we’ll talk about the meaning and consequences of these issues so you can make informed decisions in your data analysis.
We’ll also go through possible solutions, including post-hoc tests and the false discovery rate.
Note: This training is an exclusive benefit to members of the Statistically Speaking Membership Program and part of the Stat’s Amore Trainings Series. Each Stat’s Amore Training is approximately 90 minutes long.
Karen Grace-Martin helps statistics practitioners gain an intuitive understanding of how statistics is applied to real data in research studies.
She has guided and trained researchers through their statistical analysis for over 15 years as a statistical consultant at Cornell University and through The Analysis Factor. She has master’s degrees in both applied statistics and social psychology and is an expert in SPSS and SAS.
Just head over and sign up for Statistically Speaking. You'll get access to this training webinar, 130+ other stats trainings, a pathway to work through the trainings that you need — plus the expert guidance you need to build statistical skill with live Q&A sessions and an ask-a-mentor forum.Not a Member Yet?
It’s never too early to set yourself up for successful analysis with support and training from expert statisticians.