Post-hoc tests, pairwise or other linear contrasts, are typical in an analysis of variance (ANOVA) setting to understand which group means differ. They incorporate p-value adjustments to avoid concluding that group means differ when they actually do not. There are several adjustments that can be considered for conducting multiple post-hoc tests, including single-step and stepwise adjustments. (more…)
adjustments
Member Training: ANOVA Post-hoc Tests: Practical Considerations
October 1st, 2021 by TAF SupportMember Training: Adjustments for Multiple Testing: When and How to Handle Multiplicity
May 3rd, 2018 by guest contributer A research study rarely involves just one single statistical test. And multiple testing can result in more statistically significant findings just by chance.
After all, with the typical Type I error rate of 5% used in most tests, we are allowing ourselves to “get lucky” 1 in 20 times for each test. When you figure out the probability of Type I error across all the tests, that probability skyrockets.
(more…)
Member Training: A Gentle Introduction to Propensity Score Adjustments and Analysis
December 1st, 2015 by guest contributerSo you can’t randomize people into THAT condition? Now what?
Let’s say you’re investigating the impact of smoking on social outcomes like depression, poverty, or quality of life. Your IRB, with good reason, won’t allow random assignment of smoking status to your participants.
But how can you begin to overcome the self selected nature of smoking among the study participants? What if self-selection is driving differences in outcomes? Well, one way is to use propensity score matching and analysis as a framework for your investigation.
The propensity score is the probability of group assignment conditional on observed baseline characteristics. In this way, the propensity score is a balancing score: conditional on the propensity score, the distribution of observed baseline covariates will be similar between treated and untreated subjects.
In this webinar, we’ll describe broadly what this method is and discuss different matching methods that can be used to create balanced samples of “treated” and “non-treated” participants. Finally, we’ll discuss some specific software resources that can be found to perform these analyses.
Note: This training is an exclusive benefit to members of the Statistically Speaking Membership Program and part of the Stat’s Amore Trainings Series. Each Stat’s Amore Training is approximately 90 minutes long.