Here’s a little reminder for those of you checking assumptions in regression and ANOVA:
The assumptions of normality and homogeneity of variance for linear models are not about Y, the dependent variable. (If you think I’m either stupid, crazy, or just plain nit-picking, read on. This distinction really is important). (more…)
A polynomial term–a quadratic (squared) or cubic (cubed) term turns a linear regression model into a curve. But because it is X that is squared or cubed, not the Beta coefficient, it still qualifies as a linear model. This makes it a nice, straightforward way to model curves without having to model complicated non-linear models.
But how do you know if you need one–when a linear model isn’t the best model? (more…)
If your graduate statistical training was anything like mine, you learned ANOVA in one class and Linear Regression in another. My professors would often say things like “ANOVA is just a special case of Regression,” but give vague answers when pressed.
It was not until I started consulting that I realized how closely related ANOVA and regression are. They’re not only related, they’re the same thing. Not a quarter and a nickel–different sides of the same coin.
So here is a very simple example that shows why. When someone showed me this, a light bulb went on, even though I already knew both ANOVA and multiple linear (more…)
A Linear Regression Model with an interaction between two predictors (X1 and X2) has the form:
Y = B0 + B1X1 + B2X2 + B3X1*X2.
It doesn’t really matter if X1 and X2 are categorical or continuous, but let’s assume they are continuous for simplicity.
One important concept is that B1 and B2 are not main effects, the way they would be if (more…)
Back when I was doing psychology research, I knew ANOVA pretty well. I’d taken a number of courses on it and could run it backward and forward. I kept hearing about ANCOVA, but in every ANOVA class that was the last topic on the syllabus, and we always ran out of time.
The other thing that drove me crazy was those stats professors kept saying “ANOVA is just a special case of Regression.” I could not for the life of me figure out why or how.
It was only when I switched over to statistics that I finally took a regression class and figured out what ANOVA was all about. And only when I started consulting, and seeing hundreds of different ANOVA and regression models, that I finally made the connection.
But if you don’t have the driving curiosity about ANOVA and regression, why should you, as a researcher in Psychology, Education, or Agriculture, who is trained in ANOVA, want to learn regression? There are 3 main reasons.
1. There a many, many continuous independent variables and covariates that need to be included in models. Without the tools to analyze them as continuous, you are left forcing them into ANOVA using an arbitrary technique like median splits. At best, you’re losing power. At worst, you’re not publishing your article because you’re missing real effects.
2. Having a solid understanding of the General Linear Model in its various forms equips you to really understand your variables and their relationships. It allows you to try a model different ways–not for data fishing, but for discovering the true nature of the relationships. Having the capacity to add an interaction term or a squared term allows you to listen to your data and makes you a better researcher.
3. The multiple linear regression model is the basis for many other statistical techniques–logistic regression, multilevel and mixed models, Poisson regression, Survival Analysis, and so on. Each of these is a step (or small leap) beyond multiple regression. If you’re still struggling with what it means to center variables or interpret interactions, learning one of these other techniques becomes arduous, if not painful.
Having guided thousands of researchers through their statistical analysis over the past 10 years, I am convinced that having a strong, intuitive understanding of the general linear model in its variety of forms is the key to being an effective and confident statistical analyst. You are then free to learn and explore other methodologies as needed.
There are two reasons to center predictor variables in any type of regression analysis–linear, logistic, multilevel, etc.
1. To lessen the correlation between a multiplicative term (interaction or polynomial term) and its component variables (the ones that were multiplied).
2. To make interpretation of parameter estimates easier.
I was recently asked when is centering NOT a good idea? (more…)