Which Test When?

When do I perform a t-test?

There are actually three different types of t-tests, depending upon your research question and the groups you are interested in testing. For this test you need interval or ratio data as the outcome (dependent variable) so that you can test for mean differences between groups and a categorical independent variable.

Use an independent samples t-test if you want to know if the mean differences between two normally distributed independent groups are reliably different from each other. In other words, the independent variable (IV) is categorical and has two levels.

For example, if you want to know the difference in weight loss (DV) between those who exercise and those who do not (IV), you would use this test. So, all things being equal (that is, those in both groups being similar in weight, following the same diet, etc.), will adding exercise to diet make a significant difference in weight loss?

IV ? two independent groups
DV ? one interval/ratio outcome

Use a dependent (or paired) samples t-test if you want to know if there is change in the same group before and after something occurs to the entire group. Use this test when the same group is measured at two different times.

Example: Will balance be impaired after drinking alcohol? In this case we would test balance before and then after alcohol consumption. If we continue to use our weight loss example, we could have everyone follow our special diet for a period of time, check their weight loss, and have the entire group add exercise to their diet regimen and test the group again a second time. Exercise is the IV, and the amount of weight loss between Time 1 and Time 2 is the DV.

IV ? same group before and after
DV ? change in score from time one to time two

Use a One Sample t-test when you have one group?s mean you wish to compare to a known population mean. For example, if you wish to know if your class? mean IQ score differs significantly from the standard mean IQ, you could compare those two scores.

When do I perform a one way ANOVA?

While this test can get quite complicated, and there are several types of ANOVA, a one way ANOVA is the test most commonly used to determine whether there are any significant differences between the means of three or more independent (unrelated) groups. Like the t-test, there are assumptions that must be met (independence, normality, homogeneity of variance, etc.), and the DV must be interval or ratio level data. Remember, though, that it is an omnibus test (that is, it tests overall mean differences between groups), so post hoc tests will be needed to tell exactly where the differences actually exist.

For a one way ANOVA, only one continuous DV and one categorical IV can be tested, but there can be any number of levels on the IV (unlike the t-test, which only allows two). If we continue with our weight loss example, we could test for mean differences between three groups this time: diet only, exercise only, and diet and exercise combined. The results, if significant, will tell us there is a difference somewhere, but we won?t know where, so that?s when a post hoc test will be necessary.

IV ? Weight loss group (diet only, exercise only, diet and exercise combined group)
DV ? Amount of weight lost