Experimental Designs Week 4 Solution Experimental Designs 2. Explain the difference between multiple independent variables and multiple levels of independent variables. Which is better? Answer: The general purpose of multivariate analysis of variance (MANOVA) is to determine whether multiple levels of independent variables on their own or in combination with one another have an effect on the dependent variables. MANOVA requires that the dependent variables meet parametric requirements. MANOVA is used under the same circumstances as ANOVA but when there are multiple dependent variables as well as independent variables within the model which the researcher wishes to test. MANOVA is also considered a valid alternative to the repeated measures ANOVA when sphericity is violated. Like an ANOVA, MANOVA examines the degree of variance within the independent variables and determines whether it is smaller than the degree of variance between the independent variables. If the within subjects variance is smaller than the betw- een subjects variance it means the independent variable has had a significant effect on the dependent variables. There are two main differences between MANOVAs and ANOVAs. The first is that MANOVAs are able to take into account multiple independent and multiple dependent variables within the same model, permitting greater complexity. Secondly,rather than using the F value as the indicator of significance a number of multivariate measures. MANOVAs the independent variables relevant to each main effect are weighted to give them priority in the calculations performed. In interactions the independent variables are equally weighted to determine whether or not they have an additive effect in terms of the combined variance they account for in the dependent variable/s. The main effects of the independent variables and of the interactions are examined with all else held constant. The effect of each of the independent variables is tested separately. Any multiple interactions are tested separately from one another and from any significant main effects. Assuming there are equal sample sizes both in the main effects and the inter- actions, each test performed will be independent of the next or previous calculation (exce- pt for the error term which is calculated across the independent variables). 3. What is blocking and how does it reduce “noise”? What is a disadvantage of blocking? Sol: The Randomized Block Design is research design's equivalent to stratified random sampling. Like stratified sampling, randomized block designs are constructed to reduce noise or variance in the data (see Classifying the Exper- imental Designs). How do they do it? They require that the researcher divide the sample into relatively homogeneous subgroups or blocks (analogous to "strata" in stratified sampling). Then, the experimental design you want to impl- ement is implemented within each block or homogeneous subgroup. The key idea is that the variability within each block is less than the variability of the entire sample. Thus each estimate of the treatment effect within a block is more efficient than estimates across the entire sample. And, when we pool these more efficient estimates across blocks, we should get an overall more efficient estimate than we would without blocking. How Blocking Reduces Noise So how does blocking work to reduce noise in the data? To see how it works, you have to begin by thinking about the non-blocked study. The figure shows the pretest-posttest distribution for a hypothetical pre-post randomized experi- mental design. We use the 'X' symbol to indicate a program group case and the 'O' symbol for a comparison group member. You can see that for any specific pretest value, the program group tends to outscore the comparison group by about 10 points on the posttest. That is, there is about a 10-point posttest mean difference. Now, let's consider an example where we divide the sample into three relatively homogeneous blocks. To see what happens graphically, we'll use the pretest mea- sure to block. This will assure that the groups are very homogeneous. Let's look at what is happening within the third block. Notice that the mean difference is still the same as it was for the entire sample about 10 points within each block. But also notice that the variability of the posttest is much less than it was for the entire samp le. Remember that the treatment effect estimate is a signal-to-noise ratio. The signal in this case is the mean difference. The noise is the variability. The two figures show that we haven't changed the signal in moving to blocking there is still about a 10-point posttest difference. But, we have changed the noise -- the variability on the posttest is much smaller within each block that it is for the entire sample. So, the treatment effect will have less noise for the same signal.