Controlling extraneous variables is important in terms of eliminating confounds and reducing noise. Here I identify five methods of controlling extraneous variables.
Constancy. Here you hold the value of an extraneous variable constant across all subjects. If the EV is not variable, it cannot contribute to the variance in the DV. For example, you could choose to use only female subjects in your research, eliminating any variance in the DV that could be attributable to gender. Do keep in mind that while such noise reduction will increase the statistical “power” of your analysis (the ability to detect an effect of the IV, even if that effect is not large), it comes at a potential cost of external validity. If your subjects are all female, you remain uncertain whether or not your results generalize to male individuals.
Balancing. Here you assign subjects to treatment groups in such a way that the distribution of the EV is the same in each group. For example, if 60% of the subjects in the experimental group are female, then you make sure that 60% of the subjects in the control group are female. While this will not reduce noise and enhance power, it will prevent the EV from being confounded with the IV.
Randomization. If you randomly assign subjects to treatment groups, they should be balanced on subject characteristics (those EVs that subjects bring to the experiment with themselves).
Matching. Here we are talking about the research design commonly know as the randomized blocks or split plot design. On one or more EVs, thought to be well correlated with the DV, we match subjects up in blocks of k, where k is the number of treatment groups. Within each block, the subjects are identical or nearly identical on the matching variable(s). Within each block, one subject is (randomly) assigned to each treatment group. This will, of course, balance the distribution of the EV across groups, but it will also allow a statistical analysis which removes from the DV the effect of the matching variable, reducing noise and increasing power.
Statistical control. Suppose you were going to evaluate the effectiveness of three different methods of teaching young children the alphabet. To enhance power, you wish to use a randomized blocks design. You administer to every potential subject a test of readiness to learn the alphabet, and then you match (block) subjects on that variable. Next you randomly assign them (within each block) to groups. In your statistical analysis, the effect of the matching/blocking variable is taken out of what would otherwise be “error variance” in your statistical model. Such error variance is generally the denominator of the ratio that you use as the test statistic for a test of statistical significance, and the numerator of that ratio is generally a measure of the apparent magnitude of the treatment effect. Lets look at that ratio.
, for example, , or .
Look at this pie chart, in which I have partitioned the total variance in the DV into variance due to the treatment, due to the blocking variable, and due to everything else (error). If we had just ignored the blocking variable, rather than controlling it by using the randomized blocks design, the variance identified as due to blocks would be included in the error variance. Look back at the test statistic ratio. Since error variance is in the denominator, removing some of it makes the absolute value of the test statistic greater, giving you more power (a greater probability of obtaining a significant result).
Another statistical way to reduce noise and increase power is to have available for every subject data on one or more covariate. Each covariate should be an extraneous variable which is well correlated with the dependent variable. We can then use an ANCOV (analysis of covariance) to remove from the error term that variance due to the covariate (just like the randomized blocks analysis does), but we don’t need to do the blocking and random assignment within blocks. This analysis is most straightforward when we are using it along with random assignment of subjects to groups, rather than trying to use ANCOV to “unconfound” a static-group design (more on this later in the semester).
If the EV you wish to control is a categorical variable, one method to remove its effect from the error variance is just to designate the EV as being an IV in a factorial ANOVA. More on this later in the semester.
I used the term “split-plot” design earlier as a synonym for a randomized blocks design. The term “split plot” comes from agricultural research, in which a field is divided into numerous plots and each plot is divided into k parts, where k is the number of treatments. Within each plot, treatments are randomly assigned to each part -- for example, seed type A to one part, seed type B to a second part, etc. Statistically, the plots here are just like the blocks we use in a randomized blocks design.
Some of you have already studied “repeated measures” or “within subjects” designs, where each subject is tested under each treatment condition. This is really just a special case of the randomized blocks design, where subjects are blocked up on all subject variables.
Constancy. Here you hold the value of an extraneous variable constant across all subjects. If the EV is not variable, it cannot contribute to the variance in the DV. For example, you could choose to use only female subjects in your research, eliminating any variance in the DV that could be attributable to gender. Do keep in mind that while such noise reduction will increase the statistical “power” of your analysis (the ability to detect an effect of the IV, even if that effect is not large), it comes at a potential cost of external validity. If your subjects are all female, you remain uncertain whether or not your results generalize to male individuals.
Balancing. Here you assign subjects to treatment groups in such a way that the distribution of the EV is the same in each group. For example, if 60% of the subjects in the experimental group are female, then you make sure that 60% of the subjects in the control group are female. While this will not reduce noise and enhance power, it will prevent the EV from being confounded with the IV.
Randomization. If you randomly assign subjects to treatment groups, they should be balanced on subject characteristics (those EVs that subjects bring to the experiment with themselves).
Matching. Here we are talking about the research design commonly know as the randomized blocks or split plot design. On one or more EVs, thought to be well correlated with the DV, we match subjects up in blocks of k, where k is the number of treatment groups. Within each block, the subjects are identical or nearly identical on the matching variable(s). Within each block, one subject is (randomly) assigned to each treatment group. This will, of course, balance the distribution of the EV across groups, but it will also allow a statistical analysis which removes from the DV the effect of the matching variable, reducing noise and increasing power.
Statistical control. Suppose you were going to evaluate the effectiveness of three different methods of teaching young children the alphabet. To enhance power, you wish to use a randomized blocks design. You administer to every potential subject a test of readiness to learn the alphabet, and then you match (block) subjects on that variable. Next you randomly assign them (within each block) to groups. In your statistical analysis, the effect of the matching/blocking variable is taken out of what would otherwise be “error variance” in your statistical model. Such error variance is generally the denominator of the ratio that you use as the test statistic for a test of statistical significance, and the numerator of that ratio is generally a measure of the apparent magnitude of the treatment effect. Lets look at that ratio.
, for example, , or .
Look at this pie chart, in which I have partitioned the total variance in the DV into variance due to the treatment, due to the blocking variable, and due to everything else (error). If we had just ignored the blocking variable, rather than controlling it by using the randomized blocks design, the variance identified as due to blocks would be included in the error variance. Look back at the test statistic ratio. Since error variance is in the denominator, removing some of it makes the absolute value of the test statistic greater, giving you more power (a greater probability of obtaining a significant result).
Another statistical way to reduce noise and increase power is to have available for every subject data on one or more covariate. Each covariate should be an extraneous variable which is well correlated with the dependent variable. We can then use an ANCOV (analysis of covariance) to remove from the error term that variance due to the covariate (just like the randomized blocks analysis does), but we don’t need to do the blocking and random assignment within blocks. This analysis is most straightforward when we are using it along with random assignment of subjects to groups, rather than trying to use ANCOV to “unconfound” a static-group design (more on this later in the semester).
If the EV you wish to control is a categorical variable, one method to remove its effect from the error variance is just to designate the EV as being an IV in a factorial ANOVA. More on this later in the semester.
I used the term “split-plot” design earlier as a synonym for a randomized blocks design. The term “split plot” comes from agricultural research, in which a field is divided into numerous plots and each plot is divided into k parts, where k is the number of treatments. Within each plot, treatments are randomly assigned to each part -- for example, seed type A to one part, seed type B to a second part, etc. Statistically, the plots here are just like the blocks we use in a randomized blocks design.
Some of you have already studied “repeated measures” or “within subjects” designs, where each subject is tested under each treatment condition. This is really just a special case of the randomized blocks design, where subjects are blocked up on all subject variables.
Thanks
0 comments:
Post a Comment