Thursday, November 8, 2007

Cost vs. ANCOVA

My supervisor just asked me an interesting question on email. In RT research, it's quite common to "control" for baseline response time by either subtracting it from experimental condition RTs or run an ANCOVA using baseline RT as covariate. She wants to know if these two control methods are the same.

Here's my response:
Mathematically, they are not the same. Imagine a distribution of response time, subtracting control RT from experimental RT is like shifting the distribution down by a constant. But of course, this substraction is done at a mean level for each individual, so individual differences would be included in the costs (e.g., someone is exceptionally interfered by the flankers, regardless of whether it is congruent or incongruent, would incur a much larger cost than someone who is not interfered by the mere appearance of flankers). The end result is a much lower mean and more variability.

On the other hand, running an ANCOVA using control RT as covariate is like generating a new distribution, which would give you a different mean (central tendency measure) and a different standard error (variability). In this case, however, the variability would be smaller. This is because each individual's performance in each condition is considered as a distribution. ANCOVA partials out the common variability between control trials and each of the congruent and incongruent trials. The end result, in this case, is a mean that is analogous to the raw mean, but a tighter variability. Sometimes changes in ANCOVA is very small (if the control is not very correlated with the experimental conditions).

Any thoughts?

1 comment:

Manolo said...

Well... I found a nice reference in which you can read the effects of violating the assumptions of the ANCOVA and how it is good to run an ANOVA of the covariate to be sure it is independent from the grouping variable. This makes sense to me, because any pre-treatment difference (not only in the covariate, that in your case is the baseline) would compromise any findings later.
I guess that if there are any assumptions violation the subtraction method would be more appropriate. Does this make sense at all?

The reference is:
Elashoff, J.D. (1969). Analysis of Covariance: A delicate instrument. American Educational Research Journal, 6(3): 383-401.