©Richard Lowry, 1999-
All rights reserved.
¶Example 2. Three Methods of Instruction for Elementary Computer Programming
Method A | Method B | Method C | ||
29 24 14 27 27 28 27 32 13 35 32 17 |
15 28 13 36 29 27 31 33 32 15 30 26 |
32 27 15 23 26 17 25 14 29 22 30 25 | ||
means |
25.4 |
26.3 |
23.8 |
Given the differences among the means of the three groups, you might think at first glance that Method B has the edge over Method A, and that Methods B and A are both superior to Method C. As it happens, however, these differences, considered in and of themselves, are well within the range of mere random variability. A simple one-way ANOVA performed on this set of data would yield a miniscule
The reason for this shortfall is of course the degree of variability within the groups, which is quite large in comparison with the mean differences that appear between the groups. Well aware of the broad range of pre-existing individual differences that are likely to be found in situations of this sort, our curriculum researcher took the precaution of measuring her subjects beforehand with respect to their pre-existing levels of basic computer familiarity. The rationale is fairly obvious: the more familiar a subject is with basic computer procedures, the more of a head start he or she will have in learning the elements of computer programming; remove the effects of this covariate, and you thereby remove a substantial portion of the extraneous individual differences. Our researcher was also well aware that her three groups, drawn from three different schools, might be starting out with substantially different levels of basic computer familiarity, in consequence of the average socio-economic differences that she knows to exist among the schools. The groups instructed by Methods A and B both reside in fairly affluent neighborhoods, while the group instructed by Method C comes from a less privileged part of town.
The following table shows the measures on both variables laid out in a form suitable for an analysis of covariance, with
X = | the prior measure of basic computer familiarity [The covariate whose effects the investigator wishes to remove from the analysis.] | |
and | ||
Y = | the measure of how well the subject has learned the elementary programming material [The dependent variable in which the investigator is chiefly interested.] |
Method A | Method B | Method C | |||||||||
Sub- ject | X_{a} | Y_{a} | Sub- ject | X_{b} | Y_{b} | Sub- ject | X_{c} | Y_{c} | |||
a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 |
14 10 7 18 14 16 13 15 5 18 16 10 |
29 24 14 27 27 28 27 32 13 35 32 17 |
b1 b2 b3 b4 b5 b6 b7 b8 b9 b10 b11 b12 |
6 16 9 19 13 14 15 18 17 8 15 16 |
15 28 13 36 29 27 31 33 32 15 30 26 |
c1 c2 c3 c4 c5 c6 c7 c8 c9 c10 c11 c12 |
15 9 7 12 12 9 12 3 13 10 11 8 |
32 27 15 23 26 17 25 14 29 22 30 25 | |||
Means | 13.0 | 25.4 | 13.8 | 26.3 | 10.1 | 23.8 |
1. Calculations for the Dependent Variable Y
Values of Y along with the several summary statistics required for the calculation of SS_{T(Y)}, SS_{wg(Y)}, and SS_{bg(Y)}:
Y_{a} | Y_{b} | Y_{c} | ||||
29 24 14 27 27 28 27 32 13 35 32 17 |
15 28 13 36 29 27 31 33 32 15 30 26 |
32 27 15 23 26 17 25 14 29 22 30 25 | for total array of data | |||
N | 12 | 12 | 12 | 36 | ||
.Y_{i} | 305 | 315 | 285 | 905 | ||
.Y_{i}^{2} | 8315 | 8919 | 7143 | 24377 | ||
SS | 562.9 | 650.3 | 374.3 | |||
Mean | 25.4 | 26.3 | 23.8 | 25.1 |
Values of X along with the several summary statistics required for the calculation of SS_{T(X)} and SS_{wg(X)}.
X_{a} | X_{b} | X_{c} | ||||
14 10 7 18 14 16 13 15 5 18 16 10 |
6 16 9 19 13 14 15 18 17 8 15 16 |
15 9 7 12 12 9 12 3 13 10 11 8 | for total array of data | |||
N | 12 | 12 | 12 | 36 | SS_{T(X)} = 581.6 SS_{wg(X)} = 488.6 | |
.X_{i} | 156 | 166 | 121 | 443 | ||
.X_{i}^{2} | 2220 | 2482 | 1331 | 6033 | ||
SS | 192.0 | 185.7 | 110.9 | |||
Mean | 13.0 | 13.8 | 10.1 | 12.3 |
Cross-products of X_{i} and Y_{i} for each subject in each of the three groups, along with other summary data required for the calculation of SC_{T} and SC_{wg}:
A | B | C | |
X_{a}Y_{a} | X_{b}Y_{b} | X_{c}Y_{c} | |
(14)(29) = 406 (10)(24) = 240 (7)(14) =98 (18)(27) = 486 (14)(27) = 378 (16)(28) = 448 (13)(27) = 351 (15)(32) = 480 (5)(13) =65 (18)(35) = 630 (16)(32) = 512 (10)(17) = 170 |
(6)(15) =90 (16)(28) = 448 (9)(13) = 117 (19)(36) = 684 (13)(29) = 377 (14)(27) = 378 (15)(31) = 465 (18)(33) = 594 (17)(32) = 544 (8)(15) = 120 (15)(30) = 450 (16)(26) = 416 |
(15)(32) = 480 (9)(27) = 243 (7)(15) = 105 (12)(23) = 276 (12)(26) = 312 (9)(17) = 153 (12)(25) = 300 (3)(14) =42 (13)(29) = 377 (10)(22) = 220 (11)(30) = 330 (8)(25) = 200 | for total array of data |
.∑(X_{ai}Y_{ai}) =4264 | .∑(X_{bi}Y_{bi}) =4683 | .∑(X_{ci}Y_{ci}) =3038 | .∑(X_{Ti}Y_{Ti}) =11985 |
o∑X_{ai}=156 o∑Y_{ai}=305 | o∑X_{bi}=166 o∑Y_{bi}=315 | o∑X_{ci}=121 o∑Y_{ci}=285 | o∑X_{Ti}=443 o∑Y_{Ti}=905 |
SC_{T} | = ∑(X_{Ti}Y_{Ti}) — | (∑X_{Ti})(∑Y_{Ti}) N_{T} |
SC_{T} | = 11985 — | (443)(905) 36 | = 848.5 |
The components for each group ("g") are calculated as:
SC_{wg(g)} | = ∑(X_{gi}Y_{gi}) — | (∑X_{gi})(∑Y_{gi}) N_{g} |
SC_{wg(a)} | = 4264 — | (156)(305) 12 | = 299.0 |
SC_{wg(b)} | = 4683 — | (166)(315) 12 | = 325.5 |
SC_{wg(c)} | = 3036 — | (121)(285) 12 | = 164.3 |
SC_{wg} | = SC_{wg(a)} + SC_{wg(b)} + SC_{wg(c)} | |
= 299.0 + 325.5 + 164.3 = 788.8 |
Here again is a summary of the values of SS and SC obtained so far. Recall that Y is the variable in which we are chiefly interested, and X is the covariate whose effects we are seeking to remove.
X | Y | Covariance | |||
SS_{T(X)} = 581.6 SS_{wg(X)} = 488.6 |
SS_{T(Y)} = 1626.3 SS_{wg(Y)} = 1587.4 SS_{bg(Y)} = 38.9 |
SC_{T} = 848.5 SC_{wg} = 788.8 | |||
For handy reference, click here to place a version of this table into the frame on the left.] |
As indicated in connection with Example 1, the overall correlation between X and Y (all three groups combined) can be calculated as
r_{T} | = |
SC_{T} sqrt[SS_{T(X)} x SS_{T(Y)}] | |||
= |
848.5 sqrt[581.6 x 1626.31] | ||||
= | +.872 |
(r_{T})^{2} = (+.872)^{2} = .760 |
[adj]SS_{T(Y)} | = SS_{T(Y)}— |
(SC_{T})^{2} SS_{T(X)} | |
= 1626.3 — | (848.5)^{2} 581.6 | ||
= 388.4 |
Similarly, the aggregate correlation between X and Y within the three groups can be calculated as
r_{wg} | = |
SC_{wg} sqrt[SS_{wg(X)} x SS_{wg(Y)}] | |||
= |
788.8 sqrt[488.6 x 1587.4] | ||||
= | +.896 |
(r_{wg})^{2} = (+.896)^{2} = .803 |
[adj]SS_{wg(Y)} | = SS_{wg(Y)}— |
(SC_{wg})^{2} SS_{wg(X)} | |
= 1587.4 — | (788.8)^{2} 488.6 | ||
= 314.0 |
The adjusted value of SS_{bg(Y)} can then again be obtained through simple subtraction as
[adj]SS_{bg(Y)} | = |
[adj]SS_{T(Y)}
— [adj]SS_{wg(Y)} | |
= | 388.4 — 314.0 = 74.4 |
The average relationship between
b_{wg} | = | SC_{wg} SS_{wg(X)} | ||||
= | 788.8 488.6 | = +1.61 | ||||
M_{X} | observed M_{Y} | adjusted M_{Y} | |
group A | 13.0 | 25.4 | 24.3 |
group B | 13.8 | 26.3 | 23.9 |
group C | 10.1 | 23.8 | 27.3 |
combined | 12.3 | 25.1 |
[adj]M_{Ya} | = M_{Ya} — b_{wg}(M_{Xa}—M_{XT}) | ||
= 25.4 — 1.61(13.0—12.3) | |||
= 24.3 |
[adj]M_{Yb} | = M_{Yb} — b_{wg}(M_{Xb}—M_{XT}) | ||
= 26.3 — 1.61(13.8—12.3) | |||
= 23.9 |
[adj]M_{Yc} | = M_{Yc} — b_{wg}(M_{Xc}—M_{XT}) | ||
= 23.8 — 1.61(10.1—12.3) | |||
= 27.3 | |||
As illustrated in the adjacent graph, the adjusted group means paint quite a different picture. When the different pre-existing levels of basic computer familiarity are taken into account, Method C appears to have a substantial edge over both of the other instructional methods. |
The simple computational format for this step is the same as for Example 1. All we need do here is lay out its results in the form of an ANCOVA summary table:
Source | SS | df | MS | F | P | |
adjusted means [between-groups effect] | 74.4 | 2 | 37.2 | 3.8 | <.05 | |
adjusted error [within-groups] | 314.0 | 32 | 9.8 | |||
adjusted total | 388.4 | 34 |
[Recall that df_{bg(Y)}=k—1 and [adj]df_{wg(Y)}=N_{T}—k—1] |
df denomi- nator | df numerator | ||
1 | 2 | 3 | |
32 | 4.15 7.50 | 3.29 5.34 | 2.90 4.46 |
As mentioned toward the end of Part 2, the analysis of covariance assumes that the slopes of the regression lines for each of the groups considered separately do not significantly differ from the slope of the overall within-groups regression, which for the present example
In the adjacent graph you can see that the three separate group regression lines, in red, do appear to be in close parallel with the blue line for the overall within-groups regression. However, here as elsewhere in the domain of statistical inference, a merely intuitive impression of this sort is not sufficient. For the naked eye is simply not capable of making the fine-grained distinctions that need to be made with complex sets of numerical data.
The procedure for rigorously determining whether this "homogeneity of regression" assumption is satisfied involves yet another variation on the theme of analysis of variance. As in all versions of ANOVA, the end result is an
F = | MS_{effect} MS_{error} | = | SS_{effect} /df_{effect} SS_{error} /df_{error} |
(SC_{g})^{2} SS_{Xg} |
Note thatSC_{g}/SS_{Xg} would give you the slope for group g. |
(SC_{wg})^{2} SS_{wg(X)} |
Note thatSC_{wg}/SS_{wg(X)} = b_{wg} the slope for the overall within-groups regression. |
SS_{b-reg} = | (SC_{g})^{2} SS_{Xg} | — | (SC_{wg})^{2} SS_{wg(X)} |
SS_{b-reg} | = | (SC_{a})^{2} SS_{Xa} | + | (SC_{b})^{2} SS_{Xb} | + | (SC_{c})^{2} SS_{Xc} | — | (SC_{wg})^{2} SS_{wg(X)} |
SS_{b-reg} | = | (299.0)^{2} 192.0 | + | (325.5)^{2} 185.7 | + | (164.3)^{2} 110.9 | — | (788.8)^{2} 488.6 |
SS_{b-reg} | = | 6.1 |
SS_{Y(remainder)} | = [adj]SS_{wg(Y)} — SS_{b-reg} | |
= 314.0 — 6.1 = 307.9 | ||
= 307.9 |
[adj]df_{wg(Y)} | = N_{T} — k — 1 | |
= 36 — 3 — 1 | ||
= 32 |
df_{b-reg} | = k — 1 | |
= 3 — 1 | ||
= 2 |
df_{Y(remainder)} | = [adj]df_{wg(Y)} — df_{b-reg} | |
= (N_{T}—k —1)—(k —1) | ||
= 32 — 2 = 30 |
F | = | MS_{b-reg} MS_{Y(remainder)} | = | SS_{b-reg} /df_{b-reg} SS_{Y(remainder)} /df_{Y(remainder)} | |
= | 6.1/2 307.9/30 | ||||
= | 0.30 | [with df = 2,30] |
df denomi- nator | df numerator | ||
1 | 2 | 3 | |
30 | 4.17 7.56 | 3.32 5.39 | 2.92 4.51 |
Home | Click this link only if the present page does not appear in a frameset headed by the logo Concepts and Applications of Inferential Statistics |