ANCOVA

This page repeats some of the discussions in the analysis of covariance panels of Multiple Regression Explained Page and uses the same data set as example. The essentials will be repeated here, but the calculations will be carried out in R.

For those not familiar with R, an introduction is provided in R Programming Language Explained Page

For those who do not have a clear understanding of Analysis of Covariance, the following minimal and very basic terms and descriptions may be useful.

The Analysis of Variance partitions the variance of the dependent variable according to those factors that influence it.
- In the simplest model, the analysis of variance is summarized as the t test. For example, how is the variance in birth weight influenced by the sex (male or female) of the baby, a single comparison of the two sexes
- When there are more than two groups, the general model of One Way Analysis of Variance is used. For example, how do three different ethnic origin (say Greeks, Germans, and Slavs) influence the birth weight of the baby. with three groups there are 3 comparisons, Greek vs Germans, Greek vs Slavs, and German vs Slavs.
- When Two sets of influences (Factors) are involved (say sex and ethnicity), then a Two Way Analysis of Variance is used. With more, Multiway Analysis of Variance. However, there may be systematic or accidental correlations between factors, (say Greeks have more girls than Germans), and these are called Interactions between Factors. The analysis of Variance which separates those variances unique to each factor, and those that overlapped between factors is known as the Factorial Model of Analysis of Variance.
If, on top of all of this, as is usually the case, there are other influences to be taken into consideration, such as differences in birth weights must be corrected by the gestational age, then one or more of these corrections are termed covariates, and the combination of the analysis becomes Covariance Analysis.
Things now starts to become a bit complicated, because each covariate may act differently in different factors, say German babies grow faster than Slav babies near term. This is call an Interaction between a factor and a covariate.
The total number of interactions are therefore a multiple of covariates and factors. As these increases, the model becomes complex confusing.
To be correct, the results of a covariate analysis is only valid if all possible interactions are tested and found to be trivial (not statistically significant). In a review of the literature however, most do not bother and assumes that interactionsare either irrelevant or do not exist.

Sex	Ethnicity	Gest	BWt
Girl	Greek	37	3048
Boy	German	36	2813
Girl	French	41	3622
Girl	Greek	36	2706
Boy	German	35	2581
Boy	French	39	3442
Girl	Greek	40	3453
Boy	German	37	3172
Girl	French	35	2386
Boy	Greek	39	3555
Girl	German	37	3029
Boy	French	37	3185
Girl	Greek	36	2670
Boy	German	38	3314
Girl	French	41	3596
Boy	Greek	38	3312
Girl	German	39	3200
Boy	French	41	3667
Boy	Greek	40	3643
Girl	German	38	3212
Girl	French	38	3135
Girl	Greek	39	3366

For this exercise in R, we will use the same data as that in Multiple Regression Explained Page , as shown in the table to the right. The dataset was artificially generated by the computer to demonstrate the procedures, and they do not represent reality. In a real analysis, the sample sizewould be larger, and there may be additional factors and covariates.

The dependent variable is birth weight, in g
The two factors are sex (Boy or Girl), and Ethnicity (German, French, or Greek)
For this exercise we will use only one covariate, gestation in weeks.

There are 4 German boys (red) and 3 German girls (maroon), 3 Greek boys (light green) and 5 Greek girls (dark green), 3 French boys (blue) and 4 French girls (navy).

Step 1. Data input


Input = ("
Sex	Ethn	Gest	BWt
         Girl	Greek	37	3048
         Boy	German	36	2813
         Girl	French	41	3622
         Girl	Greek	36	2706
         Boy	German	35	2581
         Boy	French	39	3442
         Girl	Greek	40	3453
         Boy	German	37	3172
         Girl	French	35	2386
         Boy	Greek	39	3555
         Girl	German	37	3029
         Boy	French	37	3185
         Girl	Greek	36	2670
         Boy	German	38	3314
         Girl	French	41	3596
         Boy	Greek	38	3312
         Girl	German	39	3200
         Boy	French	41	3667
         Boy	Greek	40	3643
         Girl	German	38	3212
         Girl	French	38	3135
         Girl	Greek	39	3366
         ")
Data = read.table(textConnection(Input),header=TRUE)

Note : This is an example of direct data entry. For data in a large excel file in the current working directory


install.packages("XLConnect")
library('XLConnect')
Data = readWorksheetFromFile("MyANCOVAData.xlsx", sheet="MySheet") #header=TRUE is default

Step 2 : Initial plotting of data


par(pin=c(4.2, 3))              # set plotting window to 4.2x3 inches
plot(x   = Data$Gest,           # Gest on the x axis
     y   = Data$BWt,            # BWt on the y axis
     col = Data$Sex:Data$Ethn,  # in 6 colors (sex=2,Ethn=3)
     pch = 16,                  # size of dot
     xlab = "Gestation",        # x label
     ylab = "Birth Weight")     # y lable
#legend is optional
legend('bottomright',           # placelegendon bottomright
       legend = levels(Data$Sex:Data$Ethn), #all combinations
       col = 1:6,               # 6 colors for 6groups
       cex = 1,   
       pch = 16)

This produces a plot of all data points,in 6 colors according to sex:Ethn groupings

Step 3 : Analysis of Covariance testing for interactions between all variables

The test is arbitrarily called model.1, dependent variable=BWt, independent variables= all 3 of sex,Rthn, Gest, and the interaction term Sex:Ethn:Gest

The package car is necessary. It actually resulted in installing a whole lot of packages, probably all the supportive subroutines

I do not know why type="II", but it is necessary. I tried I and III and merely triggered an error statement


model.1 = lm (BWt ~ Sex + Ethn + Gest + Sex:Ethn:Gest, data = Data)
install.packages("car")
library('car')
Anova(model.1, type="II")

The results are shown as follows, the point to make is that there is no no significant interaction between the covariate Gest with the two factors Sex and Ethn.


Anova Table (Type II tests)

Response: BWt
               Sum Sq Df  F value    Pr(>F)    
Sex            145915  1  13.3676  0.003289 ** 
Ethn            22244  2   1.0189  0.390201    
Gest          2272527  1 208.1910 6.053e-09 ***
Sex:Ethn:Gest   21007  5   0.3849  0.849758    
Residuals      130987 12                       
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Step 4 : Analysis of Covariance testing for interactions between factors

Having been satisfied that there is no interaction involving the covariate, a second test to make sure no interaction between Sex and Ethn exists, after correction for Gest.


model.2 = lm (BWt ~ Sex + Ethn + Gest + Sex:Ethn, data = Data)
Anova(model.2, type="II")

The results shows that there is no interaction between the two factors Sex and Ethn


Anova Table (Type II tests)

Response: BWt
           Sum Sq Df  F value    Pr(>F)    
Sex        145915  1  15.3288  0.001378 ** 
Ethn        22244  2   1.1684  0.337612    
Gest      1914186  1 201.0904 4.279e-10 ***
Sex:Ethn     9208  2   0.4837  0.625801    
Residuals  142785 15                       
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Note : To be precise, interactions at all combinations and all levels must be tested and excluded before drawing the final conclusion. However, in many publications, interactions are often not tested,or if tested, only at the bottom level such as in step 3

Step 5 : Final Analysis of Covariance assuming absence of any interactions

This step should only be carried out in the absence of any interactions. However, in many publications, absence of interaction is presumed, and this final analysis is done primarily


model.3 = lm (BWt ~ Sex + Ethn + Gest, data = Data)
Anova(model.3, type="II")

The results are shown as follows. The covariate Gest has a significant effect on BWt. After correction for Gest, Sex has, but Ethn has no significant effect on BWt.


Anova Table (Type II tests)

Response: BWt
           Sum Sq Df F value    Pr(>F)    
Sex        145915  1  16.320 0.0008506 ***
Ethn        22244  2   1.244 0.3131907    
Gest      2272527  1 254.175 1.172e-11 ***
Residuals  151994 17                      
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The analysis can end at this point. However, if there is a need to calculate the BWt from sex, Ethn, and Gest, the following command can be used.


summary(model.2)

The results are as follows


Call:
lm(formula = BWt ~ Sex + Ethn + Gest, data = Data)

Residuals:
    Min      1Q  Median      3Q     Max 
-127.51  -76.71    3.61   75.65  153.75 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) -4021.50     467.16  -8.608 1.33e-07 ***
SexGirl      -165.93      41.07  -4.040 0.000851 ***
EthnGerman     58.49      54.91   1.065 0.301682    
EthnGreek      77.14      49.75   1.551 0.139430    
Gest          190.61      11.96  15.943 1.17e-11 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 94.56 on 17 degrees of freedom
Multiple R-squared:  0.9462,	Adjusted R-squared:  0.9335 
F-statistic: 74.75 on 4 and 17 DF,  p-value: 1.472e-10

This means a French Boy, at 36 weeks gestation, should weigh -4022+36*191 = 2854g.

For every week of gestation after 36 weeks, the baby gains 190.6g
If the baby is a girl, it weighs 165.9g less
If the mother is Greek, the baby weighes 77.1g more, if German 58.5g more.

[https://rcompanion.org/rcompanion/e_04.html] An R Companion for the Handbook of Biological Statistics by Salvatore S. Mangiafico. Chapter on analysis of covariance

[https://en.wikipedia.org/wiki/Analysis_of_covariance] ABCOVA by Wikipedia

Overall JE and Klett CJ (1972) Applied Multivariate Analysis. McGraw Hill Series in Psychology. McGraw Hill Book Company New York. Library of Congress No. 73-14716407-047935-6 p.415-440.

Steel RGD, Torrie JH, Dickey DA (1997) Principles and procedures of statistics. A biomedical approach. 3rd Ed. McGraw-Hill Inc New York NY 10020 ISBN 0-07-061028-2 p. 322-351

Pedhazur E. (1997) Multiple Regression in Behavioral Research. Explanation and Prediction.(3rd. ed.) Harcourt Brace College Publishers, Fort Worth, USA. ISBN 0-03-072831-2 p.181-196.