Building on the work we have done in SPSS, lets explore how to use categorical variables as IVs in multiple regression (linear modeling). First, realize that in R, we usually want to code “factors” as string variables. They can be numeric codes, but it is nice to use the string values.
For the Cereals data set system, the data file was exported from SPSS, saving the value labels instead of codes for the shelf variable. When we import the file, the read.csv function interprets this variable as a factor. In R, the lm function does not need numerically coded dummy vectors. It creates them automatically for factors.
# load packages and import data
library(beeswarm)
library(car)
library(ggplot2)
library(ggthemes)
library(psych)
library(knitr)
library(gt)
The data set is the same N=74 data set that where we evaluated dummy and effect coding with SPSS. However, upon export to a .csv, only a subset of the variables were included in order to be efficient. This document only examines the sugar content as a function of shelf position. Only the “cold” cereals were examined and exported.
cereal1 <- read.csv(file="cereal_cold_sugar_shelf.csv")
# attach is used to simplify in this short set of illustrations
cereal1 <- read.csv(file="cereal_cold_sugar_shelf.csv")
attach(cereal1)
# the first few lines of the data frame are displayed
# note that I have begun using the 'gt' function from the **gt** package to make nicer tables
gt(head(cereal1))
| name | mfr | type | sugars | shelf | rating |
|---|---|---|---|---|---|
| 100%_Bran | N | C | 6 | top | 68.40297 |
| 100%_Natural_Bran | Q | C | 8 | top | 33.98368 |
| All-Bran | K | C | 5 | top | 59.42551 |
| All-Bran_with_Extra_Fiber | K | C | 0 | top | 93.70491 |
| Almond_Delight | R | C | 8 | top | 34.38484 |
| Apple_Cinnamon_Cheerios | G | C | 10 | bottom | 29.50954 |
Basic description of the variables as a function of shelf is accomplished the the ‘describeBy’ function form psych.
describeBy(sugars, group=shelf, type=2)
##
## Descriptive statistics by group
## group: bottom
## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 19 5.11 4.48 3 4.82 4.45 0 15 15 0.73 -0.53 1.03
## --------------------------------------------------------
## group: middle
## vars n mean sd median trimmed mad min max range skew kurtosis
## X1 1 19 10.47 3.26 12 10.65 1.48 3 15 12 -0.93 0.09
## se
## X1 0.75
## --------------------------------------------------------
## group: top
## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 36 6.53 3.84 6 6.47 4.45 0 14 14 0.2 -0.61 0.64
Two simple plots are useful.
stripchart(sugars~shelf, method="stack")
boxplot(sugars~shelf)
beeswarm(sugars~shelf,add=T)
The expected kind of plot for this arrangment of data would be a bar graph of the sugar means for each shelf, with error bars representing the std error of the mean. Doing this in ggplot2 requires a bit of work. A function (‘summarySE’) is called from an external file. It extracts the means and std errors from the data file and then the output is sent to the “vals” object here. the ‘ggplot’ function operates on this summary table rather than the whole data set.
source("cereal_tools.R")
vals <- summarySE(cereal1, measurevar="sugars",groupvars="shelf")
vals
## shelf N sugars sd se ci
## 1 bottom 19 5.105263 4.483237 1.0285251 2.160851
## 2 middle 19 10.473684 3.255225 0.7467999 1.568968
## 3 top 36 6.527778 3.835817 0.6393028 1.297854
dodge <- position_dodge(width = 0.9)
p <- ggplot(data = vals, aes(x = shelf, y = sugars, fill = shelf ))
p <- p + geom_bar(stat = "identity", position = dodge,alpha=.6,colour="black") +
geom_errorbar(aes(ymax = sugars + se,
ymin = sugars - se),
position = dodge, width = 0.25) +
scale_fill_brewer(palette = "Greys") +
theme_classic() +
theme(axis.text.x=element_blank(), axis.ticks.x=element_blank(),
axis.title.x=element_blank())
p
We can use the familiar lm function to fit the regression model
# fit the linear model using the "factor" shelf as IV
fit1 <- lm(sugars~shelf,data=cereal1)
summary(fit1)
##
## Call:
## lm(formula = sugars ~ shelf, data = cereal1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -7.4737 -3.1053 -0.0278 2.5263 9.8947
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 5.1053 0.8896 5.739 2.17e-07 ***
## shelfmiddle 5.3684 1.2580 4.267 6.01e-05 ***
## shelftop 1.4225 1.0995 1.294 0.2
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 3.878 on 71 degrees of freedom
## Multiple R-squared: 0.2212, Adjusted R-squared: 0.1993
## F-statistic: 10.08 on 2 and 71 DF, p-value: 0.0001398
Examine the standard set of diagnostic plots. I’ve suppressed the printout here to save space. The diagnostic plots look like those we have already examined with SPSS.
plot(fit1)
The results from the first fit didn’t match results we obtained in SPSS when we used dummy coding. Why? We need to explore how R is doing the coding.
In our SPSS work shelves were coded 1,2, and 3 for bottom, middle, and top, respectively. Our reference category was the top shelf when we did the dummy coding in SPSS.
Also, on reading the output above for fit1 …… The first dummy vector is labeled “shelfmiddle”. R uses this labeling because the first coding vector compares the middle category to the reference category and the second vector compares top to the reference category. How do we know this?????……see next section.
We can ask R to tell us what the coding scheme for this factor became when the data were first read into the data frame by read.csv. It uses the term “contrasts” generically for any coding vectors, dummy, effect, or analytical contrasts.
# do these coefficients match the SPSS output?
# what are the dummy coefficients chosen by R?
contrasts(cereal1$shelf)
## middle top
## bottom 0 0
## middle 1 0
## top 0 1
Can we change the coding??
# can we change the coding?
contrasts(cereal1$shelf) <- contr.treatment(3, base=3)
contrasts(cereal1$shelf)
## 1 2
## bottom 1 0
## middle 0 1
## top 0 0
Now rerun the model
fit2 <- lm(sugars~shelf,data=cereal1)
# note in the ouptut that the way R labels the vectors now changes
# be careful, "shelf1" and "shelf2" now refer to vectors 1 and 2
summary(fit2)
##
## Call:
## lm(formula = sugars ~ shelf, data = cereal1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -7.4737 -3.1053 -0.0278 2.5263 9.8947
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 6.5278 0.6463 10.101 2.25e-15 ***
## shelf1 -1.4225 1.0995 -1.294 0.199946
## shelf2 3.9459 1.0995 3.589 0.000607 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 3.878 on 71 degrees of freedom
## Multiple R-squared: 0.2212, Adjusted R-squared: 0.1993
## F-statistic: 10.08 on 2 and 71 DF, p-value: 0.0001398
NOTE HOW R NOW LABELS THE VECTORS!! “shelf1” and “shelf2” now refer to the first and second coding vectors for shelf, regardless of what those vectors actually mean. I’m not sure why R switches this labeling approach – potentially confusing unless you are expecting it.
IMPORTANT POINT: Again, as we saw in our SPSS work, the Multiple R squared, and the F test of the whole equation was the same with both fit1 and fit2. This is expected, since Yhats and residuals are the invariant to the “flavors” of dummy coding schemes (reference category). But, the regression coefficients (and intercept) do change, in expected ways. They test the difference in the mean of the category coded with a 1 on that vector and the mean of the reference category.
Many software packages call effect coding “deviation” coding. This is the case in some places with R, but here we want to specify the “contrasts” as effect codes and we use the contr.sum() specifier to do that
# now switch to effect coding
# contr.sum produces deviation contrasts, however.....
# the reference category is the last category and is not changeable in contr.sum
contrasts(cereal1$shelf) <- contr.sum(3)
contrasts(cereal1$shelf)
## [,1] [,2]
## bottom 1 0
## middle 0 1
## top -1 -1
This shows that the reference category is the last category (top for our example given the alphabetical ordering of the three levels).
Now fit the model
fit3 <- lm(sugars~shelf,data=cereal1)
summary(fit3)
##
## Call:
## lm(formula = sugars ~ shelf, data = cereal1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -7.4737 -3.1053 -0.0278 2.5263 9.8947
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 7.3689 0.4714 15.631 < 2e-16 ***
## shelf1 -2.2636 0.6972 -3.247 0.00178 **
## shelf2 3.1048 0.6972 4.453 3.08e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 3.878 on 71 degrees of freedom
## Multiple R-squared: 0.2212, Adjusted R-squared: 0.1993
## F-statistic: 10.08 on 2 and 71 DF, p-value: 0.0001398
Unfortunately there is not a method for changing the reference category with effect coding if we use the contr.sum() function directly. We can, however, use our matrix manipulation knowledge to solve this problem. The “coding” vectors we saw above all came in pairs that comprised a matrix. We simply create this matrix on our own and the respecify the contrasts for our shelf factor to reflect this matrix. So, here we can force the bottom shelf to be the reference category.
# so how do we set a different category as the reference category for
# deviation or effect coding?
# I manually define my own matrix of coefficients
contrasts.dev1 <- matrix(c(-1,1,0,-1,0,1),ncol=2)
contrasts(cereal1$shelf) <- contrasts.dev1
contrasts(cereal1$shelf)
## [,1] [,2]
## bottom -1 -1
## middle 1 0
## top 0 1
Now refit the model with the new effect coding vectors.
fit4 <- lm(sugars~shelf,data=cereal1)
summary(fit4)
##
## Call:
## lm(formula = sugars ~ shelf, data = cereal1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -7.4737 -3.1053 -0.0278 2.5263 9.8947
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 7.3689 0.4714 15.631 < 2e-16 ***
## shelf1 3.1048 0.6972 4.453 3.08e-05 ***
## shelf2 -0.8411 0.6012 -1.399 0.166
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 3.878 on 71 degrees of freedom
## Multiple R-squared: 0.2212, Adjusted R-squared: 0.1993
## F-statistic: 10.08 on 2 and 71 DF, p-value: 0.0001398