Section 19.3 Classifying Spam Email with kernlab
To get started with support vector machines, we can load one of the R packages that supports this technique. We will use the "kernlab" package. Use the commands below:
install.packages("kernlab")
library(kernlab)
I found that it was important to use the double quotes in the first command, but not in the second command. The data set that we want to use is built into this package. The data comes from a study of spam emails received by employees at the Hewlett-Packard company. Load the data with the following command:
data(spam)
This command does not produce any output. We can now inspect the "spam" dataset with the
str command:
str(spam)
'data.frame': 4601 obs. of 58 variables: $ make : num 0 0.21 0.06 0 0 0 0 0 0.15 0.06 ... $ address : num 0.64 0.28 0 0 0 0 0 0 0 0.12 ... $ all : num 0.64 0.5 0.71 0 0 0 0 0 0.46 0.77 ... $ num3d : num 0 0 0 0 0 0 0 0 0 0 ... . . . $ charDollar : num 0 0.18 0.184 0 0 0 0.054 0 0.203 0.081 ... $ charHash : num 0 0.048 0.01 0 0 0 0 0 0.022 0 ... $ capitalAve : num 3.76 5.11 9.82 3.54 3.54 ... $ capitalLong : num 61 101 485 40 40 15 4 11 445 43 ... $ capitalTotal : num 278 1028 2259 191 191 ... $ type : Factor w/ 2 levels "nonspam","spam": 2 2 2 2 2 2 2 2 2 2 ...
Some of the lines of output have been elided from the material above. You can also use the
dim function to get a quick overview of the data structure:
dim(spam)
[1] 4601 58
The
dim function shows the "dimensions" of the data structure. The output of this dim function shows that the spam data structure has 4601 rows and 58 columns. If you inspect a few of the column names that emerged from the str command, you may see that each email is coded with respect to its contents. There is lots of information available about the data set here: http://archive.ics.uci.edu/ml/datasets/Spambase
For example, just before "type" at the end of the output of the
str command on the previous page, we see a variable called "capitalTotal." This is the total number of capital letters in the whole email. Right after that is the criterion variable, "type," that indicates whether an email was classified as spam by human experts. Let’s explore this variable a bit more:
table(spam$type)
nonspam spam 2788 1813
We use the table function because type is a factor rather than a numeric variable. The output shows us that there are 2788 messages that were classified by human experts as not spam, and 1813 messages that were classified as spam. What a great dataset!
To make the analysis work we need to divide the dataset into a training set and a test set. There is no universal way to do this, but as a rule of thumb, you can use two thirds of the data set to train and the remainder to test. Let’s first generate a randomized index that will let us choose cases for our training and test sets. In the following command, we create a new list/vector variable that samples at random from a list of numbers ranging from 1 to the final element index of the spam data (4601).
randIndex <- sample(1:dim(spam)[1])
summary(randIndex)
Min. 1st Qu. Median Mean 3rd Qu. Max. 1 1151 2301 2301 3451 4601
length(randIndex)
4601
The output of the
summary() and length() commands above show that we have successfully created a list of indices ranging from 1 to 4601 and that the total length of our index list is the same as the number of rows in the spam dataset: 4601. We can confirm that the indices are randomized by looking at the first few cases:
head(randIndex)
[1] 2073 769 4565 955 3541 3357
It is important to randomize your selection of cases for the training and test sets in order to ensure that there is no systematic bias in the selection of cases. We have no way of knowing how the original dataset was sorted (if at all) - in case it was sorted on some variable of interest we do not just want to take the first 2/3rds of the cases as the training set.
Next, let’s calculate the "cut point" that would divide the spam dataset into a two thirds training set and a one third test set:
cutPoint2_3 <- floor(2 * dim(spam)[1] / 3)
cutPoint2_3
[1] 3067
The first command in this group calculates the two-thirds cut point based on the number of rows in spam (the expression dim(spam)[1] gives the number of rows in the spam dataset). The second command reveals that that cut point is 3067 rows into the data set, which seems very sensible given that there are 4601 rows in total. Note that the
floor() function chops off any decimal part of the calculation. We want to get rid of any decimal because an index variable needs to be an integer.
Now we are ready to generate our test and training sets from the original spam dataset. First we will build our training set from the first 3067 rows:
trainData <- spam[randIndex[1:cutPoint2_3],]
We make the new data set, called trainData, using the randomized set of indices that we created in the randIndex list, but only using the first 3067 elements of randIndex (The inner expression in square brackets, 1:cutPoint2_3, does the job of selecting the first 3067 elements. From here you should be able to imagine the command for creating the test set:
testData <- spam[randIndex[(cutPoint2_3+1):dim(spam)[1]],]
The inner expression now selects the rows from 3068 all the way up to 4601 for a total of 1534 rows. So now we have two separate data sets, representing a two-thirds training and one third test breakdown of the original data.
