Section 19.1 Supervised vs. Unsupervised Learning
From the previous chapter you may remember that data mining techniques fall into two large categories: supervised learning techniques and unsupervised learning techniques. The association rules mining examined in the previous chapter was an unsupervised technique. This means that there was no particular criterion that we were trying to predict, rather we were just looking for patterns that would emerge from the data naturally.
In the present chapter we will examine a supervised learning technique called "support vector machines." Why the technique is called this we will examine shortly. The reason this is considered a supervised learning technique is that we "train" the algorithm on an initial set of data (the "supervised" phase) and then we test it out on a brand new set of data. If the training we accomplished worked well, then the algorithm should be able to predict the right outcome most of the time in the test data.
Take the weather as a simple example. Some days are cloudy, some are sunny. The barometer rises some days and fall others. The wind may be strong or weak and it may come from various directions. If we collect data on a bunch of days and use those data to train a machine learning algorithm, the algorithm may find that cloudy days with a falling barometer and the wind from the east may signal that it is likely to rain. Next, we can collect more data on some other days and see how well our algorithm does at predicting rain on those days. The algorithm will make mistakes. The percentage of mistakes is the error rate, and we would like the error rate to be as low as possible.
This is the basic strategy of supervised machine learning: Have a substantial number of training cases that the algorithm can use to discover and mimic the underlying pattern and then use the results of that process on a test data set in order to find out how well the algorithm and parameters perform in a "cross validation." Cross validation, in this instance, refers to the process of verifying that the trained algorithm can carry out is prediction or classification task accurately on novel data.
In this chapter, we will develop a "support vector machine" (SVM) to classify emails into spam or not spam.
