Section 19.2 How Support Vector Machines Work
An SVM maps a low dimensional problem into a higher dimensional space with the goal of being able to describe geometric boundaries between different regions. The input data (the independent variables) from a given case are processed through a "mapping" algorithm called a kernel (the kernel is simply a formula that is run on each case’s vector of input data), and the resulting kernel output determines the position of that case in multidimensional space.
A simple 2D-3D mapping example illustrates how this works: Imagine looking at a photograph of a snow-capped mountain photographed from high above the earth such that the mountain looks like a small, white circle completely surrounded by a region of green trees. Using a pair of scissors, there is no way of cutting the photo on a straight line so that all of the white snow is on one side of the cut and all of the green trees are on the other. In other words there is no simple linear separation function that could correctly separate or classify the white and green points given their 2D position on the photograph.
Next, instead of a piece of paper, think about a realistic threedimensional clay model of the mountain. Now all the white points occupy a cone at the peak of the mountain and all of the green points lie at the base of the mountain. Imagine inserting a sheet of cardboard through the clay model in a way that divides the snowcapped peak from the green-tree-covered base. It is much easier to do now because the white points are sticking up into the high altitude and the green points are all on the base of the mountain.
The position of that piece of cardboard is the planar separation function that divides white points from green points. A support vector machine analysis of this scenario would take the original two dimensional point data and search for a projection into three dimensions that would maximize the spacing between green points and white points. The result of the analysis would be a mathematical description of the position and orientation of the cardboard plane. Given inputs describing a novel data point, the SVM could then map the data into the higher dimensional space and then report whether the point was above the cardboard (a white point) or below the cardboard (a green point). The so called support vectors contain the coefficients that map the input data for each case into the high dimensional space.
