Some machine learning algorithms are surprisingly intuitive. In particular, the K-Nearest-Neighbors (KNNs) model determines the classification of a point in space by checking the classification of the K closest data points to that point and taking a vote. In its simplest form, Support Vector Machines (SVMs) classify a point by drawing a line separating the classes of data points and checking which side of the line the data point falls on. My project aims to visualize the basic inner-workings of these models to provide a basis for understanding the mathematics behind them.
KNN: First, select the number of data points in the training (already classified) dataset. Select how many classifications you want the points to be divided into, and how clumped you want the points to be. Once you're ready, click 'Generate Data'. Next, set the value for K, the number of nearest neighbors considered in the tally. If you want, choose a weighting for how much each nearest neighbor's vote counts in the tally. Then, click 'Run KNN' to run the model on the generated dataset.
SVM: Like KNN, select the number of training points and the clumping of those points, then click 'Generate Data'. Select a kernel function, then 'Run SVM'.
For both models, try dragging points around to see how the classification changes as the data changes.
While these models are intuitive, the math behind them allows for extensive customization that can increase their complexity. For example, while SVM strives simply to maximize the margin between the dividing line and the data points, there are many ways to go about determining the line. This project uses gradient descent, which can be used to iteratively find the minimum of a function. In this case, we attempt to minimize the loss function (limit the number of points on the wrong side of the dividing line) while simultaneously maximizing the margin between the dividing line and the data points. However, it is more common for SVMs to use Quadratic Programming, Sequential Minimal Optimization, or Coordinate Descent for this optimization problem.
Now, consider the scenario where one classification of points was clumped in the middle of the canvas, and the other classification was spread around the perimeter of the canvas. In this case, using a dividing line to separate one class from another doesn’t really work (how can you possibly separate a circle and an inclosed circle with a single straight line?). In situations like this one (and others when a single straight line cannot reasonably separate two classifications of data), a kernel function can be used. Described simply, a kernel function maps data points into a higher-dimensional space where a single plane (hyperplane) can be used to effectively separate the data points.
Number of points:
Number of categories:
Cluster %:
Number of Nearest Neighbors (k):
Weighting:
Number of points:
Cluster %:
Kernel: