Classification Basics: Separating Classes and Generalization

Classification assigns an object to a known class. In Introduction to Data Mining (2nd Edition) — Tan, Steinbach, Karpatne, and Kumar, a separating hyperplane illustrates how a model can divide classes in attribute space.

Core ideas

• Binary classification separates two classes using features and a decision boundary.
• Linear separability exists when one hyperplane can separate the classes.
• Training error alone is insufficient: a model can fit training data perfectly and still generalize poorly.
• Model complexity, training size, selection, and performance evaluation all affect generalization.

Prefer a model that performs well on unseen data, not merely one that memorizes the training set. Watch for overfitting and underfitting, and use suitable evaluation procedures.