📚 Data Mining Tutorial

Data Mining

Learn Data Mining concepts with simple explanations, examples, techniques and practical notes.

← Back to Data Mining

Basic Concepts

Classification is a supervised learning technique used to categorize data into predefined classes or labels.

In other words, classification is a data mining technique that categorizes items in a collection based on some predefined properties.

What is Classification?

Classification is used when data needs to be assigned to known categories or class labels. A model, also called a classifier, is constructed to predict the class label of given data.

Examples of Classification

  • A bank loans officer analyzes data to learn which loan applicants are “safe” and which are “risky” for the bank.
  • A marketing manager analyzes data to predict whether a customer with a given profile will buy a new computer.
  • A medical researcher analyzes breast cancer data to predict which of three specific treatments a patient should receive.

In these examples, a model or classifier predicts class labels such as:

  • “Safe” or “risky” for loan application data.
  • “Yes” or “no” for marketing data.
  • “Treatment A,” “Treatment B,” or “Treatment C” for medical data.

These categories or class labels can be represented by discrete values, where the ordering among values has no meaning.

How Does Classification Work?

Data classification is a two-step process. It consists of a learning step, where a classification model is constructed, and a classification step, where the model is used to predict class labels for given data.

1. Learning Step (Training Phase)

In this step, a classification algorithm builds the classifier or model by analyzing, or “learning from,” a training dataset and its associated class labels. The learned or trained model may be represented in the form of classification rules.

Classification learning step diagram

Fig: Learning step

2. Classification Step (Using the Model in Prediction)

In this step, test data are used to estimate the accuracy of the classification rules. If the accuracy is considered acceptable, the rules can be applied to classify new data tuples. The classifier or model predicts class labels for the new data.

Classification step using the trained model to predict class labels

Fig: Classification step

What About Classification Accuracy?

The model or classifier is used for classification, and its predictive accuracy is estimated first. If the training dataset were used to measure accuracy, the estimate would likely be optimistic because the classifier may overfit the training data. During learning, it may incorporate anomalies in the training data that are not present in the general dataset.

Therefore, a separate test dataset, made up of test tuples and their associated class labels, is used to estimate classification accuracy.

The accuracy of a classifier on a given test dataset is the percentage of test-set tuples that are correctly classified. The associated class label of each test tuple is compared with the class prediction made by the learned classifier.

Diagram illustrating the estimation of classification accuracy

Fig: Estimate accuracy