Lesson · 40 min · Free
Multi-Class Classification
Multi-Class Classification - AI in Drug Discovery body { font-family: sans-serif; line-height: 1.6; margin: 20px; } h1, h2 { color: #2c3e50; } pre { background-color: #ecf0f1; padding: 15px; border-radius: 5px; overflow-
Multi-Class Classification
In the realm of AI and machine learning, particularly within drug discovery, we frequently encounter scenarios where we need to categorize data into more than two distinct groups. This is precisely where multi-class classification comes into play. Unlike binary classification, which distinguishes between only two classes (e.g., active/inactive, toxic/non-toxic), multi-class classification aims to assign an input instance to one of several possible classes. For instance, in drug discovery, a multi-class classification model might be used to predict the specific therapeutic class of a novel compound (e.g., antibiotic, anti-inflammatory, anticancer, antiviral). Another application could be classifying compounds based on their mechanism of action (e.g., enzyme inhibitor, receptor agonist, DNA intercalator), or even predicting the specific disease indication a compound is most likely to treat from a predefined list.
Common Strategies for Multi-Class Classification
Several strategies exist for tackling multi-class classification problems, often building upon binary classification techniques. The two most prominent approaches are One-vs-Rest (OvR) and One-vs-One (OvO).
One-vs-Rest (OvR) or One-vs-All (OvA)
In the OvR strategy, we train N separate binary classifiers, where N is the number of classes. Each classifier is trained to distinguish one class from all the other classes combined. For example, if we have classes A, B, and C, we would train: Classifier 1: Distinguishes A from (B and C) Classifier 2: Distinguishes B from (A and C) Classifier 3: Distinguishes C from (A and B) During prediction, the input instance is fed to all N classifiers, and the class that receives the highest confidence score (e.g., probability) from its respective classifier is chosen as the predicted class. This method is generally straightforward to implement and is often the default multi-class strategy for many algorithms.
One-vs-One (OvO)
The OvO strategy is another popular approach, especially for algorithms that don't naturally extend to multi-class problems (like Support Vector Machines). Here, we train a binary classifier for every unique pair of classes. If there are N classes, this results in N * (N - 1) / 2 classifiers. For classes A, B, and C, we would train: Classifier 1: Distinguishes A from B Classifier 2: Distinguishes A from C Classifier 3: Distinguishes B from C During prediction, each classifier "votes" for one of the two classes it was trained to distinguish. The class that receives the most votes across all pairwise classifiers is selected as the final prediction. While this can be computationally more intensive during training due to the larger number of classifiers, it can sometimes lead to better performance, especially when class boundaries are complex. Many modern machine learning algorithms, such as Neural Networks, Decision Trees, and Gradient Boosting Machines (e.g., XGBoost, LightGBM), inherently support multi-class classification without needing to explicitly implement OvR or OvO strategies. They often achieve this by using a multi-output layer (e.g., a softmax activation function for the output layer in neural networks) that directly predicts probabilities for each class.
Code Example: Multi-Class Classification with Scikit-learn (OvR)
Here's a simple example using Scikit-learn's LogisticRegression , which by default uses the OvR strategy for multi-class problems unless specified otherwise (e.g., multi_class='multinomial' for algorithms that support it directly). import pandas as pd from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression from sklearn.metrics import accuracy_score, classification_report from sklearn.datasets import load_iris # A classic multi-class dataset # Load the Iris dataset iris = load_iris() X = iris.data y = iris.target feature_names = iris.feature_names target_names = iris.target_names # Create a DataFrame for better readability df = pd.DataFrame(X, columns=feature_names) df['species'] = pd.Categorical.from_codes(y, iris.target_names) print("First 5 rows of the dataset:") print(df.head()) # Split data into training and testing sets X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42, stratify=y) # Initialize and train a Logistic Regression model # By default, LogisticRegression uses OvR for multi_class='auto' or 'ovr' model = LogisticRegression(max_iter=200, solver='liblinear', random_state=42) model.fit(X_train, y_train) # Make predictions on the test set y_pred = model.predict(X_test) # Evaluate the model print("\nModel Accuracy:", accuracy_score(y_test, y_pred)) print("\nClassification Report:") print(classification_report(y_test, y_pred, target_names=target_names))
Code Example: Multi-Class Classification with Keras (Deep Learning)
For deep learning models, the output layer typically uses a softmax activation function to produce probabilities for each class, directly handling multi-class scenarios. import numpy as np from sklearn.model_selection import train_test_split from sklearn.preprocessing import StandardScaler, LabelEncoder from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Dense from tensorflow.keras.utils import to_categorical from sklearn.datasets import load_iris # Load the Iris dataset iris = load_iris() X = iris.data y = iris.target # Encode target labels to one-hot vectors (required for Keras multi-class output) # Example: class 0 becomes [1, 0, 0], class 1 becomes [0, 1, 0], etc. y_one_hot = to_categorical(y) # Split data into training and testing sets X_train, X_test, y_train, y_test = train_test_split(X, y_one_hot, test_size=0.3, random_state=42, stratify=y) # Standardize features (important for neural networks) scaler = StandardScaler() X_train_scaled = scaler.fit_transform(X_train) X_test_scaled = scaler.transform(X_test) # Build a simple sequential neural network model model = Sequential([ Dense(10, activation='relu', input_shape=(X_train_scaled.shape[1],)), # Input layer with 10 neurons Dense(10, activation='relu'), # Hidden layer with 10 neurons Dense(y_one_hot.shape[1], activation='softmax') # Output layer with softmax for multi-class ]) # Compile the model model.compile(optimizer='adam', loss='categorical_crossentropy', # Use categorical_crossentropy for one-hot encoded labels metrics=['accuracy']) # Train the model history = model.fit(X_train_scaled, y_train, epochs=50, batch_size=5, verbose=0, validation_split=0.2) # Evaluate the model on the test set loss, accuracy = model.evaluate(X_test_scaled, y_test, verbose=0) print(f"\nTest Loss: {loss:.4f}") print(f"Test Accuracy: {accuracy:.4f}") # Make predictions (get probabilities) predictions = model.predict(X_test_scaled) # Convert probabilities to class labels predicted_classes = np.argmax(predictions, axis=1) true_classes = np.argmax(y_test, axis=1) print("\nFirst 10 True Classes:", true_classes[:10]) print("First 10 Predicted Classes:", predicted_classes[:10])
Key Takeaways
Multi-class classification involves categorizing data into more than two distinct classes. Common strategies include One-vs-Rest (OvR) , where a binary classifier is trained for each class against all others, and One-vs-One (OvO) , where a classifier is trained for each pair of classes. Many advanced algorithms (e.g., Neural Networks, Decision Trees, Gradient Boosting) have native support for multi-class classification. In deep learning, the output layer typically uses a softmax activation function for multi-class probability prediction, often paired with categorical cross-entropy loss . Evaluation metrics like accuracy, precision, recall, and F1-score (often calculated per-class and then averaged) are crucial for assessing multi-class models.
Practice Exercise
Imagine you are working on a project to classify novel drug compounds based on their primary target protein family. You have a dataset of compounds labeled with one of five protein families: "Kinase Inhibitor", "GPCR Modulator", "Ion Channel Blocker", "Nuclear Receptor Agonist", and "Protease Inhibitor". Describe how you would approach building a multi-class classification model for this task. Specifically, discuss: What features (molecular descriptors, etc.) would you likely use as input to your model? Which multi-class strategy (OvR, OvO, or native multi-class support) might be most appropriate for a complex dataset like this, and why? What specific machine learning algorithm(s) would you consider, and why? What key metrics would you use to evaluate the performance of your model, and why are they important in this context?
Watch the full lesson — free
This topic is part of AI in Drug Discovery, a complete AI-narrated video course. Press play once and watch the entire lecture like a movie.
Start the course free →