Lesson · 40 min · Free
Classification Algorithms with Python
Classification Algorithms with Python body { font-family: sans-serif; line-height: 1.6; margin: 20px; } h1, h2 { color: #2c3e50; } pre { background-color: #ecf0f1; padding: 15px; border-radius: 5px; overflow-x: auto; mar
Classification Algorithms with Python
Welcome to this lesson on Classification Algorithms using Python, a crucial skill for pharmaceutical research. In drug discovery and development, we frequently encounter scenarios where we need to categorize data into predefined classes. For instance, classifying compounds as active or inactive, predicting patient response to a treatment (responder vs. non-responder), or diagnosing diseases based on clinical markers. Classification algorithms provide the statistical and computational tools to perform these tasks systematically and robustly. At an upper-undergraduate level, it's important to understand not just how to run these algorithms, but also their underlying principles and when to apply them. We'll focus on practical applications using Python's powerful libraries, particularly scikit-learn , which provides a unified interface for many machine learning models.
Introduction to Classification Algorithms
Classification is a supervised machine learning task where the goal is to predict a categorical label for new, unseen data based on patterns learned from labeled training data. Unlike regression, which predicts continuous values, classification predicts discrete classes. Common classification algorithms include Logistic Regression, Decision Trees, Random Forests, Support Vector Machines (SVMs), and K-Nearest Neighbors (KNN). For pharmaceutical applications, the choice of algorithm often depends on the nature of the data, the interpretability requirements, and the desired performance metrics. For example, a highly interpretable model like a Decision Tree might be preferred for explaining drug-response mechanisms, while an ensemble method like Random Forest might offer higher predictive accuracy for screening large compound libraries. Let's start with a simple example: classifying whether a patient is likely to respond to a new experimental drug based on some biological markers. We'll use a synthetic dataset for demonstration purposes. import numpy as np import pandas as pd from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression from sklearn.metrics import accuracy_score, classification_report # 1. Generate synthetic data for demonstration np.random.seed(42) num_samples = 100 data = { 'Bio_Marker_A': np.random.normal(loc=5, scale=1.5, size=num_samples), 'Bio_Marker_B': np.random.normal(loc=10, scale=2.0, size=num_samples), 'Age': np.random.randint(20, 70, size=num_samples), 'Drug_Response': np.random.choice([0, 1], size=num_samples, p=[0.4, 0.6]) # 0: Non-responder, 1: Responder } df = pd.DataFrame(data) # Introduce some correlation: higher Bio_Marker_A and lower Age tend to be responders df.loc[df['Drug_Response'] == 1, 'Bio_Marker_A'] += np.random.normal(loc=1, scale=0.5, size=df['Drug_Response'].sum()) df.loc[df['Drug_Response'] == 1, 'Age'] -= np.random.randint(0, 10, size=df['Drug_Response'].sum()) # 2. Prepare the data X = df[['Bio_Marker_A', 'Bio_Marker_B', 'Age']] # Features y = df['Drug_Response'] # Target variable # Split data into training and testing sets X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42, stratify=y) print("Training set shape:", X_train.shape, y_train.shape) print("Testing set shape:", X_test.shape, y_test.shape) # 3. Choose and train a classification model (Logistic Regression) model = LogisticRegression(random_state=42) model.fit(X_train, y_train) # 4. Make predictions y_pred = model.predict(X_test) # 5. Evaluate the model accuracy = accuracy_score(y_test, y_pred) report = classification_report(y_test, y_pred) print(f"\nModel Accuracy: {accuracy:.2f}") print("\nClassification Report:\n", report) # Example of predicting a new patient new_patient_data = pd.DataFrame([[6.5, 11.0, 45]], columns=['Bio_Marker_A', 'Bio_Marker_B', 'Age']) prediction = model.predict(new_patient_data) prediction_proba = model.predict_proba(new_patient_data) print(f"\nPrediction for new patient (0: Non-responder, 1: Responder): {prediction[0]}") print(f"Probability of being Non-responder: {prediction_proba[0][0]:.2f}") print(f"Probability of being Responder: {prediction_proba[0][1]:.2f}") In this example, we used Logistic Regression . Despite its name, Logistic Regression is a classification algorithm that models the probability of a binary outcome. It's often a good starting point due to its simplicity and interpretability. We split our data into training and testing sets to ensure our model generalizes well to unseen data, a critical step to avoid overfitting. Evaluation metrics like accuracy and the classification report (which includes precision, recall, and F1-score) are essential for understanding model performance, especially in imbalanced datasets where simple accuracy can be misleading. For instance, in drug screening, false negatives (missing an active compound) might be more costly than false positives, requiring a focus on recall. Let's consider another powerful algorithm: Random Forest . Random Forests are ensemble learning methods that construct a multitude of decision trees at training time and output the class that is the mode of the classes (classification) or mean prediction (regression) of the individual trees. They are known for their high accuracy and ability to handle complex datasets, including those with many features and non-linear relationships, which are common in biological data. import numpy as np import pandas as pd from sklearn.model_selection import train_test_split from sklearn.ensemble import RandomForestClassifier from sklearn.metrics import accuracy_score, classification_report, roc_auc_score, roc_curve import matplotlib.pyplot as plt # For plotting ROC curve # Re-using the synthetic data from the previous example # (Assuming df, X, y, X_train, X_test, y_train, y_test are already defined) # 1. Choose and train a classification model (Random Forest) rf_model = RandomForestClassifier(n_estimators=100, random_state=42, class_weight='balanced') rf_model.fit(X_train, y_train) # 2. Make predictions y_pred_rf = rf_model.predict(X_test) y_proba_rf = rf_model.predict_proba(X_test)[:, 1] # Probability of the positive class (Responder) # 3. Evaluate the model accuracy_rf = accuracy_score(y_test, y_pred_rf) report_rf = classification_report(y_test, y_pred_rf) roc_auc_rf = roc_auc_score(y_test, y_proba_rf) print(f"\nRandom Forest Model Accuracy: {accuracy_rf:.2f}") print("\nRandom Forest Classification Report:\n", report_rf) print(f"\nRandom Forest ROC AUC Score: {roc_auc_rf:.2f}") # Plotting ROC Curve (Receiver Operating Characteristic) fpr, tpr, thresholds = roc_curve(y_test, y_proba_rf) plt.figure(figsize=(8, 6)) plt.plot(fpr, tpr, color='darkorange', lw=2, label=f'ROC curve (area = {roc_auc_rf:.2f})') plt.plot([0, 1], [0, 1], color='navy', lw=2, linestyle='--') plt.xlim([0.0, 1.0]) plt.ylim([0.0, 1.05]) plt.xlabel('False Positive Rate') plt.ylabel('True Positive Rate') plt.title('Receiver Operating Characteristic (ROC) Curve') plt.legend(loc="lower right") plt.grid(True) plt.show() # Feature Importance (unique to tree-based models) feature_importances = pd.Series(rf_model.feature_importances_, index=X.columns).sort_values(ascending=False) print("\nFeature Importances from Random Forest:\n", feature_importances) The Random Forest example introduces n_estimators (number of trees) and class_weight='balanced' , which is useful for handling imbalanced datasets where one class has significantly fewer samples than others – a common scenario in pharmaceutical research (e.g., rare diseases, successful drug candidates). We also introduced the ROC AUC score and plot, which are critical for evaluating classifiers, especially when dealing with varying thresholds and the trade-off between sensitivity and specificity. Understanding feature importance from models like Random Forest can provide valuable insights into which biological markers or patient characteristics are most predictive of the outcome, aiding in mechanistic understanding or biomarker discovery.
Key Takeaways
Classification Algorithms predict categorical labels and are vital for tasks like disease diagnosis, drug response prediction, and compound screening in pharmaceutical research. Python's scikit-learn library provides a comprehensive and user-friendly toolkit for implementing various classification models. Logistic Regression is a simple yet powerful baseline classifier, modeling the probability of a binary outcome. Random Forest is an ensemble method known for high accuracy, robustness, and ability to handle complex, non-linear data; it also provides feature importance. Data Splitting (Train/Test) is crucial to evaluate model generalization and prevent overfitting. Evaluation Metrics like accuracy, precision, recall, F1-score, and ROC AUC are essential for a comprehensive assessment of model performance, especially in imbalanced datasets. Understanding the strengths and weaknesses of different algorithms helps in choosing the most appropriate model for a specific pharmaceutical problem.
Practice Exercise
Using the provided synthetic dataset (or creating your own with similar structure), implement a K-Nearest Neighbors (KNN) classifier. Train the model, make predictions, and evaluate its performance using accuracy and a classification report. Compare its performance to the Logistic Regression and Random Forest models. Consider how you might choose the optimal 'k' (number of neighbors) for KNN and discuss its potential advantages or disadvantages in a pharmaceutical context, particularly regarding interpretability or handling noisy data.
Watch the full lesson — free
This topic is part of Python for Pharmaceutical Research, a complete AI-narrated video course. Press play once and watch the entire lecture like a movie.
Start the course free →