Lesson · 40 min · Free
Feature Attribution: SHAP and Partial Dependence
Feature Attribution: SHAP and Partial Dependence body { font-family: sans-serif; line-height: 1.6; margin: 20px; } h1, h2 { color: #2c3e50; } pre { background-color: #ecf0f1; padding: 15px; border-radius: 5px; overflow-x
Feature Attribution: SHAP and Partial Dependence
Welcome to this lesson on Feature Attribution, a critical component of building Trustworthy AI in healthcare. As pharmacists and biotech professionals, understanding why an AI model makes a particular prediction is as important as the prediction itself, especially when clinical decisions or drug discovery efforts are at stake. This lesson will introduce you to two powerful techniques for interpreting complex machine learning models: SHAP (SHapley Additive exPlanations) and Partial Dependence Plots (PDPs). In healthcare, AI models are increasingly used for tasks like disease diagnosis, patient risk stratification, and predicting drug efficacy or adverse events. However, the inherent "black box" nature of many sophisticated models (e.g., deep neural networks, complex ensemble methods) can hinder their adoption and trust. Feature attribution methods aim to open this black box, providing insights into which input features contribute most to a model's output, and how they influence it.
Understanding Model Decisions with SHAP and Partial Dependence
Partial Dependence Plots (PDPs) offer a way to visualize the marginal effect of one or two features on the predicted outcome of a machine learning model. They show whether the relationship between the target and a feature is linear, monotonic, or more complex. For example, a PDP could reveal how the probability of a patient developing a certain disease changes as their age increases, while averaging out the effects of all other features. PDPs are intuitive and easy to understand, making them valuable for communicating model insights to clinicians and researchers. However, PDPs assume that the features are independent, which is often not the case in real-world healthcare data. They also provide a global interpretation, showing the average effect across the entire dataset, rather than individual predictions. This is where SHAP values come into play. SHAP (SHapley Additive exPlanations) is a game-theoretic approach to explain the output of any machine learning model. It connects optimal credit allocation with local explanations using Shapley values from cooperative game theory. SHAP values quantify the contribution of each feature to a single prediction, indicating how much each feature pushes the prediction from the base value (the average prediction) to the actual predicted value. This local interpretability is incredibly powerful in healthcare, allowing us to understand why an AI predicted a specific diagnosis for a particular patient, considering their unique clinical profile. SHAP values offer several advantages: they are locally accurate (the sum of SHAP values for all features equals the difference between the prediction and the base value), consistent (if a feature contributes more to one model than another, its SHAP value will reflect that), and model-agnostic (can be applied to any machine learning model). They can be used to generate global interpretations by aggregating local SHAP values, such as SHAP summary plots or dependence plots, which overcome some limitations of traditional PDPs by accounting for feature interactions.
Code Example: Partial Dependence Plot
Let's consider a simplified example where we train a Random Forest Regressor to predict a patient's response to a new drug, based on their age and BMI. We'll use synthetic data for demonstration. import numpy as np import pandas as pd from sklearn.ensemble import RandomForestRegressor from sklearn.inspection import plot_partial_dependence import matplotlib.pyplot as plt # Generate synthetic data for drug response prediction np.random.seed(42) n_samples = 1000 age = np.random.randint(20, 80, n_samples) bmi = np.random.uniform(18, 35, n_samples) # Simulate a non-linear relationship and interaction drug_response = (0.5 * age + 2 * bmi - 0.01 * age**2 + np.random.normal(0, 5, n_samples) + (0.1 * age * (bmi > 25)) ) X = pd.DataFrame({'Age': age, 'BMI': bmi}) y = drug_response # Train a Random Forest Regressor model = RandomForestRegressor(n_estimators=100, random_state=42) model.fit(X, y) # Plot Partial Dependence for 'Age' and 'BMI' fig, ax = plt.subplots(figsize=(10, 5)) plot_partial_dependence(model, X, features=['Age', 'BMI'], target=y, grid_resolution=50, ax=ax, feature_names=['Age (years)', 'BMI (kg/m^2)']) fig.suptitle("Partial Dependence Plots for Drug Response") plt.tight_layout(rect=[0, 0.03, 1, 0.95]) plt.show() This code will generate two plots, showing how the predicted drug response changes as 'Age' varies (while 'BMI' is averaged out) and how it changes as 'BMI' varies (while 'Age' is averaged out). You might observe non-linear relationships, suggesting that the drug's effect isn't constant across all ages or BMI ranges.
Code Example: SHAP Explanations
Now, let's use SHAP to explain a specific prediction from our trained model. We'll explain why a particular patient received a certain drug response prediction. import shap # Assuming 'model', 'X', 'y' are already defined from the previous example # Create a SHAP explainer for tree-based models explainer = shap.TreeExplainer(model) # Select a specific patient to explain (e.g., the 5th patient in our dataset) patient_index = 4 patient_data = X.iloc[[patient_index]] # Calculate SHAP values for this specific patient shap_values = explainer.shap_values(patient_data) # Visualize the explanation for this patient # The base value is the average model output over the training dataset base_value = explainer.expected_value print(f"Base value (average drug response): {base_value:.2f}") print(f"Actual prediction for patient {patient_index}: {model.predict(patient_data)[0]:.2f}") print(f"SHAP values for patient {patient_index}:") for feature, shap_val in zip(X.columns, shap_values[0]): print(f" {feature}: {shap_val:.2f}") shap.initjs() # For interactive JS plots in notebooks shap.plots.force(base_value, shap_values[0], patient_data, matplotlib=True) plt.title(f"SHAP Explanation for Patient {patient_index}") plt.show() # Another common plot: SHAP summary plot for global understanding # This shows feature importance and impact direction across the dataset shap_values_all = explainer.shap_values(X) shap.summary_plot(shap_values_all, X, plot_type="bar", show=False) plt.title("SHAP Feature Importance (Average Absolute SHAP Value)") plt.show() shap.summary_plot(shap_values_all, X, show=False) plt.title("SHAP Summary Plot (Feature Impact and Direction)") plt.show() The force plot for a single patient will visually show how each feature's value pushes the prediction away from the base value. For example, it might show that for this specific patient, their high BMI increased the predicted drug response, while their age had a slightly negative impact. The summary plots provide a global view: the bar plot shows average absolute SHAP values (feature importance), and the dot plot shows the distribution of SHAP values for each feature, indicating whether a high or low feature value tends to increase or decrease the prediction. In a healthcare context, this level of detail is invaluable. For instance, if an AI model predicts a high risk of adverse drug reaction for a patient, SHAP can pinpoint exactly which patient characteristics (e.g., specific genetic markers, co-morbidities, or drug interactions) are driving that prediction. This allows clinicians to validate the AI's reasoning, intervene appropriately, or even refine treatment plans. Key Takeaways: Feature attribution methods are crucial for building trust and transparency in AI models, especially in high-stakes domains like healthcare. Partial Dependence Plots (PDPs) show the average marginal effect of one or two features on the model's prediction, providing a global, intuitive understanding of feature relationships. SHAP (SHapley Additive exPlanations) provides local, individual-prediction explanations by calculating Shapley values, quantifying each feature's contribution to a specific prediction. SHAP offers consistency, local accuracy, and model-agnosticism, making it a robust choice for explaining complex AI models in healthcare. Both PDPs and SHAP can reveal non-linear relationships and feature interactions, helping healthcare professionals understand the nuances of AI predictions.
Practice Exercise:
Imagine you are developing an AI model to predict the likelihood of a patient developing antibiotic resistance based on their medical history (e.g., number of previous antibiotic courses, duration of hospital stays, presence of certain comorbidities). Discuss how you would use both Partial Dependence Plots and SHAP values to interpret this model for a team of infectious disease specialists. Specifically, explain what insights each method would provide and how those insights could be used to improve clinical decision-making or model trustworthiness. Consider a scenario where a patient has an unexpectedly high predicted risk of resistance – how would SHAP help you explain this to the treating physician?
Watch the full lesson — free
This topic is part of AI in Healthcare: Diagnosis to Drug Discovery — The Trustworthy AI Track, a complete AI-narrated video course. Press play once and watch the entire lecture like a movie.
Start the course free →