Lesson · 40 min · Free
TF & PyTorch Model Creation
TF & PyTorch Model Creation body { font-family: sans-serif; line-height: 1.6; margin: 20px; } h1, h2 { color: #2c3e50; } pre { background-color: #ecf0f1; padding: 15px; border-radius: 5px; overflow-x: auto; } code { font
TF & PyTorch Model Creation
In the rapidly evolving field of AI in Drug Discovery, the ability to construct and deploy machine learning models is paramount. Two of the most dominant and widely used frameworks for deep learning are TensorFlow (TF) and PyTorch. Both offer robust ecosystems for building, training, and evaluating complex neural networks, but they approach model definition and execution with slightly different philosophies. Understanding their core principles and practical application is crucial for any biotech or pharmacy professional looking to leverage AI for drug development. This lesson will introduce you to the fundamental concepts of defining neural network architectures using both TensorFlow's Keras API and PyTorch's torch.nn module. We'll focus on building simple feed-forward neural networks, which form the basis for more complex architectures often used in areas like molecular property prediction, target identification, and drug-target interaction modeling. While the examples here are simplified, the underlying principles extend directly to more sophisticated models.
Building Neural Networks with TensorFlow Keras and PyTorch
TensorFlow, particularly through its high-level Keras API, emphasizes a user-friendly and declarative approach to model building. Keras allows you to stack layers sequentially or define more complex architectures using the Functional API. This makes it very intuitive for beginners to quickly prototype models. PyTorch, on the other hand, is known for its imperative and Pythonic style, offering more fine-grained control over the computation graph. While Keras might feel more "plug-and-play," PyTorch often provides greater flexibility for custom layers, loss functions, and training loops, which can be beneficial for cutting-edge research in drug discovery. Let's start by looking at how to define a simple feed-forward neural network for a hypothetical binary classification task (e.g., predicting if a compound is active or inactive against a target).
TensorFlow Keras Example: Sequential Model
In Keras, the Sequential model is the simplest way to build a neural network where layers are stacked one after the other. import tensorflow as tf from tensorflow.keras import layers, models # Define a simple feed-forward neural network using the Sequential API tf_model = models.Sequential([ layers.Input(shape=(128,)), # Input layer expecting 128 features (e.g., molecular descriptors) layers.Dense(64, activation='relu', name='hidden_layer_1'), # First hidden layer with 64 neurons, ReLU activation layers.Dropout(0.2), # Dropout layer for regularization layers.Dense(32, activation='relu', name='hidden_layer_2'), # Second hidden layer with 32 neurons, ReLU activation layers.Dense(1, activation='sigmoid', name='output_layer') # Output layer for binary classification, sigmoid activation ]) # Compile the model tf_model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy']) # Print a summary of the model architecture tf_model.summary() In this TensorFlow example, we define an input layer for 128 features, two hidden layers with ReLU activation, a dropout layer to prevent overfitting, and an output layer with a single neuron and sigmoid activation for binary classification. The model is then compiled, specifying the optimizer, loss function, and evaluation metrics.
PyTorch Example: torch.nn.Module
In PyTorch, models are typically defined by creating a class that inherits from torch.nn.Module . You define the layers in the __init__ method and specify the forward pass (how data flows through the network) in the forward method. import torch import torch.nn as nn import torch.optim as optim # Define a simple feed-forward neural network using nn.Module class PyTorchModel(nn.Module): def __init__(self, input_size=128, hidden_size_1=64, hidden_size_2=32, output_size=1): super(PyTorchModel, self).__init__() self.fc1 = nn.Linear(input_size, hidden_size_1) # First fully connected layer self.relu1 = nn.ReLU() # ReLU activation self.dropout = nn.Dropout(0.2) # Dropout layer self.fc2 = nn.Linear(hidden_size_1, hidden_size_2) # Second fully connected layer self.relu2 = nn.ReLU() # ReLU activation self.fc3 = nn.Linear(hidden_size_2, output_size) # Output fully connected layer self.sigmoid = nn.Sigmoid() # Sigmoid activation for binary classification def forward(self, x): x = self.fc1(x) x = self.relu1(x) x = self.dropout(x) x = self.fc2(x) x = self.relu2(x) x = self.fc3(x) x = self.sigmoid(x) return x # Instantiate the model pytorch_model = PyTorchModel() # Print the model architecture print(pytorch_model) # Define a loss function and an optimizer (similar to Keras compile step) criterion = nn.BCELoss() # Binary Cross-Entropy Loss optimizer = optim.Adam(pytorch_model.parameters(), lr=0.001) The PyTorch example achieves the same network architecture. Notice how the layers are defined as attributes in __init__ and then explicitly called in the forward method, defining the computational flow. This imperative style provides immense flexibility, allowing for complex data-dependent operations within the forward pass that might be more challenging to express in Keras's declarative style. Both frameworks provide extensive documentation and community support, and the choice between them often comes down to personal preference, project requirements, and existing team expertise. For drug discovery, both have been successfully applied to a wide array of problems, from virtual screening to de novo drug design.
Key Takeaways
TensorFlow (via Keras) and PyTorch are leading deep learning frameworks. Keras offers a high-level, declarative API for quick model prototyping (e.g., Sequential model). PyTorch provides an imperative, Pythonic style for more fine-grained control and flexibility (e.g., nn.Module ). Both frameworks allow defining common neural network layers like Dense (TF/Keras) or Linear (PyTorch), activation functions (e.g., ReLU, Sigmoid), and regularization techniques (e.g., Dropout). Model compilation (TF/Keras) or defining loss and optimizer (PyTorch) are crucial steps before training.
Practice Exercise
Modify one of the provided code examples (either TensorFlow Keras or PyTorch) to create a neural network for a multiclass classification task. Assume you are classifying a compound into one of 5 different therapeutic classes. This would involve changing the output layer's number of neurons and activation function, as well as the appropriate loss function. Briefly explain the changes you made and why they are necessary for multiclass classification.
Watch the full lesson — free
This topic is part of AI in Drug Discovery, a complete AI-narrated video course. Press play once and watch the entire lecture like a movie.
Start the course free →