Lesson · 40 min · Free
MLP & Own Framework
MLP & Own Framework body { font-family: sans-serif; line-height: 1.6; margin: 20px; } h1, h2 { color: #2c3e50; } pre { background-color: #ecf0f1; padding: 15px; border-radius: 5px; overflow-x: auto; } code { font-family:
MLP & Own Framework
Welcome to this advanced topic in our Python Programming course, tailored for pharmacy and biotech students. Today, we'll delve into the fascinating world of Multilayer Perceptrons (MLPs) and, more importantly, explore the foundational concepts by building a simplified neural network framework from scratch. While libraries like TensorFlow and PyTorch abstract away much of the complexity, understanding the underlying mechanics is crucial for effective model design, debugging, and for appreciating the power of these tools in drug discovery, genomics, and personalized medicine. An MLP is a type of artificial neural network characterized by multiple layers of perceptrons (neurons) arranged in a feedforward manner. This means information flows in one direction, from the input layer, through one or more hidden layers, to the output layer. Each connection between neurons has an associated weight, and each neuron applies an activation function to the weighted sum of its inputs. This architecture allows MLPs to learn complex non-linear relationships in data, making them highly versatile for tasks like classifying patient data, predicting drug efficacy, or identifying biomarkers. Our "own framework" will be a minimalistic Python implementation that demonstrates the core components: layers, activation functions, forward propagation, and a simplified backpropagation for learning. We won't implement full-fledged optimization algorithms like Adam or RMSprop, but focus on the fundamental matrix operations and gradient calculations that form the backbone of neural network training.
Building Blocks of Our Neural Network Framework
Let's start by defining a simple Dense (fully connected) layer. This layer will take inputs, multiply them by a weight matrix, and add a bias vector. We'll also need an activation function. For simplicity, we'll use the Sigmoid function, which squashes values between 0 and 1, often used in the output layer for binary classification, or in hidden layers in older networks. import numpy as np # --- Activation Functions --- class Sigmoid: def forward(self, x): self.output = 1 / (1 + np.exp(-x)) return self.output def backward(self, grad_output): # Derivative of sigmoid: sigma * (1 - sigma) return grad_output * self.output * (1 - self.output) # --- Layers --- class Dense: def __init__(self, input_size, output_size, learning_rate=0.01): self.weights = np.random.randn(input_size, output_size) * 0.01 # Small random weights self.biases = np.zeros(output_size) self.learning_rate = learning_rate def forward(self, input_data): self.input = input_data # Store input for backward pass self.output = np.dot(self.input, self.weights) + self.biases return self.output def backward(self, grad_output): # Calculate gradients grad_weights = np.dot(self.input.T, grad_output) grad_biases = np.sum(grad_output, axis=0) grad_input = np.dot(grad_output, self.weights.T) # Update weights and biases (simplified gradient descent) self.weights -= self.learning_rate * grad_weights self.biases -= self.learning_rate * grad_biases return grad_input Now, let's assemble these components into a simple MLP. We'll define a Network class that will hold our layers and manage the forward and backward passes. For our example, we'll implement a simple Mean Squared Error (MSE) loss function, which is suitable for regression tasks, and its corresponding derivative. # --- Loss Function --- class MSE: def forward(self, y_true, y_pred): return np.mean(np.power(y_true - y_pred, 2)) def backward(self, y_true, y_pred): return 2 * (y_pred - y_true) / y_true.size # --- Network Class --- class Network: def __init__(self): self.layers = [] self.loss_function = MSE() def add(self, layer): self.layers.append(layer) def predict(self, input_data): output = input_data for layer in self.layers: output = layer.forward(output) return output def train(self, X_train, y_train, epochs): for i in range(epochs): # Forward pass y_pred = self.predict(X_train) # Calculate loss loss = self.loss_function.forward(y_train, y_pred) # Backward pass grad = self.loss_function.backward(y_train, y_pred) for layer in reversed(self.layers): grad = layer.backward(grad) if i % 100 == 0: print(f"Epoch {i}, Loss: {loss:.4f}") # --- Example Usage --- # Simple dataset for demonstration (e.g., predicting a drug's absorption based on two features) X_train = np.array([[0,0], [0,1], [1,0], [1,1]]) y_train = np.array([[0], [1], [1], [0]]) # XOR-like problem, requires non-linearity # Create and train the network mlp = Network() mlp.add(Dense(2, 3, learning_rate=0.1)) # Input layer (2 features) to hidden layer (3 neurons) mlp.add(Sigmoid()) # Activation for hidden layer mlp.add(Dense(3, 1, learning_rate=0.1)) # Hidden layer (3 neurons) to output layer (1 neuron) mlp.add(Sigmoid()) # Activation for output layer print("Training our custom MLP...") mlp.train(X_train, y_train, epochs=2000) print("\nPredictions after training:") for x in X_train: prediction = mlp.predict(x.reshape(1, -1)) # Reshape for single input print(f"Input: {x}, Predicted Output: {prediction[0][0]:.4f}") In the example above, we've created a simple MLP with one hidden layer to tackle an XOR-like problem. This problem is famously not linearly separable, demonstrating the need for non-linear activation functions like Sigmoid. The train method iterates through epochs, performing a forward pass to get predictions, calculating the loss, and then executing a backward pass to update the weights and biases of each layer using the computed gradients. This iterative process of minimizing the loss function is how neural networks "learn." For pharmacy and biotech applications, you would replace this toy dataset with real-world data, such as patient demographics and genetic markers to predict disease susceptibility, or molecular descriptors to predict drug-target interactions. The principles of layers, activation functions, forward propagation, and backpropagation remain the same, though the scale and complexity would increase dramatically, necessitating the use of specialized libraries.
Key Takeaways:
MLPs are feedforward neural networks with multiple layers, capable of learning complex non-linear relationships. Core components include Dense layers (weight matrices, bias vectors) and activation functions (e.g., Sigmoid, ReLU). Forward propagation computes outputs by passing data through layers sequentially. Backward propagation (backpropagation) calculates gradients of the loss with respect to weights and biases, enabling parameter updates. Building a simple framework from scratch helps to demystify how neural networks work at a fundamental level. This foundational understanding is invaluable even when using high-level libraries for real-world pharmacy/biotech problems. Practice Exercise: Modify the provided Network class and its components. Instead of the Sigmoid activation function, implement and integrate the Rectified Linear Unit (ReLU) activation function. The ReLU function is defined as f(x) = max(0, x) and its derivative is 1 for x > 0 and 0 for x . After implementing ReLU, replace the Sigmoid layers in the example usage with your new ReLU layer and observe how the training loss and predictions change. What advantages might ReLU offer over Sigmoid in certain scenarios, especially considering its derivative?
Watch the full lesson — free
This topic is part of Python Programming - Basics, a complete AI-narrated video course. Press play once and watch the entire lecture like a movie.
Start the course free →