Working with Pandas DataFrames

Working with Pandas DataFrames

Introduction to Pandas DataFrames

In the realm of data science, particularly within pharmacy and biotechnology, you'll frequently encounter tabular data – think patient records, clinical trial results, or gene expression matrices. Python's Pandas library provides a powerful and intuitive data structure called the DataFrame, which is specifically designed to handle such data effectively. A Pandas DataFrame can be thought of as a two-dimensional, labeled data structure with columns of potentially different types. It's akin to a spreadsheet or a SQL table, but with much more powerful analytical capabilities integrated directly into Python. DataFrames are built on top of NumPy arrays, inheriting their efficiency, but adding features like row and column labels (indices), which make data manipulation and analysis much more readable and flexible. Understanding how to create, inspect, and manipulate DataFrames is fundamental to almost any data-driven project in your field.

Creating a DataFrame

There are several ways to create a DataFrame. One common method is from a dictionary of lists, where the keys become the column names and the lists provide the data for each column. Another frequent approach is to load data from external files like CSV (Comma Separated Values) or Excel spreadsheets. import pandas as pd import numpy as np # Creating a DataFrame from a dictionary data = { 'Patient_ID': ['P001', 'P002', 'P003', 'P004', 'P005'], 'Age': [34, 56, 29, 48, 62], 'Gender': ['Female', 'Male', 'Female', 'Male', 'Female'], 'Drug_Dosage_mg': [100, 150, 75, 120, 200], 'Response_Score': [8.5, 6.2, 9.1, 7.5, 5.8] } df_patients = pd.DataFrame(data) print("DataFrame created from dictionary:") print(df_patients) # Creating a DataFrame from a CSV file (assuming 'clinical_trial_data.csv' exists) # For demonstration, let's create a dummy CSV file first csv_content = """Trial_ID,Drug_A_Dose,Drug_B_Dose,Outcome_Measure,Side_Effects T001,50,10,12.5,None T002,75,15,10.1,Nausea T003,60,12,14.2,Fatigue T004,80,18,9.8,Headache """ with open('clinical_trial_data.csv', 'w') as f: f.write(csv_content) df_trials = pd.read_csv('clinical_trial_data.csv') print("\nDataFrame loaded from CSV:") print(df_trials.head()) # .head() shows the first 5 rows

Inspecting and Accessing DataFrames

Once you have a DataFrame, you'll want to inspect its contents, dimensions, and data types. Pandas provides several useful methods for this. You can access columns using dictionary-like notation or dot notation (if column names are valid Python identifiers). Rows can be accessed by their integer position or by their label using .loc[] and .iloc[] . # Inspecting the DataFrame print("\nDataFrame Info:") df_patients.info() print("\nDataFrame Description (summary statistics):") print(df_patients.describe()) print("\nColumn 'Age':") print(df_patients['Age']) print("\nMultiple Columns ('Patient_ID', 'Drug_Dosage_mg'):") print(df_patients[['Patient_ID', 'Drug_Dosage_mg']]) print("\nFirst row (using .iloc[0]):") print(df_patients.iloc[0]) print("\nRows with index 1 to 3 (exclusive of 4) (using .iloc[1:4]):") print(df_patients.iloc[1:4]) print("\nRows with specific Patient_ID (using boolean indexing):") print(df_patients[df_patients['Patient_ID'] == 'P003']) # Adding a new column df_patients['Dosage_per_kg'] = df_patients['Drug_Dosage_mg'] / 70 # Assuming average 70kg print("\nDataFrame with new 'Dosage_per_kg' column:") print(df_patients.head())

Key Takeaways

Pandas DataFrames are 2D labeled data structures, similar to spreadsheets, ideal for tabular data in biotech/pharmacy. They can be created from various sources, including dictionaries and external files like CSVs. .info() provides a summary of the DataFrame, including data types and non-null counts. .describe() offers descriptive statistics for numerical columns. Columns can be selected using bracket notation ( df['column'] ) or dot notation ( df.column ). Rows can be selected by integer position ( .iloc[] ) or by label ( .loc[] ). New columns can be added easily by assigning a Series or an array to a new column name.

Practice Exercise

Imagine you have collected preliminary data from a small-scale clinical trial for a new oncology drug. The data includes patient IDs, their tumor size at baseline (in cm³), tumor size after 4 weeks of treatment, and whether they experienced a severe adverse event (True/False). Create a Pandas DataFrame to store this data. Then, calculate a new column called 'Tumor_Reduction_Percent' which represents the percentage reduction in tumor size. Finally, display the average 'Tumor_Reduction_Percent' for patients who did NOT experience a severe adverse event.