World Robot Olympiad Online Coaching for Southeast Asia Students

1. Introduction: The Most Popular Data Structure in Pandas

Building on the one-dimensional Pandas Series from the previous module, this lecture covers the central and most widely used structure in Pandas: the DataFrame.

In AI and machine learning, datasets rarely arrive as isolated lists of numbers; they come as multi-attribute tables from clients and databases. The DataFrame is designed to import, store, manipulate, and prepare these datasets for AI and ML workflows.

2. What Is a DataFrame?

A DataFrame is a two-dimensional (2D) labeled data structure:

• Tabular arrangement: Data is organized in rows and columns, like a spreadsheet or an SQL table.
• Mixed data types: Unlike homogeneous numerical matrices, each column can have its own type (text, integers, decimals, etc.).
• Real-world suitability: Names, dates, quantities, and monetary values can live side by side, which matches how real datasets are structured.

3. Creating a DataFrame from Scratch in Jupyter Notebook

Import Pandas as pd, put the data in a dictionary, and pass it to pd.DataFrame():

import pandas as pd

# 1. Defining a dictionary containing data for 3 individuals
data = {
    "Name": ["Ali", "Sara", "Ahmed"],
    "Age": [25, 30, 22],
    "City": ["Lahore", "Karachi", "Islamabad"]
}

# 2. Converting the dictionary into a Pandas DataFrame
df = pd.DataFrame(data)

# 3. Displaying the resulting table
print(df)

Resulting structure:

• Column labels come from the dictionary keys: Name, Age, City.
• Row labels (index) default to 0, 1, 2 when no custom index is given.

    Name  Age       City
0    Ali   25     Lahore
1   Sara   30    Karachi
2  Ahmed   22  Islamabad

4. Loading Real-World Datasets from Disk (pd.read_csv)

DataFrames can load large files from disk. CSV (Comma-Separated Values) is a common format. pd.read_csv() reads a CSV into a DataFrame in memory:

# Loading a large CSV dataset from disk
df_sales = pd.read_csv("retail_sales.csv")

In a sample file such as retail_sales.csv, Pandas can load on the order of 1,825 rows across six columns—for example Date, Category, Sales, Quantity, Profit, and Region. In AI and machine learning these columns are called features.

5. Essential Inspection Methods and Attributes

Once data is in a DataFrame, built-in tools reveal structure, size, and distributions:

A. Checking Dataset Architecture: df.info()

# Displaying a structural overview of the dataset
df_sales.info()

• Confirms column count and indices (e.g. 0–5 for six columns).
• Reports non-null counts per column.
• Identifies types: e.g. Date and Category as object (text), Sales and Profit as floats, Quantity as integers, Region as object.

B. Checking Dimensions & Types: df.shape and df.dtypes

# 1. Checking the exact dimensions of the dataset
print("Shape (Rows, Columns):", df_sales.shape)

# 2. Inspecting the data types of all feature columns
print("\nData Types:")
print(df_sales.dtypes)

• df.shape returns a tuple such as (1825, 6) — 1,825 rows and 6 columns.
• df.dtypes lists the type of each column (e.g. float64, int64, object).

C. Statistical Summaries of Numerical Features: df.describe()

# Generating descriptive statistics for numerical features
print(df_sales.describe())

.describe() summarizes numerical columns (e.g. Sales, Quantity, Profit) and skips non-numeric text columns. For each numerical feature it reports:

1. count — number of valid observations
2. mean — arithmetic average
3. std — standard deviation (spread)
4. min — minimum value
5. 25% — first quartile
6. 50% — median
7. 75% — third quartile
8. max — maximum value

This gives a fast view of ranges and distributions for numerical data.