Data Analysis with pandas · pandas Fundamentals
Adding and Transforming Columns
Create new columns with vectorised arithmetic, conditions with np.where and cut, mappings, and method chains with assign, avoiding slow row loops.
What you'll build
Derived columns for the orders table: line totals, VAT, a size category, a weekday, and a region looked up from a mapping, written as one readable method chain.
The idea
- Vectorised arithmetic on columns is fast and readable:
df["total"] = df.qty * df.price. - Conditions:
np.where(cond, a, b)for two outcomes,pd.cutfor numeric ranges,Series.map(dict)for lookups. .dtaccessor for dates:.dt.day_name(),.dt.month.assignadds columns in a chain without modifying the original; lambdas refer to the DataFrame being built.- Avoid
apply(axis=1)anditerrows()for maths: they run Python code per row and are much slower.
The code
import numpy as np
from io import StringIO
import pandas as pd
csv = StringIO("""order_id,date,customer,country,product,quantity,unit_price
1001,2026-01-03,ana,DE,mug,2,8.50
1002,2026-01-03,ben,US,teapot,1,24.00
1003,2026-01-04,chen,DE,kettle,1,31.90
1004,2026-01-05,ana,DE,teapot,1,24.00
1005,2026-01-07,dara,FR,mug,4,8.50
1006,2026-01-07,ben,US,mug,1,8.50
""")
orders = pd.read_csv(csv, parse_dates=["date"])
regions = {"DE": "EU", "FR": "EU", "US": "NA"}
enriched = orders.assign(
total=lambda d: d["quantity"] * d["unit_price"],
vat=lambda d: np.where(d["country"].map(regions) == "EU", (d["total"] * 0.19).round(2), 0.0),
size=lambda d: pd.cut(d["total"], bins=[0, 10, 30, np.inf], labels=["small", "medium", "large"]),
weekday=lambda d: d["date"].dt.day_name(),
region=lambda d: d["country"].map(regions),
)
print(enriched[["order_id", "total", "vat", "size", "weekday", "region"]])
Output
order_id total vat size weekday region 0 1001 17.0 3.23 medium Saturday EU 1 1002 24.0 0.00 medium Saturday NA 2 1003 31.9 6.06 large Sunday EU 3 1004 24.0 4.56 medium Monday EU 4 1005 34.0 6.46 large Wednesday EU 5 1006 8.5 0.00 small Wednesday NA
Common mistakes
for i, row in df.iterrows(): df.loc[i, "total"] = ..., which is orders of magnitude slower than one vectorised line.mapwith a dict that misses keys, producing NaN silently; check withisna().sum()or use.map(d).fillna("other").pd.cutbins that don't cover all values, leaving NaN categories.
Key points
- Prefer vectorised column operations over loops and apply.
- Use np.where, pd.cut and map for conditional and lookup columns.
- assign builds new columns in a readable chain.
Further reading: Essential basic functionality · DataFrame.assign
Practice questions
Why avoid iterrows() for calculations?
Answer
It runs Python code row by row; vectorised column operations run in optimised native code and are typically 10-1000x faster.
What does pd.cut do?
Answer
It bins numeric values into intervals (optionally labelled categories), such as small/medium/large by total.
What happens to keys missing from the dict passed to Series.map?
Answer
They become NaN; check for missing results or fill them with a default.
Open in the app to mark lessons complete, take notes and track your progress.