Data Analysis with pandas · pandas Fundamentals

Adding and Transforming Columns

Create new columns with vectorised arithmetic, conditions with np.where and cut, mappings, and method chains with assign, avoiding slow row loops.

Create a free account to save this page, take notes and track your progress. Sign in

What you'll build

Derived columns for the orders table: line totals, VAT, a size category, a weekday, and a region looked up from a mapping, written as one readable method chain.

The idea

  • Vectorised arithmetic on columns is fast and readable: df["total"] = df.qty * df.price.
  • Conditions: np.where(cond, a, b) for two outcomes, pd.cut for numeric ranges, Series.map(dict) for lookups.
  • .dt accessor for dates: .dt.day_name(), .dt.month.
  • assign adds columns in a chain without modifying the original; lambdas refer to the DataFrame being built.
  • Avoid apply(axis=1) and iterrows() for maths: they run Python code per row and are much slower.

The code

A method chain of new columnsTry it yourself
import numpy as np
from io import StringIO
import pandas as pd

csv = StringIO("""order_id,date,customer,country,product,quantity,unit_price
1001,2026-01-03,ana,DE,mug,2,8.50
1002,2026-01-03,ben,US,teapot,1,24.00
1003,2026-01-04,chen,DE,kettle,1,31.90
1004,2026-01-05,ana,DE,teapot,1,24.00
1005,2026-01-07,dara,FR,mug,4,8.50
1006,2026-01-07,ben,US,mug,1,8.50
""")
orders = pd.read_csv(csv, parse_dates=["date"])

regions = {"DE": "EU", "FR": "EU", "US": "NA"}
enriched = orders.assign(
    total=lambda d: d["quantity"] * d["unit_price"],
    vat=lambda d: np.where(d["country"].map(regions) == "EU", (d["total"] * 0.19).round(2), 0.0),
    size=lambda d: pd.cut(d["total"], bins=[0, 10, 30, np.inf], labels=["small", "medium", "large"]),
    weekday=lambda d: d["date"].dt.day_name(),
    region=lambda d: d["country"].map(regions),
)
print(enriched[["order_id", "total", "vat", "size", "weekday", "region"]])

Output

   order_id  total   vat    size    weekday region
0      1001   17.0  3.23  medium   Saturday     EU
1      1002   24.0  0.00  medium   Saturday     NA
2      1003   31.9  6.06   large     Sunday     EU
3      1004   24.0  4.56  medium     Monday     EU
4      1005   34.0  6.46   large  Wednesday     EU
5      1006    8.5  0.00   small  Wednesday     NA

Common mistakes

  • for i, row in df.iterrows(): df.loc[i, "total"] = ..., which is orders of magnitude slower than one vectorised line.
  • map with a dict that misses keys, producing NaN silently; check with isna().sum() or use .map(d).fillna("other").
  • pd.cut bins that don't cover all values, leaving NaN categories.

Key points

  • Prefer vectorised column operations over loops and apply.
  • Use np.where, pd.cut and map for conditional and lookup columns.
  • assign builds new columns in a readable chain.

Further reading: Essential basic functionality · DataFrame.assign

Practice questions

Why avoid iterrows() for calculations?

Answer

It runs Python code row by row; vectorised column operations run in optimised native code and are typically 10-1000x faster.

What does pd.cut do?

Answer

It bins numeric values into intervals (optionally labelled categories), such as small/medium/large by total.

What happens to keys missing from the dict passed to Series.map?

Answer

They become NaN; check for missing results or fill them with a default.

Open in the app to mark lessons complete, take notes and track your progress.

Was this page helpful?