Machine Learning Fundamentals for Engineers · Foundations: Models, Metrics and Pipelines
Preprocessing Pipelines Without Leakage
Scale numbers, encode categories and impute missing values in a ColumnTransformer inside a Pipeline, so preprocessing never leaks test data.
What you'll build
A churn model for a table with numeric columns (some missing) and text categories, with all preprocessing inside one Pipeline. You'll also see how fitting a scaler on all data before splitting inflates scores, a subtle form of data leakage.
The idea
- Models need numbers: impute missing values, scale numeric features, one-hot encode categories.
ColumnTransformerapplies different steps to different columns;Pipelinechains preprocessing and model sofitlearns everything from training data only.handle_unknown="ignore"keeps predictions working when a new category appears in production.- The pipeline is one object to cross-validate, tune, save and deploy.
The code
import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline, make_pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
rng = np.random.default_rng(1)
n = 1200
df = pd.DataFrame({
"tenure_months": rng.integers(1, 60, n).astype(float),
"monthly_fee": rng.choice([9.0, 19.0, 49.0], n),
"support_tickets": rng.poisson(1.5, n).astype(float),
"plan": rng.choice(["basic", "pro", "team"], n),
"country": rng.choice(["DE", "US", "FR", "IN"], n),
})
risk = 0.05 * df["support_tickets"] - 0.04 * df["tenure_months"] + (df["plan"] == "basic") * 0.8
df["churned"] = (risk + rng.normal(0, 0.6, n) > -0.6).astype(int)
df.loc[rng.choice(n, 80, replace=False), "tenure_months"] = np.nan # missing values
numeric = ["tenure_months", "monthly_fee", "support_tickets"]
categorical = ["plan", "country"]
preprocess = ColumnTransformer([
("num", make_pipeline(SimpleImputer(strategy="median"), StandardScaler()), numeric),
("cat", OneHotEncoder(handle_unknown="ignore"), categorical),
])
model = Pipeline([("prep", preprocess), ("clf", LogisticRegression(max_iter=1000))])
X_train, X_test, y_train, y_test = train_test_split(df[numeric + categorical], df["churned"], test_size=0.25, random_state=1, stratify=df["churned"])
model.fit(X_train, y_train)
print("test accuracy:", round(model.score(X_test, y_test), 3))
print("features after preprocessing:", len(model.named_steps["prep"].get_feature_names_out()))
new_customer = pd.DataFrame([{"tenure_months": np.nan, "monthly_fee": 9.0, "support_tickets": 4.0, "plan": "basic", "country": "BR"}])
print("churn probability for a new customer:", round(model.predict_proba(new_customer)[0, 1], 3))
Output
test accuracy: 0.75 features after preprocessing: 10 churn probability for a new customer: 0.794
The new customer has a missing tenure and a country (BR) the model never saw; the pipeline still predicts.
Common mistakes
- Fitting scalers, imputers or encoders on the full dataset before splitting: test statistics leak into training.
- Preprocessing by hand in a notebook and again (differently) in production; ship the whole pipeline.
- Encoding categories as integers (
basic=0, pro=1, team=2) for linear models, implying an order that doesn't exist.
Key points
- Put all preprocessing inside a Pipeline with the model.
- ColumnTransformer handles numeric and categorical columns differently.
- The fitted pipeline is the unit you evaluate, tune and deploy.
Further reading: Pipelines and composite estimators · Common pitfalls: data leakage
Practice questions
Why fit preprocessing inside a Pipeline?
Answer
So scalers, imputers and encoders learn their statistics from training data only, avoiding leakage, and the same steps are applied in production.
What does handle_unknown='ignore' do in OneHotEncoder?
Answer
Categories not seen during fitting are encoded as all zeros instead of raising an error.
Why not encode plan names as 0, 1, 2 for a linear model?
Answer
It implies an order and equal spacing between plans; one-hot encoding represents categories without a false order.
Open in the app to mark lessons complete, take notes and track your progress.