The Machine Learning Workflow

What you'll build

A first model that predicts whether a customer will buy, trained on part of the data and evaluated on data it has never seen, compared against a "dumb" baseline. Every example in this course runs in your browser (click Run) with scikit-learn 1.8 in Pyodide, the same environment that produced the outputs shown.

The idea

• Supervised learning: learn a function from examples with known answers (labels), then predict answers for new examples.
• Features (X): the inputs. Target (y): what you predict. Regression predicts numbers; classification predicts categories.
• Train/test split: fit on one part, measure on the held-out part; scores on training data are always optimistic.
• Baseline: a trivial model (predict the most common class, or the mean). If your model doesn't beat it clearly, it isn't learning anything useful.
• scikit-learn's API is the same for every model: fit(X, y), predict(X), score(X, y).

The code

from sklearn.datasets import make_classification
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

X, y = make_classification(n_samples=1000, n_features=8, n_informative=4, n_clusters_per_class=1,
                           class_sep=1.2, weights=[0.7], random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, stratify=y, random_state=42)
print("train/test sizes:", len(X_train), len(X_test), "| share of buyers:", round(y.mean(), 2))

baseline = DummyClassifier(strategy="most_frequent").fit(X_train, y_train)
model = LogisticRegression(max_iter=1000).fit(X_train, y_train)

print("baseline accuracy:", round(baseline.score(X_test, y_test), 3))
print("model accuracy:   ", round(model.score(X_test, y_test), 3))
print("model on training data:", round(model.score(X_train, y_train), 3))
print("first 5 predictions:", model.predict(X_test[:5]), "actual:", y_test[:5])

train/test sizes: 750 250 | share of buyers: 0.3
baseline accuracy: 0.7
model accuracy:    0.94
model on training data: 0.928
first 5 predictions: [1 0 0 0 0] actual: [1 0 0 0 0]

Common mistakes

• Reporting accuracy on the training data.
• No baseline: 70% accuracy sounds good until you notice 70% of customers never buy.
• Splitting without stratify on imbalanced labels, giving test sets with very different class shares.
• Choosing ML when a simple rule or SQL query answers the question.

Key points

• Fit on training data, evaluate on held-out test data.
• Always compare with a baseline.
• fit, predict and score work the same for every scikit-learn model.

Further reading: Getting started with scikit-learn · Model evaluation