Machine Learning Fundamentals for Engineers · Foundations: Models, Metrics and Pipelines
Regression: Predicting Numbers
Predict numeric values with linear regression, measure errors with MAE, RMSE and R2, and read residuals to see where a model fails.
What you'll build
A model that predicts delivery time in hours from distance and package weight, evaluated with error metrics people can understand ("off by about 2 hours on average"), and a residual check that reveals a pattern the model misses.
The idea
- Linear regression fits
y = w1*x1 + w2*x2 + b; coefficients are interpretable (hours per kilometre). - MAE (mean absolute error): average size of the error, in the target's units. RMSE punishes large errors more. R2: share of variance explained (1 is perfect, 0 equals predicting the mean).
- Residuals (actual minus predicted) should look like random noise; patterns mean the model is missing something.
The code
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, r2_score, root_mean_squared_error
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(0)
distance = rng.uniform(1, 300, 500) # km
weight = rng.uniform(0.1, 30, 500) # kg
hours = 2 + 0.05 * distance + 0.1 * weight + 0.00008 * distance**2 + rng.normal(0, 1.5, 500)
X = np.column_stack([distance, weight])
X_train, X_test, y_train, y_test = train_test_split(X, hours, test_size=0.2, random_state=0)
model = LinearRegression().fit(X_train, y_train)
pred = model.predict(X_test)
print("coefficients (h/km, h/kg):", model.coef_.round(3), "intercept:", round(model.intercept_, 2))
print(f"MAE {mean_absolute_error(y_test, pred):.2f} h | RMSE {root_mean_squared_error(y_test, pred):.2f} h | R2 {r2_score(y_test, pred):.3f}")
residuals = y_test - pred
far = X_test[:, 0] > 200
print(f"mean residual, short trips: {residuals[~far].mean():+.2f} h | long trips: {residuals[far].mean():+.2f} h")
Output
coefficients (h/km, h/kg): [0.074 0.1 ] intercept: 0.95 MAE 1.29 h | RMSE 1.59 h | R2 0.942 mean residual, short trips: -0.30 h | long trips: +0.49 h
The residuals show the model underestimates long trips: the real relationship isn't a straight line (there's a squared distance term). A model with a curved feature, or a tree-based model, would fix it.
Common mistakes
- Reporting only R2; stakeholders understand "off by 1.3 hours" better.
- Ignoring residual patterns, which reveal missing features or non-linear effects.
- Extrapolating: linear models happily predict for inputs far outside the training range.
Key points
- Linear regression is a strong, interpretable starting point.
- Report MAE or RMSE in the target's units, plus R2.
- Check residuals for patterns the model misses.
Further reading: Linear models · Regression metrics
Practice questions
What's the difference between MAE and RMSE?
Answer
Both are in the target's units; RMSE squares errors before averaging, so it weighs large errors more heavily.
What does an R2 of 0 mean?
Answer
The model explains none of the variance; it's no better than always predicting the mean.
What do patterned residuals tell you?
Answer
The model systematically misses something, such as a non-linear relationship or a missing feature.
Open in the app to mark lessons complete, take notes and track your progress.