What machine learning is, and when not to use it

Machine learning is fitting a function from examples instead of writing the rules by hand. You show the computer thousands of past cases with known answers; it finds the pattern that maps inputs to answers; you then apply that pattern to new cases.

The vocabulary, once

• Features (X) — the input columns. Attendance, age, previous mark.
• Target (y) — the column you want to predict. Final mark, or pass/fail.
• Training — showing the model examples so it can adjust itself.
• Inference — using the fitted model on new rows.
• Generalisation — performing well on data it has never seen. The only thing that counts.

The three families

• Supervised — you have known answers. Split again by target type:
  • Regression — the target is a number. "What mark will this student get?"
  • Classification — the target is a category. "Will this student pass?"
• Unsupervised — no answers, only structure. Clustering students into groups, reducing forty columns to two for a chart.
• Reinforcement learning — an agent learning from rewards. Powerful, rarely what a business problem needs, and out of scope here.

The question to ask first: is this even an ML problem?

Machine learning is the right tool when all of these hold:

• The pattern exists but is too complex or too shifting to write as rules.
• You have enough labelled historical examples — hundreds at minimum, usually thousands.
• The future will resemble the past closely enough for a fitted pattern to hold.
• An imperfect answer is still useful. A model that is right 85% of the time must be worth having.

The baseline you must beat

Before any model, compute what a trivial answer achieves. If you cannot beat it, you have nothing.

import numpy as np
from sklearn.dummy import DummyClassifier, DummyRegressor
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# classification: always predict the most common class
dummy = DummyClassifier(strategy="most_frequent").fit(X_train, y_train)
print("baseline accuracy:", dummy.score(X_test, y_test))

# regression: always predict the mean
dummy = DummyRegressor(strategy="mean").fit(X_train, y_train)
print("baseline R2:", dummy.score(X_test, y_test))

If 92% of loans are repaid, "predict repaid" scores 92% accuracy while identifying not a single default. Any real model must be compared against that, and accuracy is clearly the wrong metric — which is the subject of a later lesson.

Framing: the target is a decision

"Predict student performance" is not a task. These are:

• Predict the final mark (regression) — to allocate revision resources.
• Predict pass/fail (binary classification) — to trigger an intervention.
• Predict dropout within one term (binary, time-boxed) — to schedule a home visit.

Each needs a different target column, a different metric, and a different definition of an acceptable mistake. Decide what action the prediction triggers before you write any code; that decision determines everything else.

Set up

pip install scikit-learn pandas matplotlib seaborn
python -c "import sklearn; print(sklearn.__version__)"

The scikit-learn contract

Every model in the library exposes the same four methods. Learn one and you have learned all of them:

model.fit(X_train, y_train)     # learn
model.predict(X_test)            # predict
model.predict_proba(X_test)      # class probabilities (classifiers)
model.score(X_test, y_test)      # a default metric

X is always 2-D — rows by features. y is always 1-D. A "Reshape your data" error means you passed a single column as 1-D; use X[["column"]] with two brackets.

Try it yourself

Write down one prediction problem from your own data. State: the target column, whether it is regression or classification, what action the prediction triggers, what a false positive costs, what a false negative costs, and the baseline score. Keep this note — you will check every model in this track against it.