Framing: the work before the code

Most failed data-science projects were not badly modelled. They were badly framed: the wrong target, a metric nobody cared about, or a prediction that arrived after the decision had already been made. Framing is the highest-leverage work you will do, and it is done before any code.

The six questions

1. What decision changes because of this prediction? If the answer is "none", stop. You are building a dashboard, and that is fine — just say so.
2. Who acts on it, and when? A churn prediction that arrives the day after the customer left is worthless regardless of its accuracy.
3. What exactly is the target? "Churn" needs a definition: no purchase in 90 days? cancelled subscription? Write the SQL that produces the label.
4. What does each error cost? In money, time, or harm to a person. This sets your metric and your threshold.
5. What is the baseline? The current process. Often a rule, sometimes a person's judgement — measure it before you compete with it.
6. What is the bar for shipping? Agreed in advance, in writing. Otherwise "good enough" is decided retrospectively by whoever is most senior in the room.

The leakage question, asked at framing time

For each candidate feature, ask: at the moment the prediction must be made, is this value known? Draw the timeline.

enrolment ----- term 1 ----- PREDICT HERE ----- term 2 ----- final mark
                                  |
              anything to the right of this line is not a feature

Do this on paper. It catches more errors than any code review, and it catches them before you have wasted a fortnight.

Write the brief

PROJECT: At-risk student flag

Decision      Head of studies assigns tutoring slots each Monday.
Timing        Prediction needed by Friday, from data available Thursday.
Target        student fails (final mark < 50) at end of current term.
              Label SQL: SELECT id, (final_mark < 50) AS y FROM marks
              WHERE term = :term
Population    All enrolled students with at least 3 weeks attendance.
              Excludes: new transfers (no history) -- handled by a rule.
Features      Attendance to date, term-1 mark, assignment submission rate,
              days since last submission, district, school type.
              NOT: final mark, exam attendance, any post-cutoff record.
Baseline      Current rule "attendance < 60%": recall 0.44, precision 0.51.
Metric        Recall at precision >= 0.50. A wasted tutoring slot costs
              little; a missed failing student costs a term.
Bar to ship   Recall >= 0.65 at precision >= 0.50, held-out, and no
              district below recall 0.50.
Refresh       Retrain each term; monitor weekly.
Owner         Named person for the model, named person for the decision.

One page. It settles every argument that would otherwise happen in week six, and the fairness constraint in the "Bar to ship" line means nobody has to raise it awkwardly at the end.

Estimate the value before you build

students_at_risk = 200
current_recall, model_recall = 0.44, 0.65
extra_caught = students_at_risk * (model_recall - current_recall)   # 42
intervention_success = 0.35
extra_passes = extra_caught * intervention_success                  # ~15

Fifteen more students passing per term. That is a sentence a head of studies can act on, and it is a far better justification than an F1 score. If the arithmetic comes out trivial, you have learned something important cheaply.

Scope down, deliberately

Ship the simplest thing that can create value, then improve it:

1. Week 1 — the existing rule, measured properly. Now you have a baseline nobody disputes.
2. Week 2 — logistic regression on five features, evaluated honestly.
3. Week 3 — put it in front of the decision-maker as a list, not an API.
4. Later — boosting, more features, automation, monitoring.

Teams that spend eight weeks on step 4 before showing anyone step 3 are the ones whose models never ship.

Try it yourself

Write the one-page brief above for a real problem in your own organisation or study. Include the label SQL, the timeline diagram, the measured baseline, the metric with its justification, the ship bar with a fairness constraint, and the value estimate. Then show it to whoever would act on the output and correct it from what they say — that conversation is the most valuable hour in the project.