A driving school in Astana advertises on Instagram and in Google search. The owner opens the ad accounts and messages you: "Instagram leads are half the price. Let's put the whole budget there." You could open the report and start arguing, but a table with twenty columns has something in it for everyone. So before the numbers, you write down three things: the question, the metric and the rule the decision will follow. The numbers in this lesson are made up.
A question that changes something
"How are our ads doing?" and "we need a dashboard" are not questions for data: whatever the answer, nobody will do anything differently. A question is worth asking if the answer leads to an action and someone will carry it out. The driving school owner's question is this: should part of the budget move from Instagram to Google next month?
Next to it, write down the options (leave it as it is, move part, move all of it), who decides (the owner) and by when (when the ad accounts are topped up at the start of the month). If there is only one option, there is nothing to decide, and no data is needed.
The metric: which number answers the question
For a budget question, cost per lead will not do. The owner pays for ads to get students, and a lead is only a step towards one. What he needs is ad spend per student who paid for the course. So that a colleague gets the same number a month later, write the metric out in full:
- spend: what went on ads in the month, according to the ad account;
- students: those who paid for the course within 14 days of their lead, according to the CRM;
- channel: from the UTM tags the form saved with the lead; a person with two leads counts once, by the first;
- period: leads for the calendar month, with 14 days allowed for each one to pay.
Both channels get the same 14-day window. Otherwise the channel where people take longer to decide loses simply because its students have not paid yet. End-to-end analytics is what links a CRM payment to an ad tag.
Write the decision rule before the report
Now the important part: agree in advance which result leads to which action. After paying instructors, fuel and the practice ground, the driving school keeps 60,000 ₸ per student. The owner is willing to spend half of that on advertising, so up to 30,000 ₸ per student. The IELTS prep centre works out its limit on the cost of a student the same way.
The driving school's rule:
If one channel's spend per student is above 30,000 ₸ and the other's is below, move 20% of the expensive channel's budget to the cheaper one for a month. If either channel has fewer than five students, change nothing and collect another month of data. After a month, calculate the same metric and the total number of students across both channels, and if the total has fallen, put the budget back as it was.
The rule has three parts: a threshold taken from the economics, not from last month's report; a minimum amount of data, below which no decision is made; and a step size with the date of the next review.
It has to be written before the numbers, because once people see them, each picks the metric that suits their view. The owner likes CPL, since Instagram wins on it. The specialist running search ads likes cost per student. A rule written in advance settles that argument: the result either meets the condition or it does not. Experience still matters. Without it you cannot come up with the question or the threshold, and the data is there to check whether your instinct was right this time.
The month's numbers
Leads for September, with payments within 14 days:
| Instagram (channel A) | Google (channel B) | |
|---|---|---|
| Ad spend | 200,000 ₸ | 160,000 ₸ |
| Leads | 50 | 20 |
| CPL | 4,000 ₸ | 8,000 ₸ |
| Paid for the course | 5 | 8 |
| Spend per student | 40,000 ₸ | 20,000 ₸ |
On CPL the owner is right: an Instagram lead costs half as much. On the metric chosen in advance it is the other way round: a student from Instagram costs 40,000 ₸ and is over the threshold, one from Google costs 20,000 ₸. Both channels have at least five students, so the rule applies: 40,000 ₸, a fifth of the Instagram budget, moves to Google for a month.
Five students is not many, and the rule allows for that through the size of the step. With six students on Instagram, each would have cost 33,333 ₸; with four, 50,000 ₸. One person swings the number, which is why a fifth of the budget moves rather than all of it, and why you look again after a month.
What the data will not tell you
The table describes September. Whether September will repeat once the money moves, it does not say, and here is why.
Correlation is not cause. People who come through search are already looking for a driving school, while Instagram ads also reach people who are only thinking about a licence. That may be exactly why Google is cheaper per student, and an extra 40,000 ₸ will not add searchers: there are only so many in Astana. Or the owner might notice that in months when Instagram got more money, Google also brought in more students. That happens when Instagram nudges people to search for the school, and it also happens when both channels grow before the summer for a third reason. Only a test can tell these explanations apart, one where a randomly chosen share of cities or weeks gets the change, as in an A/B test.
Attribution is not cause either. Under the attribution rule a student is credited to Google, though before the lead he may have spent a month watching the school's videos on Instagram. That is why the decision rule includes the total number of students; how many students the channel itself brings in only a test with a control group can show.
Where, not why. Data shows at which step people drop off. Five of the 50 Instagram leads paid; why the other 45 did not is something the reasons for rejection in the CRM and conversations with people will tell you, not the ad account.
The decision record
All of this fits in five lines kept next to the report:
| Line | At the driving school |
|---|---|
| Question | Should part of the budget move from Instagram to Google next month |
| Metric | Ad spend per student who paid within 14 days of the lead, from the CRM and the lead's tags |
| Rule | Threshold of 30,000 ₸ per student, at least five students per channel, a step of 20% of the budget for a month |
| Result for September | Instagram 40,000 ₸ per student, Google 20,000 ₸ |
| Decision | 40,000 ₸ moves from Instagram to Google for a month. Review once this month's leads have had their 14 days: spend per student by channel and the total number of students |
A month later the record shows why this was the decision; without it, nobody would remember. And if the new numbers miss the expectation, it shows what let you down: the rule, the data or the conditions themselves.
Exercise: the rule before the report
Take a question that you or a client have to settle in the next two weeks: the budget, a new creative, a second landing page, a manager's working hours. Without opening any reports, write down:
Then open the report and apply the rule.
If your question came out as something like "how are the ads working", add the possible actions to it: without them there is nothing to answer. If the metric is CPL and the question is about budget, ask yourself what the client is really paying for, leads or payments. A threshold taken from last month ("it was 25,000 ₸, let's keep it") tells you what happened, not how much you can afford, so take it from the client's economics. And if, once the report is open, you want to change the rule, do not change it now. Note what bothered you and adjust the rule for the next review; otherwise it is a decision by gut feel again, just with a table attached.