Skip to content
Back to Academia

Effectiveness analysis: what ROAS tells you and what the ads added

04.09The digital ecosystem
On this page

A week after its Meta test, the IELTS prep centre got its first student from the ads: he paid for one month of the course, 45,000 ₸. The media buyer divides that by the 9,860 ₸ of spend and messages the owner: "ROAS 4.6". The owner replies: "So the ads have already made us 35,000 ₸ on top. Let's double the budget." The division is right. The conclusion is not.

An effectiveness review answers four questions in order: what happened, how much money is left, over what period and under what rule the revenue was credited to ads, and what the ads added. Below are all four, worked through on the centre's numbers from the metrics glossary and the scaling calculation. The numbers are an example.

What happened: count each sale once

Facts come from your list of leads and payments, not from the ad account. The test gave the centre 11 leads; three people came to the free trial lesson and one paid for the first month. Ads Manager credits leads to the ads under its own rules, and it knows nothing about the payment unless you send it there.

So sales are counted in the CRM and checked against incoming payments and refunds, and ad account reports are never added up. Each ad account credits a sale to its own ads, and once there are two of them, one payment easily lands in both reports.

Ad accounts A and B both claim the same order 042, while for the business it is one purchase.
Ad accounts A and B both claim the same order 042, while for the business it is one purchase.

The overall picture without double counting is all revenue from the CRM divided by all marketing spend for the same period. This ratio is called MER (marketing efficiency ratio). Double counting across ad accounts does not get into it, but it will not tell you which channel worked either.

ROAS: revenue, not profit

ROAS is revenue credited to ads divided by ad spend. For the centre that is 45,000 / 9,860 = 4.6. The number itself tells you one thing: the revenue credited to ads under the chosen attribution model, over the chosen period, is 4.6 times what the ads cost. The centre's model is simple: a payment counts as ads revenue if the lead came through a link tagged paid_social. The period is the first month of payment. Without those two labels a ROAS cannot be read.

The second justified conclusion is about what the number leaves out: to judge profit you need the margin and the other costs. ROAS does not include the cost of goods, delivery, refunds or any other expense, and the same revenue leaves a different contribution depending on the product, the discount and the cost of fulfilling the order. That holds for any ROAS. If a client writes "the campaign has a ROAS of 5", it means the revenue credited to the ads is five times the spend, and nothing more. In the metrics glossary, a campaign with a ROAS of 5 spent 100,000 ₸ on ads, took 500,000 ₸ in revenue and still ended 50,000 ₸ in the red: variable costs took 450,000 ₸.

Here is what the centre kept from the student's first month:

LineAmount
Revenue for the first month45,000 ₸
Teacher's pay and materials29,000 ₸
Contribution the centre keeps16,000 ₸
Ads9,860 ₸
Contribution after ads6,140 ₸

Instead of the 35,000 ₸ the owner counted, the contribution after ads is 6,140 ₸, and the centre still pays rent and the administrator's salary out of it.

That gives the breakeven ROAS. The centre keeps 36% of revenue, so the ads pay for themselves when ROAS is at least 45,000 / 16,000 = 2.81. The test is at 4.6, so there is headroom. But the threshold depends on what was actually sold. Say the student got the first month at a discount, for 35,000 ₸. ROAS becomes 3.5, still above 2.81, yet the contribution drops to 6,000 ₸ and the result after ads is minus 3,860 ₸. With the discount the centre needs a ROAS of at least 5.8. That is why ROAS is compared neither with 1 nor with the campaign next door, but with the breakeven point for the margin you actually got.

One ROAS, different periods and rules

ROAS moves with the period the revenue is taken over. For the first month the centre has 4.6. If the student stays all three months, the same spend brings 135,000 ₸, ROAS becomes 13.7 and the contribution after ads is 48,000 − 9,860 = 38,140 ₸. Whether he will stay is not known yet: in the cohort analysis, 70% of the centre's August students reached the third month. So the report gives ROAS for the period that has already passed, and puts the forecast on a separate line, called a forecast.

It also moves with the rule. While there is one channel the rule hardly matters: in GA4, last click gave the test all 11 leads and the data-driven model gave it 10.6. Once search is added, the same payment may move to another channel's report. If the model changes, the report notes the date from which the new rule applies. Otherwise, a month later, someone will decide that one channel grew and another collapsed, when all that changed is who the payments are credited to.

And one more thing: the ROAS of 4.6 rests on one person. Had he not paid, ROAS would be 0. Had a second person from the trial lesson paid, it would be 9.1. One payment tells you that the first student covered the cost of the test. What ROAS will be at twice the budget, one payment cannot tell you.

What the ads added

Attribution splits a payment among the touchpoints the system saw. It does not answer whether the student would have bought the course without the ads. Maybe he had been choosing between centres for a while and the post just reminded him of yours. Maybe without the post he would never have heard of the centre. In the report both cases look the same: an ad click, a lead, a payment.

The only answer comes from comparing with people who did not see the ads. The platforms can run such experiments themselves. In a Conversion Lift test, Meta randomly holds back part of the audience from the ads and measures lift as the difference in conversions between the groups. The guideline for running one yourself is a campaign started in the past year with at least $5,000 of spend and at least 500 conversions. Google's similar tool is called Conversion lift: groups are split by users or by regions, but it is not available in every account, and you get it through a Google account specialist.

With 9,860 ₸ and 11 leads, the centre is nowhere near those thresholds. Comparing regions will not work either: the centre operates in one city. What remains is a before-and-after comparison, two weeks with ads against two weeks without. It is useful as a hint, not as proof: over those weeks exam dates may get closer, autumn enrolment may begin or competitors may change prices, and the whole difference will be put down to the ads.

For a small budget there is a simpler method. The manager asks every new student how they heard about the centre and records the answer in the CRM. The ad account will never see a friend's recommendation, but the student will mention it. This does not measure the effect of the ads, but if half of the "ad" students say a friend sent them, ROAS 4.6 deserves a more careful reading. In the report, write "revenue credited to ads", not "revenue the ads brought in".

A conclusion you can test

A weak conclusion: "ROAS 4.6, the ads paid off, let's double". It has a ROAS with no period and no rule, treats one student as a pattern, and has neither a check nor a decision date.

One payment is not enough to name a cause, but the test numbers do suggest a hypothesis to check. The "Problem" ad brought 7 leads and one person at the trial lesson: perhaps without a price in the text, the form gets submitted by people who cannot afford the course. The second test checks exactly that: the price goes into "Problem" and nothing else changes. If you also raise the budget, switch the audience and rewrite the creatives, the result will not tell you which of these worked. The media buyer keeps the table, and the centre's manager marks the same day who came, who paid and how they heard about the centre.

The decision goes into the same five lines as any data-driven decision:

LineFor the centre
QuestionShould the daily budget go up after the Meta test
MetricCPL and the cost of a person at the trial lesson from the lead list, ROAS on the first month of payment for leads tagged paid_social
RuleThe daily budget goes up after two weeks on one step if CPL is at least 30% below the 2,000 ₸ target, a person at the trial lesson costs up to 8,000 ₸ and ROAS is at least 2.81
Test resultFive days: CPL 896 ₸, a person at the trial lesson 3,287 ₸, ROAS 4.6, contribution after ads 6,140 ₸. Behind it are 11 leads and one student
DecisionThe budget is not doubled. A second test for two weeks: 2,000 ₸ a day, about 28,000 ₸, with the price added to "Problem". Review a week after the test ends, when the payments become visible

Exercise: review your own test

Take the campaign from week 2 of the Start track or a client's campaign and go through the four questions.

If there are no payments yet, there is no ROAS to calculate; say so. Instead, work out how many payments the ads need to pay off: the centre would have needed just one, because a student's first-month contribution, 16,000 ₸, is more than the test cost. The client may not know what share of revenue the business keeps. Then ask how much of each sale goes to the person doing the work and to materials: without that figure there is nothing to compare ROAS with. If the revenue in your review comes from two ad accounts and does not match the CRM, start by looking for sales recorded in both. And if your conclusion says "the ads brought in", change it to "credited to ads": your test did not measure what the ads added.

Video: attribution and incrementality

Inside Google Marketing: Attribution & Incrementality

Need help applying this to your analytics? Discuss your project with Goodlabs →