A shop owner looks at a report: revenue has risen, the share of repeat buyers has fallen, and the newest cohort shows zero. He proposes raising the budget because revenue is growing, and rebuilding the website urgently while he is at it. The analyst needs to establish what actually changed and which decisions the evidence supports.
This is a written self-check. Keep your calculations and explanations, then compare them with the analysis. The figures are illustrative; repeat purchases are measured within 30 days.
Case: three cohorts from one shop
In May, June and July, 100 people made their first purchase each month. Each person belongs to one cohort, defined by the month of their first payment. Retention is the share of buyers with at least one further paid, non-refunded order within 30 days of that payment. Someone with several repeat orders adds only one person to the numerator.
At the reporting date, everyone in May and June has completed the observation window. In May, 30 people bought again; in June, 20 did. Some July buyers have not yet reached 30 days. The analyst recorded 0% for July despite the incomplete observation.
Which values can appear as completed results? How should July be displayed? Explain separately whether the May-to-June difference can be described as a consequence of a worse product.
Analysis: the same age and the same action
May is 30 / 100 = 30%. June is 20 / 100 = 20%. The difference is 10 percentage points. July needs an incomplete-window label, the reporting date and the point when the last buyer's window will close. Zero would mean that the full window had elapsed and nobody had bought again.
Waiting for maturity does not require suspending all analysis. The team can keep monitoring orders and errors while comparing a separately defined, equally short horizon across groups. Its name and duration must distinguish it from the final 30-day metric. One column cannot mix results observed at different buyer ages.
Selecting only July buyers who have completed the window produces a subset of the cohort. It can be shown separately with its own denominator, but it cannot be labelled as the result for all 100 people. Earlier entrants may have come from different campaigns or bought different products.
The difference between completed cohorts is an observation. Explaining it requires the buyer mix, channels, categories, stock, delivery terms and history of measurement changes. Cohort segmentation helps separate changes in composition from changes within groups. A causal claim about a new product version needs an appropriate comparison, beyond adjacent months.
The action in the definition matters too. Here it is a paid repeat purchase within the window. Opening an app, returning exactly on the final day and buying after the window closes answer different questions.
Case: revenue rises while fewer buyers return
Add the economics of the same two completed cohorts. Each first purchase generated 10,000 ₸ in revenue and incurred 6,000 ₸ in variable product and fulfilment costs. Every repeat buyer placed exactly one further order within the window.
| Repeat-order measure | May | June |
|---|---|---|
| Buyers with a repeat order | 30 | 20 |
| Revenue per repeat order | 10,000 ₸ | 20,000 ₸ |
| Variable costs per repeat order | 6,000 ₸ | 16,000 ₸ |
All included orders are paid, with no refunds in this case. Advertising and fixed costs are excluded from the table. Calculate each cohort's revenue from first and repeat purchases, then contribution after the stated variable costs. What would you tell an owner who considers revenue growth sufficient grounds for increasing the budget?
Analysis: separate revenue from contribution
First purchases generate 100 × 10,000 = 1,000,000 ₸ of revenue for each cohort. Their contribution is identical: 100 × (10,000 - 6,000) = 400,000 ₸.
May's repeat orders add 30 × 10,000 = 300,000 ₸ of revenue and 30 × 4,000 = 120,000 ₸ of contribution. June's produce 20 × 20,000 = 400,000 ₸ of revenue, but only 20 × 4,000 = 80,000 ₸ of contribution: variable costs increased alongside order value.
| Cohort total over the same horizon | May | June |
|---|---|---|
| Revenue from first and repeat purchases | 1,300,000 ₸ | 1,400,000 ₸ |
| Contribution after variable costs | 520,000 ₸ | 480,000 ₸ |
Revenue is 100,000 ₸ higher; contribution is 40,000 ₸ lower. Fewer buyers returned, while a larger order value lifted revenue. Under these conditions, revenue growth did not improve contribution. The reason for the change in order value still needs investigation: products and order terms may have differed.
These are cohort totals over a defined horizon, rather than the shop's entire revenue in a calendar month. Calling them profit would also be premature: advertising, fixed costs and other items are absent. Acquisition decisions need the cost of new buyers, later repeat purchases and capacity to fulfil additional orders. Raising the budget just because revenue grew is as unsupported as switching campaigns off based on one aggregate chart.
The message to the owner can be brief: “Among completed cohorts, repeat purchasing within 30 days fell from 30% to 20%. Revenue rose through higher-value repeat orders, but contribution after variable costs fell by 40,000 ₸. We will break the change down by product and channel and add acquisition costs; July will remain without a final assessment.”
Case: sign-ups are called customer value
Another product helps people book services. The team proposes “accounts created” as its North Star Metric. Yet a person can register without choosing a service or a time.
Suggest a measure closer to the completed task, identify its evidence source and name quality guardrails. Explain what to do if the service knows a booking was created but does not receive a reliable status confirming that the visit took place.
Analysis: respect what the metric can observe
The candidate depends on the product's purpose. For a service that helps people book, counting people with a successfully created booking can be useful, excluding tests, duplicates and cancellations under an agreed rule. If the task includes receiving the service itself, a confirmed visit is closer to the outcome. That measure requires the status to be collected reliably.
With incomplete visit data, the measure must remain labelled “created bookings”. Show confirmation coverage alongside it, and assess no-shows separately where they are recorded. Treating a missing status as a successful visit would exaggerate the result.
The main product metric still needs supporting measures: booking errors, cancellations, task completion time, complaints and retention. Activity can increase without creating value, so the guardrails belong in the decision.
Exercise: prepare one line of your own report
Choose a project metric and document its unit, event, numerator, denominator, window, source, refresh date and owner. Attach a calculation from available data without personal details. Mark incomplete windows, name the decision the metric supports and identify information still missing for that decision.
A colleague should be able to reproduce the calculation and read the completed line the same way. If “return” means a login while the decision concerns repeat payment, the definition needs repair. If a fresh cohort displays zero, its observation window needs a separate label. In a dashboard that supports decisions, a deviation comes with a next step: who will investigate the data, in which breakdown and when they will report back.