Skip to content
Back to Academia

Test: repeat purchases, reminders and discounts

18.06Assessment and testing
On this page

A customer has paid, then receives an email saying, “You forgot to complete your purchase.” The marketer sees an automation running; the customer sees a mistake. A CRM flow needs the customer's data, the action and the offer terms to agree.

These problems use a separate shop with a 20,000 ₸ basket. The numbers and situations are illustrative. Write your solutions before reading the explanations: this is a written self-assessment with no automatic score.

Problem 1: a hundred buyers are labelled lost

An export contains 100 buyers who have made just one purchase so far. For 40, the first purchase was less than a week ago; for the other 60, more than 30 days have passed. The team calls all hundred lost and proposes sending a repeat-order discount.

Identify what is known about these people and what the team has assumed. What information is needed to compare repeat purchases? Can the repeat-purchase rate be calculated from this export?

For this problem, the team wants to measure a repeat paid, non-cancelled order within 30 days of the first purchase. Explain which records belong in the numerator and denominator, and how to handle people whose window has not ended.

Problem 2: payment arrives after the email is queued

A visitor left a cart, and the system scheduled a reminder. They later paid for the order. The purchase is confirmed and linked to the same customer, but the unfinished-purchase email went out anyway.

Describe where the condition needs to be checked again and what should remove the person from this flow. Name cases for testing the fix. What information about the sending queue and payment delivery is missing before anyone can promise to cancel every such email?

Problem 3: a discount is expected to increase orders

An item costs 20,000 ₸. Variable costs of the goods and fulfilment are 12,000 ₸ per order and, under the problem's assumptions, remain unchanged by the discount. The team expects 100 orders over the chosen period without the promotion. With a 10% discount, it forecasts 120.

Calculate contribution per order and total contribution in both scenarios, before advertising and fixed costs. Find the smallest whole number of discounted orders that preserves the original contribution. Then list the assumptions that still need evidence before launching a promotion on this forecast.

Customer explanation: observation time and selection

One purchase describes order history at the export date. It does not explain why the person has not returned. They may still be using the product, may have found it unsuitable, or may have another order under a different ID. Those causes call for different actions.

The forty recent buyers have not completed the chosen observation window. Their result can be marked incomplete, with their first-purchase dates retained. Giving them a final zero for 30 days that have not elapsed would be wrong. The other sixty have completed the window, but their orders still need to be linked correctly and cancellations and refunds handled under an agreed rule.

The export also has a selection limit: it contains only people with one purchase. Repeat buyers have already been excluded, so these hundred rows cannot provide the overall repeat-purchase rate. That requires the full first-purchase cohort, including people who have returned.

The denominator contains all buyers in the selected cohort with a complete observation window. The numerator contains people from that same group with at least one qualifying repeat order within 30 days of their first. A person with several repeats appears in the numerator once. Dates and payment status come from an agreed order register; cancellations and refunds are handled consistently across groups.

Thirty days is specified for this problem. Another product needs a window chosen for the task and consumption data, with comparisons on an equal horizon. A result measured over 60 days or a return on precisely day 30 uses a different definition and cannot be substituted here.

Choose the group and the reason for contacting it before deciding on a message. A discount does not replace help with using a product. If a purchase was already planned, the discount may reduce contribution without adding an order. Sending also requires appropriate communication terms and permission for the selected type of message.

Flow explanation: the state before sending

A condition checked only on entry becomes stale while an email waits. Before sending, the flow needs to retrieve the order's state again and exclude the reminder if the relevant purchase is confirmed. It also needs a way for the person to exit.

A purchase is confirmed between the queued reminder and its send; the pre-send check cancels the email.
A purchase is confirmed between the queued reminder and its send; the pre-send check cancels the email.

This is a requirement for the flow. Whether it can be met depends on when the system learns about payment and how late it permits cancellation. The developer and CRM specialist should trace one order through payment confirmation, the order register, transfer to the CRM, customer matching and the queue's decision.

Once a message has been handed off for delivery, the required cancellation may no longer be available. If the payment event is delayed, the check will see old data. A promise that no reminder will ever arrive after payment therefore needs support from the capabilities of the entire process, and any discovered delay must be addressed in the design.

Testing should cover a cart without a purchase, payment before sending, a delayed payment event, a duplicate event and the person's re-entry into the flow. Keep the order ID, event sequence and sending decision for each case. For a completed purchase, the expected result is suppression of the reminder about that purchase.

A payment confirmation or another message the buyer needs remains a separate process. A global “send nothing after a purchase” rule could fix one complaint by creating another: the person may miss information about their order.

Discount explanation: revenue and contribution move differently

Without the promotion, contribution per order is 20,000 - 12,000 = 8,000 ₸. At a hundred orders, that is 800,000 ₸ before advertising and fixed costs.

The 10% discount reduces the price to 18,000 ₸. Costs remain 12,000 ₸, leaving 6,000 ₸ in contribution. At the forecast 120 orders, total contribution is 120 × 6,000 = 720,000 ₸.

MetricNo discount10% discount
Order price20,000 ₸18,000 ₸
Variable cost per order12,000 ₸12,000 ₸
Contribution per order8,000 ₸6,000 ₸
Forecast orders100120
Revenue2,000,000 ₸2,160,000 ₸
Total contribution800,000 ₸720,000 ₸

Forecast revenue rises by 160,000 ₸, while contribution falls by 80,000 ₸, or 10%. A report showing only orders and revenue would hide that difference.

Preserving 800,000 ₸ takes 800,000 / 6,000 = 133.33 orders. Round up to the next whole number, 134, which produces 804,000 ₸. At 133, contribution is only 798,000 ₸. This threshold excludes additional promotion costs; adding them raises the volume required.

Both 100 orders without the promotion and 120 with it are forecasts. Evaluating the effect requires a suitable comparison and a period that reveals whether purchases were added or merely brought forward. Refunds, a change in basket mix and the cost of preparing and running the promotion also belong in the calculation. Future repeat orders can appear in a separate scenario, but cannot be recorded in realised contribution in advance.

Exercise: describe your flow and discount decision

Choose one CRM flow in your own project. Write down entry, the check before the action and the exit condition. Add the source of the current status, the person responsible for transferring the data and an example record that lets you trace the outcome. Use permitted test data.

Then write a short decision on the shop's promotion: is the forecast of 120 orders enough to preserve contribution? State what changes if orders reach 134.

An actionable flow description identifies when the check happens and which data it uses. “Watch for purchases” is insufficient: the specialist needs the ID and status used to exclude a reminder and a way to see that it was excluded.

For the promotion, 120 orders produce 80,000 ₸ less than the original contribution. At 134, orders cover the original 800,000 ₸ only under the problem's assumptions, leaving 4,000 ₸ before additional expenses. This is a calculated threshold, not an established discount effect. The next decision needs evidence about demand, all relevant costs and purchases that would have happened without the promotion.

Need help applying this to your analytics? Discuss your project with Goodlabs →