An analytics course shortened its enquiry form by removing an optional budget question. Leads rose from 50 to 80. The marketer wants to keep the new form, the manager dislikes the lead quality, and the owner asks about payments. The old group has ten; the new group has eight. Everyone is discussing a different number because the team never wrote down how it would make the decision.
It is the same course as in the end-to-end analytics lesson: the price is 25,000 ₸ and contribution per sale is 15,000 ₸. The form test was invented for this lesson, and its figures are made up. HADI connects the question, change, data and decision in advance: Hypothesis, Action, Data, Insights. Sber's guide uses the same expansion.
H: the hypothesis explains the obstacle
"Remove the field" describes the developer's work. A hypothesis explains why that work should help a buyer. For the course, it might read:
People do not yet know how much they are willing to spend on training and stop at the budget question. Removing it from the first form will help more visitors complete an enquiry, while lead quality holds and the share of visitors who pay increases.
This names a cause, a change and an expected outcome. First look for evidence of the cause in enquiries and observations of people filling out the form. If visitors complain about a phone-number error but skip the budget field easily, the hypothesis needs a different basis.
A change is ready for testing when the team understands who it is for and what that person will see. Here, one field in the first form changes. Course prices, advertising, the offer and the manager's work stay the same. Removing the price from the page at the same time would make it impossible to attribute the result to one change.
A: a plan you can launch
Randomly assign eligible new visitors to A and B at the same time. Preserve the assignment so returning visitors see the same form. Exclude staff and test submissions under a rule written before launch. The team verifies the link between a visitor, enquiry and payment using its own test orders.
The primary metric is the share of assigned visitors who pay for the course within 30 days of entering the test. This example uses a 30-day window; a real project needs to justify it by the delay between a first visit and payment. The manager qualifies leads the same way in both groups: the course programme, format and price fit the person's needs.
Lead count helps explain whether the field affected completion. Qualified leads show whom the change brought in. Complaints, refunds and the manager's time constrain the cost of improvement. If the forms generate more enquiries but sales staff spend all day talking to people with a different need, that belongs in the test result.
Before publication, the card includes group sizes, the minimum useful effect, analysis method, recruitment end and readout date. The A/B test planner gives a starting point for comparing two proportions. The image's 500 visitors are there to explain the table; they are not a ready-made sample size for any course.
D: keep one comparison base throughout
The results below give every visitor a full 30-day window. Repeat enquiries from one person have been merged. Payments are linked to the original variant assignment, and cancellations and refunds are excluded under the same rule.
| Metric | Form A | Form B |
|---|---|---|
| Assigned visitors | 500 | 500 |
| Visitors with an enquiry | 50 | 80 |
| Visitors with a qualified enquiry | 30 | 32 |
| Visitors with a payment | 10 | 8 |
| Share with an enquiry | 10% | 16% |
| Share with a qualified enquiry | 6% | 6.4% |
| Share with a payment | 2% | 1.6% |
| Revenue at 25,000 ₸ | 250,000 ₸ | 200,000 ₸ |
| Contribution at 15,000 ₸ per sale | 150,000 ₸ | 120,000 ₸ |
There are 30 extra leads but only two extra qualified enquiries. The lead rate rose by 6 percentage points, or 60% relative to the previous 10%. The observed paying-visitor rate fell by 0.4 points. The money rows describe those same payments, before acquisition costs and the cost of changing the form.
The manager benefits from knowing the qualified share of leads: 30 / 50 = 60% for A and 32 / 80 = 40% for B. For the main comparison, however, keep every assigned visitor. The form has already affected who submits an enquiry; using only those people as the denominator compares subsets with different compositions.
Before estimating the effect, the analyst reconciles assignments, duplicate IDs, payment links and participants' observation time. If some group B payments were lost in the CRM transfer, restore the data first. Statistics applied to those gaps would measure the difference between reports rather than forms.
I: answer the original question
The table supports this record: "B produced more leads with almost the same number of qualified enquiries. The observed payment rate is 1.6%, against 2% for A. We will estimate the payment effect using the method chosen in advance, including its uncertainty." A difference of two payments is too fragile to pick a winner, just as a gap of one or two customers is in the creative review.
It is also too early to call B harmful because it has two fewer payments. If the planned test has ended and the estimated range includes both useful growth and a decline, record an inconclusive result. The team can keep the previous form and study the new enquiries separately. A new test needs its own plan; adding visitors to the old one until the answer becomes convenient is not acceptable.
The log keeps the original hypothesis, both forms, audience, dates, metrics, final table, data problems and the owner's decision. Keeping the previous form is a valid decision: the team is not expanding the flow of weak enquiries without a clear benefit. The next cycle tests a new explanation.
A cycle does not have to fit into a week
The length of a cycle is set by the outcome window and the flow of people, not by the sprint calendar. The course waits 30 days for payment, while a reminder test measured by second orders within 60 days takes many months; the calculation is in the retention test lesson. A promise of "one confirmed hypothesis every week" will not increase the flow of customers.
Exercise: complete the cycle before launch
Choose a change to a form, message or product step. Record the suspected obstacle, audience, single element being changed, primary metric with its window, and guardrails. Add group sizes, assignment rule, recruitment and readout dates, and the decision owner. State what the team will do with a useful, harmful or inconclusive result.
Then review the course's card: "Two weeks after the start, form B has 40 leads and form A has 25. Switch everyone to B." Which decision does the written plan allow at this point?
The primary metric is payment within 30 days of entering the test, and after 14 days that window has not closed for any participant. Leads show only form completion, and in the final table more leads did not bring more payments. Under the plan, the team continues recruitment, monitors group assignment, duplicates and manager time, and reads the result on the scheduled date. If a guardrail threshold is reached, for example on complaints, the test stops under that rule without declaring a winner.
Video: HADI in practice
A video from the Marketing WHOyarketing channel. Compare the discussion with your card: does it state the decision the team is collecting data to make?