After six in-depth interviews, the marketer at an IELTS prep centre in Almaty has customers' words but no proportions. Four of the six brought up pauses in speaking before anyone asked, and three were afraid of signing up for three months at once. Six conversations aren't enough for the owner, and he has already polled his followers in Stories: "What's stopping you from starting IELTS prep?" 240 people voted, and 132 of them tapped "Too expensive". "55%, there's our answer. We're cutting the price to 40,000 ₸ a month."
A survey really does answer "how many", but first you need to know who was counted. Below, the centre builds a questionnaire, sends it out and reads the answers. The centre, the people and their answers are made up; the formulas and the law are real.
Who the Stories poll counted
The 240 votes came from followers who opened Stories that day and felt like tapping an option: school students, parents, former students, people who want English with no IELTS at all, teachers from other centres. People preparing for the exam who don't follow the centre aren't there at all.
So the 55% describes the followers who voted, and for them the number is useful: the centre's posts should explain more often what the price is made of. But it doesn't measure market demand, and it doesn't explain why applicants don't pay after the trial lesson. The question was different, too: it asked what stops people from starting, while the centre needs to know what stopped them from signing up. Before you carry a conclusion over to other groups, you ask those groups directly.
A survey counts, an interview explains
A survey is the right tool when the answer options are already known and what matters is how often each one comes up. A questionnaire won't find a reason you didn't think of: people pick from what you offer. Pew Research Center's guide on how to word survey questions has an example from 2008. Asked which issue mattered most in deciding their vote for president, 58% chose the economy when it was on the list of options, and 35% when people answered in their own words. That's why the options for the questionnaire come from the interviews.
The centre wrote its decision down back in its research plan: what to fix first, the manager's first message, the trial lesson or the price. Pauses in speaking can be counted without a survey, since the teacher logs them in the CRM at the trial lesson. Why someone didn't sign up after the lesson isn't in the CRM, and that's what the questionnaire will ask.
Whom to ask
Before you open a form builder, answer three questions. The American Association for Public Opinion Research (AAPOR), in its best practices for survey research, asks for the answers to them in the survey report.
Who the conclusion is about. For the centre, it's everyone who came to a trial lesson from August to October: 140 people, of whom 47 paid for the course and 93 didn't. The same 47 payments appear in the centre's cohort table: 20, 15 and 12 new students over three months.
Which list reaches them. The list is the CRM, where everyone has a WhatsApp number. Anyone missing from the list is invisible to the survey, however many answers come in.
Everyone or a sample. 140 people isn't many, so the marketer sent the questionnaire to everyone. You sample when the list is large: from a base of 5,000 customers you take, say, 400 random rows by sorting the sheet on a column of random numbers. "The first 400" or "the most active" will tilt the result towards people who are already closer to you. A link in an open channel or in Stories works the other way round: whoever feels like it replies, and you know nothing about those who scrolled past.
The questionnaire: questions about what already happened
The marketer's first draft had to be rewritten almost from scratch.
| Draft | What's wrong | What went into the questionnaire |
|---|---|---|
| Are you happy with the teacher and the schedule? | Two questions in one: the teacher may have been fine and the time not | The schedule became an option in the question about the reason |
| Did you like the useful speaking review? | The word "useful" suggests the answer | Did you find out at the trial lesson where you lose points? |
| Would you sign up if a month of the course cost 40,000 ₸? | A promise about the future costs the person nothing | What most influenced your decision not to sign up for the course? |
| Did you know about our convenient monthly payment? | An ad inside the question | Did you know the course is paid month by month, not for all three months at once? |
Two more tips from the Pew guide. Answer options don't overlap and cover every case, which is why the lists include "Other", and questions about facts also get "Don't remember". And options with no natural order get shuffled: people filling in a questionnaire on their own tend to pick the items at the top.
Those who didn't pay got six questions:
- When is your exam?
- How are you mainly preparing now? (On my own: books, videos, apps / With a tutor / At another centre / Not preparing yet / Other)
- What most influenced your decision not to sign up for the course? One option out of seven.
- Did you know the course is paid month by month, not for all three months at once? (Yes / No / Don't remember)
- Did you find out at the trial lesson where you lose points? (Yes / Partly / No / Don't remember)
- What else should we know? Free text, optional.
Those who paid got a short version: questions 1, 5 and 6.
Before sending it out, the marketer gave the full questionnaire to two people who hadn't paid and one student from the September group, and asked them to retell the questions in their own words. Two asked whether a phone app counts as "On my own", so "books, videos, apps" was added to that option. Pew calls this kind of pretesting an essential step, especially for new questions.
If you need a rating on a scale, use a known one. NPS, under the Bain method, is the share of 9-10 scores minus the share of 0-6 scores on the question "How likely are you to recommend us to a friend or colleague?", answered on a scale from 0 to 10. Passives stay in the calculation, or the same people get a different NPS, as in the twenty-response example in the NPS and CSAT lesson.
How many answers you need
The margin of error for a share at a 95% confidence level is usually worked out with the normal-approximation formula. The NIST handbook calls it the most frequently used, but recommends the Wilson interval: for 13 of 31 it gives 26-59%, close to the formula below. The formula:
margin = ±1.96 × √(p × (1 − p) / n),
where p is the share giving an answer and n is the number of answers. The margin is largest at p = 50%, so that's what you plan with:
| Answers | Margin at a 50% share |
|---|---|
| 30 | ±17.9 pp |
| 100 | ±9.8 pp |
| 400 | ±4.9 pp |
| 1,000 | ±3.1 pp |
The margin is stated in percentage points: "±3 pp" at a 40% share means 37% to 43%. To find how many answers you need, turn the same formula around: n = 1.96² × p × (1 − p) / e², where e is the margin you can live with. For ±5 pp that's 3.8416 × 0.25 / 0.0025 = 384.2, so 385 answers; for ±10 pp, 97.
The formula has four limits:
- It holds for a random sample. According to AAPOR's explainer, it doesn't apply to surveys people come to on their own, such as Stories or an open link.
- A subgroup has its own margin: in a survey of 1,000 people at ±3 pp, a subgroup of 200 gives ±6.9 pp.
- When a noticeable part of a small list replies, the margin is multiplied by √((N − n) / (N − 1)), where N is the list size. According to MeasuringU, the correction becomes noticeable once at least 10% of the list has replied.
- The formula can't see bad questions or the people who didn't answer. Wording errors, AAPOR says, can't be measured, only reduced. The skew from those who stayed silent partly shows when you compare the respondents with the list, as below.
How to read the answers
The questionnaire went out on WhatsApp, with one reminder three days later: AAPOR advises against stopping at a single message, or you can lose whole groups of people. 62 of the 140 replied, 44%, but the groups responded differently:
| Group | Sent | Replied | Response rate |
|---|---|---|---|
| Paid for the course | 47 | 31 | 66% |
| Didn't pay | 93 | 31 | 33% |
| Total | 140 | 62 | 44% |
Those who paid replied twice as readily, and it shows straight away on question 5. "Yes, I found out where I lose points" was the answer of 27 of the 31 who paid and 17 of the 31 who didn't. Together that's 44 of 62, or 71%. But payers are half of those who replied and only a third of those who came to the lesson. Weight each group's share by its size in the list: 47 × 27 / 31 ≈ 41 and 93 × 17 / 31 = 51, so 92 of 140, about 66%. Even this recount rests on the assumption that the silent people in each group would have answered like those who replied.
The main question, number 3, went only to those who didn't pay:
| What most influenced the decision not to sign up | Answers out of 31 |
|---|---|
| Didn't want to sign up for three months at once | 13 |
| Too expensive | 5 |
| Chose a tutor or another centre | 5 |
| Inconvenient schedule | 3 |
| Decided to prepare on my own | 2 |
| Postponed the exam or the application | 2 |
| Other | 1 |
Thirteen of 31 named the three months, 42%. The margin for that share at 31 answers is ±17 pp, and about ±14 pp with the correction for a list of 93. If the respondents were random, the share among all 93 would most likely sit somewhere between 28% and 56%. But they chose themselves, 62 of the 93 stayed silent, and someone who went to a competitor may simply not have wanted to explain why. So the conclusion is modest: among those who replied, fear of the three months comes up more often than any other reason, and five chose "Too expensive".
Question 4 hints at where the fear comes from. Of the 13 who named the three months, 11 didn't know the course is paid month by month. They may have thought they'd have to pay 135,000 ₸ up front, when the first month costs 45,000 ₸.
The fix is cheap: the manager's first message and the teacher at the end of the trial lesson both say the course is paid monthly. Whether it works will show in payments: how many of those who come pay for the first month and how many of them for the second, as the marketer wrote down after the interviews. The centre leaves the price alone for now. "Too expensive" came from 5 of 31, and a discount to 40,000 ₸ a month, as the centre worked out with the five forces, would need 54 students instead of 37 to keep the same contribution.
In Stories, 55% chose "Too expensive"; here, 16%. The numbers don't contradict each other: these are different people answering different questions.
The tool and personal data
Any form builder will handle a six-question questionnaire: Google Forms, Typeform, Yandex Forms. Look at the response limit on the free plan and at where the answers are stored.
The centre used Google Forms and made two copies of the questionnaire: one went to those who paid, the other to everyone else. That way the groups are told apart without asking for names or phone numbers. The marketer didn't tick "Limit to 1 response": with it, the form can be filled in only after signing in to a Google Account, which is an extra step for someone who opened the link from WhatsApp.
Messaging numbers from the CRM is processing personal data, governed by the Law on Personal Data and Its Protection (as amended up to 14 September 2026). What follows is our reading of the law, not legal advice.
- The phone number isn't on the list in Article 6, but it's safer to treat it as personal data.
- The number was left to book a trial lesson, and a survey is a different purpose (Art. 14), so you can message that number if the consent names this purpose. The example rests on one condition: the centre's consent at booking includes a line about short questions after the lesson. If yours doesn't, decide with a lawyer.
- Proving consent falls to whoever collected the data (Art. 25(2)(5)).
- Databases with personal data are kept in Kazakhstan (Art. 12(2)), while Google Forms answers sit with Google. One more reason not to ask for a name, phone number or email in the questionnaire.
Exercise: a survey for your own project
Take a "how many" question left over after interviews or chats with customers.
If the first line has no decision in it, the survey will produce interesting numbers you can't do anything with: put it off until you have a choice between two or three options. If your list is followers or an open link, write "out of 240 followers who voted" in the report, not "55% of customers", and leave out the margin of error. At 30 answers the margin is about ±18 pp: a survey like that will tell a reason named by half from a reason named by two people, but not 30% from 40%. If the questionnaire went to different groups, work out the response rate in each one first. And rewrite questions with "would you", "convenient" or "useful" into questions about what the person has already done.
Video: NPS and CSI
Sources
- Pew Research Center: Writing Survey Questions.
- AAPOR: Best Practices for Survey Research.
- AAPOR: Margin of Sampling Error.
- NIST/SEMATECH: Confidence intervals for a proportion.
- MeasuringU: How to Use the Finite Population Correction.
- Bain & Company: Measuring Your Net Promoter Score.
- Law of the Republic of Kazakhstan on Personal Data and Its Protection.