Update in 2026: I wrote this years before I did the research for my own book on surveys. At that time, I thought absolute response rates were important – but now I understand that the representativeness of the response is more important than the response rate. This version now reflects my current views.
In the 1950s, a well-designed survey could often achieve over 90% response rates. Since then, response rates have consistently declined.
But I was still a bit shocked the other day when a post on a usability discussion group quoted a ‘typical response rate of 2%’ as if that were something we all knew as a fact. And it’s true: sometimes we can get 2%, or even lower, as a response rate.
A survey with a low response rate has a greater chance of being unrepresentative
2% can be a worrying response rate. Why? Because there’s such a big chance that the people who filled in the survey are different from the people who didn’t respond in a way that can affect the result of the survey. (Technically, ‘non-response bias’).
With any survey, we need to look at the profile of the people who responded and satisfy ourselves that they are about the same as the people who didn’t respond – and also, that they’re about the same as the overall population that you’re sampling.
So we try really hard to design a sampling method that gives everyone in the population an equal chance of being selected as a potential respondent. We are trying to make sure that the sample we get has good coverage from the population, and also that it is representative of the population.
If the overall response rate is poor, that can be a flag that we’ve got some systematic non-response that might harm the representativeness. It’s not absolutely inevitable: I have got good results from surveys with terrible response rates (banners at the top of high traffic websites, with maybe 0.01% response rates), but it’s harder to convince myself, and the people who might act on the results, that the response is truly representative with these very low response rates.
Improve your response rate by better survey design
Here are some tips for improving your response rate.
- Ask fewer people. Choose a small sample, make sure that those people know that they have been specially selected, and spend a bit of time and effort on following up with each of them. Feeling special makes people more likely to respond.
- Ditch the prize draw and use the money for an incentive that they get before filling in the questionnaire. A dollar bill sent with a mail survey gets a better response rate than ten dollars guaranteed on returning the survey. Prize draws can have surprisingly little effect on response rates.
- Make the questionnaire SHORT (yup, I’m yelling ‘short’ at you). Longer = more off-putting.
- Make the questionnaire interesting. A topic that people want to talk about will get a better response than being asked, again, to rate some routine transaction such as buying something from a vendor that you use frequently.
- Test, test and test again. That’s how you’ll find out whether it’s short enough and interesting enough to get a response.
- Read a good book on the topic. When I first wrote this in 2005, I recommended Don Dillman (2000) Mail and Internet Surveys: The Tailored Design Method and I’m still very happy to recommend it – or the more recent Dillman, Smyth and Christian (2014) Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method. Or even consider buying my book on surveys.
A different version of this article first appeared in Usability News, the online magazine of BCS HCI group (now BCS Interaction specialist group) which is no longer online.
#surveys #surveysthatwork