Surveys often include questions about satisfaction. But what is satisfaction anyway? And are there better ways to ask about it?
To measure customer satisfaction, we need to consider the customer’s starting point and the comparisons that drive whatever emotion the customer have might been feeling at the time, and also whether they can recall the emotion at the time we ask about it.
In this post, I’ll reflect on:
- My lack of satisfaction with a satisfaction survey
- The emotions that might be part of satisfaction
- Possible comparisons and the resulting emotions
- The complexity of memorable experiences
Then I’ll recommend that you focus your satisfaction survey on small, vivid, recent parts of customer experience.
I got an unsatisfying request to say whether I was satisfied
Several years ago, I started to think a lot about asking about satisfaction when I received an email message after an encounter with a customer support facility, complete with its odd repetition of the question. Here is a screenshot:
The response options in this message equated good support with satisfied; bad support with unsatisfied. But my polite and charming support person had explained a deeply annoying business rule to me. I was a mixture of conflicting emotions and neither response option worked for me.
Satisfaction is a variety of emotions
My research led me to Richard L. Oliver’s classic text Satisfaction: A Behavioral Perspective on the Consumer. He points out that the state of satisfaction may include a variety of emotions and that their intensity may vary according to how much you care—or putting that into psychological language, on a person’s level of engagement (arousal) or disengagement (quietude). This can make it tricky to understand what satisfaction means:
“It appears that emotional extremes including delight, excitement, and distress are naturally occurring emotions in satisfaction … If a consumer responds that he or she is satisfied, does it mean that this consumer is in a state of contentment, or of pleasure, or of delight?”— Richard L. Oliver
For example: if you’re really interested in something, being satisfied with it might feel really thrilling. If you’re not that interested, maybe being satisfied is only a little better than feeling nothing. So we need to know the customer’s starting point in order to understand the range of emotions that we might be measuring.
Satisfaction depends on comparisons
Let’s think a bit more about the comparisons. To be satisfied, you’ve got to do some mental processing of an experience—to compare it with some sort of mental idea or standard of what the experience ought to have been like.
For example, bronze medal winners tend to be happier than silver medal winners, as described by Victoria Husted Medvec, Scott F. Madey, and Thomas V. H. Gilovich in their 1995 study, When Less Is More: Counterfactual Thinking and Satisfaction Among Olympic Medallists.
A 2006 replication of the study explained it like this:
“Winning any medal in Olympic competition is an amazing accomplishment. Folk beliefs suggest a linear decrease in positive emotions, with the gold medallists experiencing the most, followed by the silver medallists, and then bronze. However, in reality, [in Judo] the silver medallist loses the gold medal match, and the bronze medallists win their last match, capturing the bronze and avoiding going home without a medal…. Bronze medallists from the 1992 Barcelona Games appeared happier than silver medallists at the end of the match and on the podium, and … silver medallists’ comments were much more characterised by counterfactual thinking about ‘what might have been’ and were associated with greater regret and less enjoyable emotion.”— From D. Matsumoto and B. Willingham’s ‘The Thrill of Victory and the Agony of Defeat: Spontaneous Expressions of Medal Winners of the 35 2004 Athens Olympic Games’, from the Journal of Personality and Social Psychology, September 2006 (link to pdf).
Richard L. Oliver delves into the varieties of factual and cognitive comparisons that can contribute to satisfaction or dissatisfaction, which I have summarised drastically in the table below. He points out that, if you don’t process the experience at all, the result is indifference—not dissatisfaction.
| An experience compared to … | A person’s resulting thoughts … |
|---|---|
| Nothing | Indifference |
| Expectations | Better / worse / different |
| Needs | Met / not met / a mixture of the two |
| Excellence / the ideal product | Good / poor quality / good enough |
| Fairness | Privileged / equal / disadvantaged / discriminated against |
| Events that might have been | Vindication / regret |
Memorable experiences are also complex
If the emotions in satisfaction are complicated, what about the experience that gives rise to them?
Let me tell you a story about a rather simple experience. A few years ago, I had my usual day of using a bunch of websites: I browsed some stuff, I bought something or other, I caught up with social media, I checked the news. But the experience that still stands out as memorable: I signed up for the website where I planned to log my routes and training for a half-marathon. Years later, much about that half-marathon is still memorable while many other events of that year are not.
Was the experience of getting started on using a website for my half-marathon training a satisfying one? Well, yes and no.
- I expected it to be straightforward. There are plenty of training websites available, so this one had to be simple.
- I needed it to be reassuring. This half-marathon was double the distance I’d ever attempted.
- Ideally, I wanted it to be inspiring—to suggest ways of conquering this training challenge in ways that I hadn’t yet thought of.
- I feared that it would be too US-centric and wouldn’t understand that I need to be treated a bit differently because I’m a UK-based person.
- If I hadn’t signed up for this site, I’d have had to go hunting for another one. Plus, over a few months, I’d already invested effort getting familiar with this site as an unregistered guest, so looking elsewhere seemed like a lot of work.
And where did I finish? With mixed results. It was mostly easy, but part-way through, there was a bit where I ended up paying a very modest fee to remove an ad. I had thought I was paying to use the site ad free, but then there appeared to be a fee to remove a particular ad rather than remove all advertising. Grumpily, I thought about spending time to find a different option. Later, I realised that I needed to sign in, so the site would know that I’d paid. Just a few bumps along the road, but they tangled me up and undermined the initial experience. Looking back, I had:
- multiple goals—signing up, intentions about long-term use, meeting a personal challenge
- multiple levels of effectiveness—completing the signup, paying the fee, my initial use of the service, my long-term use
- a variety of types of resources—the actual payment, my time, my emotional reaction to the mix-up
- multiple contexts of use—while it was just me at my desk, there are so many different contexts of who I am as a person: beginning runner, UK person, UX consultant, supporter of a particular charity.
So, on a scale of 1 to 10, how satisfied am I? You answer for me! Your guess will be at least as accurate as mine. The point here: memorable experiences tend to be complex ones.
Focus on a brief question about something specific
With all this bad news about complexities, is there any point trying to ask about satisfaction in a survey? My view: yes, definitely. The key is to pick one small aspect of satisfaction, and ask about that. Briefly. And provide space for a free-form comment.
Find out about starting points, a bit at a time
We need to learn something about the customer’s starting point, whatever it was that they are comparing against.
When I was first collecting stories about how UX experts use surveys in their work, Suzanne Boyd of Anthro-Tech told me about asking one simple question:
“Why did you come to this website today?”
Suzanne Boyd, Anthro-Tech
Anthro-Tech has got great results from asking this question, and that makes a lot of sense to me. It’s an easy question, and it explores one of the most important aspects of customer experience: their goals.
Of course, it’s tempting to go on to ask about all the other aspects of a visitor’s experience at the same time. Many of the worst surveys that I see do that —apparently taking the view that, if they’ve managed to grab my attention “to answer this short survey,” I’ll be prepared to answer screen after screen of questions about all sorts of details.
Don’t fall into that trap. Instead, keep people happy by asking just a bit at a time—and not every single time they visit your site either. You’ll keep your overall response rates much higher, and you’ll gradually build up the details that you need through patchworking your insights together.
Ask about their recent, vivid experiences
People are much better at reporting how they feel right now and what they’re doing right now than on their experiences from days or weeks ago. Try to answer these two questions:
- “Did you print anything in the last hour?”
- “How many pages did you print last month?”
Most of us find it much easier to answer the question about the last hour and are much more likely to give an accurate answer. But an even easier question might be:
- “Did you buy a printer this week?”
For most of us, printing is an infrequent but still somewhat trivial task, whereas buying a printer is a rather unusual, more important, and therefore, more vivid task. The more trivial the experience, the less memorable—so the less accuracy you’ll get in people’s answers about it. I call this the “Approximate Curve of Forgetting” and there’s more about it in my book Surveys that work.
Try to ask about recent, vivid experiences.
Ask a sample, not everyone
If you ask every customer to provide feedback, every time they visit your site, that’s a feedback form, not a survey. You know that, your customers know that, and this contributes to declining response rates and poorer quality data—for both your feedback forms and your surveys.
Ask fewer people, and make it clear that you’re deliberately asking only a specific few. They’ll feel more special and be more likely to respond.
For any survey: interview first
Often, stakeholders say “Let’s do a survey, then do some follow-up interviews.”
Survey methodologists agree that a successful survey includes an interview stage. But they recommend interviewing early in the survey process. Interviewing first saves time because:
- It helps you avoid problems like asking questions that users don’t understand or don’t have answers for.
- It gives you preliminary data from the interviews that may be enough to move your project forward without the expense of an actual survey.
Of course, I’m not trying to stop you from doing more interviews after the survey. Listening to your customers is always a good idea. Do lots of it.
Summary: Keep your satisfaction questions short and specific
Satisfaction is a complex emotion, and memorable experiences are also complex.
Don’t try the patience of your survey respondents with long questionnaires that probe every aspect of those complexities. Instead:
- Find out about users’ contexts—a bit at a time.
- Ask about their recent, vivid experiences.
- Ask a sample, not everyone.
- And as with any survey: interview first.
Acknowledgement—Thanks to Francis Rowland for the sketches. Image of printer by Kevin Cortopassi, creative commons
History: this blog post was originally an article published in 2012 on another website. I updated this version in 2025.
If you’d like to know more about the survey as a process and the way in which our memories retain vivid experiences better than trivial ones, there’s more about it in my book Surveys that work
If you would like to see similar advice as a presentation, here are some slides: