A few years ago, a rather dull conference speaker spoke one phrase that stuck in my mind. He said: “I was at this meeting, and I heard a voice going ‘blah blah blah’. And then I realised: it was me”. Since then, I’ve tried to listen to myself in meetings – and even tried to listen to the other people, much harder. But this month it was something I said that surprised me. In two separate meetings. Both times, I heard myself say: “I really don’t think you need any more data at this point”.
Why no more data is surprising
A constant theme of my life as a usability consultant has been persuading clients to get real data:
- To get them to try some usability testing.
- To go to observe users in their natural habitat.
- To monitor the incoming post, the telephone calls, the drop-in centres and appointments: any contact they have with users.
- To get good web logs and statistics.
What happened? Why did I say ‘don’t do more testing’ at one client meeting? And at another meeting, why did I turn down the opportunity to do one of my favourite things: analysing a thousand emails for common patterns? (I’m not kidding: I really, really enjoy hunting through and extracting insights from large data sets). The common theme for both these clients is that they have already done enough data gathering for the moment.
User-centred design needs concentrated thought
One client has just completed a very complex usability test of a new information architecture for an important, rich site that is highly valued by its many users. The test went well, and most things either did or did not work. But now they are stuck on a couple of problems where the findings weren’t all that clear.
They have learned how valuable usability testing can be, so their first impulse is rush back to the users and try some more testing. But when we look at the design, none of us thinks it’s quite right. It’s not too bad, it’s not too good. No wonder the findings were mixed.
So truly, I don’t think more testing would help them right now. What they need to do is some hard thinking, to come up with a solution that they believe will really work.
User-centred design is design
Another client has recently implemented a major, painful back-end system change. They knew all along that the user interface wasn’t getting the attention it deserved, but the stability of the back-end was more crucial. Problem is: they have been swamped by a swarm of complaints. They have a lot of users, all of whom know how to use email. And a significant proportion have opinions about the correct operation of websites. Oops.
This isn’t news to us: the findings of several rounds of testing said exactly what was wrong with that user interface. We’re all together in the crisis meeting, and someone says “We ought to analyse those emails to find out exactly what they don’t like. Well, OK, a sample, say 1000. We need that data”. So, so tempting.
Not only the fun of reducing emails to order, but also the fun of proving that yes, we really were right in our predictions. But I resist. They don’t need any more data right now: they need design. Time to produce some wireframes and requirements that will deal with the most obvious problems. It really doesn’t matter if it’s 7% or 17% of the users who are complaining – even 1% would be far too many.
Getting the balance right
Spending all your time on design without exposing your ideas to users is definitely bad. Ignoring real-world data will bite you in the end. But it’s all a balance: the right amount of data, the right amount of insight, and the right amount of getting down to doing something about it.
This article first appeared in Usability News, 3 February 2008