Doing research with people who are not users: consultation

User research is central to our work in User Experience: doing research to find out about our users, then acting on what we’ve learned—or persuading our colleagues to act on a shared understanding of what we have learned.

But what about doing research with other people? This article, co-written with Naintara Land for the August UX Matters, focuses on exactly that: conducting research with people who are not users—that is, people who aren’t part of the target audience for the product or service you’re creating. We call this type of research consultation.

 We admit that, in the past, we’ve often battled with managers who believe that consultation is all that matters and can replace user research. We’ve lost count of the number of conversations we’ve had that went like this:

User researcher: “We need a budget for recruiting users to be research participants.”

Manager: “Why? We have Subject Matter Experts (SMEs) on the team who have years of experience using this product. Why can’t you rely on them for the usability testing?”

Both of us have worked a lot for the UK government, for which a key method of gathering information about a proposed new thing is to publish a consultation paper. The purpose of a consultation is to obtain the views of invited special-interest groups and, in theory at least, interested members of the general public. So another conversation we’ve had too many times goes like this:

User researcher: “We need to do some research with real users.”

Manager: “Why? We’ve received dozens of comments on the recent consultation paper.”

Of course, we understand that part of our role is to explain why we need to do user research throughout a project. However, more recently, we’ve come to see that there is a place for consultation alongside user research. For example, consultation can get you up to speed on a new subject area more quickly. Plus, it can help you identify edge cases and scenarios that you need to accommodate in ways that would be hard to identify through conventional user research. So don’t confuse consultation with user research, but do use both approaches.

In this column, we’ll explain what we mean by consultation, why it’s different from user research, and how we think it can add value.

Consultation involves people who aren’t users

One of the key differences between user research and consultation is who you involve in a research activity.

When we do user research, we work hard to ensure that we find participants who belong to the exact target audience for the product or service we’re working on.

We use the term consultation for any type of research with anyone else. For example, a senior manager decides to try using the product your team is working on. Might that manager be part of the actual target audience? Maybe so, if the product is for busy professionals. But the manager might be nothing at all like an actual user—for example, if the product is for teenagers who are trying to get their first job. In either case, the senior manager’s views are well worth paying attention to, but this type of consultation is not user research.

We’ll focus on consultation with two sorts of people:

  • subject-matter experts (SMEs): often their expertise comes from previous experience as users, but they do not currently belong to the actual target audience.
  • user advocates: these are people from charities, professional organisations, or pressure groups who represent  specific point of view or people with particular needs.
We’ve chosen to focus on consultation with SMEs because many product teams correctly place a lot of value on having people working on their team—whether occasionally or permanently—who have first-hand knowledge and experience of being users, as illustrated in Figure 1. Some people—especially within the disability rights movement—argue that a team cannot be truly user-centred unless it includes people who belong to the exact target audience. The slogan “Nothing about us without us” captures this idea. Demetrius Madrigal and Bryan McClain explore some of the arguments for and against having SMEs on a team in their UXmatters column “Subject-Matter Experts: Putting Users at the Center of the Design Process.”
Figure 1—Many SMEs have extensive experience as users
Many SMEs have extensive experience as users

Image credit: “Users and Experts,” by Naintara Land

We’ve also chosen to consider user advocates because of the fascinating work they do around raising awareness of and creating understanding of the needs of the groups they represent. For example, if you wanted to understand the impact of a new piece of legislation on people in the UK who are deaf or have hearing loss, it would be a good idea to ask Action on Hearing Loss for their views.

Clearly, there are many other variations on these themes. The senior manager we mentioned earlier undoubtedly has valuable subject-matter expertise. SMEs may have been user advocates in the past or continue to combine user-advocacy activities with their current role.

And there’s a variation of this that often applies to us personally, in our work as UX consultants: negotiating the overlaps between our role as consultants—whether as internal employees or outside experts—and consultation, by which we mean providing our knowledge and expertise. Plus, sometimes, to complicate matters further, we happen to actually belong to the target audience for the product or service, so are also users. We deal with this variation by making sure that we frequently ask ourselves, “Which role am I playing right now?”

User research aims to discover what users do

Our overall aim for consultation is to discover what people know about a product or service, a business, or organisational needs. We might conduct a workshop with stakeholders to identify key priorities and areas of interest and concern. We may have one-on-one meetings with SMEs to learn about the history of the product or service—focusing on successes and failures so we can try to avoid repeating old mistakes. Or we might have a public meeting or call for interested parties to comment on proposed changes.

How a consultation differs from user research

Consultation can help you improve the user experience

If you followed the links to investigate the Ministry of Justice consultation, you discovered that the reports are full of very technical language and detailed questions. You may be surprised that we think consultation can improve the user experience of your product or service, but we do, and that’s what we will focus on for the rest of this column.

SMEs help you understand the domain much more quickly

For example, when Caroline was working on a system for farmers in the UK, she found it tremendously helpful to have discussions with an SME who explained many of the technical terms the farmers used in that specific context. For example, the term feature did not mean something the system could do, but instead referred to a permanent structure on the land—such as a water tower. The SME also explained some sensitivities about an earlier version of the system that helped her to prepare for negative reactions to the proposed new system from some users during her subsequent user research.

Referring back to our Ministry of Justice example, a new user researcher working in the area of fees might find it helpful to know that the courts use the term fee remission to mean a reduction in the court fee for a person on a low income.

Advocate groups help you quickly discover the problems of hard-to-reach groups

“A free Web application that monitors your friends’ Tweets, alerting you if it spots anyone who may be struggling to cope. The app gives users a second chance to see potentially worrying Tweets, which might have otherwise been missed.”

Although the original press release for the app mentioned consultation with academic experts, there was an outcry about the potential for those with less than wholesome intentions—such as stalkers, trolls or bullies—to exploit the app. The Samaritans recalled the app nine days after its launch. While we expect that concerns over privacy might well have emerged eventually through user research, perhaps the particular threat that stalkers and bullies posed could have been identified more quickly through consultation with other organisations that support vulnerable people.

Looking again at the Ministry of Justice example, one of the questions that came up during the consultation was specifically about the impact of changes to fees on “court users who have protected characteristics.” In the UK, the Equality Act 2010 uses the term protected characteristics to refer collectively to nine areas in which discrimination is illegal: disability, gender reassignment; marriage and civil partnership; pregnancy and maternity; race; religion or belief; sex, and sexual orientation.

Using the sort of participant recruitment that is typical of user research, it might have been difficult to find someone on a low income who needs help with their court fees and is also willing to disclose their sexual orientation in court. Reaching out to some LBGT+ advocacy groups might be a good way to discover specific concerns of people in those circumstances and would help the team to decide how best to involve them in user research.

Consultation can be misleading

SME experience can become dated or overly focused on your project

SMEs and advocates who are also users may not be typical

Similarly, we’ve seen challenges arise when a particular advocacy group captured stakeholders’ attention to the exclusion of other people who do not share their point of view. For example, one service became fixated on the needs of people with deep concerns about privacy and data sharing—even though the majority of users did not care about this. (Maybe the users ought to have been more worried about these issues, but the fact was that they weren’t.)

Consultation saves time and can deliver better value from user research

You need to do both user research and consultation to resolve the tension between designing for the lowest common denominator, in terms of skill and knowledge, and supporting niche scenarios and edge cases. Consultation can save you time and money by flagging issues before you’ve committed too much effort to designing a solution that may just not be viable.

This article first appeared in the August edition of UX Matters.