Definitions chapter: “What is a survey?” and the Survey Octopus

My definition of a survey is modelled on the one in (Groves et al. 2009):

a systematic method for gathering information from (a sample of) entities for the purpose of constructing quantitative descriptors of the attributes of the larger population of which the entities are members. (Groves et al. 2009)

The survey methodologists are mainly writing for people who are, or are training as survey methodologists, often working in large organisations and sometimes in National Statistical Institutes. Survey methodologists typically work on descriptive surveys, which is why their definition has “for the purpose of constructing quantitative descriptors of the attributes of a larger population of which the entities are members”. They are also likely to survey organisations such as businesses, which is why their definition refers to “entities” – a term that can include both individual people and organisations. 

My book is aimed at user experience (UX) professionals, market researchers, and other people who are not survey methodologists and so I wanted to move to a definition that reflects current practice in more informal settings. My changes are:

  • A systematic method” becomes “process”, because the word “process” is more familiar in business settings.
  • Gathering information from” becomes “asking questions that are answered by”, because these days we have a variety of passive methods of gathering information such as web analytics, customer usage patterns, and to me the essential part of a survey is that is asks questions.
  • Entities” becomes “a defined group of people”, because I wanted to make it clear that people will answer, but not just any people. I return to the topic of how to define a group of people in Chapter 1, “Goals” and again in Chapter 2, “Sample”.
  • constructing quantitative descriptors” becomes “get numbers”. The original definition is clearly more precise, but I found that the word “numbers” worked better in conversations with UX professionals and stakeholders
  • “the attributes of the larger population of which the entities are members” becomes “you can use to make decisions”. This is really quite a departure from the original. The survey methodologists are typically aiming to create quantitative descriptors for the population, and then leave it to the people who will use those descriptors (known in this context as “data users” to decide exactly how they will use them: for example, for academic purposes, for planning government decisions, or for triangulating with other data sets. So there is an element of decision-making in there, but it’s a long way from the definition. I discovered that when I’m working with people who are doing surveys, it really helps them to do better surveys when I bring “to make decisions” right up into the definition, so that’s what I did. 

My definition is:

A process of asking questions that are answered by a sample of a defined group of people to get numbers that you can use to make decisions. 

I’ll leave it to you to decide whether my ‘numbers you can use to make decisions’ is a pragmatic approximation of the rest of their definition, or an inappropriate over-simplification.  (Jarrett, 2021)

References:

Groves et al, 2009: Robert M. Groves,  Floyd J. Fowler, Mick P. Couper, James. M. Lepkowski, Eleanor Singer, Roger Tourangeau, Survey Methodology, 2nd Edition, WIley

If you’re only going to tackle one book on survey methodology, get this one. It’s packed with references to take you to the next step in your search. Stocked by many libraries; expensive new, relatively easy to find second-hand at an affordable price.