This is the sixth in a series of articles on the topic of Respondent Engagement and Respondent Survey Satisfaction.
As we continue our exploration of respondent engagement, in this edition we’ll look at how games are introduced in the context of an online survey. As we’ve mentioned in the past, the best game is one that has a high degree of “visual intuity,” meaning that you can tell how it works just by looking at it. But in some cases, when a gamified element is not familiar, it needs some sort of introduction to play it successfully.
In general, there are four approaches that survey game designers take to prepare respondents for the exercise:
- A written description of the game play
- A series of rules that defines “success” in the outcome
- A demonstration of the game that visually conveys the way the game works
- A test-run of the game before the actual exercise commences
As we study the evolution of survey game design, it is clear that the designer’s notion of how something is supposed to work is not always how the respondent wishes to play. In many interactive game situations, every element in the environment is potentially a clickable object. So while we may want the participant to focus on certain tasks, extraneous items may be distracting. Therefore, simplicity and straight-forward play is a primary design goal.
Many games are simple enough to have just a very quick written introduction to the desired action. For instance, here is an instruction for a packaging “shelf-impact” exercise: “In this game, we will show you a very brief look at a store shelf. Then we will ask you to put the items back on the shelf where you think you saw them. When you are ready, click ‘GO’.” This is usually enough information for simple tasks in a generally recognizable “drag-and-drop” interface.
A rule of thumb is that if the task requires fewer than three actions and the game interface is a familiar Web-based action, a written instruction is probably sufficient. This of course assumes that people will read a sentence or two before wanting to jump in. If the instruction gets to be a paragraph or longer, you will probably need to move to one of the other introductory approaches.
A second, simple type of introduction is one that sets rules for the game. Reg Baker recently sent me a very interesting Web video by SSI’s Pete Cape. In his NewMR presentation, Pete discusses how setting rules for answering creates a game situation in which the respondent is given a task wherein a specific goal defines success. One example he gives is, “You will have 60 seconds to write down all the brands of mobile phones you can think of; you need to get at least 5.” This automatically conveys the notion that you need to meet a certain threshold of response in order to “win.” In the video Pete suggests that this helps defeat survey answer satisficing (where people put forward just enough effort to appear that they have responded).
Chart 1: Illustration from SSI’s Pete Cape’s Presentation; Gamification: The Challenge of Rules
You’ll note that this approach doesn’t require animations or a gaming environment, but rather simply uses a time rule to guide the task completion.
When we start to get into more complex interactions (e.g. more than three actions or an unfamiliar interface), a brief demo of the game may be advisable. In one example, researchers wanted to enforce top-of-mind perceptions of brand associations. For this game, a new interface was created in which attributes would drop from the top of the screen and respondents needed to press a button related to degree of agreement that the brand was described by the attribute before it hit the bottom of the screen—at which point the attribute would explode (indicating a critical failure to make any association). Demonstrating the game play on a brand that was known gave respondents the sense of what needed to be accomplished prior to introducing the real brand being tested.
Illustration 1: Brand Explosion Game Demonstration
Interface Failure to Associate
Finally, when a game is particularly complex or new, a test run-through on the interface may be advisable. German game designers call this a Probespiel, or “audition,” which allows the respondent a real-time experience with the interface prior to the test in which the true stimulus is introduced. When creating a demo of this nature, it is advisable to make it as generic as possible, so that it can be re-used in other gamified surveys. We have found that a demo lasting more than 20 seconds is probably overkill; or conversely, if it takes more than 20 seconds to introduce a concept, it is probably too complex to be a good gamified element.
Note that the test-run that is shown here is for a price estimation game, the total time for the real test is one minute and thirty seconds. In the demo portion however, after the respondent makes the first entry (usually within five seconds), the demo ends and the real game begins.
Illustration 2: Price Estimation Probespiel
Game Instructions Game Play with Test Brands
Note that like Pete Cape’s rules model, this game also had a time limit, but upped the ante by increasing the incentive for correct (real retail price match) answers.
As always, we value your comments and ideas for creating new and successful survey gaming elements!
Bill MacElroy is Chairman of Socratic Technologies, Inc. www.sotech.com