The phrase “gamification” first emerged around the turn of the century and is generally credited to British game creator Nick Pelling. As online gaming platforms began to appear, such as massively multiplayer online role-playing games (MMORPG) like EverQuest (1999), and virtual reality systems like Second Life (2003), etc., the research community began to imagine how surveys could be adapted to feature similar game-play elements. Early attempts at “gamification” featured building game characters and creating virtual interviewing systems.
I remember at the time that many were saying that this “virtualization” of research would mean the end of Web-based surveys as we knew them. Being an avid gamer, I was very enthusiastic about this concept.
However, when many such technological discontinuities occur (Kuhn, T. 1996) the first notions as to how the paradigm will eventually shift, tend to be wrong. As many people began to experiment with transforming research into games, we began to reach the same conclusion: simply turning surveys into games does work very well.
One of our game clients in the early 2000s, decided that their surveys needed to display the same sort of “fun and entertaining” elements as their products. As a result, we experimented with having survey questions surrounded by animated flying saucers, meteors and space ships. When we looked at the results, we found that the results were not very good (the answers had a poor level of internal consistency). In addition, we noted that the average (mode) time to complete a survey was much longer than it should have been. Because the respondents were customers, we were able to re-contact them to ask some follow-up questions and found that a great deal of time had been spent trying to figure out how to shoot down the animated elements and that the recall of the survey topic was very low.
The Second Life virtual reality avatar experiments in the mid-2000s were even less fruitful. When we asked respondents to create avatars and take place in virtual conversations, the time-cost-effort curve completely overwhelmed the value of the results. Clearly, creating whole gaming environments to support surveys was not economically feasible.
A very good journal article summarizing a great many of the early failures in this arena is: “Myths and Realities of Respondent Engagement in Online Surveys” (Downes-LeGuin, T., and et. al., 2012.) In this paper, they did side-by-side comparisons of traditional forms of online survey questions versus “game element” environments including functionally illustrated surveys (slider bars, drag-and-drop exercises) and full avatar creations. As seen in previous research, they concluded that the time-cost-effort was not worth it and that, in fact, the extra time it took to answer questions was perceived to be a burden, not an intrinsic enhancement to the survey experience.
So while we find that turning surveys into games does not work very well, there are elements of the gaming activity that do yield positive results. This involves taking questions and transforming them into micro-games that can be quickly mastered with little or no explanation. In design, this is called “visual intuity,” i.e. you know what to do just by looking at it.
Some of the gamified elements that have been successful have been gleaned from great, old focus group and intercept techniques that are simply being transferred online. One area that has been found to be very fruitful is turning dull, associative type questions into concentration tasks. These tasks, which involve memory and concentration increase the level of presence and create a pleasurable experience when correct associations are made.
The field of packaging and design research has a rich history of data collection tools that were fun. Some of these date back to the 1950s when mechanical devices were used to measure the power and memorability of graphic art. One such device was the Blur Meter—which used a lens to distort a design and then slowly bring the art into focus until the brand was recognizable. In the online version, some researchers use Gaussian blur and a timed resolution to accomplish the same measurement. The result is that the researcher can now use an old technique in a new way that is faster, more standardized, and provides quantitative level sample sizes that would have been prohibitively expensive in the past.
Blur Meter Threshold of Logo Recognition
Another such mechanical device was the Tachistoscope (Greek for “rapid vision”). This mall intercept tool was an aperture that fit over the end of a 35mm projector; it had a hand-held plunger that would open and close the aperture for just a second, giving respondents just a glimpse of the art being projected. The average number (or length) of exposures that was required for the “threshold of recognition” for the brand showed how impactful the design was in breaking through a cluttered competitive shelf environment. In the online version, researchers can now use HTML5’s “timed events” to perfectly recreate a timed exposure and capture similar data.
T-Scope Shelf Reconstruction Following 1-Sec Exposure
Both of these examples demonstrate that successful gamification does not attempt to transform a survey into a game, but rather takes interesting and engaging tasks and uses the power of Web interactivity to collect data in entertaining ways. Levels of stated respondent satisfaction with this type of survey is significantly higher than text-only surveys.
Please comment with other ideas and to share your experiences.
Kuhn, T. (1996) The Structure of Scientific Revolutions, Third Edition, University of Chicago Press.
Downes-Le Guin, T., Baker, R., Mechling, J. & Ruylea, E. (2012) Myths and realities of respondent engagement in online surveys. International Journal of Market Research, Vol. 54. Issue 5.
Bill MacElroy is Chairman of Socratic Technologies, Inc. www.sotech.com.