Data Gathering Methods for Evaluating Playability of Pervasive Mobile Games

Abstract

This thesis presents new game genre, pervasive games, that mix reality and virtual reality by using context in their gameplay. In this thesis I discuss the data gathering methods that are commonly used when evaluating games, and how adequate they are for pervasive game evaluation. I use two case studies, Coup and Insectopia, as well as thirteen other pervasive game evaluation studies to demonstrate what kind of information can be collected with different data gathering methods, and discuss their benefits and disadvantages for the evaluation.

The results of the example game evaluations show that it is seldom sufficient to use just one method when gathering data for evaluation. Instead, it is recommendable to use at least two methods, one for quantitative and one for qualitative data. Evaluation should be part of the whole game development process from the game idea to the final game product. Different data gathering methods proved to be valuable in different phases of the game development process. Whereas game scenarios and target group discussions can be good for getting information about the game idea and the acceptance of the game, questionnaires, interviews, and log files are good sources to get knowledge about the final game product. The results also show that it is hard to evaluate pervasive aspects of a game since even the evaluator may not know the issues to concentrate on when planning an evaluation. Also it is hard for participants to imagine a pervasive game without any previous experiences of them. Improved practices and new methods for evaluating pervasive games are needed.

Keywords and -phrases: Pervasive games, Playability, Evaluation, Data gathering methods, Player experience, Mobile games