The design of puzzle selection strategies for GWAP systems

The Games With A Purpose (GWAP) genre is a type of Human Computation that outsources certain steps of the computational process to humans. By taking advantage of people's desire to be entertained, GWAP attracts people to play voluntarily, and also produce useful metadata as a by-product. The games have shown promise in solving a variety of problems, which computer computation has been unable to resolve completely thus far. In this paper, we propose a metric, called system gain, for evaluating the performance of GWAP systems, and also use analysis to study the properties of GWAP systems. We argue that it is important for GWAP systems to implement proper puzzle selection strategies in order to collect human intelligence in a more efficient manner. Therefore, based on our analysis, we implement an Optimal Puzzle Selection Strategy (OPSA) to improve GWAP systems. Using a comprehensive set of simulations, we demonstrate that the proposed OPSA approach can effectively improve the system gain of GWAP systems, as long as the number of puzzles in the system is sufficiently large.