The design of puzzle selection strategies for ESP-like GWAP systems

Ling-Jyh Chen, Bo Chun Wang, Wen Yuan Zhu

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)

Abstract

The games with a purpose (GWAP) genre is a type of human computation that outsources certain steps of the computational process to humans. Although most GWAP studies focus on the design and analysis of GWAP systems, a systematic and thorough evaluation of existing systems is lacking. We address the issue in this paper. Taking the ESP game as an example, we propose a metric, called system utility, for evaluating the performance of GWAP systems, and use analysis to study the properties of the ESP game. We argue that GWAP systems should be designed and played with strategies. To this end, based on our analysis, we implement an optimal puzzle selection strategy (OPSA) to improve GWAP systems. Using a comprehensive set of simulations, we show that the proposed OPSA approach can improve the system utility of the ESP game significantly. In addition, we implement a quasi ESP game, called ESP Lite, which embeds three puzzle selection algorithms transparently and records the complete game trace for evaluation and further research. During a one-month experiment, we have investigated the inner properties of the three strategies in real-world GWAP systems, and verified that the OPSA scheme achieves the best system utility for the ESP game. The results of this study demonstrate that GWAP systems are more efficient if they are designed and played with strategies.

Original languageEnglish
Article number5460962
Pages (from-to)120-130
Number of pages11
JournalIEEE Transactions on Computational Intelligence and AI in Games
Volume2
Issue number2
DOIs
Publication statusPublished - 2010 Jun 1

Keywords

  • Games with a purpose (GWAP)
  • human computation
  • tagging

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Artificial Intelligence
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'The design of puzzle selection strategies for ESP-like GWAP systems'. Together they form a unique fingerprint.

Cite this