Progression in a Language Annotation Game with a Purpose
View/ Open
Published version
Embargoed until: 2222-01-01
Embargoed until: 2222-01-01
Publisher
Publisher URL
Location
Metadata
Show full item recordAbstract
Within traditional games design, incorporating progressivedifficulty is considered of fundamental importance. But de-spite the widespread intuition that progression could haveclear benefits in Games-With-A-Purpose (GWAPs)–e.g., fortraining non-expert annotators to produce more complexjudgements– progression is not in fact a prominent featureof GWAPs; and there is even less evidence on its effects. Inthis work we present an approach to progression in GWAPsthat generalizes to different annotation tasks with minimal, ifany, dependency on gold annotated data. Using this methodwe observe a statistically significant increase in accuracy overrandomly showing items to annotators.