TurKit: human computation algorithms on mechanical turk
G. Little, L. Chilton, M. Goldman, и R. Miller. Proceedings of the 23nd annual ACM symposium on User interface software and technology, стр. 57--66. New York, NY, USA, ACM, (2010)
Аннотация
Mechanical Turk (<i>MTurk</i>) provides an on-demand source of human computation. This provides a tremendous opportunity to explore algorithms which incorporate human computation as a function call. However, various systems challenges make this difficult in practice, and most uses of MTurk post large numbers of independent tasks. TurKit is a toolkit for prototyping and exploring algorithmic human computation, while maintaining a straight-forward imperative programming style. We present the crash-and-rerun programming model that makes TurKit possible, along with a variety of applications for human computation algorithms. We also present case studies of TurKit used for real experiments across different fields.
