TY  - CONF
AU  - Parameswaran, Aditya
AU  - Polyzotis, Neoklis
A2  - 
T1  - Answering Queries using Humans, Algorithms and Databases
T2  - Conference on Inovative Data Systems Research (CIDR 2011)
PB  - 
CY  - 
PY  - 2011/01
M2  - 
VL  - 
IS  - 
SP  - 160
EP  - 166
UR  - http://ilpubs.stanford.edu:8090/986/
M3  - 
KW  - algorithm
KW  - cirg
KW  - collective
KW  - computing
KW  - database
KW  - human
KW  - intelligence
KW  - social
L1  - 
SN  - 
N1  - 
N1  - 
AB  - For some problems, human assistance is needed in addition to automated (algorithmic) computation. In sharp contrast to existing data management approaches, where human input is either ad-hoc or is never used, we describe the design of the first declarative language involving human-computable functions, standard relational operators, as well as algorithmic computation. We consider the challenges involved in optimizing queries posed in this language, in particular, the tradeoffs between uncertainty, cost and performance, as well as combination of human and algorithmic evidence. We believe that the vision laid out in this paper can act as a road-map for a new area of data management research where human computation is routinely used in data analytics.
ER  -

TY  - CONF
AU  - Little, Greg
AU  - Chilton, Lydia B.
AU  - Goldman, Max
AU  - Miller, Robert C.
A2  - 
T1  - TurKit: human computation algorithms on mechanical turk
T2  - Proceedings of the 23nd annual ACM symposium on User interface software and technology
PB  - ACM
CY  - New York, NY, USA
PY  - 2010/
M2  - 
VL  - 
IS  - 
SP  - 57
EP  - 66
UR  - http://doi.acm.org/10.1145/1866029.1866040
M3  - 10.1145/1866029.1866040
KW  - algorithm
KW  - cirg
KW  - collective
KW  - computing
KW  - human
KW  - intelligence
KW  - social
KW  - turkit
L1  - 
SN  - 978-1-4503-0271-5
N1  - 
N1  - 
AB  - Mechanical Turk (<i>MTurk</i>) provides an on-demand source of human computation. This provides a tremendous opportunity to explore algorithms which incorporate human computation as a function call. However, various systems challenges make this difficult in practice, and most uses of MTurk post large numbers of independent tasks. TurKit is a toolkit for prototyping and exploring algorithmic human computation, while maintaining a straight-forward imperative programming style. We present the crash-and-rerun programming model that makes TurKit possible, along with a variety of applications for human computation algorithms. We also present case studies of TurKit used for real experiments across different fields.
ER  -