User Feedback as a First Class Citizen in Information Integration Systems. Proceedings of the Conference on Innovative Data Systems Research (CIDR), 175--183, 2011. [PUMA: information ie feedback human intelligence social computing cirg collective extraction] URL