PUMA publications for /tag/crowdsourcing%20computinghttps://puma.uni-kassel.de/tag/crowdsourcing%20computingPUMA RSS feed for /tag/crowdsourcing%20computing2024-03-29T03:30:08+01:00Attribute Exploration on the Webhttps://puma.uni-kassel.de/bibtex/245e900e280661d775d8da949baee3747/stummestumme2013-12-16T17:19:49+01:002013 acquisition analysis attribute computing concept crowdsourcing data exploration fca formal human information ir iteg knowledge l3s linked lod open retrieval search sparql web <span class="authorEditorList"><span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Robert Jäschke" itemprop="url" href="/author/Robert%20J%c3%a4schke"><span itemprop="name">R. Jäschke</span></a></span>, und <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Sebastian Rudolph" itemprop="url" href="/author/Sebastian%20Rudolph"><span itemprop="name">S. Rudolph</span></a></span>. </span><span itemtype="http://schema.org/Book" itemscope="itemscope" itemprop="isPartOf"><em><span itemprop="name">Contributions to the 11th International Conference on Formal Concept Analysis</span>, </em></span><em>Seite <span itemprop="pagination">19--34</span>. </em><em>Technische Universität Dresden, </em>(<em><span>Mai 2013<meta content="Mai 2013" itemprop="datePublished"/></span></em>)Mon Dec 16 17:19:49 CET 2013Contributions to the 11th International Conference on Formal Concept Analysismay19--34Attribute Exploration on the Web20132013 acquisition analysis attribute computing concept crowdsourcing data exploration fca formal human information ir iteg knowledge l3s linked lod open retrieval search sparql web We propose an approach for supporting attribute exploration by web information retrieval, in particular by posing appropriate queries to search engines, crowd sourcing systems, and the linked open data cloud. We discuss underlying general assumptions for this to work and the degree to which these can be taken for granted.Learning From Crowdshttps://puma.uni-kassel.de/bibtex/214220abe8babfab01c0cdd5ebd5e4b7c/jaeschkejaeschke2012-06-20T11:57:50+02:00cirg collective computing crowdsourcing extraction human ie information intelligence learning machine ml social <span class="authorEditorList"><span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Vikas C. Raykar" itemprop="url" href="/author/Vikas%20C.%20Raykar"><span itemprop="name">V. Raykar</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Shipeng Yu" itemprop="url" href="/author/Shipeng%20Yu"><span itemprop="name">S. Yu</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Linda H. Zhao" itemprop="url" href="/author/Linda%20H.%20Zhao"><span itemprop="name">L. Zhao</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Gerardo Hermosillo Valadez" itemprop="url" href="/author/Gerardo%20Hermosillo%20Valadez"><span itemprop="name">G. Valadez</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Charles Florin" itemprop="url" href="/author/Charles%20Florin"><span itemprop="name">C. Florin</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Luca Bogoni" itemprop="url" href="/author/Luca%20Bogoni"><span itemprop="name">L. Bogoni</span></a></span>, und <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Linda Moy" itemprop="url" href="/author/Linda%20Moy"><span itemprop="name">L. Moy</span></a></span>. </span><span itemtype="http://schema.org/PublicationIssue" itemscope="itemscope" itemprop="isPartOf"><span itemtype="http://schema.org/Periodical" itemscope="itemscope" itemprop="isPartOf"><span itemprop="name"><em>Journal of Machine Learning Research</em></span></span> </span>(<em><span>August 2010<meta content="August 2010" itemprop="datePublished"/></span></em>)Wed Jun 20 11:57:50 CEST 2012Journal of Machine Learning Researchaug1297--1322Learning From Crowds112010cirg collective computing crowdsourcing extraction human ie information intelligence learning machine ml social For many supervised learning tasks it may be infeasible (or very expensive) to obtain objective and reliable labels. Instead, we can collect subjective (possibly noisy) labels from multiple experts or annotators. In practice, there is a substantial amount of disagreement among the annotators, and hence it is of great practical interest to address conventional supervised learning problems in this scenario. In this paper we describe a probabilistic approach for supervised learning when we have multiple annotators providing (possibly noisy) labels but no absolute gold standard. The proposed algorithm evaluates the different experts and also gives an estimate of the actual hidden labels. Experimental results indicate that the proposed method is superior to the commonly used majority voting baseline.Information Extraction Meets Crowdsourcing: A Promising Couplehttps://puma.uni-kassel.de/bibtex/237cc8f1d19105a073544d6594fbbc033/jaeschkejaeschke2012-06-20T09:49:59+02:00cirg collective computing crowdsourcing extraction human ie information intelligence social <span class="authorEditorList"><span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Christoph Lofi" itemprop="url" href="/author/Christoph%20Lofi"><span itemprop="name">C. Lofi</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Joachim Selke" itemprop="url" href="/author/Joachim%20Selke"><span itemprop="name">J. Selke</span></a></span>, und <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Wolf-Tilo Balke" itemprop="url" href="/author/Wolf-Tilo%20Balke"><span itemprop="name">W. Balke</span></a></span>. </span><span itemtype="http://schema.org/PublicationIssue" itemscope="itemscope" itemprop="isPartOf"><span itemtype="http://schema.org/Periodical" itemscope="itemscope" itemprop="isPartOf"><span itemprop="name"><em>Datenbank-Spektrum</em></span></span> <em><span itemtype="http://schema.org/PublicationVolume" itemscope="itemscope" itemprop="isPartOf"><span itemprop="volumeNumber">12 </span></span>(<span itemprop="issueNumber">2</span>):
<span itemprop="pagination">109--120</span></em> </span>(<em><span>2012<meta content="2012" itemprop="datePublished"/></span></em>)Wed Jun 20 09:49:59 CEST 2012Berlin/HeidelbergDatenbank-Spektrum2109--120Information Extraction Meets Crowdsourcing: A Promising Couple122012cirg collective computing crowdsourcing extraction human ie information intelligence social Recent years brought tremendous advancements in the area of automated information extraction. But still, problem scenarios remain where even state-of-the-art algorithms do not provide a satisfying solution. In these cases, another aspiring recent trend can be exploited to achieve the required extraction quality: explicit crowdsourcing of human intelligence tasks. In this paper, we discuss the synergies between information extraction and crowdsourcing. In particular, we methodically identify and classify the challenges and fallacies that arise when combining both approaches. Furthermore, we argue that for harnessing the full potential of either approach, true hybrid techniques must be considered. To demonstrate this point, we showcase such a hybrid technique, which tightly interweaves information extraction with crowdsourcing and machine learning to vastly surpass the abilities of either technique.Crowdsourced Databases: Query Processing with Peoplehttps://puma.uni-kassel.de/bibtex/229723ba38aa6039091769cd2f69a1514/jaeschkejaeschke2012-06-19T16:29:05+02:00cirg collective computing crowdsourcing database intelligence query social <span class="authorEditorList"><span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Adam Marcus" itemprop="url" href="/author/Adam%20Marcus"><span itemprop="name">A. Marcus</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Eugene Wu" itemprop="url" href="/author/Eugene%20Wu"><span itemprop="name">E. Wu</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Samuel Madden" itemprop="url" href="/author/Samuel%20Madden"><span itemprop="name">S. Madden</span></a></span>, und <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Robert C. Miller" itemprop="url" href="/author/Robert%20C.%20Miller"><span itemprop="name">R. Miller</span></a></span>. </span><span itemtype="http://schema.org/Book" itemscope="itemscope" itemprop="isPartOf"><em><span itemprop="name">Proceedings of the 5th Biennial Conference on Innovative Data Systems Research</span>, </em></span><em>Seite <span itemprop="pagination">211--214</span>. </em><em><span itemprop="publisher">CIDR</span>, </em>(<em><span>Januar 2011<meta content="Januar 2011" itemprop="datePublished"/></span></em>)Tue Jun 19 16:29:05 CEST 2012Proceedings of the 5th Biennial Conference on Innovative Data Systems Researchjan211--214Crowdsourced Databases: Query Processing with People2011cirg collective computing crowdsourcing database intelligence query social Amazon's Mechanical Turk (\MTurk") service allows users to post short tasks (\HITs") that other users can receive a small amount of money for completing. Common tasks on the system include labelling a collection of images, com- bining two sets of images to identify people which appear in both, or extracting sentiment from a corpus of text snippets. Designing a work ow of various kinds of HITs for ltering, aggregating, sorting, and joining data sources together is common, and comes with a set of challenges in optimizing the cost per HIT, the overall time to task completion, and the accuracy of MTurk results. We propose Qurk, a novel query system for managing these work ows, allowing crowd- powered processing of relational databases. We describe a number of query execution and optimization challenges, and discuss some potential solutions.Pushing the boundaries of crowd-enabled databases with query-driven schema expansionhttps://puma.uni-kassel.de/bibtex/241224a60badfeefb0fe2cea85f2a4ff0/jaeschkejaeschke2012-06-12T15:23:55+02:00cirg collective computing crowdsourcing database expansion human intelligence schema social <span class="authorEditorList"><span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Joachim Selke" itemprop="url" href="/author/Joachim%20Selke"><span itemprop="name">J. Selke</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Christoph Lofi" itemprop="url" href="/author/Christoph%20Lofi"><span itemprop="name">C. Lofi</span></a></span>, und <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Wolf-Tilo Balke" itemprop="url" href="/author/Wolf-Tilo%20Balke"><span itemprop="name">W. Balke</span></a></span>. </span><span itemtype="http://schema.org/PublicationIssue" itemscope="itemscope" itemprop="isPartOf"><span itemtype="http://schema.org/Periodical" itemscope="itemscope" itemprop="isPartOf"><span itemprop="name"><em>Proceedings of the VLDB Endowment</em></span></span> <em><span itemtype="http://schema.org/PublicationVolume" itemscope="itemscope" itemprop="isPartOf"><span itemprop="volumeNumber">5 </span></span>(<span itemprop="issueNumber">6</span>):
<span itemprop="pagination">538--549</span></em> </span>(<em><span>Februar 2012<meta content="Februar 2012" itemprop="datePublished"/></span></em>)Tue Jun 12 15:23:55 CEST 2012Proceedings of the VLDB Endowmentfeb6538--549Pushing the boundaries of crowd-enabled databases with query-driven schema expansion52012cirg collective computing crowdsourcing database expansion human intelligence schema social By incorporating human workers into the query execution process crowd-enabled databases facilitate intelligent, social capabilities like completing missing data at query time or performing cognitive operators. But despite all their flexibility, crowd-enabled databases still maintain rigid schemas. In this paper, we extend crowd-enabled databases by flexible query-driven schema expansion, allowing the addition of new attributes to the database at query time. However, the number of crowd-sourced mini-tasks to fill in missing values may often be prohibitively large and the resulting data quality is doubtful. Instead of simple crowd-sourcing to obtain all values individually, we leverage the usergenerated data found in the Social Web: By exploiting user ratings we build <i>perceptual spaces</i>, i.e., highly-compressed representations of opinions, impressions, and perceptions of large numbers of users. Using few training samples obtained by expert crowd sourcing, we then can extract all missing data automatically from the perceptual space with high quality and at low costs. Extensive experiments show that our approach can boost both performance and quality of crowd-enabled databases, while also providing the flexibility to expand schemas in a query-driven fashion.Deco: Declarative Crowdsourcinghttps://puma.uni-kassel.de/bibtex/24de5dd97e5466c9f1fc63c0d23b4d90a/jaeschkejaeschke2012-06-12T15:00:25+02:00cirg collective computing crowdsourcing database deco human intelligence programming social <span class="authorEditorList"><span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Aditya Parameswaran" itemprop="url" href="/author/Aditya%20Parameswaran"><span itemprop="name">A. Parameswaran</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Hyunjung Park" itemprop="url" href="/author/Hyunjung%20Park"><span itemprop="name">H. Park</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Hector Garcia-Molina" itemprop="url" href="/author/Hector%20Garcia-Molina"><span itemprop="name">H. Garcia-Molina</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Neoklis Polyzotis" itemprop="url" href="/author/Neoklis%20Polyzotis"><span itemprop="name">N. Polyzotis</span></a></span>, und <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Jennifer Widom" itemprop="url" href="/author/Jennifer%20Widom"><span itemprop="name">J. Widom</span></a></span>. </span><em>1015. </em><em><span itemprop="producer">Stanford University</span>, </em>(<em><span>2011<meta content="2011" itemprop="datePublished"/></span></em>)Tue Jun 12 15:00:25 CEST 20121015Deco: Declarative Crowdsourcing2011cirg collective computing crowdsourcing database deco human intelligence programming social Crowdsourcing enables programmers to incorporate ``human computation'' as a building block in algorithms that cannot be fully automated, such as text analysis and image recognition. Similarly, humans can be used as a building block in data-intensive applications --- providing, comparing, and verifying data used by applications. Building upon the decades-long success of declarative approaches to conventional data management, we use a similar approach for data-intensive applications that incorporate humans. Specifically, declarative queries are posed over stored relational data as well as data computed on-demand from the crowd, and the underlying system orchestrates the computation of query answers. We present Deco, a database system for declarative crowdsourcing. We describe Deco's data model, query language, and our initial prototype. Deco's data model was designed to be general (it can be instantiated to other proposed models), flexible (it allows methods for uncertainty resolution and external access to be plugged in), and principled (it has a precisely-defined semantics). Syntactically, Deco's query language is a simple extension to SQL. Based on Deco's data model, we define a precise semantics for arbitrary queries involving both stored data and data obtained from the crowd. We then describe the Deco query processor, which respects our semantics while coping with the unique combination of latency, monetary cost, and uncertainty introduced in the crowdsourcing environment. Finally, we describe our current system implementation, and we discuss the novel query optimization challenges that form the core of our ongoing work.CrowdLang - First Steps Towards Programmable Human Computers for General Computationhttps://puma.uni-kassel.de/bibtex/2fe3477c51c6a2159ec1c72ecf299f1fb/jaeschkejaeschke2012-06-05T08:23:00+02:00cirg collective computing crowdsourcing human intelligence social <span class="authorEditorList"><span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Patrick Minder" itemprop="url" href="/author/Patrick%20Minder"><span itemprop="name">P. Minder</span></a></span>, und <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Abraham Bernstein" itemprop="url" href="/author/Abraham%20Bernstein"><span itemprop="name">A. Bernstein</span></a></span>. </span><span itemtype="http://schema.org/Book" itemscope="itemscope" itemprop="isPartOf"><em><span itemprop="name">Proceedings of the 3rd Human Computation Workshop</span>, </em></span><em>Seite <span itemprop="pagination">103--108</span>. </em><em><span itemprop="publisher">AAAI Press</span>, </em>(<em><span>2011<meta content="2011" itemprop="datePublished"/></span></em>)Tue Jun 05 08:23:00 CEST 2012Proceedings of the 3rd Human Computation Workshop103--108AAAI WorkshopsCrowdLang - First Steps Towards Programmable Human Computers for General Computation2011cirg collective computing crowdsourcing human intelligence social Crowdsourcing markets such as Amazon’s Mechanical Turk provide an enormous potential for accomplishing work by combining human and machine computation. Today crowdsourcing is mostly used for massive parallel information processing for a variety of tasks such as image labeling. However, as we move to more sophisticated problem-solving there is little knowledge about managing dependencies between steps and a lack of tools for doing so. As the contribution of this paper, we present a concept of an executable, model-based programming language and a general purpose framework for accomplishing more sophisticated problems. Our approach is inspired by coordination theory and an analysis of emergent collective intelligence. We illustrate the applicability of our proposed language by combining machine and human computation based on existing interaction patterns for several general computation problems.CrowdForge: crowdsourcing complex workhttps://puma.uni-kassel.de/bibtex/2e1022258d8e73b250ff625ce2e10095b/jaeschkejaeschke2012-06-05T08:08:34+02:00cirg collective computing crowdsourcing human intelligence social <span class="authorEditorList"><span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Aniket Kittur" itemprop="url" href="/author/Aniket%20Kittur"><span itemprop="name">A. Kittur</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Boris Smus" itemprop="url" href="/author/Boris%20Smus"><span itemprop="name">B. Smus</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Susheel Khamkar" itemprop="url" href="/author/Susheel%20Khamkar"><span itemprop="name">S. Khamkar</span></a></span>, und <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Robert E. Kraut" itemprop="url" href="/author/Robert%20E.%20Kraut"><span itemprop="name">R. Kraut</span></a></span>. </span><span itemtype="http://schema.org/Book" itemscope="itemscope" itemprop="isPartOf"><em><span itemprop="name">Proceedings of the 24th annual ACM symposium on User interface software and technology</span>, </em></span><em>Seite <span itemprop="pagination">43--52</span>. </em><em>New York, NY, USA, </em><em><span itemprop="publisher">ACM</span>, </em>(<em><span>2011<meta content="2011" itemprop="datePublished"/></span></em>)Tue Jun 05 08:08:34 CEST 2012New York, NY, USAProceedings of the 24th annual ACM symposium on User interface software and technology43--52CrowdForge: crowdsourcing complex work2011cirg collective computing crowdsourcing human intelligence social Micro-task markets such as Amazon's Mechanical Turk represent a new paradigm for accomplishing work, in which employers can tap into a large population of workers around the globe to accomplish tasks in a fraction of the time and money of more traditional methods. However, such markets have been primarily used for simple, independent tasks, such as labeling an image or judging the relevance of a search result. Here we present a general purpose framework for accomplishing complex and interdependent tasks using micro-task markets. We describe our framework, a web-based prototype, and case studies on article writing, decision making, and science journalism that demonstrate the benefits and limitations of the approach.Crowdsourcing systems on the World-Wide Webhttps://puma.uni-kassel.de/bibtex/284f738a6efae5eb6612ea75e8616fecf/jaeschkejaeschke2012-05-30T10:41:17+02:00cirg collective computing crowdsourcing human intelligence social <span class="authorEditorList"><span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Anhai Doan" itemprop="url" href="/author/Anhai%20Doan"><span itemprop="name">A. Doan</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Raghu Ramakrishnan" itemprop="url" href="/author/Raghu%20Ramakrishnan"><span itemprop="name">R. Ramakrishnan</span></a></span>, und <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Alon Y. Halevy" itemprop="url" href="/author/Alon%20Y.%20Halevy"><span itemprop="name">A. Halevy</span></a></span>. </span><span itemtype="http://schema.org/PublicationIssue" itemscope="itemscope" itemprop="isPartOf"><span itemtype="http://schema.org/Periodical" itemscope="itemscope" itemprop="isPartOf"><span itemprop="name"><em>Communications of the ACM</em></span></span> <em><span itemtype="http://schema.org/PublicationVolume" itemscope="itemscope" itemprop="isPartOf"><span itemprop="volumeNumber">54 </span></span>(<span itemprop="issueNumber">4</span>):
<span itemprop="pagination">86--96</span></em> </span>(<em><span>April 2011<meta content="April 2011" itemprop="datePublished"/></span></em>)Wed May 30 10:41:17 CEST 2012New York, NY, USACommunications of the ACMapr486--96Crowdsourcing systems on the World-Wide Web542011cirg collective computing crowdsourcing human intelligence social The practice of crowdsourcing is transforming the Web and giving rise to a new field.Human computation: a survey and taxonomy of a growing fieldhttps://puma.uni-kassel.de/bibtex/23524eeb1e7a62c5bfbe0cec74a14af21/jaeschkejaeschke2012-05-25T16:25:36+02:00cirg collective computing crowdsourcing human intelligence social survey taxonomy <span class="authorEditorList"><span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Alexander J. Quinn" itemprop="url" href="/author/Alexander%20J.%20Quinn"><span itemprop="name">A. Quinn</span></a></span>, und <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Benjamin B. Bederson" itemprop="url" href="/author/Benjamin%20B.%20Bederson"><span itemprop="name">B. Bederson</span></a></span>. </span><span itemtype="http://schema.org/Book" itemscope="itemscope" itemprop="isPartOf"><em><span itemprop="name">Proceedings of the 2011 annual conference on Human factors in computing systems</span>, </em></span><em>Seite <span itemprop="pagination">1403--1412</span>. </em><em>New York, NY, USA, </em><em><span itemprop="publisher">ACM</span>, </em>(<em><span>2011<meta content="2011" itemprop="datePublished"/></span></em>)Fri May 25 16:25:36 CEST 2012New York, NY, USAProceedings of the 2011 annual conference on Human factors in computing systems1403--1412Human computation: a survey and taxonomy of a growing field2011cirg collective computing crowdsourcing human intelligence social survey taxonomy The rapid growth of human computation within research and industry has produced many novel ideas aimed at organizing web users to do great things. However, the growth is not adequately supported by a framework with which to understand each new system in the context of the old. We classify human computation systems to help identify parallels between different systems and reveal "holes" in the existing work as opportunities for new research. Since human computation is often confused with "crowdsourcing" and other terms, we explore the position of human computation with respect to these related topics.