PUMA publications for /author/Luca%20Bogonihttps://puma.uni-kassel.de/author/Luca%20BogoniPUMA RSS feed for /author/Luca%20Bogoni2024-03-29T10:50:34+01:00Learning From Crowdshttps://puma.uni-kassel.de/bibtex/214220abe8babfab01c0cdd5ebd5e4b7c/jaeschkejaeschke2012-06-20T11:57:50+02:00information crowdsourcing human learning computing collective machine extraction ie social intelligence cirg ml <span class="authorEditorList"><span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Vikas C. Raykar" itemprop="url" href="/author/Vikas%20C.%20Raykar"><span itemprop="name">V. Raykar</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Shipeng Yu" itemprop="url" href="/author/Shipeng%20Yu"><span itemprop="name">S. Yu</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Linda H. Zhao" itemprop="url" href="/author/Linda%20H.%20Zhao"><span itemprop="name">L. Zhao</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Gerardo Hermosillo Valadez" itemprop="url" href="/author/Gerardo%20Hermosillo%20Valadez"><span itemprop="name">G. Valadez</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Charles Florin" itemprop="url" href="/author/Charles%20Florin"><span itemprop="name">C. Florin</span></a></span>, <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Luca Bogoni" itemprop="url" href="/author/Luca%20Bogoni"><span itemprop="name">L. Bogoni</span></a></span>, und <span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Linda Moy" itemprop="url" href="/author/Linda%20Moy"><span itemprop="name">L. Moy</span></a></span>. </span><span itemtype="http://schema.org/PublicationIssue" itemscope="itemscope" itemprop="isPartOf"><span itemtype="http://schema.org/Periodical" itemscope="itemscope" itemprop="isPartOf"><span itemprop="name"><em>Journal of Machine Learning Research</em></span></span> </span>(<em><span>August 2010<meta content="August 2010" itemprop="datePublished"/></span></em>)Wed Jun 20 11:57:50 CEST 2012Journal of Machine Learning Researchaug1297--1322Learning From Crowds112010information crowdsourcing human learning computing collective machine extraction ie social intelligence cirg ml For many supervised learning tasks it may be infeasible (or very expensive) to obtain objective and reliable labels. Instead, we can collect subjective (possibly noisy) labels from multiple experts or annotators. In practice, there is a substantial amount of disagreement among the annotators, and hence it is of great practical interest to address conventional supervised learning problems in this scenario. In this paper we describe a probabilistic approach for supervised learning when we have multiple annotators providing (possibly noisy) labels but no absolute gold standard. The proposed algorithm evaluates the different experts and also gives an estimate of the actual hidden labels. Experimental results indicate that the proposed method is superior to the commonly used majority voting baseline.