PUMA publications for /bibtexkey/Teufel01task-basedevaluation/hothohttps://puma.uni-kassel.de/bibtexkey/Teufel01task-basedevaluation/hothoPUMA RSS feed for /bibtexkey/Teufel01task-basedevaluation/hotho2024-03-29T13:15:02+01:00Task-Based Evaluation of Summary Quality: Describing Relationships between Scientific Papershttps://puma.uni-kassel.de/bibtex/2b95470c8eae5d8f0372d20215c35f236/hothohotho2014-11-26T09:44:26+01:00dh sota structure text tp1 <span class="authorEditorList"><span itemtype="http://schema.org/Person" itemscope="itemscope" itemprop="author"><a title="Simone Teufel" itemprop="url" href="/author/Simone%20Teufel"><span itemprop="name">S. Teufel</span></a></span>. </span><span itemtype="http://schema.org/Book" itemscope="itemscope" itemprop="isPartOf"><em><span itemprop="name">In Workshop Automatic Summarization, NAACL</span>, </em></span><em>Seite <span itemprop="pagination">12--21</span>. </em>(<em><span>2001<meta content="2001" itemprop="datePublished"/></span></em>)Wed Nov 26 09:44:26 CET 2014In Workshop Automatic Summarization, NAACL12--21Task-Based Evaluation of Summary Quality: Describing Relationships between Scientific Papers2001dh sota structure text tp1 We present a novel method for task-based evalua- tion of summaries of scientific articles. The task we propose is a question-answering task, where the questions are about the relatedness of the current paper to prior research. This evaluation method is time-efficient with respect to material preparation and data collection, so that it is possible to test against many different baselines, something that is not usually feasible in evaluations by relevance decision. We use this methodology to evaluate the quality of summaries our system produces. These summaries are designed to describe the contribution of a scientific article in relation to other work. The re- sults show that this type of summary is indeed more useful than the baselines (random sentences, keyword lists and generic author-written summaries), and nearly as useful as the full texts.Task-Based Evaluation of Summary Quality: Describing Relationships between Scientific Papers