Gold Standard Online Debates Summaries and First Experiments Towards Automatic Summarization of Online Debate Data [Best Paper Award]

Abstract

Usage of online textual media is steadily increasing. Daily, more and more news stories, blog posts and scientific articles are added to the online volumes. These are all freely accessible and have been employed extensively in multiple research areas, e.g. automatic text summarization, information retrieval, information extraction, etc. Meanwhile, online debate forums have recently become popular, but have remained largely unexplored. For this reason, there are no sufficient resources of annotated debate data available for conducting research in this genre. In this paper, we collected and annotated debate data for an automatic summarization task. Similar to extractive gold standard summary generation our data contains sentences worthy to include into a summary. Five human annotators performed this task. Inter-annotator agreement, based on semantic similarity, is 36% for Cohen's kappa and 48% for Krippendorff's alpha. Moreover, we also implement an extractive summarization system for online debates and discuss prominent features for the task of summarizing online debate data automatically.

Keywords: online debate summarization, text summarization, semantic similarity, information extraction, sentence extraction

References

Download paper (PDF) Download BibTeX

Dataset: SSSD dataset

@inproceedings{2018_Sanchan_Aker_Bontcheva,
  author    = {Sanchan, Nattapong and Aker, Ahmet and Bontcheva, Kalina},
  title     = {Gold Standard Online Debates Summaries and First Experiments Towards Automatic Summarization of Online Debate Data},
  booktitle = {Computational Linguistics and Intelligent Text Processing},
  editor    = {Gelbukh, Alexander},
  year      = {2018},
  address   = {Cham},
  volume    = {10762},
  series    = {Lecture Notes in Computer Science},
  publisher = {Springer International Publishing},
  pages     = {495--505},
  isbn      = {978-3-319-77116-8}
}

Sanchan, N., Aker, A., Bontcheva, K. (2018) Gold Standard Online Debates Summaries and First Experiments Towards Automatic Summarization of Online Debate Data. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2017. Lecture Notes in Computer Science, vol 10762. Springer, Cham.

More information

This paper is a continuing work in the Domain-Focused Summarization of Polarized Debates project:

1. Understanding Human Preferences for Summary Designs in Online Debates Domain
2. Gold Standard Online Debates Summaries and First Experiments Towards Automatic Summarization of Online Debate Data
3. Automatic Summarization of Online Debates
4. An Adoption of a Contradiction Detection Task to Assist the Summarization of Online Debates