Share this page:

Content Planning for Neural Story Generation with Aristotelian Rescoring

Seraphina Goldfarb-Tarrant, Tuhin Chakrabarty, Ralph Weischedel, and Nanyun Peng, in the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020.

Download the full text


Abstract

Long-form narrative text generated from largelanguage models manages a fluent impersonation of human writing, but only at the localsentence level, and lacks structure or global cohesion. We posit that many of the problem of story generation can be addressed via high quality content planning, and present a systemthat focuses on how to learn good plot structures to guide story generation. We utilize a plot-generation language model along with an ensemble of rescoring models that each implement an aspect of good story-writing as detailed in Aristotle’s Poetics. We find that stories written with our more principled plot structure are both more relevant to a given prompt and higher quality than baselines that do not content plan, or that plan in an unprincipled way.



Bib Entry

@inproceedings{goldfarb2020content,
  title = {Content Planning for Neural Story Generation with Aristotelian Rescoring},
  author = {Goldfarb-Tarrant, Seraphina and Chakrabarty, Tuhin and Weischedel, Ralph and Peng, Nanyun},
  booktitle = {the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  pages = {4319--4338},
  slideslive_id = {38939240},
  year = {2020}
}

Related Publications

  • Re3: Generating Longer Stories With Recursive Reprompting and Revision

    Kevin Yang, Yuandong Tian, Nanyun Peng, and Dan Klein, in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022.
    Full Text BibTeX Details
    @inproceedings{yang2022re3,
      title = {Re3: Generating Longer Stories With Recursive Reprompting and Revision},
      author = {Yang, Kevin and Tian, Yuandong and Peng, Nanyun and Klein, Dan},
      booktitle = {Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
      year = {2022}
    }
    
    Details
  • Go Back in Time: Generating Flashbacks in Stories with Event Temporal Prompts

    Rujun Han, Hong Chen, Yufei Tian, and Nanyun Peng, in 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2022.
    Full Text Code BibTeX Details
    @inproceedings{han2022go,
      title = {Go Back in Time: Generating Flashbacks in Stories with Event Temporal Prompts},
      author = {Han, Rujun and Chen, Hong and Tian, Yufei and Peng, Nanyun},
      booktitle = {2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)},
      year = {2022}
    }
    
    Details
  • Fantastic Questions and Where to Find Them: FairytaleQA–An Authentic Dataset for Narrative Comprehension

    Ying Xu, Dakuo Wang, Mo Yu, Daniel Ritchie, Bingsheng Yao, Tongshuang Wu, Zheng Zhang, Toby Jia-Jun Li, Nora Bradford, Branda Sun, Tran Hoang, Yisi Sang, Yufang Hou, Xiaojuan Ma, Diyi Yang, Nanyun Peng, Zhou Yu, and Mark Warschauer, in Proceedings of the Conference of the 60th Annual Meeting of the Association for Computational Linguistics (ACL), 2022.
    BibTeX Details
    @inproceedings{xu2022fairy,
      title = {Fantastic Questions and Where to Find Them: FairytaleQA--An Authentic Dataset for Narrative Comprehension},
      author = {Xu, Ying and Wang, Dakuo and Yu, Mo and Ritchie, Daniel and Yao, Bingsheng and Wu, Tongshuang and Zhang, Zheng and Li, Toby Jia-Jun and Bradford, Nora and Sun, Branda and Hoang, Tran and Sang, Yisi and Hou, Yufang and Ma, Xiaojuan and Yang, Diyi and Peng, Nanyun and Yu, Zhou and Warschauer, Mark},
      booktitle = {Proceedings of the Conference of the 60th Annual Meeting of the Association for Computational Linguistics (ACL)},
      year = {2022}
    }
    
    Details
  • Content Planning for Neural Story Generation with Aristotelian Rescoring

    Seraphina Goldfarb-Tarrant, Tuhin Chakrabarty, Ralph Weischedel, and Nanyun Peng, in the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020.
    Full Text Slides Code Abstract BibTeX Details
    Long-form narrative text generated from largelanguage models manages a fluent impersonation of human writing, but only at the localsentence level, and lacks structure or global cohesion. We posit that many of the problem of story generation can be addressed via high quality content planning, and present a systemthat focuses on how to learn good plot structures to guide story generation. We utilize a plot-generation language model along with an ensemble of rescoring models that each implement an aspect of good story-writing as detailed in Aristotle’s Poetics. We find that stories written with our more principled plot structure are both more relevant to a given prompt and higher quality than baselines that do not content plan, or that plan in an unprincipled way.
    @inproceedings{goldfarb2020content,
      title = {Content Planning for Neural Story Generation with Aristotelian Rescoring},
      author = {Goldfarb-Tarrant, Seraphina and Chakrabarty, Tuhin and Weischedel, Ralph and Peng, Nanyun},
      booktitle = {the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
      pages = {4319--4338},
      slideslive_id = {38939240},
      year = {2020}
    }
    
    Details
  • STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generation

    Nader Akoury, Shufan Wang, Josh Whiting, Stephen Hood, Nanyun Peng, and Mohit Iyyer, in the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020.
    Full Text Code Abstract BibTeX Details
    Systems for story generation are asked to produce plausible and enjoyable stories given an input context. This task is underspecified, as a vast number of diverse stories can originate from a single input. The large output space makes it difficult to build and evaluate story generation models, as (1) existing datasets lack rich enough contexts to meaningfully guide models, and (2) existing evaluations (both crowdsourced and automatic) are unreliable for assessing long-form creative text. To address these issues, we introduce a dataset and evaluation platform built from STORIUM, an online collaborative storytelling community. Our author-generated dataset contains 6K lengthy stories (125M tokens) with fine-grained natural language annotations (e.g., character goals and attributes) interspersed throughout each narrative, forming a robust source for guiding models. We evaluate language models fine-tuned on our dataset by integrating them onto STORIUM, where real authors can query a model for suggested story continuations and then edit them. Automatic metrics computed over these edits correlate well with both user ratings of generated stories and qualitative feedback from semi-structured user interviews. We release both the STORIUM dataset and evaluation platform to spur more principled research into story generation.
    @inproceedings{akoury2020storium,
      title = {STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generation},
      author = {Akoury, Nader and Wang, Shufan and Whiting, Josh and Hood, Stephen and Peng, Nanyun and Iyyer, Mohit},
      booktitle = {the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
      slideslive_id = {38939010},
      year = {2020}
    }
    
    Details
  • Plan-And-Write: Towards Better Automatic Storytelling

    Lili Yao, Nanyun Peng, Weischedel Ralph, Kevin Knight, Dongyan Zhao, and Rui Yan, in The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19), 2019.
    Full Text BibTeX Details
    @inproceedings{yao2019plan,
      title = {Plan-And-Write: Towards Better Automatic Storytelling},
      author = {Yao, Lili and Peng, Nanyun and Ralph, Weischedel and Knight, Kevin and Zhao, Dongyan and Yan, Rui},
      booktitle = {The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19)},
      year = {2019}
    }
    
    Details
  • Plan, Write, and Revise: an Interactive System for Open-Domain Story Generation

    Seraphina Goldfarb-Tarrant, Haining Feng, and Nanyun Peng, in 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT 2019), Demonstrations Track, 2019.
    Full Text Video Code Abstract BibTeX Details
    Story composition is a challenging problem for machines and even for humans. We present a neural narrative generation system that interacts with humans to generate stories. Our system has different levels of human interaction, which enables us to understand at what stage of story-writing human collaboration is most productive, both to improving story quality and human engagement in the writing process. We compare different varieties of interaction in story-writing, story-planning, and diversity controls under time constraints, and show that increased types of human collaboration at both planning and writing stages results in a 10-50% improvement in story quality as compared to less interactive baselines. We also show an accompanying increase in user engagement and satisfaction with stories as compared to our own less interactive systems and to previous turn-taking approaches to interaction. Finally, we find that humans tasked with collaboratively improving a particular characteristic of a story are in fact able to do so, which has implications for future uses of human-in-the-loop systems.
    @inproceedings{goldfarb2019plan,
      title = {Plan, Write, and Revise: an Interactive System for Open-Domain Story Generation},
      author = {Goldfarb-Tarrant, Seraphina and Feng, Haining and Peng, Nanyun},
      booktitle = {2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT 2019), Demonstrations Track},
      volume = {4},
      pages = {89--97},
      year = {2019}
    }
    
    Details
  • Towards controllable story generation

    Nanyun Peng, Marjan Ghazvininejad, Jonathan May, and Kevin Knight, in NAACL Workshop, 2018.
    Full Text BibTeX Details
    @inproceedings{peng2018towards,
      title = {Towards controllable story generation},
      author = {Peng, Nanyun and Ghazvininejad, Marjan and May, Jonathan and Knight, Kevin},
      booktitle = {NAACL Workshop},
      year = {2018}
    }
    
    Details