Preliminary Performance Assessment on Ask4Summary’s Reading Methods for Summary Generation

Rita Kuo, Maria F. Iriarte, Di Zou, Maiga Chang

Research output: Chapter in Book/Report/Conference proceedingPublished Conference contributionpeer-review

Abstract

Ask4Summary creates summary for students’ questions based on text-based learning materials. This study conducts a preliminary assessment on Ask4Summary’s performance in terms of generating summaries with different subsets of course materials (e.g., supplement academic papers in PDF only, notes and slides in Word and PowerPoint only, and everything the teacher provides for the students) read and processed by two reading methods: the built-in algorithm based on Python NLTK and AWS Comprehend Keyphrase Extraction and Syntax Analysis. The course materials of a graduate level Academic Writing in English course in an Asian university and twenty-six common questions that students may ask in the class are provided by the course instructor. Each of the questions are read via the two methods and Ask4Summary generates the summaries with the six different datasets created by: (1) Python NLTK reading the academic papers in PDF only; (2) Python NLTK reading notes and slides in Word and PowerPoint format only; (3) Python NLTK reading every course materials; (4) AWS Comprehend reading academic papers in PDF only; (5) AWS Comprehend reading notes and slides in Word and PowerPoint format only; and (6) AWS Comprehend reading every course materials. For the 312 queries (i.e., ask 26 questions in 6 datasets with 2 methods analyzing the questions) made, 117 queries successfully generated the summary, where only 2 of them were read by AWS Comprehend. Among the rest of 115 summaries, 67 of them are from the datasets created via the built-in algorithm and 48 are from the datasets created by AWS Comprehend.

Original languageEnglish
Title of host publicationAugmented Intelligence and Intelligent Tutoring Systems - 19th International Conference, ITS 2023, Proceedings
EditorsClaude Frasson, Phivos Mylonas, Christos Troussas
Pages630-637
Number of pages8
DOIs
Publication statusPublished - 2023
Event19th International Conference on Augmented Intelligence and Intelligent Tutoring Systems, ITS 2023 - Corfu, Greece
Duration: 2 Jun. 20235 Jun. 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13891 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference19th International Conference on Augmented Intelligence and Intelligent Tutoring Systems, ITS 2023
Country/TerritoryGreece
CityCorfu
Period2/06/235/06/23

Keywords

  • AWS
  • Language Learning
  • Learning Materials
  • NLTK
  • Natural Language Processing

Fingerprint

Dive into the research topics of 'Preliminary Performance Assessment on Ask4Summary’s Reading Methods for Summary Generation'. Together they form a unique fingerprint.

Cite this