From text to multimodal: a survey of adversarial example generation in question answering systems


Yigit G., AMASYALI M. F.

Knowledge and Information Systems, vol.66, no.12, pp.7165-7204, 2024 (SCI-Expanded) identifier

  • Publication Type: Article / Review
  • Volume: 66 Issue: 12
  • Publication Date: 2024
  • Doi Number: 10.1007/s10115-024-02199-z
  • Journal Name: Knowledge and Information Systems
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, PASCAL, ABI/INFORM, Applied Science & Technology Source, Compendex, Computer & Applied Sciences, INSPEC
  • Page Numbers: pp.7165-7204
  • Keywords: Adversarial datasets, Adversarial evaluation metrics, Adversarial question generation, Question answering, Visual question generation
  • Yıldız Technical University Affiliated: Yes

Abstract

Integrating adversarial machine learning with question answering (QA) systems has emerged as a critical area for understanding the vulnerabilities and robustness of these systems. This article aims to review adversarial example-generation techniques in the QA field, including textual and multimodal contexts. We examine the techniques employed through systematic categorization, providing a structured review. Beginning with an overview of traditional QA models, we traverse the adversarial example generation by exploring rule-based perturbations and advanced generative models. We then extend our research to include multimodal QA systems, analyze them across various methods, and examine generative models, seq2seq architectures, and hybrid methodologies. Our research grows to different defense strategies, adversarial datasets, and evaluation metrics and illustrates the literature on adversarial QA. Finally, the paper considers the future landscape of adversarial question generation, highlighting potential research directions that can advance textual and multimodal QA systems in the context of adversarial challenges.