From text to multimodal: a survey of adversarial example generation in question answering systems

Yigit G., AMASYALI M. F.

Knowledge and Information Systems, cilt.66, sa.12, ss.7165-7204, 2024 (SCI-Expanded) identifier

  • Yayın Türü: Makale / Derleme
  • Cilt numarası: 66 Sayı: 12
  • Basım Tarihi: 2024
  • Doi Numarası: 10.1007/s10115-024-02199-z
  • Dergi Adı: Knowledge and Information Systems
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, PASCAL, ABI/INFORM, Applied Science & Technology Source, Compendex, Computer & Applied Sciences, INSPEC
  • Sayfa Sayıları: ss.7165-7204
  • Anahtar Kelimeler: Adversarial datasets, Adversarial evaluation metrics, Adversarial question generation, Question answering, Visual question generation
  • Yıldız Teknik Üniversitesi Adresli: Evet

Özet

Integrating adversarial machine learning with question answering (QA) systems has emerged as a critical area for understanding the vulnerabilities and robustness of these systems. This article aims to review adversarial example-generation techniques in the QA field, including textual and multimodal contexts. We examine the techniques employed through systematic categorization, providing a structured review. Beginning with an overview of traditional QA models, we traverse the adversarial example generation by exploring rule-based perturbations and advanced generative models. We then extend our research to include multimodal QA systems, analyze them across various methods, and examine generative models, seq2seq architectures, and hybrid methodologies. Our research grows to different defense strategies, adversarial datasets, and evaluation metrics and illustrates the literature on adversarial QA. Finally, the paper considers the future landscape of adversarial question generation, highlighting potential research directions that can advance textual and multimodal QA systems in the context of adversarial challenges.