Adversarial distractor generation for MCQA: Leveraging in-context learning and rule-based approaches

Yigit G., AMASYALI M. F.

Natural Language Processing Journal, vol.13, 2025 (Scopus) identifier

  • Publication Type: Article / Article
  • Volume: 13
  • Publication Date: 2025
  • Doi Number: 10.1016/j.nlp.2025.100186
  • Journal Name: Natural Language Processing Journal
  • Journal Indexes: Scopus
  • Keywords: Adversarial Distractor Generation, attacks, In-context learning, Llama-7b, Multiple Choice Question Answering
  • Yıldız Technical University Affiliated: Yes

Abstract

Adversarial distractor generation strengthens the capability of Multiple Choice Question Answering (MCQA) systems. This study introduces two distinct methods: in-context learning and rule-based techniques, which are designed explicitly for adversarial distractor generation for MCQA datasets. We leverage the LLama-7b model in the in-context learning approach, following a prompt-based strategy for adversarial distractor generation. This methodology is designed to enrich the variety and complexity of the distractors. Besides, this study presents a rule-based approach, mainly focusing on the impact of appending garbage values to the answer options. Via comprehensive experiments, we systematically evaluate the results of two approaches on the T5-Base and RoBERTa-Large across three MCQA datasets. The empirical results demonstrate a constant decrease in performance across all evaluated datasets when utilizing both the in-context learning and rule-based approaches.