Optimal RAG System Design for Turkish Textbooks: A Comprehensive Evaluation and Performance Enhancement Study

Oner E. N., Ceyhun S., Yildiz M. H., Goncharova A., Yucel T. S., KESGİN H. T., ...Daha Fazla

2025 Innovations in Intelligent Systems and Applications Conference, ASYU 2025, Bursa, Türkiye, 10 - 12 Eylül 2025, (Tam Metin Bildiri) identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Doi Numarası: 10.1109/asyu67174.2025.11208464
  • Basıldığı Şehir: Bursa
  • Basıldığı Ülke: Türkiye
  • Anahtar Kelimeler: educational AI systems, RAG systems, retrieval-augmented generation, textbook question-answering, Turkish natural language processing
  • Yıldız Teknik Üniversitesi Adresli: Evet

Özet

Retrieval-augmented generation enables precise educational question-answering by combining retrieval with natural language generation. Limited evaluation exists for educational RAG systems, particularly for curriculum-based applications in diverse languages. We introduce systematic evaluation using Turkish Ministry of Education textbooks, comparing embedding architectures, generation models, and optimization techniques across educational questions. Results establish multilingual-e5-large-instruct and BAAI/bge-m3 as optimal embedding models, demonstrate Qwen3 and Turkish-Gemma-9b's superior generation performance, and show ensemble methods yield 3.3% improvements. This provides evidence-based recommendations for educational RAG development across different languages and curricula.