Artificial Intelligence Revolution in Turkish Health Consultancy: Development of LLM-Based Virtual Doctor Assistants T rk e Saglik Danismanliginda Yapay Zeka Devrimi: LLM Tabanli Sanal Doktor Asistanlarinin Gelistirilmesi

Bulut M. K., DİRİ B.

8th International Artificial Intelligence and Data Processing Symposium, IDAP 2024, Malatya, Türkiye, 21 - 22 Eylül 2024 identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Doi Numarası: 10.1109/idap64064.2024.10710973
  • Basıldığı Şehir: Malatya
  • Basıldığı Ülke: Türkiye
  • Anahtar Kelimeler: Comparative model evaluation, Doctor-patient communication, LLM fine-tuning, Medical AI, Natural language processing, Turkish healthcare
  • Yıldız Teknik Üniversitesi Adresli: Evet

Özet

This study examines the performance of four different large language models (LLama2, LLama3, and Mistralbased) in doctor-patient written communication in Turkish health counseling. The models were trained and fine-tuned on a patient-doctor question-answer dataset [1]. The metrics used for performance evaluation include ROUGE, Elo rating, Winning percentage, and Expert evaluation. The comparative analysis results indicate that the SambaLingo-Turkish-Chat model was successful in terms of response accuracy and contextual relevance, while the Trendyol-LLM-7b-chat-v 1.8 model proved to be more successful when considering the ethical aspects of the task [14], [17]. This study demonstrates the potential of AI-powered virtual doctor assistants in Turkish healthcare services and contributes to the development of Turkish-specific medical chatbots.