Artificial Intelligence Revolution in Turkish Health Consultancy: Development of LLM-Based Virtual Doctor Assistants T rk e Saglik Danismanliginda Yapay Zeka Devrimi: LLM Tabanli Sanal Doktor Asistanlarinin Gelistirilmesi
8th International Artificial Intelligence and Data Processing Symposium, IDAP 2024, Malatya, Türkiye, 21 - 22 Eylül 2024, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Doi Numarası: 10.1109/idap64064.2024.10710973
- Basıldığı Şehir: Malatya
- Basıldığı Ülke: Türkiye
- Anahtar Kelimeler: Comparative model evaluation, Doctor-patient communication, LLM fine-tuning, Medical AI, Natural language processing, Turkish healthcare
- Yıldız Teknik Üniversitesi Adresli: Evet
Özet
This study examines the performance of four different large language models (LLama2, LLama3, and Mistralbased) in doctor-patient written communication in Turkish health counseling. The models were trained and fine-tuned on a patient-doctor question-answer dataset [1]. The metrics used for performance evaluation include ROUGE, Elo rating, Winning percentage, and Expert evaluation. The comparative analysis results indicate that the SambaLingo-Turkish-Chat model was successful in terms of response accuracy and contextual relevance, while the Trendyol-LLM-7b-chat-v 1.8 model proved to be more successful when considering the ethical aspects of the task [14], [17]. This study demonstrates the potential of AI-powered virtual doctor assistants in Turkish healthcare services and contributes to the development of Turkish-specific medical chatbots.