Cost-Efficient Hybrid Human-AI Handoff System: A Density-Based Semantic Clustering Approach for Customer Support Dialogs in the Wild Gerçek Dünya Müşteri Destek Diyaloglari için Maliyet Etkin Hibrit ?Insan-AI Devir Teslim Sistemi: Yo?gunluk Tabanli Anlamsal Kümeleme Yaklaşim
34th Signal Processing and Communications Applications Conference, SIU 2026, İstanbul, Türkiye, 7 - 10 Temmuz 2026, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Doi Numarası: 10.1109/siu71813.2026.11637097
- Basıldığı Şehir: İstanbul
- Basıldığı Ülke: Türkiye
- Anahtar Kelimeler: Customer Support Automation, LLM, OOD Detection, Semantic Routing, Unsupervised Clustering
- Yıldız Teknik Üniversitesi Adresli: Evet
Özet
Integrating Large Language Models (LLMs) into customer support workflows offers strong automation potential, yet raises risks related to hallucination, operational cost, and the handling of sensitive requests. This study proposes a label-efficient Semantic Router for hybrid human-AI handoff. The proposed approach combines unsupervised semantic clustering of historical agent responses with domain-informed Weighted Complexity Scoring (WCS) to identify latent support categories and route high-risk clusters to human operators. A distance-based out-of-distribution (OOD) filter is also introduced to detect queries that fall outside known support patterns. Experiments on an internal customer support dataset, Banking77, and CLINC150 show that the framework achieves a 0.56 Silhouette Score, 0.76 NMI, and 0.88 ROC-AUC for OOD detection. The results indicate that the proposed system can support safer and more cost-efficient human-in-the-loop customer support automation.