Evaluating Class Imbalance Handling Methods for Multi-Class IoT Intrusion Detection: A Comparative Study on the CIC-IoT 2023 Dataset

Tadeeb S. M., ÖZEN F.

13th International Conference on Electrical and Electronics Engineering, ICEEE 2026, Antalya, Türkiye, 27 - 29 Nisan 2026, ss.412-417, (Tam Metin Bildiri)

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Doi Numarası: 10.1109/iceee69936.2026.11598250
  • Basıldığı Şehir: Antalya
  • Basıldığı Ülke: Türkiye
  • Sayfa Sayıları: ss.412-417
  • Anahtar Kelimeler: CIC-IoT 2023, class imbalance, cost-sensitive learning, IoT intrusion detection system, SMOTE
  • Yıldız Teknik Üniversitesi Adresli: Evet

Özet

With growing deployments of IoT devices across various sectors, the attack ratio has significantly increased, making intrusion detection systems (IDS) essential. However, almost all the major IoT network traffic datasets have severe class imbalance, where rare but critical attack types such as malware and brute force are vastly outnumbered by dominant classes like DDoS. In order to balance the datasets, majority of the studies use SMOTE without considering alternative approaches. This paper presents a systematic comparative evaluation of four class imbalance handling methods, namely Baseline (no balancing), SMOTE, Cost-Sensitive Learning, and Threshold Optimization on the CIC-IoT 2023 dataset using Random Forest and LightGBM classifiers. Experiments are conducted on the coarse-grained 9class multi-class classification task with 19 million unique network flow records. Results demonstrate that Cost-Sensitive Learning achieves the highest minority class recall improvements, increasing malware detection from 1.87% to 51.94% with LightGBM, a 2,678% relative improvement, while requiring zero additional preprocessing time or memory overhead. SMOTE, by contrast, provides marginal improvements and cannot scale to the full dataset without subsampling. Threshold Optimization achieves the best overall Macro F1 balance. These findings offer practical guidance for IoT security practitioners in selecting appropriate imbalance handling strategies.