Interpolation Based Distributed Learning İnterpolasyon Tabanlı Paralel Öğrenme


Akdiş M., AMASYALI M. F.

32nd IEEE Conference on Signal Processing and Communications Applications, SIU 2024, Mersin, Türkiye, 15 - 18 Mayıs 2024 identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Doi Numarası: 10.1109/siu61531.2024.10601042
  • Basıldığı Şehir: Mersin
  • Basıldığı Ülke: Türkiye
  • Anahtar Kelimeler: artificial neural networks, distributed training, gradient descent, linear interpolation, parameter-loss space
  • Yıldız Teknik Üniversitesi Adresli: Evet

Özet

Artificial neural networks are trained with gradient descent-based algorithms in a complex parameter error space. As a result of the training, a point (local minimum) with a small error on the training set is obtained in this space. However, due to the random processes of the training algorithms, different local minima are reached even if starting from very close points using the same data set. When linear interpolation is made between these reached points, it is seen that the error values of the obtained points are high. In this study, it has been shown that this situation does not occur among models trained in parallel with different data after a common training, and that very successful results can be achieved when training continues from the best point obtained by interpolation. In other words, instead of training sequentially with all the data, more successful results could be obtained by adding a parallel process to the training process in approximately the same time. This approach, which does not require information sharing between models throughout the parallel process, differs from the literature in this respect and offers an innovative training method for artificial neural networks.