Comparison of Statistical and Neural Regression Using Activation Functions Derived from Swish Activation Function

Koçak Y., Üstündağ Şiray G.

ICAME’21 The Second International Conference on Applied Mathematics in Engineering, Balıkesir, Türkiye, 1 - 03 Eylül 2021, ss.56

  • Yayın Türü: Bildiri / Özet Bildiri
  • Basıldığı Şehir: Balıkesir
  • Basıldığı Ülke: Türkiye
  • Sayfa Sayıları: ss.56
  • Çukurova Üniversitesi Adresli: Evet

Özet

Artificial Neural Network (ANN) is a kind of artificial intelligence and it has been commonly used by scientists and practitioners. ANNs are computational tools that are widely accepted in many disciplines for modelling complex real-world problems such as function approximation, classification, regression, pattern recognition, and forecasting. The attractiveness of an ANN is due to its nonlinearity, parallelism, robustness, error and fault tolerance, learning, and ability to process. ANN learns from examples through iterations without demanding prior information on the relationships of parameters. The most important parameter of an ANN is the activation function (AF). AFs can significantly affect the performance of an ANN and therefore choosing a well-defined AF is important.  In this study, we investigate the effects of AFs on the performance of any ANN for regression. For this purpose, ReLu-swish AF and generalized swish AF that are derived from the swish AF are considered. For the comparison of these AFs with swish AF, mean square error, mean absolute error, and R2 metrics are utilized. To investigate the performance of these AFs, different data sets which are simulated data and some benchmark data from the University of California Irvine Machine Learning Repository are used.