Performance Improvement Method of Fully Connected Neural Network Using Combined Parametric Activation Functions


KIPS Transactions on Software and Data Engineering, Vol. 11, No. 1, pp. 1-10, Jan. 2022
https://doi.org/10.3745/KTSDE.2022.11.1.1,   PDF Download:  
Keywords: Fully Connected Neural Network, Nonlinear Activation Function, Combined Parametric Activation Function, Learning
Abstract

Deep neural networks are widely used to solve various problems. In a fully connected neural network, the nonlinear activation function is a function that nonlinearly transforms the input value and outputs it. The nonlinear activation function plays an important role in solving the nonlinear problem, and various nonlinear activation functions have been studied. In this study, we propose a combined parametric activation function that can improve the performance of a fully connected neural network. Combined parametric activation functions can be created by simply adding parametric activation functions. The parametric activation function is a function that can be optimized in the direction of minimizing the loss function by applying a parameter that converts the scale and location of the activation function according to the input data. By combining the parametric activation functions, more diverse nonlinear intervals can be created, and the parameters of the parametric activation functions can be optimized in the direction of minimizing the loss function. The performance of the combined parametric activation function was tested through the MNIST classification problem and the Fashion MNIST classification problem, and as a result, it was confirmed that it has better performance than the existing nonlinear activation function and parametric activation function.


Statistics
Show / Hide Statistics

Statistics (Cumulative Counts from September 1st, 2017)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.


Cite this article
[IEEE Style]
Y. M. Ko, P. H. Li, S. W. Ko, "Performance Improvement Method of Fully Connected Neural Network Using Combined Parametric Activation Functions," KIPS Transactions on Software and Data Engineering, vol. 11, no. 1, pp. 1-10, 2022. DOI: https://doi.org/10.3745/KTSDE.2022.11.1.1.

[ACM Style]
Young Min Ko, Peng Hang Li, and Sun Woo Ko. 2022. Performance Improvement Method of Fully Connected Neural Network Using Combined Parametric Activation Functions. KIPS Transactions on Software and Data Engineering, 11, 1, (2022), 1-10. DOI: https://doi.org/10.3745/KTSDE.2022.11.1.1.