Performance Improvement Method of Convolutional Neural Network Using Combined Parametric Activation Functions


KIPS Transactions on Software and Data Engineering, Vol. 11, No. 9, pp. 371-380, Sep. 2022
https://doi.org/10.3745/KTSDE.2022.11.9.371,   PDF Download:
Keywords: Convolutional Neural Network, Nonlinear Activation Function, Combined Parametric Activation Function, Loss Function
Abstract

Convolutional neural networks are widely used to manipulate data arranged in a grid, such as images. A general convolutional neural network consists of a convolutional layers and a fully connected layers, and each layer contains a nonlinear activation functions. This paper proposes a combined parametric activation function to improve the performance of convolutional neural networks. The combined parametric activation function is created by adding the parametric activation functions to which parameters that convert the scale and location of the activation function are applied. Various nonlinear intervals can be created according to parameters that convert multiple scales and locations, and parameters can be learned in the direction of minimizing the loss function calculated by the given input data. As a result of testing the performance of the convolutional neural network using the combined parametric activation function on the MNIST, Fashion MNIST, CIFAR10 and CIFAR100 classification problems, it was confirmed that it had better performance than other activation functions.


Statistics
Show / Hide Statistics

Statistics (Cumulative Counts from September 1st, 2017)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.


Cite this article
[IEEE Style]
Y. M. Ko, P. H. Li, S. W. Ko, "Performance Improvement Method of Convolutional Neural Network Using Combined Parametric Activation Functions," KIPS Transactions on Software and Data Engineering, vol. 11, no. 9, pp. 371-380, 2022. DOI: https://doi.org/10.3745/KTSDE.2022.11.9.371.

[ACM Style]
Young Min Ko, Peng Hang Li, and Sun Woo Ko. 2022. Performance Improvement Method of Convolutional Neural Network Using Combined Parametric Activation Functions. KIPS Transactions on Software and Data Engineering, 11, 9, (2022), 371-380. DOI: https://doi.org/10.3745/KTSDE.2022.11.9.371.