Jeon Seung Ho, Moon Jong Sub

KIPS Transactions on Software and Data Engineering, Vol. 6, No. 12, pp. 573-580, Dec. 2017
10.3745/KTSDE.2017.6.12.573,   PDF Download:
Keywords: Deep Learning, Generalization, Denoising, Mutation Operation
Abstract

Deep Neural Network(DNN) is a large layered neural network which is consisted of a number of layers of non-linear units. Deep Learning which represented as DNN has been applied very successfully in various applications. However, many issues in DNN have been identified through past researches. Among these issues, generalization is the most well-known problem. A Recent study, Dropout, successfully addressed this problem. Also, Dropout plays a role as noise, and so it helps to learn robust feature during learning in DNN such as Denoising AutoEncoder. However, because of a large computations required in Dropout, training takes a lot of time. Since Dropout keeps changing an inter-layer representation during the training session, the learning rates should be small, which makes training time longer. In this paper, using mutation operation, we reduce computation and improve generalization performance compared with Dropout. Also, we experimented proposed method to compare with Dropout method and showed that our method is superior to the Dropout one.

Statistics
Cite this article