Hohyun Sim, Horyeol Sung, Seungjae Lee, Hyeonjoong Cho

KIPS Transactions on Software and Data Engineering, Vol. 11, No. 8, pp. 325-330, Aug. 2022
https://doi.org/10.3745/KTSDE.2022.11.8.325,   PDF Download:
Keywords: Sign Language Recognition, Sign Language Translation, Sign Language Segmentation, Sign Language Dataset, Deep Learning
Abstract

This paper conducts the collection and experiment of datasets for deep learning research on sign language such as sign language recognition, sign language translation, and sign language segmentation for Korean sign language. There exist difficulties for deep learning research of sign language. First, it is difficult to recognize sign languages since they contain multiple modalities including hand movements, hand directions, and facial expressions. Second, it is the absence of training data to conduct deep learning research. Currently, KETI dataset is the only known dataset for Korean sign language for deep learning. Sign language datasets for deep learning research are classified into two categories: Isolated sign language and Continuous sign language. Although several foreign sign language datasets have been collected over time. they are also insufficient for deep learning research of sign language. Therefore, we attempted to collect a large-scale Korean sign language dataset and evaluate it using a baseline model named TSPNet which has the performance of SOTA in the field of sign language translation. The collected dataset consists of a total of 11,402 image and text. Our experimental result with the baseline model using the dataset shows BLEU-4 score 3.63, which would be used as a basic performance of a baseline model for Korean sign language dataset. We hope that our experience of collecting Korean sign language dataset helps facilitate further research directions on Korean sign language.

Statistics
Cite this article