Style Synthesis of Speech Videos Through Generative Adversarial Neural Networks


KIPS Transactions on Software and Data Engineering, Vol. 11, No. 11, pp. 465-472, Nov. 2022
https://doi.org/10.3745/KTSDE.2022.11.11.465,   PDF Download:
Keywords: Generative Adversarial Network, Video Generation, Style Transfer, Style Synthesis Network, Video Synthesis Network
Abstract

In this paper, the style synthesis network is trained to generate style-synthesized video through the style synthesis through training Stylegan and the video synthesis network for video synthesis. In order to improve the point that the gaze or expression does not transfer stably, 3D face restoration technology is applied to control important features such as the pose, gaze, and expression of the head using 3D face information. In addition, by training the discriminators for the dynamics, mouth shape, image, and gaze of the Head2head network, it is possible to create a stable style synthesis video that maintains more probabilities and consistency. Using the FaceForensic dataset and the MetFace dataset, it was confirmed that the performance was increased by converting one video into another video while maintaining the consistent movement of the target face, and generating natural data through video synthesis using 3D face information from the source video’s face.


Statistics
Show / Hide Statistics

Statistics (Cumulative Counts from September 1st, 2017)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.


Cite this article
[IEEE Style]
C. H. Jo and P. G. Man, "Style Synthesis of Speech Videos Through Generative Adversarial Neural Networks," KIPS Transactions on Software and Data Engineering, vol. 11, no. 11, pp. 465-472, 2022. DOI: https://doi.org/10.3745/KTSDE.2022.11.11.465.

[ACM Style]
Choi Hee Jo and Park Goo Man. 2022. Style Synthesis of Speech Videos Through Generative Adversarial Neural Networks. KIPS Transactions on Software and Data Engineering, 11, 11, (2022), 465-472. DOI: https://doi.org/10.3745/KTSDE.2022.11.11.465.