Donghyeop Shin, Incheol Kim

KIPS Transactions on Software and Data Engineering, Vol. 7, No. 2, pp. 69-76, Feb. 2018
10.3745/KTSDE.2018.7.2.69,   PDF Download:
Keywords: Referring Expression Comprehension, Deep Learning, Contextual Information, Weighted Composition
Abstract

Referring expressions are natural language constructions used to identify particular objects within a scene. In this paper, we propose a new deep neural network model for referring expression comprehension. The proposed model finds out the region of the referred object in the given image by making use of the rich information about the referred object itself, the context object, and the relationship with the context object mentioned in the referring expression. In the proposed model, the object matching score and the relationship matching score are combined to compute the fitness score of each candidate region according to the structure of the referring expression sentence. Therefore, the proposed model consists of four different sub-networks: Language Representation Network(LRN), Object Matching Network (OMN), Relationship Matching Network(RMN), and Weighted Composition Network(WCN). We demonstrate that our model achieves state-of-the-art results for comprehension on three referring expression datasets.

Statistics
Cite this article