Jisu Hwang, Incheol Kim

KIPS Transactions on Software and Data Engineering, Vol. 8, No. 9, pp. 379-390, Sep. 2019
https://doi.org/10.3745/KTSDE.2019.8.9.379,   PDF Download:
Keywords: Vision-and-Language Navigation, deep neural network, Landmark, Attention, Progress Monitor
Abstract

In this paper, we propose a novel deep neural network model for Vision-and-Language Navigation (VLN) named LVLN (Landmark- based VLN). In addition to both visual features extracted from input images and linguistic features extracted from the natural language instructions, this model makes use of information about places and landmark objects detected from images. The model also applies a context-based attention mechanism in order to associate each entity mentioned in the instruction, the corresponding region of interest (ROI) in the image, and the corresponding place and landmark object detected from the image with each other. Moreover, in order to improve the success rate of arriving the target goal, the model adopts a progress monitor module for checking substantial approach to the target goal. Conducting experiments with the Matterport3D simulator and the Room-to-Room (R2R) benchmark dataset, we demonstrate high performance of the proposed model.

Statistics
Cite this article