LVLN: A Landmark-Based Deep Neural Network Model for Vision-and-Language Navigation


KIPS Transactions on Software and Data Engineering, Vol. 8, No. 9, pp. 379-390, Sep. 2019
https://doi.org/10.3745/KTSDE.2019.8.9.379, Full Text:
Keywords: Vision-and-Language Navigation, deep neural network, Landmark, Attention, Progress Monitor
Abstract

In this paper, we propose a novel deep neural network model for Vision-and-Language Navigation (VLN) named LVLN (Landmark- based VLN). In addition to both visual features extracted from input images and linguistic features extracted from the natural language instructions, this model makes use of information about places and landmark objects detected from images. The model also applies a context-based attention mechanism in order to associate each entity mentioned in the instruction, the corresponding region of interest (ROI) in the image, and the corresponding place and landmark object detected from the image with each other. Moreover, in order to improve the success rate of arriving the target goal, the model adopts a progress monitor module for checking substantial approach to the target goal. Conducting experiments with the Matterport3D simulator and the Room-to-Room (R2R) benchmark dataset, we demonstrate high performance of the proposed model.


Statistics
Show / Hide Statistics

Statistics (Cumulative Counts from September 1st, 2017)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.


Cite this article
[IEEE Style]
J. Hwang and I. Kim, "LVLN: A Landmark-Based Deep Neural Network Model for Vision-and-Language Navigation," KIPS Transactions on Software and Data Engineering, vol. 8, no. 9, pp. 379-390, 2019. DOI: https://doi.org/10.3745/KTSDE.2019.8.9.379.

[ACM Style]
Jisu Hwang and Incheol Kim. 2019. LVLN: A Landmark-Based Deep Neural Network Model for Vision-and-Language Navigation. KIPS Transactions on Software and Data Engineering, 8, 9, (2019), 379-390. DOI: https://doi.org/10.3745/KTSDE.2019.8.9.379.