Single Shot Detector for Detecting Clickable Object in Mobile Device Screen


KIPS Transactions on Software and Data Engineering, Vol. 11, No. 1, pp. 29-34, Jan. 2022
https://doi.org/10.3745/KTSDE.2022.11.1.29,   PDF Download:
Keywords: Test Automation, Android Object Detection, Mobile Screen Detection, computer vision, Deep Learning
Abstract

We propose a novel network architecture and build dataset for recognizing clickable objects on mobile device screens. The data was collected based on clickable objects on the mobile device screen that have numerous resolution, and a total of 24,937 annotation data were subdivided into seven categories: text, edit text, image, button, region, status bar, and navigation bar. We use the Deconvolution Single Shot Detector as a baseline, the backbone network with Squeeze-and-Excitation blocks, the Single Shot Detector layer structure to derive inference results and the Feature pyramid networks structure. Also we efficiently extract features by changing the input resolution of the existing 1:1 ratio of the network to a 1:2 ratio similar to the mobile device screen. As a result of experimenting with the dataset we have built, the mean average precision was improved by up to 101% compared to baseline.


Statistics
Show / Hide Statistics

Statistics (Cumulative Counts from September 1st, 2017)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.


Cite this article
[IEEE Style]
M. Jo, H. Chun, S. Han, C. Jeong, "Single Shot Detector for Detecting Clickable Object in Mobile Device Screen," KIPS Transactions on Software and Data Engineering, vol. 11, no. 1, pp. 29-34, 2022. DOI: https://doi.org/10.3745/KTSDE.2022.11.1.29.

[ACM Style]
Min-Seok Jo, Hye-won Chun, Seong-Soo Han, and Chang-Sung Jeong. 2022. Single Shot Detector for Detecting Clickable Object in Mobile Device Screen. KIPS Transactions on Software and Data Engineering, 11, 1, (2022), 29-34. DOI: https://doi.org/10.3745/KTSDE.2022.11.1.29.