More Web Proxy on the site http://driver.im/

research-article

Learning to Recognize Unmodified Lights with Invisible Features

Authors:

Huanhuan Zhang,

Huadong MaAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 3, Issue 2

Article No.: 67, Pages 1 - 23

https://doi.org/10.1145/3328938

Published: 21 June 2019 Publication History

Abstract

To enable accurate indoor localization at low cost, recent research in visible light positioning (VLP) proposed to employ existing ceiling lights as location landmarks, and use smartphone cameras or light sensors to identify the different lights using statistical visual/optical features. Despite the potential, we find such solutions are unreliable: the features are easily corrupted with a slight rotation of the smartphone, and are not discriminative enough for many practical light models with different size/shape/intensity. In this work, we propose Auto-Litell to resolve these critical challenges and make VLP truly robust. Auto-Litell builds a customized deep-learning neural network model to automatically distill the "invisible" visual features from the lights, which are resilient to smartphone orientation and light models. Moreover, Auto-Litell introduces a Light-CycleGAN to generate "fake" light images to augment the training data, so as to relieve human labors in data collection and labeling. We have implemented Auto-Litell as a real-time localization and navigation system on Android. Our experiments demonstrate Auto-Litell's high accuracy in discriminating the lights in the same building, and high reliability across a variety of practical usage scenarios.

References

[1]

Amjad Almahairi, Sai Rajeshwar, Alessandro Sordoni, Philip Bachman, and Aaron C. Courville. 2018. Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data. (2018), 195--204.

[2]

Jean Armstrong, Y Ahmet Sekercioglu, and Adrian Neild. 2013. Visible light positioning: a roadmap for international standardization. IEEE Communications Magazine 51, 12 (2013), 68--73.

[3]

Paramvir Bahl and Venkata N Padmanabhan. 2000. RADAR: An in-building RF-based user location and tracking system. In Proc. of IEEE INFOCOM, Vol. 2. 775--784.

[4]

Huiwen Chang, Jingwan Lu, Fisher Yu, and Adam Finkelstein. 2018. Pairedcyclegan: Asymmetric style transfer for applying and removing makeup. In Proc. of IEEE CVPR.

[5]

Cheng-Lun Chen and Shao-Hua Lin. 2011. Intelligent color temperature estimation using fuzzy neural network with application to automatic white balance. Expert Syst. Appl. 38, 6, 7718--7728.

Digital Library

[6]

Yin Chen, Dimitrios Lymberopoulos, Jie Liu, and Bodhi Priyantha. 2012. FM-based indoor localization. In Proc. of ACM Mobisys. 169--182.

Digital Library

[7]

Yin Chen, Dimitrios Lymberopoulos, Jie Liu, and Bodhi Priyantha. 2012. FM-based indoor localization. In The 10th International Conference on Mobile Systems, Applications, and Services, MobiSys'12, Ambleside, United Kingdom - June 25 - 29, 2012. 169--182.

Digital Library

[8]

Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. 2016. Image Super-Resolution Using Deep Convolutional Networks. IEEE Trans. Pattern Anal. Mach. Intell. 38, 2 (2016), 295--307.

Digital Library

[9]

Chao Dong, Chen Change Loy, and Xiaoou Tang. 2016. Accelerating the super-resolution convolutional neural network. In ECCV. Springer, 391--407.

[10]

Hany Farid. 2001. Blind inverse gamma correction. IEEE Transactions on Image Processing 10, 10 (2001), 1428--1433.

Digital Library

[11]

Ross Girshick. 2015. Fast r-cnn. In CVPR. 1440--1448.

Digital Library

[12]

Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR. 580--587.

[13]

Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In AISTATS. 249--256.

[14]

Daniel B Goldman. 2010. Vignette and exposure calibration and compensation. IEEE transactions on pattern analysis and machine intelligence 32, 12 (2010), 2276--2288.

Digital Library

[15]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In NIPS. 2672--2680.

Digital Library

[16]

Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask r-cnn. In Proc. of IEEE ICCV. IEEE, 2980--2988.

[17]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proc. of IEEE CVPR. 770--778.

[18]

Yu-Feng Hsu and Shih-Fu Chang. 2007. Image Splicing Detection using Camera Response Function Consistency and Automatic Segmentation. In Proc. of IEEE ICME. 28--31.

[19]

Yiqing Hu, Yan Xiong, Wenchao Huang, Xiang-Yang Li, Panlong Yang, Yanan Zhang, and XuFei Mao. 2018. Lightitude: Indoor Positioning Using Uneven Light Intensity Distribution. IMWUT 2, 2 (2018), 67:1--67:25.

Digital Library

[20]

Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167 (2015).

[21]

Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. 2017. Image-to-Image Translation with Conditional Adversarial Networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017. 5967--5976.

[22]

Zhenyuan Jia, Jinghao Yang, Wei Liu, Fuji Wang, Yang Liu, Lingli Wang, Chaonan Fan, and Kai Zhao. 2015. Improved camera calibration method based on perpendicularity compensation for binocular stereo vision measurement system. Optics express 23, 12 (2015), 15205--15223.

[23]

Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. In ECCV. Springer, 694--711.

[24]

Andrew Evan Kamholz, Eric A Schilling, and Paul Yager. 2001. Optical measurement of transverse molecular diffusion in a microchannel. Biophysical Journal 80, 4 (2001), 1967--1972.

[25]

Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv:1312.6114 (2013).

[26]

Manikanta Kotaru, Kiran Raj Joshi, Dinesh Bharadia, and Sachin Katti. 2015. SpotFi: Decimeter Level Localization Using WiFi. In Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication, SIGCOMM 2015, London, United Kingdom, August 17-21, 2015. 269--282.

Digital Library

[27]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In NIPS. 1097--1105.

Digital Library

[28]

Ye-Sheng Kuo, Pat Pannuto, Ko-Jen Hsiao, and Prabal Dutta. 2014. Luxapose: Indoor positioning with mobile phones and visible light. In Proc. of ACM MobiCom. 447--458.

Digital Library

[29]

Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming-Hsuan Yang. 2017. Deep laplacian pyramid networks for fast and accurate superresolution. In Proc. of IEEE CVPR, Vol. 2. 5.

[30]

Chongxuan Li, Taufik Xu, Jun Zhu, and Bo Zhang. 2017. Triple Generative Adversarial Nets. (2017), 4091--4101.

Digital Library

[31]

Liqun Li, Pan Hu, Chunyi Peng, Guobin Shen, and Feng Zhao. 2014. Epsilon: A Visible Light Based Positioning System. In NSDI, Vol. 14. 331--343.

Digital Library

[32]

Xiang Li, Shengjie Li, Daqing Zhang, Jie Xiong, Yasha Wang, and Hong Mei. 2016. Dynamic-MUSIC: accurate device-free indoor localization. In Proc. of ACM UbiComp. 196--207.

Digital Library

[33]

Yi-Qun Li, Yi Dong, and Xiaofeng Xu. 2012. Color temperature tunable white light source. US Patent 8,203,260.

[34]

Min Lin, Qiang Chen, and Shuicheng Yan. 2013. Network in network. arXiv:1312.4400 (2013).

[35]

Makets and Markets. 2017. Indoor Location Market worth 40.99 Billion USD by 2022. https://www.marketsandmarkets.com/PressReleases/indoor-location.asp.

[36]

Xudong Mao, Qing Li, Haoran Xie, Raymond Y. K. Lau, and Zhen Wang. 2016. Multi-class Generative Adversarial Networks with the L2 Loss Function. CoRR abs/1611.04076 (2016).

[37]

Michael F Mathieu, Junbo Jake Zhao, Junbo Zhao, Aditya Ramesh, Pablo Sprechmann, and Yann LeCun. 2016. Disentangling factors of variation in deep representation using adversarial training. In NIPS. 5040--5048.

Digital Library

[38]

Kun Qian, Chenshu Wu, Zheng Yang, Yunhao Liu, and Kyle Jamieson. 2017. Widar: Decimeter-Level Passive Tracking via Velocity Monitoring with Commodity Wi-Fi. In Proceedings of the 18th ACM International Symposium on Mobile Ad Hoc Networking and Computing, Chennai, India, July 10-14, 2017. 6:1--6:10.

Digital Library

[39]

Kun Qian, Chenshu Wu, Yi Zhang, Guidong Zhang, Zheng Yang, and Yunhao Liu. 2018. Widar2.0: Passive Human Tracking with a Single Wi-Fi Link. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys 2018, Munich, Germany, June 10-15, 2018. 350--361.

Digital Library

[40]

John C Russ. 2016. The image processing handbook. CRC press.

Digital Library

[41]

Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018. Inverted residuals and linear bottlenecks: Mobile networks for classification, detection and segmentation. arXiv:1801.04381 (2018).

[42]

Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. CoRR abs/1312.6034 (2013).

[43]

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014).

[44]

Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A Alemi. 2017. Inception-v4, inception-resnet and the impact of residual connections on learning. In AAAI, Vol. 4. 12.

Digital Library

[45]

Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proc. of IEEE CVPR. 1--9.

[46]

Roger Tsai. 1987. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE Journal on Robotics and Automation 3, 4 (1987), 323--344.

[47]

Martin JL Turner, A Abbey, M Arnaud, M Balasini, M Barbera, E Belsole, PJ Bennie, JP Bernard, GF Bignami, M Boer, et al. 2001. The European photon imaging camera on XMM-Newton: the MOS cameras. Astronomy & Astrophysics 365, 1 (2001), L27--L35.

[48]

Dmitry Ulyanov, Andrea Vedaldi, and Victor S. Lempitsky. 2016. Instance Normalization: The Missing Ingredient for Fast Stylization. CoRR abs/1607.08022.

[49]

Deepak Vasisht, Swarun Kumar, and Dina Katabi. 2016. Decimeter-Level Localization with a Single WiFi Access Point. In Proc. of USENIX NSDI, Vol. 16. 165--178.

Digital Library

[50]

Allen Jong-Woei Whang, Yi-Yung Chen, and Yuan-Ting Teng. 2009. Designing uniform illumination systems by surface-tailored lens and configurations of LED arrays. Journal of display technology 5, 3 (2009), 94--103.

[51]

Bo Xie, Kongyang Chen, Guang Tan, Mingming Lu, Yunhuai Liu, Jie Wu, and Tian He. 2016. LIPS: A light intensity--based positioning system for indoor environments. ACM TOSN 12, 4 (2016), 28.

Digital Library

[52]

Bo Xie, Guang Tan, and Tian He. 2015. Spinlight: A high accuracy and robust light positioning system for indoor applications. In Proc. of ACM SenSys. 211--223.

Digital Library

[53]

Jie Xiong and Kyle Jamieson. 2013. ArrayTrack: A Fine-Grained Indoor Location System. In Proc. of USENIX NSDI. 71--84.

Digital Library

[54]

Han Xu, Zheng Yang, Zimu Zhou, Longfei Shangguan, Ke Yi, and Yunhao Liu. 2016. Indoor localization via multi-modal sensing on smartphones. In Proc. of ACM UbiComp. 208--219.

Digital Library

[55]

Qiang Xu, Rong Zheng, and Steve Hranilovic. 2015. IDyLL: indoor localization using inertial and light sensors on smartphones. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp 2015, Osaka, Japan, September 7-11, 2015. 307--318.

Digital Library

[56]

Jianwei Yang, Anitha Kannan, Dhruv Batra, and Devi Parikh. 2017. LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation. CoRR abs/1703.01560 (2017).

[57]

Zhice Yang, Zeyu Wang, Jiansong Zhang, Chenyu Huang, and Qian Zhang. 2015. Wearables can afford: Light-weight indoor positioning with visible light. In Proc. of ACM Mobisys. 317--330.

Digital Library

[58]

DING Ying, FAN Jingtao, QUAN Wei, and HAN Cheng. 2017. Nonlinear compensation for optical vignetting in vision systems. Journal of Tsinghua University (Science and Technology) 57, 7 (2017), 702--706.

[59]

Fisher Yu, Vladlen Koltun, and Thomas A Funkhouser. 2017. Dilated Residual Networks. In CVPR, Vol. 2. 3.

[60]

Chi Zhang and Xinyu Zhang. 2016. LiTell: robust indoor localization using unmodified light fixtures. In Proc. of ACM MobiCom. 230--242.

Digital Library

[61]

Chi Zhang and Xinyu Zhang. 2017. Pulsar: Towards ubiquitous visible light localization. In Proc. of ACM MobiCom. 208--221.

Digital Library

[62]

Jialiang Zhang, Chi Zhang, Xinyu Zhang, and Suman Banerjee. 2016. Towards a visible light network architecture for continuous communication and localization. In Proceedings of the 3rd Workshop on Visible Light Communication Systems, VLCS@MobiCom 2016, New York City, NY, USA, October 3-7, 2016. 49--54.

Digital Library

[63]

Zhengyou Zhang. 1999. Flexible Camera Calibration by Viewing a Plane from Unknown Orientations. In ICCV. 666--673.

[64]

Bo Zhao, Bo Chang, Zequn Jie, and Leonid Sigal. 2018. Modular Generative Adversarial Networks. (2018), 157--173.

[65]

Jian Zhao, Lin Xiong, Panasonic Karlekar Jayashree, Jianshu Li, Fang Zhao, Zhecan Wang, Panasonic Sugiri Pranata, Panasonic Shengmei Shen, Shuicheng Yan, and Jiashi Feng. 2017. Dual-agent gans for photorealistic and identity preserving profile face synthesis. In NIPS. 66--76.

Digital Library

[66]

A. Zhou, X. Zhang, and H. Ma. 2017. Beam-forecast: Facilitating mobile 60 GHz networks via model-driven beam steering. In IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[67]

Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. 2016. Learning deep features for discriminative localization. In Proc. of IEEE CVPR. 2921--2929.

[68]

Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. 2017. Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. (2017), 2242--2251.

[69]

Shilin Zhu and Xinyu Zhang. 2017. Enabling High-Precision Visible Light Localization in Today's Buildings. In Proc. of ACM MobiSys. 96--108.

Digital Library

Cited By

Niu QHe TLiu NHe SLuo XZhou F(2020)MAILProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/33973354:2(1-23)Online publication date: 15-Jun-2020
https://dl.acm.org/doi/10.1145/3397335
Xue HJiang WMiao CMa FWang SYuan YYao SZhang ASu L(2020)DeepMVProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/33809804:1(1-26)Online publication date: 14-Sep-2020
https://dl.acm.org/doi/10.1145/3380980
Zhou ASu GZhu SMa H(2019)Invisible QR Code Hijacking Using Smart LEDProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/33512843:3(1-23)Online publication date: 9-Sep-2019
https://dl.acm.org/doi/10.1145/3351284

Index Terms

Learning to Recognize Unmodified Lights with Invisible Features
1. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing design and evaluation methods

Recommendations

LiTell: indoor localization using unmodified light fixtures: demo
MobiCom '16: Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking

Owing to dense deployment of light fixtures and multipath-free propagation, visible light localization technology holds potential to overcome the reliability issue of radio localization. However, existing visible light localization systems require ...
Three-Dimensional Indoor Visible Light Localization: A Learning-Based Approach
UbiComp/ISWC '21 Adjunct: Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers

In this paper, a three-dimensional (3D) indoor visible light localization method based on machine learning and deep learning is presented, which is able to obtain accurate 3D spatial coordinates of the user, including the location on the plane and the ...
LiTell: robust indoor localization using unmodified light fixtures
MobiCom '16: Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking

Owing to dense deployment of light fixtures and multipath-free propagation, visible light localization technology holds potential to overcome the reliability issue of radio localization. However, existing visible light localization systems require ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 3, Issue 2

June 2019

802 pages

EISSN:2474-9567

DOI:10.1145/3341982

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 June 2019

Accepted: 01 April 2019

Revised: 01 February 2019

Received: 01 November 2018

Published in IMWUT Volume 3, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

the 111 Project
National Natural Science Foundation of China
the National Key Research and Development Plan under Grant

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
539
Total Downloads

Downloads (Last 12 months)48
Downloads (Last 6 weeks)11

Reflects downloads up to 18 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Niu QHe TLiu NHe SLuo XZhou F(2020)MAILProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/33973354:2(1-23)Online publication date: 15-Jun-2020
https://dl.acm.org/doi/10.1145/3397335
Xue HJiang WMiao CMa FWang SYuan YYao SZhang ASu L(2020)DeepMVProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/33809804:1(1-26)Online publication date: 14-Sep-2020
https://dl.acm.org/doi/10.1145/3380980
Zhou ASu GZhu SMa H(2019)Invisible QR Code Hijacking Using Smart LEDProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/33512843:3(1-23)Online publication date: 9-Sep-2019
https://dl.acm.org/doi/10.1145/3351284

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents