[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3009977.3010047acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicvgipConference Proceedingsconference-collections
research-article

DeepFly: towards complete autonomous navigation of MAVs with monocular camera

Published: 18 December 2016 Publication History

Abstract

Recently, the interest in Micro Aerial Vehicles (MAVs) and their autonomous flights has increased tremendously and significant advances have been made. The monocular camera has turned out to be most popular sensing modality for MAVs as it is light-weight, does not consume more power, and encodes rich information about the environment around. In this paper, we present DeepFly, our framework for autonomous navigation of a quadcopter equipped with monocular camera. The navigable space detection and waypoint selection are fundamental components of autonomous navigation system. They have broader meaning than just detecting and avoiding immediate obstacles. Finding the navigable space emphasizes equally on avoiding obstacles and detecting ideal regions to move next to. The ideal region can be defined by two properties: 1) All the points in the region have approximately same high depth value and 2) The area covered by the points of the region in the disparity map is considerably large. The waypoints selected from these navigable spaces assure collision-free path which is safer than path obtained from other waypoint selection methods which do not consider neighboring information.
In our approach, we obtain a dense disparity map by performing a translation maneuver. This disparity map is input to a deep neural network which predicts bounding boxes for multiple navigable regions. Our deep convolutional neural network with shortcut connections regresses variable number of outputs without any complex architectural add on. Our autonomous navigation approach has been successfully tested in both indoors and outdoors environment and in range of lighting conditions.

Supplementary Material

ZIP File (a59-shah.zip)
Supplemental material.

References

[1]
H. Alvarez, L. Paz, J. Sturm, and D. Cremers. Collision avoidance for quadrotors with a monocular camera. In Experimental Robotics, pages 195--209. Springer, 2016.
[2]
K. Bipin, V. Duggal, and K. M. Krishna. Autonomous navigation of generic monocular quadcopter in natural environment. In 2015 IEEE International Conference on Robotics and Automation (ICRA), pages 1063--1070. IEEE, 2015.
[3]
A. Bry, A. Bachrach, and N. Roy. State estimation for aggressive flight in gps-denied environments using onboard sensing. In Robotics and Automation (ICRA), 2012 IEEE International Conference on, pages 1--8. IEEE, 2012.
[4]
S. Daftry, S. Zeng, A. Khan, D. Dey, N. Melik-Barkhudarov, J. A. Bagnell, and M. Hebert. Robust monocular flight in cluttered outdoor environments. arXiv preprint arXiv:1604.04779, 2016.
[5]
F. Fraundorfer, L. Heng, D. Honegger, G. H. Lee, L. Meier, P. Tanskanen, and M. Pollefeys. Vision-based autonomous mapping and exploration using a quadrotor mav. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4557--4564. IEEE, 2012.
[6]
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385, 2015.
[7]
K. He, X. Zhang, S. Ren, and J. Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE International Conference on Computer Vision, pages 1026--1034, 2015.
[8]
H. Hirschmuller. Stereo processing by semiglobal matching and mutual information. IEEE Transactions on pattern analysis and machine intelligence, 30(2):328--341, 2008.
[9]
S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015.
[10]
Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM international conference on Multimedia, pages 675--678. ACM, 2014.
[11]
D. K. Kim and T. Chen. Deep neural network for real-time autonomous indoor navigation. arXiv preprint arXiv:1511.04668, 2015.
[12]
G. Klein and D. Murray. Parallel tracking and mapping for small ar workspaces. In Mixed and Augmented Reality, 2007. ISMAR 2007. 6th IEEE and ACM International Symposium on, pages 225--234. IEEE, 2007.
[13]
T. Krajník, M. Nitsche, S. Pedre, L. Přeučil, and M. E. Mejail. A simple visual navigation system for an uav. In Systems, Signals and Devices (SSD), 2012 9th International Multi-Conference on, pages 1--6. IEEE, 2012.
[14]
J. Langelaan and S. Rock. Towards autonomous uav flight in forests. In Proc. of AIAA Guidance, Navigation and Control Conference, 2005.
[15]
R. Mur-Artal, J. Montiel, and J. D. Tardós. Orb-slam: a versatile and accurate monocular slam system. IEEE Transactions on Robotics, 31(5):1147--1163, 2015.
[16]
R. A. Newcombe, S. J. Lovegrove, and A. J. Davison. Dtam: Dense tracking and mapping in real-time. In 2011 international conference on computer vision, pages 2320--2327. IEEE, 2011.
[17]
M. Nieuwenhuisen, D. Droeschel, M. Beul, and S. Behnke. Obstacle detection and navigation planning for autonomous micro aerial vehicles. In Unmanned Aircraft Systems (ICUAS), 2014 International Conference on, pages 1040--1047. IEEE, 2014.
[18]
M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, and A. Y. Ng. Ros: an open-source robot operating system. In ICRA workshop on open source software, volume 3, page 5. Kobe, Japan, 2009.
[19]
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi. You only look once: Unified, real-time object detection. arXiv preprint arXiv:1506.02640, 2015.
[20]
S. Ren, K. He, R. Girshick, and J. Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems, pages 91--99, 2015.
[21]
A. Saxena, S. H. Chung, and A. Y. Ng. Learning depth from single monocular images. In Advances in Neural Information Processing Systems, pages 1161--1168, 2005.
[22]
M. F. Selekwa, D. D. Dunlap, D. Shi, and E. G. Collins. Robot navigation in very cluttered environments by preference-based fuzzy behaviors. Robotics and Autonomous Systems, 56(3):231--246, 2008.
[23]
K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
[24]
S. Tulsiani and J. Malik. Viewpoints and keypoints. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1510--1519. IEEE, 2015.
[25]
K. Yamaguchi, D. McAllester, and R. Urtasun. Efficient joint segmentation, occlusion labeling, stereo and flow estimation. In European Conference on Computer Vision, pages 756--771. Springer, 2014.

Cited By

View all
  • (2021)Multi-obstacle detection based on monocular vision for UAV2021 IEEE 16th Conference on Industrial Electronics and Applications (ICIEA)10.1109/ICIEA51954.2021.9516384(1067-1072)Online publication date: 1-Aug-2021
  • (2021)A survey on recent optimal techniques for securing unmanned aerial vehicles applicationsTransactions on Emerging Telecommunications Technologies10.1002/ett.413332:7Online publication date: 5-Jul-2021
  • (2020)Towards Simulating Semantic Onboard UAV Navigation2020 IEEE Aerospace Conference10.1109/AERO47225.2020.9172771(1-15)Online publication date: Mar-2020
  • Show More Cited By

Index Terms

  1. DeepFly: towards complete autonomous navigation of MAVs with monocular camera

      Recommendations

      Comments

      Please enable JavaScript to view thecomments powered by Disqus.

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      ICVGIP '16: Proceedings of the Tenth Indian Conference on Computer Vision, Graphics and Image Processing
      December 2016
      743 pages
      ISBN:9781450347532
      DOI:10.1145/3009977
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      • Google Inc.
      • QI: Qualcomm Inc.
      • Tata Consultancy Services
      • NVIDIA
      • MathWorks: The MathWorks, Inc.
      • Microsoft Research: Microsoft Research

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 18 December 2016

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. autonomous navigation
      2. deep learning
      3. micro aerial vehicles

      Qualifiers

      • Research-article

      Conference

      ICVGIP '16
      Sponsor:
      • QI
      • MathWorks
      • Microsoft Research

      Acceptance Rates

      ICVGIP '16 Paper Acceptance Rate 95 of 286 submissions, 33%;
      Overall Acceptance Rate 95 of 286 submissions, 33%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)9
      • Downloads (Last 6 weeks)1
      Reflects downloads up to 12 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2021)Multi-obstacle detection based on monocular vision for UAV2021 IEEE 16th Conference on Industrial Electronics and Applications (ICIEA)10.1109/ICIEA51954.2021.9516384(1067-1072)Online publication date: 1-Aug-2021
      • (2021)A survey on recent optimal techniques for securing unmanned aerial vehicles applicationsTransactions on Emerging Telecommunications Technologies10.1002/ett.413332:7Online publication date: 5-Jul-2021
      • (2020)Towards Simulating Semantic Onboard UAV Navigation2020 IEEE Aerospace Conference10.1109/AERO47225.2020.9172771(1-15)Online publication date: Mar-2020
      • (2020)Obstacle Recognition and Avoidance for UAVs Under Resource-Constrained EnvironmentsIEEE Access10.1109/ACCESS.2020.30206328(169408-169422)Online publication date: 2020
      • (2019)Depth Estimation and Semantic Segmentation from a Single RGB Image Using a Hybrid Convolutional Neural NetworkSensors10.3390/s1908179519:8(1795)Online publication date: 15-Apr-2019
      • (2019)Learning a Controller Fusion Network by Online Trajectory Filtering for Vision-Based UAV Racing2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW.2019.00083(573-581)Online publication date: Jun-2019
      • (2019)Design Challenges of Multi-UAV Systems in Cyber-Physical Applications: A Comprehensive Survey and Future DirectionsIEEE Communications Surveys & Tutorials10.1109/COMST.2019.292414321:4(3340-3385)Online publication date: Dec-2020
      • (2019)Teaching UAVs to Race: End-to-End Regression of Agile Controls in SimulationComputer Vision – ECCV 2018 Workshops10.1007/978-3-030-11012-3_2(11-29)Online publication date: 29-Jan-2019
      • (2018)Hybridnet for Depth Estimation and Semantic Segmentation2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP.2018.8462433(1563-1567)Online publication date: Apr-2018
      • (2018)Sim4CVInternational Journal of Computer Vision10.1007/s11263-018-1073-7126:9(902-919)Online publication date: 1-Sep-2018
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media