[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

Path selection in disaster response management based on Q-learning

  • Published:
International Journal of Automation and Computing Aims and scope Submit manuscript

Abstract

Suitable rescue path selection is very important to rescue lives and reduce the loss of disasters, and has been a key issue in the field of disaster response management. In this paper, we present a path selection algorithm based on Q-learning for disaster response applications. We assume that a rescue team is an agent, which is operating in a dynamic and dangerous environment and needs to find a safe and short path in the least time. We first propose a path selection model for disaster response management, and deduce that path selection based on our model is a Markov decision process. Then, we introduce Q-learning and design strategies for action selection and to avoid cyclic path. Finally, experimental results show that our algorithm can find a safe and short path in the dynamic and dangerous environment, which can provide a specific and significant reference for practical management in disaster response applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. A. Mansourian, A. Rajabifard, M. J. V. Zoej, I. Williamson. Using SDI and web-based system to facilitate disaster management. Computers and Geosciences, vol. 32, no. 3, pp. 303–315, 2006.

    Article  Google Scholar 

  2. K. Friesen, D. Bell. Regional health authorities, disaster management, and geomatics: Opportunities and barriers. International Journal of Emergency Management, vol.4, no. 2, pp. 141–165, 2007.

    Article  Google Scholar 

  3. W. E. Roper. Waste management policy revisions: Lessons learned from the Katrina disaster. International Journal of Environmental Technology and Management, vol.8, no. 2/3, pp. 275–309, 2008.

    Article  Google Scholar 

  4. M. H. Tsai, L. Y. Liu, F. P. Mora, C. Arboleda. A preliminary design of disaster-survivable building blackbox system for Urban disaster response. Electronic Journal of Information Technology in Construction, vol. 13, no. 2, pp. 179–192, 2008.

    Google Scholar 

  5. P. G. Bello, I. Aedo, F. Sainz, P. Diaz, J. Munnelly, S. Clarke. Improving communication for mobile devices in disaster response. In Proceedings of the 1st International Conference on Mobile Information Technology for Emergency Response, Lecture Notes in Computer Science, Springer, vol. 4458, pp. 126–134, 2007.

    Google Scholar 

  6. F. R. Yu, J. Zhang, H. Tang, H. C. B. Chan, V. C. M. Leung. Enhancing interoperability in heterogeneous mobile wireless networks for disaster response. In Proceedings of IEEE Military Communications Conference, Orlando, Florida, USA, pp. 1–7, 2007.

  7. Télécoms Sans Frontières. Emergency Communications Aid Disaster Response, Geneva: International Telecommunications Union, pp. 34, 2005.

    Google Scholar 

  8. S. Thompson, N. Altay, W. G. Green III, J. Lepetina. Improving disaster response efforts with decision support systems. International Journal of Emergency Management, vol. 3, no. 4, pp. 250–263, 2006.

    Article  Google Scholar 

  9. Y. L. Tu, W. J. Zhang, X. Liu, W. Li, C. L. Chai, R. Deters. A disaster response management system based on the control systems technology. International Journal of Critical Infrastructures, vol. 4, no. 3, pp. 274–295, 2008.

    Article  Google Scholar 

  10. Z. W. Hu, X. J. Li, Y. H. Sun, L. Zhu. Flood disaster response and decision-making support system based on remote sensing and GIS. In Proceedings of International Geoscience and Remote Sensing Symposium, Barcelona, Spain, pp. 2435–2438, 2008.

  11. F. Fiedrich, P. Burghardt. Agent-based systems for disaster management. Communications of the ACM, vol. 50, no. 3, pp. 41–42, 2007.

    Article  Google Scholar 

  12. L. L. Yang, R. Prasanna, M. King. On-site information systems design for emergency first responders. Journal of Information Technology Theory and Application, vol. 10, no. 1, pp. 5–27, 2009.

    Google Scholar 

  13. M. Turoff, M. Chumer, B. Van deWalle, X. Yao. The design of a dynamic emergency response management information system. Journal of Information Technology Theory and Application, vol. 5, no. 4, pp. 1–36, 2004.

    Google Scholar 

  14. A. F. Chandio, L. Y. Shu, N. M. Memon, A. Khawaja. GIS based route guiding system for optimal path planning in disaster/crisis management. In Proceedings of the 10th IEEE International Multitopic Conference, Islamabad, Pakistan, pp. 207–210, 2006.

  15. L. Ozdamar, W. Yi. Greedy neighborhood search for disaster relief and evacuation logistics. IEEE Intelligent Systems, vol. 13, no. 1, pp. 14–23, 2008.

    Google Scholar 

  16. Y. C. Chiou, Y. H. Lai. An integrated multi-objective model to determine the optimal rescue path and traffic controlled arcs for disaster relief operations under uncertainty environments. Journal of Advanced Transportation, vol. 42, no. 4, pp. 493–519, 2008.

    Article  Google Scholar 

  17. Y. Yuan, D. W. Wang. Path selection model and algorithm for emergency logistics management. Computers and Industrial Engineering, vol. 56, no. 3, pp. 1081–1094, 2009.

    Article  Google Scholar 

  18. E. G. Lopez. Efficient graph-based genetic programming representation with multiple outputs. International Journal of Automation and Computing, vol. 5, no. 1, pp. 81–89, 2008.

    Article  Google Scholar 

  19. L. P. Kaelbling, M. L. Littman, A. W. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, vol. 4, no. 1, pp. 237–285, 1996.

    Google Scholar 

  20. M. Kumar, A. K. Verma, A. Srividya. Analyzing effect of demand rate on safety of systems with periodic proof-tests. International Journal of Automation and Computing, vol.4, no. 4, pp. 335–341, 2007.

    Article  Google Scholar 

  21. C. J. C. H. Watkins, P. Dayan. Technical note: Q-learning. Machine Learning, vol. 8, no. 3–4, pp. 279–292, 1992.

    MATH  Google Scholar 

  22. V. S. Borkar. Q-learning for risk-sensitive control, Mathematics of Operations Research, vol. 27, no. 2, pp. 294–311, 2002.

    Article  MathSciNet  MATH  Google Scholar 

  23. M. Andrecut, M. K. Ali. Q learning in the minority game. Physical Review E, vol. 64, no. 6, pp. 1–4, 2001.

    Article  MathSciNet  Google Scholar 

  24. J. G. Jiang, Z. P. Su, M. B. Qi, G. F. Zhang. Multi-task coalition parallel formation strategy based on reinforcement learning. Acta Automatica Sinica, vol. 34, no. 3, pp. 349–352, 2008.

    Article  Google Scholar 

  25. S. M. Lucas. Computational intelligence and games: Challenges and opportunities. International Journal of Automation and Computing, vol. 5, no. 1, pp. 45–57, 2008.

    Article  Google Scholar 

  26. S. Q. Yu, H. Q. Wang, F. M. Ye, S. Mabu, K. Shimada, K. Hirasawa. A Q value-based dynamic programming algorithm with Boltzmann distribution for optimizing the global traffic routing strategy. In Proceedings of SICE Annual Conference, Tokyo, Japan, pp. 619–622, 2008.

  27. K. Takadama, T. Kawai, Y. Koyama. Can agents acquire human-like behaviors in a sequential bargaining game? — Comparison of Roth’s and Q-learning agents. In Proceedings of the 7th International Workshop on Multi-agent-based Simulation, Hakodate, Japan, pp. 156–171, 2006.

Download references

Author information

Authors and Affiliations

  1. Key Laboratory of Special Display Technology (Hefei University of Technology), Ministry of Education, Hefei, 230009, PRC

    Zhao-Pin Su, Jian-Guo Jiang & Guo-Fu Zhang

  2. School of Computer and Information, Hefei University of Technology, Hefei, 230009, PRC

    Zhao-Pin Su, Jian-Guo Jiang, Chang-Yong Liang & Guo-Fu Zhang

  3. Postdoctoral Research Station for Management Science and Engineering, Hefei University of Technology, Hefei, 230009, PRC

    Zhao-Pin Su & Chang-Yong Liang

  4. Engineering Research Center of Safety Critical Industrial Measurement and Control Technology, Ministry of Education, Hefei, 230009, PRC

    Jian-Guo Jiang & Guo-Fu Zhang

Authors
  1. Zhao-Pin Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian-Guo Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chang-Yong Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guo-Fu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhao-Pin Su.

Additional information

This work was supported by National Basic Research Program of China (973 Program) (No. 2009CB326203), National Natural Science Foundation of China (No. 61004103), the National Research Foundation for the Doctoral Program of Higher Education of China (No. 20100111110005), China Postdoctoral Science Foundation (No. 20090460742), National Engineering Research Center of Special Display Technology (No. 2008HGXJ0350), Natural Science Foundation of Anhui Province (No. 090412058, No. 070412035), Natural Science Foundation of Anhui Province of China (No. 11040606Q44, No. 090412058), Specialized Research Fund for Doctoral Scholars of Hefei University of Technology (No.GDBJ2009-003, No.GDBJ2009-067)

Zhao-Pin Su received the B. Sc. and Ph.D. degrees in computer science from Hefei University of Technology, Hefei, PRC in 2004 and 2008, respectively. Currently, she is a lecturer in the School of Computer and Information, Hefei University of Technology. Also, she is now working together with Guo-Fu Zhang to model and solve disaster response coalition formation for unconventional emergency at Postdoctoral Research Station for Management Science and Engineering in Hefei University of Technology.

Her research interests include autonomous agent, reinforcement learning, and immune algorithm.

Jian-Guo Jiang received the M. Sc. degree in computer science from Hefei University of Technology, Hefei, PRC in 1989. He is currently a professor in the School of Computer and Information, Hefei University of Technology. He is head of the Texas Instruments-Hefei University of Technology DSPS Laboratory in Engineering Research Center of Safety Critical Industrial Measurement and Control Technology, Ministry of Education.

His research interests include automatic control, image processing, and software engineering.

Chang-Yong Liang received the Ph.D. degree from Harbin Institute of Technology, PRC in 2001. He is currently a professor in the School of Management, Hefei University of Technology, PRC.

His research interests include collaborative filtering and intelligent decision support system.

Guo-Fu Zhang received the B. Sc. and Ph.D. degrees in computer science from Hefei University of Technology, Hefei, PRC in 2002 and 2008, respectively. He is currently a lecturer in the School of Computer and Information, Hefei University of Technology.

His research interests include evolutionary computation, intelligent agent, and multi-agent systems, especially in coalition formation.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Su, ZP., Jiang, JG., Liang, CY. et al. Path selection in disaster response management based on Q-learning. Int. J. Autom. Comput. 8, 100–106 (2011). https://doi.org/10.1007/s11633-010-0560-2

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11633-010-0560-2

Keywords

Search

Navigation