[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content

Advertisement

Springer Nature Link
Log in

A distributed reinforcement learning based sensor node scheduling algorithm for coverage and connectivity maintenance in wireless sensor network

  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

The fundamental challenge for randomly deployed resource-constrained wireless sensor network is to enhance the network lifetime without compromising its performance metrics such as coverage rate and network connectivity. One way is to schedule the activities of sensor nodes and form scheduling rounds autonomously in such a way that each spatial point is covered by at least one sensor node and there must be at least one communication path from the sensor nodes to base station. This autonomous activity scheduling of the sensor nodes can be efficiently done with Reinforcement Learning (RL), a technique of machine learning because it does not require prior environment modeling. In this paper, a Nash Q-Learning based node scheduling algorithm for coverage and connectivity maintenance (CCM-RL) is proposed where each node autonomously learns its optimal action (active/hibernate/sleep/customize the sensing range) to maximize the coverage rate and maintain network connectivity. The learning algorithm resides inside each sensor node. The main objective of this algorithm is to enable the sensor nodes to learn their optimal action so that the total number of activated nodes in each scheduling round becomes minimum and preserves the criteria of coverage rate and network connectivity. The comparison of CCM-RL protocol with other protocols proves its accuracy and reliability. The simulative comparison shows that CCM-RL performs better in terms of an average number of active sensor nodes in one scheduling round, coverage rate, and energy consumption.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Wang, X., Xing, G., Zhang, Y., Lu, C., Pless, R., & Gill, C. (2003). Integrated coverage and connectivity configuration in wireless sensor networks. In Proceedings of the 1st international conference on embedded networked sensor systems (pp. 28–39).

  2. Zhang, H., & Hou, J. C. (2005). Maintaining sensing coverage and connectivity in large sensor networks. Ad Hoc and Sensor Wireless Networks,1, 89–124.

    Google Scholar 

  3. Onur, E., Ersoy, C., Deliҫ, H., & Akarun, L. (2007). Surveillance wireless sensor networks: Deployment quality analysis. IEEE Network,21(6), 48–53.

    Google Scholar 

  4. Mini, S., Udgata, S. K., & Sabat, S. L. (2014). Sensor deployment and scheduling for target coverage problem in wireless sensor networks. IEEE Sensor Journal,14(3), 636–644.

    Google Scholar 

  5. Yu, J., Wan, S., Cheng, X., & Yu, D. (2017). Coverage contribution area based k-coverage for wireless sensor networks. IEEE Transactions on Vehicular Technology,66(9), 8510–8523.

    Google Scholar 

  6. Jiang, B., Ravindran, B., & Cho, H. (2012). Probability-based prediction and sleep scheduling for energy-efficient target tracking in sensor networks. IEEE Transactions on Mobile Computing,12(4), 735–747.

    Google Scholar 

  7. Zorbas, D., Glynos, D., Kotzanikolaou, P., & Douligeris, C. (2010). Solving coverage problems in wireless sensor networks using cover sets. Ad Hoc Networks,8(4), 400–415.

    Google Scholar 

  8. Mohamadi, H., Ismail, A. S., & Salleh, S. (2014). Solving target coverage problem using cover sets in wireless sensor networks based on learning automata. Wireless Personal Communications,75(1), 447–463.

    Google Scholar 

  9. Seah, M. W. M., Tham, C. K., Srinivasan, V., & Xin, A. (2007). Achieving coverage through distributed reinforcement learning in wireless sensor networks. In Proceedings of the 3rd international conference on intelligent sensors, sensor networks and information (pp. 425–430).

  10. Esnaashari, M., & Meybodi, M. R. (2010). A learning automata based scheduling solution to the dynamic point coverage problem in wireless sensor networks. Computer Networks,54(14), 2410–2438.

    MATH  Google Scholar 

  11. Chen, H., Li, X., & Zhao, F. (2016). A reinforcement learning-based sleep scheduling algorithm for desired area coverage in solar-powered wireless sensor networks. IEEE Sensors Journal,16(8), 2763–2774.

    Google Scholar 

  12. Mostafaei, H., Montieri, A., Persico, V., & Pescapé, A. (2017). A sleep scheduling approach based on learning automata for WSN partial coverage. Journal of Network and Computer Applications,80, 67–78.

    Google Scholar 

  13. Esnaashari, M., & Meybodi, M. R. (2010). Data aggregation in sensor networks using learning automata. Wireless Networks,16(3), 687–699.

    MATH  Google Scholar 

  14. Lu, Y., Zhang, T., He, E., & Comşa, I. S. (2018). Self-learning-based data aggregation scheduling policy in wireless sensor networks. Journal of Sensors,2018, 9647593.

  15. Shahina, K., & Vaidehi, V. (2018). Clustering and data aggregation in wireless sensor networks using machine learning algorithms. In Proceedings of the 2018 international conference on recent trends in advance computing (pp. 109–115).

  16. Forster, A., & Murphy, A. L. (2009). CLIQUE: Role-free clustering with Q-learning for wireless sensor networks. In Proceedings of the 29th IEEE international conference on distributed computing systems (pp. 441–449).

  17. Aoudia, F. A., Gautier, M., & Berder, O. (2018). RLMan: An energy manager based on reinforcement learning for energy harvesting wireless sensor networks. IEEE Transactions on Green Communications and Networking,2(2), 408–417.

    Google Scholar 

  18. Guo, W., & Zhang, W. (2014). A survey on intelligent routing protocols in wireless sensor networks. Journal of Network and Computer Applications,38, 85–201.

    Google Scholar 

  19. Ye, D., & Zhang, M. (2017). A self-adaptive sleep/wake-up scheduling approach for wireless sensor networks. IEEE Transactions on Cybernetics,48(3), 979–992.

    Google Scholar 

  20. Savaglio, C., Pace, P., Aloi, G., Liotta, A., & Fortino, G. (2019). Lightweight reinforcement learning for energy efficient communications in wireless sensor networks. IEEE Access,7, 29355–29364.

    Google Scholar 

  21. Raj, A. B., Ramesh, M. V., Kulkarni, R. V., & Hemalatha, T. (2012). Security enhancement in wireless sensor networks using machine learning. In Proceedings of the IEEE 14th international conference on high performance computing and communication (pp. 1264–1269).

  22. Kulkarni, R. V., Forster, A., & Venayagamoorthy, G. K. (2010). Computational intelligence in wireless sensor networks: A survey. IEEE Communications Surveys & Tutorials,13(1), 68–96.

    Google Scholar 

  23. Hu, J., & Wellman, M. P. (2003). Nash Q-learning for general-sum stochastic games. Journal of Machine Learning Research,4(Nov), 1039–1069.

    MathSciNet  MATH  Google Scholar 

  24. Gosavi, A. (2009). Reinforcement learning: A tutorial survey and recent advances. INFORMS Journal on Computing,21(2), 178–192.

    MathSciNet  MATH  Google Scholar 

  25. Bu, L., Babu, R., & De Schutter, B. (2008). A comprehensive survey of multiagent reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews),38(2), 56–172.

    Google Scholar 

  26. Yau, K. L. A., Komisarczuk, P., & Teal, P. D. (2012). Reinforcement learning for context awareness and intelligence in wireless networks: Review, new features and open issues. Journal of Network and Computer Applications,35(1), 253–267.

    Google Scholar 

  27. Yau, K. L. A., Goh, H. G., Chieng, D., & Kwong, K. H. (2015). Application of reinforcement learning to wireless sensor networks: models and algorithms. Computing,97(11), 1045–1075.

    MathSciNet  MATH  Google Scholar 

  28. Al-Karaki, J. N., & Gawanmeh, A. (2017). The optimal deployment, coverage, and connectivity problems in wireless sensor networks: Revisited. IEEE Access,5, 8051–18065.

    Google Scholar 

  29. Borasia, S., & Raisinghani, V. (2011). A review of congestion control mechanisms for wireless sensor networks. In Technology systems and management (pp. 201-206). Berlin: Springer.

  30. Ishmanov, F., Malik, A. S., & Kim, S. W. (2011). Energy consumption balancing (ECB) issues and mechanisms in wireless sensor networks (WSNs): A comprehensive overview. European Transactions on Telecommunications,22(4), 151–167.

    Google Scholar 

  31. Du, X., & Chen, H. H. (2008). Security in wireless sensor networks. IEEE Wireless Communications,15(4), 60–66.

    Google Scholar 

  32. Wang, B. (2011). Coverage problems in sensor networks: A survey. ACM Computing Surveys (CSUR),43(4), 1–53.

    Google Scholar 

  33. Yetgin, H., Cheung, K. T. K., El-Hajjar, M., & Hanzo, L. H. (2017). A survey of network lifetime maximization techniques in wireless sensor networks. IEEE Communications Surveys & Tutorials,19(2), 828–854.

    Google Scholar 

  34. More, A., & Raisinghani, V. (2017). A survey on energy efficient coverage protocols in wireless sensor networks. Journal of King Saud University-Computer and Information Sciences,29(4), 428–448.

    Google Scholar 

  35. Cardei, M., Wu, J., & Lu, M. (2006). Improving network lifetime using sensors with adjustable sensing ranges. International Journal of Sensor Networks,1(1–2), 41–49.

    Google Scholar 

  36. Kumar, D. P., Amgoth, T., & Annavarapu, C. S. R. (2019). Machine learning algorithms for wireless sensor networks: A survey. Information Fusion,49, 1–25.

    Google Scholar 

  37. Hefeeda, M., & Ahmadi, H. (2007). A probabilistic coverage protocol for wireless sensor networks. In Proceedings of IEEE international conference on network protocol (pp. 41–50).

  38. Mohamadi, H., Salleh, S., Razali, M. N., & Marouf, S. (2015). A new learning automata-based approach for maximizing network lifetime in wireless sensor networks with adjustable sensing ranges. Neurocomputing,153, 11–19.

    Google Scholar 

  39. Chenait, M., Zebbane, B., Benzaid, C., & Badache, N. (2015). Sleep scheduling with predictive coverage redundancy check in wireless sensor networks. In Proceedings of the international symposium on wireless communication systems (pp. 366–370).

  40. Basheer, S., Mathew, R. M., Ranjith, D., Sathish Kumar, M., Sundar, P., & Balajee, J. M. (2019). An analysis on barrier coverage in wireless sensor networks. Journal of Computational and Theoretical Nanoscience,16(5–6), 2599–2603.

    Google Scholar 

  41. Niculescu, D., & Nath, B. (2001). Ad hoc positioning system (APS). In Proceedings of the IEEE global telecommunications conference, vol. 5 (pp. 2926–2931).

  42. Zhu, J., & Papavassiliou, S. (2003). On the energy-efficient organization and the lifetime of multi-hop sensor networks. IEEE Communications Letters,7(11), 537–539.

    Google Scholar 

  43. Lauer, M., & Riedmiller, M. (2000). An algorithm for distributed reinforcement learning in cooperative multi-agent systems. In Proceedings of the 17th international conference on machine learning.

  44. Littman, M. L. (2001). Friend-or-foe Q-learning in general-sum games. ICML,1, 322–328.

    Google Scholar 

  45. Greenwald, A., Hall, K., & Serrano, R. (2003). Correlated Q-learning. In ICML, vol. 3 (pp. 242–249).

  46. Littman, M. L. (1994). Markov games as a framework for multi-agent reinforcement learning. In Machine learning proceedings (pp. 157–163).

  47. Wang, X., & Sandholm, T. (2003). Reinforcement learning to play an optimal Nash equilibrium in team Markov games. In Advances in neural information processing systems (pp. 1603–1610).

Download references

Author information

Authors and Affiliations

  1. Computer Science and Engineering Department, National Institute of Technology, Hamirpur, Himachal Pradesh, 177005, India

    Anamika Sharma & Siddhartha Chauhan

Authors
  1. Anamika Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Siddhartha Chauhan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anamika Sharma.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sharma, A., Chauhan, S. A distributed reinforcement learning based sensor node scheduling algorithm for coverage and connectivity maintenance in wireless sensor network. Wireless Netw 26, 4411–4429 (2020). https://doi.org/10.1007/s11276-020-02350-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11276-020-02350-y

Keywords