[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

Reinforcement-Learning-Based Double Auction Design for Dynamic Spectrum Access in Cognitive Radio Networks

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

In cognitive radio networks, an important issue is to share the detected available spectrum among different secondary users to improve the network performance. Although some work has been done for dynamic spectrum access, the learning capability of cognitive radio networks is largely ignored in the previous work. In this paper, we propose a reinforcement-learning-based double auction algorithm aiming to improve the performance of dynamic spectrum access in cognitive radio networks. The dynamic spectrum access process is modeled as a double auction game. Based on the spectrum access history information, both primary users and secondary users can estimate the impact on their future rewards and then adapt their spectrum access or release strategies effectively to compete for channel opportunities. Simulation results show that the proposed reinforcement-learning-based double auction algorithm can significantly improve secondary users’ performance in terms of packet loss, bidding efficiency and transmission rate or opportunity access.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Haykin S. (2005) Cognitive radio: Brain-empowered wireless communications. IEEE Journal on Selected Areas in Communications 23(2): 201–220

    Article  Google Scholar 

  2. Yucek T., Arslan H. (2009) A survey of spectrum sensing algorithms for cognitive radio applications. IEEE Communications Surveys & Tutorials 11(1): 116–130

    Article  Google Scholar 

  3. Haykin S., Thomson D. J., Reed J. H. (2009) Spectrum sensing for cognitive radio. IEEE Proceedings 97(5): 849–877

    Article  Google Scholar 

  4. Yu F. R., Huang M., Tang H. (2010) Biologically inspired consensus-based spectrum sensing in mobile ad hoc networks with cognitive radios. IEEE Networks 24(3): 26–30

    Article  Google Scholar 

  5. Yiping X., Chandramouli R., Stefan M., Sai Shankar N. (2006) Dynamic spectrum access in open spectrum wireless networks. IEEE Journal on Selected Areas in Communications, 24(3): 626–637

    Article  Google Scholar 

  6. Zhu J., Liu K. J. R. (2007) Cognitive radios for dynamic spectrum access—dynamic spectrum sharing: A game theoretical overview. IEEE Communications Magazine 45(5): 1–5

    Article  Google Scholar 

  7. Wang B., Wu Y., Ji Z., Liu K. J. R., Clancy T. C. (2008) Game theoretical mechanism design methods. IEEE Signal Processing Magazine 25(6): 74–84

    Article  Google Scholar 

  8. Krishnamurthy V. (2009) Decentralized spectrum access amongst cognitive agents—an interacting multivariate global games approach. IEEE Transactions on Signal Processing 57(10): 3999–4013

    Article  MathSciNet  Google Scholar 

  9. Niyato D., Hossain E. (2008) Competitive spectrum sharing in cognitive radio networks: A dynamic game approach. IEEE Transactions on Wireless Communications 7(7): 2651–2660

    Article  Google Scholar 

  10. Zhu J., Liu K. J. R. (2008) Multi-stage pricing game for collusion-resistant dynamic spectrum allocation. IEEE Journal on Selected Areas in Communications 26(1): 182–191

    Article  Google Scholar 

  11. Zhu, J., & Liu, K. J. R. (2006). Belief-assisted pricing for dynamic spectrum allocation in wireless networks with selfish users. In Proceedings of IEEE SECON’06, pp. 119–127.

  12. Niyato D., Hossain E. (2008) Market-equilibrium, competitive, and cooperative pricing for spectrum sharing in cognitive radio networks: Analysis and comparison. IEEE Transactions on Wireless Communications 7(11): 4273–4283

    Article  Google Scholar 

  13. Niyato D., Hossain E. (2008) Spectrum trading in cognitive radio networks: A market-equilibrium-based approach. IEEE Wireless Communications 15(6): 71–80

    Article  Google Scholar 

  14. Bobrow D. G. (1994) Artificial intelligence in perspective. MIT Press, Cambridge, MA

    Google Scholar 

  15. Turing, A. M. (1950). Computing machinery and intelligence. Mind, 59(236), 71–80, 1–5.

    Google Scholar 

  16. Sutton R. S., Barto A. G. (1998) Reinforcement learning. MIT Press, Cambridge, MA

    Google Scholar 

  17. Yu F. R., Wong V. W. S., Leung V. C. M. (2008) A new QoS provisioning method for adaptive multimedia in wireless networks. IEEE Transactions on Vehicular Technology 57(3): 1899–1909

    Article  Google Scholar 

  18. Bernardo F., Agusti R., Perez-Romero J., Sallent O. (2011) Intercell interference management in OFDMA networks: A decentralized approach based on reinforcement learning. IEEE Transactions on Systems, Man and Cybernetics, Part C, 41(6): 1–9

    Article  Google Scholar 

  19. Gibbons R. D. (1992) Game theory for applied economists. Princeton University Presss, Princeton

    Google Scholar 

  20. Shankar, S., Chou, C. T., Challapali, K., & Mangold, S. (2005). Spectrum agile radio: Capacity and QoS implications of dynamic spectrum assignment. Proceedings of IEEE Globecom’05, pp. 2510–2516.

  21. Fangwen F., van der Schaar M. (2009) Learning to compete for resources in wireless stochastic games. IEEE Transactions on Vehicular Technology 58(4): 1904–1919

    Article  Google Scholar 

  22. Niyato D., Hossain E., Han Z. (2009) Dynamics of multiple-seller and multiple-buyer spectrum trading in cognitive radio networks: A game-theoretic modeling approach. IEEE Transactions on Mobile Computing 8(8): 1009–1022

    Article  Google Scholar 

  23. Roy, N., Roy, A., & Das, S. K. (2005). A cooperative learning framework for Mobility-aware resource management in multi-inhabitant smart momes. In Proceedings of IEEE MobiQuitous’05, pp. 393–403.

  24. Jakkola T., Singh S.P. (1994) On the convergence of stochastic iterative dynamic programming algorithms. Neural Computation 6(6): 1185–1201

    Article  Google Scholar 

  25. Venkatesh T., Kiran Y. V., Murthy C. S. R. (2009) Joint path and wavelength selection using Q-learning in Optical burst switching networks. In: Proceedings of IEEE Globecom’09 (Vol. 15, no 6, pp. 1–5).

Download references

Author information

Authors and Affiliations

  1. Beijing University of Posts and Telecommunications, Beijing, China

    Yinglei Teng, Ke Han, Yifei Wei & Yong Zhang

  2. Defense R&D Canada, Ottawa, ON, Canada

    F. Richard Yu

Authors
  1. Yinglei Teng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. F. Richard Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ke Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yifei Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yinglei Teng.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Teng, Y., Yu, F.R., Han, K. et al. Reinforcement-Learning-Based Double Auction Design for Dynamic Spectrum Access in Cognitive Radio Networks. Wireless Pers Commun 69, 771–791 (2013). https://doi.org/10.1007/s11277-012-0611-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-012-0611-9

Keywords

Search

Navigation