Abstract
As the scarce spectrum resource is becoming over-crowded, cognitive wireless mesh networks have great flexibility to improve the spectrum utilization by opportunistically accessing the licensed frequency bands. One of the critical challenges for realizing such network is how to adaptively allocate transmit powers and frequency resources among secondary users (SUs) of the licensed frequency bands while maintaining the quality-of-service (QoS) requirement of the primary users (PUs). In this paper, we consider the power control problem in the context of cognitive wireless mesh networks formed by a number of clusters under the total transmit power constraint by each SU as well as the mean-squared error (MSE) constraint by PUs. The problem is modeled as a non-cooperative game. A distributed iterative power allocation algorithm is designed to reach the Nash equilibrium (NE) between the coexisting interfered links. It offers an opportunity for SUs to negotiate the best use of power and frequency with each other. Furthermore, how to adaptively negotiate the transmission power level and spectrum usage among the SUs according to the changing networking environment is discussed. We present an intelligent policy based on reinforcement learning to acquire the stochastic behavior of PUs. Based on the learning approach, the SUs can adapt to the dynamics of the interference environment state and reach new NEs quickly through partially cooperative information sharing via a common control channel. Theoretical analysis and numerical results both show effectiveness of the intelligent policy.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Chen, T., Zhang, H., Maggio, G. M., & Chlamtac, I. (2007). CogMesh: A cluster-based cognitive radio network. In Proceedings of IEEE DySPAN (pp. 168–178).
Chen, X., Zhao, Z., Jiang, T., Grace, D., & Zhang, H. (2009). Intercluster connection in cognitive wireless mesh networks based on intelligent network coding. EURASIP Journal on Advances in Signal Processing, Article ID 141097.
Chung, S. T., Kim, S. J., Lee, J., & Cioffi, J. M. (2003). A game theoretic approach to power allocation in frequency–selective gaussian interference channels. In Proceedings of IEEE International Symposium Information theory (pp. 316–316).
Etkin R., Parekh A., Tse D. (2007) Spectrum sharing for unlicensed bands. IEEE Journal of Selected Areas in Communications 25(3): 517–528
Federal Communications Commission. (2002). Spectrum policy task force. Rep. ET Docket No. 02–135, November 2002.
Fu F., Schaar M. (2009) Learning to compete for resources in wireless stochastic games. IEEE Transactions on Vehicular Technology 58: 1904–1919
Haykin S. (2005) Cognitive radio: Brain-empowered wireless communications. IEEE Journal of Selected Areas in Communications 23(2): 201–220
Mitola J., Maguire G. Q. (1999) Cognitive radios: Making software radios more personal. IEEE Personal Communications 6(4): 13–18
Nie J., Haykin S. (1999) A Q-learning-based dynamic channel assignment technique for mobile communication systems. IEEE Transaction on Vehicular Technology 48(5): 1676–1687
Shi, Y., & Hou, T. (2008). A distributed optimization algorithm for multi-hop cognitive radio networks. In Proceedings of IEEE INFOCOM (pp. 1292–1300).
Song Y., Zhang C., Fang Y. (2010) Stochastic traffic engineering in multi-hop cognitive wireless mesh networks. IEEE Transaction on Mobile Computing 9(3): 305–316
Sutton R. S., Barto A. G. (1998) Reinforcement learning: An introduction. MIT Press, Cambridge, MA
Wang F., Krunz M., Cui S. (2008) Price-based spectrum management in cognitive radio networks. IEEE Journal of Selected Topics in Signal Processing 2: 74–87
Yu W., Ginis G., Cioffi J. M. (2002) Distributed multiuser power control for digital subscriber lines. IEEE Journal on Selected Areas in Communications 20(5): 1105–1115
Zhang, L., Xin, Y., & Liang, Y. (2007). Power allocation for multi-antenna multiple access channels in cognitive radio networks. In Proceedings of 41st annual conference on information sciences and systems (pp. 351–356).
Zhang, W., & Mitra, U. (2008). A spectrum-shaping perspective on cognitive radio. In Proceedings of IEEE DySPAN (pp. 1–12).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Chen, X., Zhao, Z., Zhang, H. et al. Reinforcement Learning Enhanced Iterative Power Allocation in Stochastic Cognitive Wireless Mesh Networks. Wireless Pers Commun 57, 89–104 (2011). https://doi.org/10.1007/s11277-010-0008-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11277-010-0008-6