[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

A Spatial Cognitive Model that Integrates the Effects of Endogenous and Exogenous Information on the Hippocampus and Striatum

  • Research Article
  • Published:
International Journal of Automation and Computing Aims and scope Submit manuscript

Abstract

Reproducing the spatial cognition of animals using computational models that make agents navigate autonomously has attracted much attention. Many biologically inspired models for spatial cognition focus mainly on the simulation of the hippocampus and only consider the effect of external environmental information (i.e., exogenous information) on the hippocampal coding. However, neurophysiological studies have shown that the striatum, which is closely related to the hippocampus, also plays an important role in spatial cognition and that information inside animals (i.e., endogenous information) also affects the encoding of the hippocampus. Inspired by the progress made in neurophysiological studies, we propose a new spatial cognitive model that consists of analogies between the hippocampus and striatum. This model takes into consideration how both exogenous and endogenous information affects coding by the environment. We carried out a series of navigation experiments that simulated a water maze and compared our model with other models. Our model is self-adaptable and robust and has better performance in navigation path length. We also discuss the possible reasons for the results and how our findings may help us understand real mechanisms in the spatial cognition of animals.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. M. Contreras, T. Pelc, M. Llofriu, A. Weitzenfeld, J. M. Fellous. The ventral hippocampus is involved in multi-goal obstacle-rich spatial navigation. Hippocampus, vol. 28, no. 12, pp. 853–866, 2018. DOI: https://doi.org/10.1002/hipo.22993.

    Article  Google Scholar 

  2. I. Ardiyanto, J. Miura. Time-space viewpoint planning for guard robot with chance constraint. International Journal of Automation and Computing, vol. 16, no. 4, pp. 475–490, 2019. DOI: https://doi.org/10.1007/s11633-018-1146-7.

    Article  Google Scholar 

  3. A. D. Rast, S. V. Adams, S. Davidson, S. Davies, M. Hopkins, A. Rowley, A. B. Stokes, T. Wennekers, S. Furber, A. Cangelosi. Behavioral learning in a cognitive neuromorphic robot: An integrative approach. IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 12, pp. 6132–6144, 2018. DOI: https://doi.org/10.1109/TNNLS.2018.2816518.

    Article  Google Scholar 

  4. N. Cazin, P. Scleidorovich, A. Weitzenfeld, P. F. Dominey. Real-time sensory-motor integration of hippocampal place cell replay and prefrontal sequence learning in simulated and physical rat robots for novel path optimization. Biological Cybernetics, vol. 114, no. 2, pp. 249–268, 2020. DOI: https://doi.org/10.1007/s00422-020-00820-2.

    Article  MATH  Google Scholar 

  5. J. Nassour, T. D. Hoa, P. Atoofi, F. Hamker. Concrete action representation model: From neuroscience to robotics. IEEE Transactions on Cognitive and Developmental Systems, vol. 12, no. 2, pp. 272–284, 2020. DOI: https://doi.org/10.1109/TCDS.2019.2896300.

    Article  Google Scholar 

  6. J. O’keefe, L. Nadel The Hippocampus as a Cognitive Map, Oxford, UK: Oxford University Press, 1978.

    Google Scholar 

  7. T. Hafting, M. Fyhn, S. Molden, M. B. Moser, E. I. Moser. Microstructure of a spatial map in the entorhinal cortex. Nature, vol. 436, no. 7052, pp. 801–806, 2005. DOI: https://doi.org/10.1038/nature03721.

    Article  Google Scholar 

  8. G. Tejera, M. Llofriu, A. Barrera, T. Alfredo. Bio-inspired robotics: a spatial cognition model integrating place cells, grid cells and head direction cells. Journal of Intelligent & Robotic Systems, vol. 91, no. 1, pp. 85–99, 2018. DOI: https://doi.org/10.1007/s10846-018-0852-2.

    Article  Google Scholar 

  9. J. O’Keefe, N. Burgess. Geometric determinants of the place fields of hippocampal neurons. Nature, vol. 381, no. 6581, pp. 425–428, 1996. DOI: https://doi.org/10.1038/381425a0.

    Article  Google Scholar 

  10. H. Wagatsuma, Y. Yamaguchi. Neural dynamics of the cognitive map in the hippocampus. Cognitive Neurodynamics, vol. 1, no. 2, pp. 119–141, 2007. DOI: https://doi.org/10.1007/s11571-006-9013-6.

    Article  Google Scholar 

  11. Y. F. Zuo, F. Luo, C. L. Cui. Processing of spatial information by hippocampal place cells. Progress in Physiological Sciences, vol. 37, no. 1, pp. 6–10, 2006. (in Chinese)

    Google Scholar 

  12. J. W. Liu. Study on Navigation Methods of Intelligent Robots Based on Cognitive Map, Master dissertation, Northeastern University, China, 2015. (in Chinese)

    Google Scholar 

  13. E. I. Moser, M. B. Moser, P. Lipa, M. Newton, F. P. Houston, C. A. Barnes, B. L. McNaughton. A test of the reverberatory activity hypothesis for hippocampal “place” cells. Neuroscience, vol. 130, no. 2, pp. 519–526, 2005. DOI: https://doi.org/10.1016/e.neuroscience.2004.09.044.

    Article  Google Scholar 

  14. C. Barry, C. Lever, R. Hayman, T. Hartley, S. Burton, J. O’Keefe, K. Jeffery, N. Burgess. The boundary vector cell model of place cell firing and spatial memory. Reviews in the Neurosciences, vol. 17, no. 1–2, pp. 71–97, 2006. DOI: https://doi.org/10.1515/revneuro.2006.17.1-2.71.

    Google Scholar 

  15. C. Lever, S. Burton, A. Jeewajee, J. O’Keefe, N. Burgess. Boundary vector cells in the subiculum of the hippocampal formation. Journal of Neuroscience, vol. 29, no. 31, pp. 9771–9777, 2009. DOI: https://doi.org/10.1523/JNEUROSCI.1319-09.2009.

    Article  Google Scholar 

  16. R. M. Grieves, É. Duvelle, P. A. Dudchenko. A boundary vector cell model of place field repetition. Spatial Cognition & Computation, vol. 18, no. 3, pp. 217–256, 2018. DOI: https://doi.org/10.1080/13875868.2018.1437621.

    Article  Google Scholar 

  17. E. T. Rolls, S. M. Stringer, T. Elliot. Entorhinal cortex grid cells can map to hippocampal place cells by competitive learning. Network: Computation in Neural Systems, vol. 17, no. 4, pp. 447–465, 2006. DOI: https://doi.org/10.1080/09548980601064846.

    Article  Google Scholar 

  18. N. G. Yu, Y. S. Liao, X. G. Zheng. A spatial cognition model based on the selection mechanism of hippocampus place cells. Journal of Biomedical Engineering, vol. 37, no. 1, pp. 27–37, 2020. DOI: https://doi.org/10.7507/1001-5515.201901044. (in Chinese)

    Google Scholar 

  19. Y. Zhou, D. W. Wu. Location estimation model based on the transformation from grid cells to place cells. Journal of Electronics & Information Technology, vol. 39, no. 9, pp. 2272–2276, 2017. DOI: https://doi.org/10.11999/JEIT161284. (in Chinese)

    Google Scholar 

  20. R. S. Sutton, A. G. Barto. Reinforcement Learning: An Introduction, 2nd ed., Cambridge, UK: MIT Press, 2018.

    MATH  Google Scholar 

  21. F. Chersi, N. Burgess. The cognitive architecture of spatial navigation: Hippocampal and striatal contributions. Neuron, vol. 88, no. 1, pp. 64–77, 2015. DOI: https://doi.org/10.1016/j.neuron.2015.09.021.

    Article  Google Scholar 

  22. C. M. A. Pennartz, R. Ito, P. F. M. J. Verschure, F. P. Battaglia, T. W. Robbins The hippocampal-striatal axis in learning, prediction and goal-directed behavior Trends in Neurosciences, vol. 34, no. 10, pp. 548–559, 2011 DOI: https://doi.org/10.1016/j.tins.2011.08.001.

    Article  Google Scholar 

  23. T. V. Maia. Reinforcement learning, conditioning, and the brain: Successes and challenges. Cognitive, Affective, & Behavioral Neuroscience, vol. 9, no. 4, pp. 343–364, 2009. DOI: https://doi.org/10.3758/CABN.9.4.343.

    Article  Google Scholar 

  24. T. Tanaka, K. Nishida, T. Kurita. Navigation of mobile robot using location map of place cells and reinforcement learning. Systems and Computers in Japan, vol. 38, no. 7, pp. 65–75, 2007. DOI: https://doi.org/10.1002/scj.20632.

    Article  Google Scholar 

  25. F. Chersi, G. Pezzulo. Using hippocampal-striatal loops for spatial navigation and goal-directed decision-making. Cognitive processing, vol. 13, no. 1, pp. 125–129, 2012.

    Article  Google Scholar 

  26. T. Madl, K. Chen, D. Montaldi, R. Trappl. Computational cognitive models of spatial memory in navigation space: A review. Neural Networks, vol. 65, pp. 18–43, 2015. DOI: https://doi.org/10.1016/j.neunet.2015.01.002.

    Article  Google Scholar 

  27. S. Doboli, A. A. Minai, P. J. Best. A computational model of the interaction between external and internal cues for the control of hippocampal place cells. Neurocomputing, vol. 52–54, pp. 371–379, 2003. DOI: https://doi.org/10.1016/S0925-2312(02)00843-3.

    Article  Google Scholar 

  28. A. Aggarwal. The sensori-motor model of the hippocampal place cells. Neurocomputing, vol. 185, pp. 142–152, 2016. DOI: https://doi.org/10.1016/j.neucom.2015.12.044.

    Article  Google Scholar 

  29. T. Madl, S. Franklin, K. Chen, D. Montaldi, R. Trappl. Bayesian integration of information in hippocampal place cells. PLoS One, vol. 9, no. 3, Article number e89762, 2014. DOI: https://doi.org/10.1371/journal.pone.0089762.

    Google Scholar 

  30. A. Arleo, W. Gerstner. Spatial cognition and neuro-mimetic navigation: A model of hippocampal place cell activity. Biological Cybernetics, vol. 83, no. 3, pp. 287–299, 2000. DOI: https://doi.org/10.1007/s004220000171.

    Article  Google Scholar 

  31. M. G. Packard. Anxiety, cognition, and habit: A multiple memory systems perspective. Brain Research, vol. 1293, pp. 121–128, 2009. DOI: https://doi.org/10.1016/j.brainres.2009.03.029.

    Article  Google Scholar 

  32. N. Arias, M. Méndez, J. L. Arias. Brain networks underlying navigation in the Cincinnati water maze with external and internal cues. Neuroscience Letters, vol. 576, pp. 68–72, 2014. DOI: https://doi.org/10.1016/j.neulet.2014.05.064.

    Article  Google Scholar 

  33. D. Sukumar, S. Chakravarthy. A computational neuromotor model of the role of basal ganglia and hippocampus in spatial navigation. In Proceedings of the 22th International Conference on Artificial Neural Networks, Springer, Berlin, Heidelberg, Germany, pp. 216–221, 2010. DOI: https://doi.org/10.1007/978-3-642-15822-3_28.

    Google Scholar 

  34. L. F. Hu, K. R. Hao, X. Cai, L. Chen. A spatial cognitive cells inspired goal-directed navigation model. In Proceedings of IEEE International Conference on Artificial Intelligence and Computer Applications, Dalian, China, pp. 211–215, 2019. DOI: https://doi.org/10.1109/ICAICA.2019.8873496.

  35. J. O’Keefe, J. Dostrovsky. The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. Brain Research, vol. 34, no. 1, pp. 171–175, 1971. DOI: https://doi.org/10.1016/0006-8993(71)90358-1.

    Article  Google Scholar 

  36. R. G. M. Morris, P. Garrud, J. N. P. Rawlins, J. O’Keefe. Place navigation impaired in rats with hippocampal lesions. Nature, vol. 297, no. 5868, pp. 681–683, 1982. DOI: https://doi.org/10.1038/297681a0.

    Article  Google Scholar 

  37. T. Kulvicius, M. Tamosiunaite, J. Ainge, P. Dudchenko, F. Wörgötter. Odor supported place cell model and goal navigation in rodents. Journal of Computational Neuroscience, vol. 25, no. 3, pp. 481–500, 2008. DOI: https://doi.org/10.1007/s10827-008-0090-x.

    Article  Google Scholar 

  38. J. J. Knierim, H. S. Kudrimoti, B. L. McNaughton. Place cells, head direction cells, and the learning of landmark stability. Journal of Neuroscience, vol. 15, no. 3, pp. 1648–1659, 1995. DOI: https://doi.org/10.1523/JNEUROSCI.15-03-01648.1995.

    Article  Google Scholar 

  39. A. J. Hill, P. J. Best. Effects of deafness and blindness on the spatial correlates of hippocampal unit activity in the rat. Experimental Neurology, vol. 74, no. 1, pp. 204–217, 1981. DOI: https://doi.org/10.1016/0014-4886(81)90159-X.

    Article  Google Scholar 

  40. G. J. Quirk, R. U. Muller, J. L. Kubie. The firing of hippocampal place cells in the dark depends on the rat’s recent experience. Journal of Neuroscience, vol. 10, no. 6, pp. 2008–2017, 1990. DOI: https://doi.org/10.1523/JNEUROSCI.10-06-02008.1990.

    Article  Google Scholar 

  41. R. Moussa, B. Poucet, M. Amalric, F. Sargolini. Contributions of dorsal striatal subregions to spatial alternation behavior. Learning & Memory, vol. 18, no. 7, pp. 444–451, 2011. DOI: https://doi.org/10.1101/lm.2123811.

    Article  Google Scholar 

  42. C. H. Donahue, M. Liu, A. C. Kreitzer. Distinct value encoding in striatal direct and indirect pathways during adaptive learning. [Online], Available: https://www.biorxiv.org/content/biorxiv/early/2018/03/07/277855.full.pdf, May, 2020.

  43. E. L. Thorndike, Jelliffe. Animal intelligence: Experimental studies. Journal of Nervous & Mental Disease, vol. 39, no. 5, Article number 357, 1912.

    Google Scholar 

  44. J. Huang, X. G. Ruan, N. G. Yu, Q. W. Fan, J. M. Li, J. X. Cai. A cognitive model based on neuromodulated plasticity. Computational Intelligence and Neuroscience, vol. 2016, Article number 4296356, 2016. DOI: https://doi.org/10.1155/2016/4296356.

  45. D. Sheynikhovich, R. Chavarriaga, T. Strösslin, W. Gerstner. Spatial representation and navigation in a bio-inspired robot. In Proceedings of Biomimetic Neural Learning for Intelligent Robots: Intelligent Systems, Cognitive Robotics, and Neuroscience, Springer, Berlin, Heidelberg, Germany, pp. 245–264, 2005. DOI: https://doi.org/10.1007/11521082_15.

    MATH  Google Scholar 

  46. M. Y. Zhang, G. H. Tian, C. C. Li, J. Gong. Learning to transform service instructions into actions with reinforcement learning and knowledge base. International Journal of Automation and Computing, vol. 15, no. 5, pp. 582–592, 2018. DOI: https://doi.org/10.1007/s11633-018-1128-9.

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Nos. 61773027 and 62076014), National Key Research and Development Program Project (No. 2020YFB1005903), and Industrial Internet Innovation and Development Project (No. 135060009002).

Author information

Authors and Affiliations

  1. Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China

    Jing Huang, He-Yuan Yang, Xiao-Gang Ruan, Nai-Gong Yu, Guo-Yu Zuo & Hao-Meng Liu

  2. Beijing Key Laboratory of Computational Intelligence and Intelligence System, Beijing, 100124, China

    Jing Huang, He-Yuan Yang, Xiao-Gang Ruan, Nai-Gong Yu, Guo-Yu Zuo & Hao-Meng Liu

Authors
  1. Jing Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. He-Yuan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiao-Gang Ruan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nai-Gong Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guo-Yu Zuo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hao-Meng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jing Huang.

Additional information

Recommended by Associate Editor Bin Luo

Colored figures are available in the online version at https://link.springer.com/journal/11633

Jing Huang received the Ph. D. degree in pattern recognition and intelligent system from Beijing University of Technology, China in 2016. Now she is an associate professor in Faculty of Information Technology, Beijing University of Technology, China.

Her research interests include cognitive robotics, machine learning, and artificial Intelligence.

He-Yuan Yang received the B. Sc. degree in automation from North China University of Water Resources and Electric Power (NCWU), China in 2019. He is currently a master student in control science and engineering at Faculty of Information Technology of Beijing University of Technology, China.

His research interest is cognitive robotics.

Xiao-Gang Ruan received the Ph. D. degree in control science and engineering from Zhejiang University, China in 1992. Now he is a professor of Beijing University of Technology, and he is also as a director of Institute of Artificial Intelligent and Robots (IAIR).

His research interests include automatic control, artificial intelligence, and intelligent robot.

Nai-Gong Yu received the B. Eng. degree in information processing display and recognition from Harbin Institute of Technology, China in 1989, the M. Eng. degree in control science and engineering from Shanghai Jiao Tong University, China in 1996, and the Ph. D. degree in pattern recognition and intelligent systems from Beijing University of Technology, China in 2005. He worked as a visiting scholar in University of Alberta, Canada in 2011. He is currently a professor with Faculty of Information Technology, Beijing University of Technology, China.

His research interests include computational intelligence, intelligent systems and robotics.

Guo-Yu Zuo received the Ph. D. degree in cybernetics from Beijing University of Technology, China in 2005. He is currently an associate professor and head of Intelligent Robot Laboratory of Beijing University of Technology, China. He has published over 50 journal and conference articles and achieved over 20 Chinese patents in artificial intelligence and robotics. His research interests include computational intelligence, robot learning, robot control, and human-robot interaction.

Hao-Meng Liu received the B. Sc. degree in computer science and technology from Beijing University of Technology (BJUT), China in 2019. He is currently a master student in control engineering at Faculty of Information Technology of Beijing University of Technology, China.

His research interest is industrial big data.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, J., Yang, HY., Ruan, XG. et al. A Spatial Cognitive Model that Integrates the Effects of Endogenous and Exogenous Information on the Hippocampus and Striatum. Int. J. Autom. Comput. 18, 632–644 (2021). https://doi.org/10.1007/s11633-021-1286-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11633-021-1286-z

Keywords

Search

Navigation