[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to main content

Advertisement

Springer Nature Link
Log in

Modeling Axonal Plasticity in Artificial Neural Networks

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

A Correction to this article was published on 15 June 2021

This article has been updated

Abstract

Axonal growth and pruning are the brain’s primary method of controlling the structured sparsity of its neural circuits. Without long-distance axon branches connecting distal neurons, no direct communication is possible. Artificial neural networks have almost entirely ignored axonal growth and pruning, instead relying on implicit assumptions that prioritize dendritic/synaptic learning above all other concerns. This project proposes a new model called the axon game, which allows biologically-inspired axonal plasticity dynamics to be incorporated into most artificial neural network models in a computationally efficient manner. First, we demonstrate that the axon game replicates multiple previously defined pre-synaptic cortical maps. Second, we demonstrate that the axon game integrated with a synaptic learning model similar to the Laterally Interconnected Synergetically Self-Organizing Map (LISSOM), can simulate the interaction of axonal plasticity and synaptic plasticity within one model creating both pre-synaptic and post-synaptic cortical maps. Finally, it is shown that pre-synaptic and post-synaptic maps can be decoupled from one another. This decoupling depends on the relative sizes of dendritic and axonal arbors, and indicates a novel theoretical prediction about how axonal and synaptic dynamics interact.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Change history

References

  1. Yates PA, Holub AD, McLaughlin T, Sejnowski TJ, O’Leary DDM (2004) Computational modeling of retinotopic map development to define contributions of EphA-ephrinA gradients, axon-axon interactions, and patterned activity. J Neurobiol 59(1):95–113

    Article  Google Scholar 

  2. Wandell BA, Dumoulin SO, Brewer AA (2007) Visual field maps in human cortex. Neuron 56(2):366–383

    Article  Google Scholar 

  3. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  4. Sperry RW (1963) Chemoaffinity in the orderly growth of nerve fiber patterns and connections. Proc Natl Acad Sci 50(4):703–710

    Article  Google Scholar 

  5. Simon DK, O’Leary DDM (1992) Responses of retinal axons in vivo and in vitro to position-encoding molecules in the embryonic superior colliculus. Neuron 9(5):977–989

    Article  Google Scholar 

  6. Simon DK, Leary DD (1992) Development of topographic order in the mammalian retinocollicular projection. J Neurosci 12(4):1212–1232

    Article  Google Scholar 

  7. Qiu A, Rosenau BJ, Greenberg AS, Hurdal MK, Barta P, Yantis S, Miller MI (2006) Estimating linear cortical magnification in human primary visual cortex via dynamic programming. NeuroImage 31(1):125–138

    Article  Google Scholar 

  8. Qiao Q, Ma L, Li W, Tsai J-W, Yang G, Gan W-B (2016) Long-term stability of axonal boutons in the mouse barrel cortex. Dev Neurobiol 76(3):252–261

    Article  Google Scholar 

  9. Obermayer K, Blasdel GG (1993) Geometry of orientation and ocular dominance columns in monkey striate cortex. J Neurosci 13(10):4114–4129

    Article  Google Scholar 

  10. Le Vay S, Wiesel TN, Hubel DH (1980) The development of ocular dominance columns in normal and visually deprived monkeys. J Comp Neurol 191(1):1–51

    Article  Google Scholar 

  11. Issa NP, Trepel C, Stryker MP (2000) Spatial frequency maps in cat visual cortex. J Neurosci 20(22):8504–8514

    Article  Google Scholar 

  12. Innocenti GM, Price DJ (2005) Exuberance in the development of cortical networks. Nat Rev Neurosci 6(12):955–965

    Article  Google Scholar 

  13. Hubel DH, Wiesel TN (1972) Laminar and columnar distribution of geniculo-cortical fibers in the macaque monkey. J Comp Neurol 146(4):421–450

    Article  Google Scholar 

  14. Hubel DH, Wiesel TN (1962) Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J Physiol 160(1):106–154

    Article  Google Scholar 

  15. Hamasaki T, Leingärtner A, Ringstedt T, O’Leary DDM (2004) EMX2 regulates sizes and positioning of the primary sensory and motor areas in neocortex by direct specification of cortical progenitors. Neuron 43(3):359–372

    Article  Google Scholar 

  16. Goodhill GJ, Cimponeriu A (2000) Analysis of the elastic net model applied to the formation of ocular dominance and orientation columns. Net Comput Neural Syst 11(2):153–168

    Article  MATH  Google Scholar 

  17. Gierer A (1987) Directional cues for growing axons forming the retinotectal projection. Development 101(3):479–489

    Article  Google Scholar 

  18. Gebhardt C, Bastmeyer M, Weth F (2012) Balancing of ephrin/Eph forward and reverse signaling as the driving force of adaptive topographic mapping. Development 139(2):335–345

    Article  Google Scholar 

  19. Fukushima K (1980) Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybern 36(4):193–202

    Article  MATH  Google Scholar 

  20. Fukuchi-Shimogori T, Grove EA (2001) Neocortex patterning by the secreted signaling molecule FGF8. Science 294(5544):1071–1074

    Article  Google Scholar 

  21. Elston GN, Rosa MG (1997) The occipitoparietal pathway of the macaque monkey: comparison of pyramidal cell morphology in layer III of functionally related cortical visual areas. Cereb Cortex 7(5):432–452

    Article  Google Scholar 

  22. Durbin R, Mitchison G (1990) A dimension reduction framework for understanding cortical maps. Nature 343(6259):644–647

    Article  Google Scholar 

  23. De Paola V, Holtmaat A, Knott G, Song S, Wilbrecht L, Caroni P, Svoboda K (2006) Cell type-specific structural plasticity of axonal branches and boutons in the adult neocortex. Neuron 49(6):861–875

    Article  Google Scholar 

  24. Chklovskii DB (2000) Optimal sizes of dendritic and axonal arbors in a topographic projection. J Neurophysiol 83(4):2113–2119

    Article  Google Scholar 

  25. Burt P, Adelson E (1983) The Laplacian Pyramid as a compact image code. IEEE Trans Commun 31(4):532–540

    Article  Google Scholar 

  26. Bonhoeffer T, Grinvald A (1991) Iso-orientation domains in cat visual cortex are arranged in pinwheel-like patterns. Nature 353(6343):429–431

    Article  Google Scholar 

  27. Bishop KM, Goudreau G, O’Leary DDM (2000) Regulation of area identity in the mammalian neocortex by Emx2 and Pax6. Science 288(5464):344–349

    Article  Google Scholar 

  28. Bishop KM, Garel S, Nakagawa Y, Rubenstein JLR, O’Leary DDM (2003) Emx1 and Emx2 cooperate to regulate cortical size, lamination, neuronal differentiation, development of cortical efferents, and thalamocortical pathfinding. J Comp Neurol 457(4):345–360

    Article  Google Scholar 

  29. Bishop KM, Rubenstein JLR, O’Leary DDM (2002) Distinct actions of Emx1, Emx2, andPax6 in regulating the specification of areas in the developing neocortex. J Neurosci 22(17):7627–7638

    Article  Google Scholar 

  30. Yamins DL, DiCarlo JJ (2016) Using goal-driven deep learning models to understand sensory cortex. Nat Neurosci 19:356–365

    Article  Google Scholar 

  31. von der Malsburg C (1973) Self-organization of orientation sensitive cells in the striate cortex. Kybernetik 14(2):85–100

    Article  Google Scholar 

  32. Stevens JLR, Law JS, Antolík J, Bednar JA (2013) Mechanisms for stable, robust, and adaptive development of orientation maps in the primary visual cortex. J Neurosci 33(40):15747–15766

    Article  Google Scholar 

  33. Sirosh J, Miikkulainen R (1994) Cooperative self-organization of afferent and lateral connections in cortical maps. Biol Cybern 71(1):65–78

    Article  MATH  Google Scholar 

  34. Simpson HD, Goodhill GJ (2011) A simple model can unify a broad range of phenomena in retinotectal map development. Biol Cybern 104(1):9–29

    Article  MathSciNet  MATH  Google Scholar 

  35. Simon DK, O’Leary DDM (1992) Influence of position along the medial-lateral axis of the superior colliculus on the topographic targeting and survival of retinal axons. Dev Brain Res 69(2):167–172

    Article  Google Scholar 

  36. Ribot J, Aushana Y, Bui-Quoc E, Milleret C (2013) Organization and origin of spatial frequency maps in cat visual cortex. J Neurosci 33(33):13326–13343

    Article  Google Scholar 

  37. Quinlan PT (1998) Structural change and development in real and artificial neural networks. Neural Net 11(4):577–599

    Article  Google Scholar 

  38. Price DJ, Kennedy H, Dehay C, Zhou L, Mercier M, Jossin Y, Goffinet AM, Tissir F, Blakey D, Molnár Z (2006) The development of cortical connections. Eur J Neurosci 23(4):910–920

    Article  Google Scholar 

  39. McLaughlin T, O’Leary DDM (2005) Molecular gradients and development of retinotopic maps. Annu Rev Neurosci 28(1):327–355

    Article  Google Scholar 

  40. Marik SA, Yamahachi H, McManus JNJ, Szabo G, Gilbert CD (2010) Axonal dynamics of excitatory and inhibitory neurons in somatosensory cortex. PLoS Biol 8(6):1–16

    Article  Google Scholar 

  41. Levy M, Lu Z, Dion G, Kara P (2014) The shape of dendritic arbors in different functional domains of the cortical orientation map. J Neurosci 34(9):3231–3236

    Article  Google Scholar 

  42. LeVay S, Hubel DH, Wiesel TN (1975) The pattern of ocular dominance columns in macaque visual cortex revealed by a reduced silver stain. J Comp Neurol 159(4):559–575

    Article  Google Scholar 

  43. A Krizhevsky, I Sutskever and GE Hinton, 2012 ImageNet classification with deep convolutional neural networks, in Advances in Neural Information Processing Systems

  44. Issa NP, Rosenberg A, Husson TR (2008) Models and Measurements of Functional Maps in V1. J Neurophysiol 99(6):2745–2754

    Article  Google Scholar 

  45. Humphrey AL, Sur M, Uhlrich DJ, Sherman SM (1985) Projection patterns of individual X- and Y-cell axons from the lateral geniculate nucleus to cortical area 17 in the cat. J Comp Neurol 233(2):159–189

    Article  Google Scholar 

  46. Hubel DH, Wiesel TN, Stryker MP (1978) Anatomical demonstration of orientation columns in macaque monkey. J Comp Neurol 177(3):361–379

    Article  Google Scholar 

  47. Hubel DH, Wiesel TN (1965) Receptive fields and function architecture in two nonstriate visual ares (18 and 19) or the cat. J Neurophysiol 28(2):229–289

    Article  Google Scholar 

  48. Horton JC, Hocking DR (1996) Intrinsic variability of ocular dominance column periodicity in normal macaque monkeys. J Neurosci 16(22):7228–7339

    Article  Google Scholar 

  49. Godfrey KB, Eglen SJ, Swindale NV (2009) A multi-component model of the developing retinocollicular pathway incorporating axonal and synaptic growth. PLoS Comput Biol 5(12):1–22

    Article  MathSciNet  Google Scholar 

  50. A. Gierer and W. Lewis, 1983 Model for the retino-tectal projection, Proceedings of the Royal Society of London. Series B. Biological Sciences, 218, 1210, 77–93

  51. Fukuchi-Shimogori T, Grove EA (2003) Emx2 patterns the neocortex by regulating FGF positional signaling. Nat Neurosci 6(8):825–831

    Article  Google Scholar 

  52. Fraser SE, Perkel DH (1990) Competitive and positional cues in the patterning of nerve connections. J Neurobiol 21(1):51–72

    Article  Google Scholar 

  53. Engel SA, Glover GH, Wandell BA (1997) Retinotopic organization in human visual cortex and the spatial precision of functional MRI. Cereb Cortex 7(2):181–192

    Article  Google Scholar 

  54. Dougherty RF, Koch VM, Brewer AA, Fischer B, Modersitzki J, Wandell BA (2003) Visual field representations and locations of visual areas v1/2/3 in human visual cortex. J Vis 3(10):586–598

    Article  Google Scholar 

  55. Chapman B, Jacobson MD, Reiter HO, Stryker MP (1986) Ocular dominance shift in kitten visual cortex caused by imbalance in retinal electrical activity. Nature 324(6093):154–156

    Article  Google Scholar 

  56. Blasdel GG (1992) Orientation selectivity, preference, and continuity in monkey striate cortex. J Neurosci 12(8):3139–3161

    Article  Google Scholar 

  57. Crair MC, Gillespie DC, Stryker MP (1998) The role of visual experience in the development of columns in cat visual cortex. Science 279(5350):566–570

    Article  Google Scholar 

  58. Crowley JC, Katz LC (2000) Early development of ocular dominance columns. Science 290(5495):1321–1324

    Article  Google Scholar 

  59. Benson DL, Colman DR, Huntley GW (2001) Molecules, maps and synapse specificity. Nat Rev Neurosci 2(12):899–909

    Article  Google Scholar 

  60. Bosking WH, Zhang Y, Schofield B, Fitzpatrick D (1997) Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex. J Neurosci 17(6):2112–2127

    Article  Google Scholar 

  61. Miikkulainen R, Bednar JA, Choe Y, Sirosh J (2005) Computational maps in visual cortex. Springer, New York

    Google Scholar 

  62. Ruthazer ES, Akerman CJ, Cline HT (2003) Control of axon branch dynamics by correlated activity in vivo. Science 301(5629):66–70

    Article  Google Scholar 

  63. Portera-Cailliau C, Weimer RM, De Paola V, Caroni P, Svoboda K (2005) Diverse modes of axon elaboration in the developing neocortex. PLoS Biol 3(8):8

    Article  Google Scholar 

  64. Meyer MP, Smith SJ (2006) Evidence from in vivo imaging that synaptogenesis guides the growth and branching of axonal arbors by two distinct mechanisms. J Neurosci 26(13):3604–3614

    Article  Google Scholar 

  65. Gogolla N, Galimberti I, Caroni P (2007) Structural plasticity of axon terminals in the adult. Curr Opin Neurobiol 17:516–524

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Behavioral and Brain Sciences, University of Texas at Dallas, Richardson, TX, USA

    James Ryland

Authors
  1. James Ryland
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to James Ryland.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix 1

Appendix 1

1.1 Simulation Settings for Sect. 6

This section contains the parameter settings for the axon game used in the section titled “Simulated Axonal Maps”. The version of the axon game implemented for this paper uses a simple auto-scale feature whereby some of the input parameters are adjusted to produce results that can be compared to a standard 100 × 100 scale simulation. The \({\alpha }_{APDV}\) is multiplied by a factor of the largest simulation dimension divided by 100 when an axis of the simulation is larger than 100. Additionally, the starting exuberance \(E{X}_{0}\) is also scaled by the same factor.

Symbol

Value

Res

170 × 250

Co-Act

Covariance

\(Seed\)

True

\({\sigma }_{seed}\)

0

\({\sigma }_{y}^{diff}\)

10

\({\sigma }_{\eta }^{diff}\)

1

\({\alpha }_{N}\)

6

\({\alpha }_{S}^{0}\)

0.002

\({\alpha }_{S}^{1}\)

0.04

\({\alpha }_{R}\)

0.08

\({\alpha }_{C}\)

50

\({\alpha }_{global}\)

0.3

\({\alpha }_{local}\)

0.08

\({\beta }_{s}\)

0.1

\(B\)

8

\(k\)

200

\({t}_{end}\)

100

\(E{X}_{0}\)

1.5

\(E{X}_{k}\)

0.025

\({P}_{0}^{grow}\)

1

\({P}_{1}^{grow}\)

1

1.2 Simulation settings for Sect. 7

This section contains the simulation parameters used for the second set of demonstrations that combined the Axon game with H-LISSOM Tables 3 and 4

Table 3 Axon game
Full size table
Table 4 H-LISSOM
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ryland, J. Modeling Axonal Plasticity in Artificial Neural Networks. Neural Process Lett 53, 1119–1146 (2021). https://doi.org/10.1007/s11063-021-10433-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-021-10433-w

Keywords

Search

Navigation