[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

Symbols as Self-emergent Entities in an Optimization Process of Feature Extraction and Predictions

  • Prospects
  • Published:
Biological Cybernetics Aims and scope Submit manuscript

Abstract

In the mammalian cortex the early sensory processing can be characterized as feature extraction resulting in local and analogue low-level representations. As a direct consequence, these map directly to the environment, but interpretation under natural conditions is ambiguous. In contrast, high-level representations for cognitive processing, e.g. language, require symbolic representations characterized by expression and syntax. The representations are binary, structured and disambiguated. However, do these fundamental functional distinctions translate into a fundamental distinction of the respective brain areas and their anatomical and physiological properties? Here we argue that the distinction between early sensory processing and higher cognitive functions may not be based on structural differences of cortical areas; instead similar learning principles acting on input signals with different statistics give rise to the observed variations of function. Firstly, we give an account of present research describing neuronal properties at early stages of sensory systems as a consequence of an optimization process over the set of natural stimuli. Secondly, addressing a stage following early visual processing we suggest to extend the unsupervised learning scheme by including predictive processes. These contain the widely used objective of temporal coherence as a special case and are a powerful approach to resolve ambiguities. Furthermore, in combination with a prior on the bandwidth of information exchange between units it leads to a condensation of information. Thirdly, as a crucial step, not only are predictive units optimized, but the selectivity of the feature extractors are adapted to allow optimal predictability. Thus, over and beyond making useful predictions, we propose that the predictability of a stimulus be in itself a selection criterion for further processing. In a hierarchical system the combined optimization process leads to entities that represent condensed pieces of knowledge and that are not analogue anymore. Instead, these entities work as arguments in a framework of transformations that realize predictions. Thus, the criteria of predictability and condensation in an optimization of sensory representations relate directly to the two defining properties of symbols of expression and syntax. In this paper, we sketch an unsupervised learning process that gradually transforms analogue local representations into discrete binary representations by means of four hypotheses. We propose that in this optimization process acting in a hierarchical system, entities emerge at, higher levels that fulfil the criteria defining symbols, instantiating qualitatively different representations at similarly structured low and high levels.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aloimonos J, Shulman D (1989) Integration of visual modules: an extension of the marr paradigm. Academic Press, Boston

    Google Scholar 

  • Ashby FG, Maddox WT (2005) Human category learning. Annu Rev Psychol 56:149–178

    Article  PubMed  Google Scholar 

  • Barlow H, Blakemore C, Pettigrew JD (1967) The neural mechanisms of binocular depth discrimination. J Physiol (Lond) 193:327–342

    CAS  Google Scholar 

  • Barlow HB (1961) Possible principles underlying the transformation of sensory messages. In: Rosenblith WA (eds). Sensory communication, vol 1961. MIT, Cambridge, pp 217–234

    Google Scholar 

  • Barlow HB (2001) Redundancy reduction revisited. Network Comput Neural Syst 12(3):241–254

    Article  CAS  Google Scholar 

  • Berkes P, Wiskott L (2005) Slow feature analysis yields a rich repertoire of complex cell properties. J Vision 5(6):579–602

    Article  Google Scholar 

  • Berry MJ II, Brivanlou IH, Jordan TA, Meister M (1999) Anticipation of moving stimuli by the retina. Nature 398:334–338

    Article  PubMed  CAS  Google Scholar 

  • Betsch BY, Einhäuser W, Körding KP, König P (2004) The world from a cat’s perspective-statistics of natural videos. Biol Cybern 90(1): 41–50

    Article  PubMed  Google Scholar 

  • Bialek W, Nemenman I, Tishby N (2001) Predictability, complexity, and learning. Neural Comput 13(11):2409–2463

    Article  PubMed  CAS  Google Scholar 

  • Braitenberg V, Schüz A (1991) Anatomy of the cortex. Springer, Berlin Heidelberg New york

    Google Scholar 

  • Brodmann K (1906) Beiträge zur histologischen Lokalisation der Grosshirnrinde. Fünfte Mitteilung: über den allgemeinen Bauplan des Cortex pallii bei den Mammalieren und zwei homologe Rindenfelder im besonderen. Zugleich ein Beitrag zur Furchenlehre. J Psychol Neurol 6:275–400

    Google Scholar 

  • Buonomano DV, Merzenich MM (1998) Cortical plasticity: from synapses to maps. Annu Rev Neurosci 21:149–186

    Article  PubMed  CAS  Google Scholar 

  • Callaway EM (1998) Local circuits in primary visual cortex of the macaque monkey. Annu Rev Neurosci 21:47–74

    Article  PubMed  CAS  Google Scholar 

  • Chichocki A, Amari S-I (2002) Adaptive blind signal and image processing. In: Learning algorithms and applications. Wiley, New York

  • Douglas RJ, Martin KA (2004) Neuronal circuits of the neocortex. Annu Rev Neurosci 27:419–451

    Article  PubMed  CAS  Google Scholar 

  • Einhäuser W, Kayser C, Körding KP, König P (2003) Learning distinct and complementary feature selectivities from natural colour videos. Rev Neurosci 14(1–2):43–52

    PubMed  Google Scholar 

  • Elder JH, Goldberg RM (2002) Ecological statistics of Gestalt laws for the perceptual organization of contours. J Vision 2(4):324–353

    Article  Google Scholar 

  • Field D (1987) Relations between the statistics of natural images and the response properties of cortical cells. J Opt Soc of Am 4(12): 2379–2394

    Article  CAS  Google Scholar 

  • Geisler WS, Perry JS, Super BJ, Gallogly DP (2001) Edge co-occurrence in natural images predicts contour grouping performance. Vis Res 41:711–724

    Article  PubMed  CAS  Google Scholar 

  • Gibson JJ (1979) The ecological approach to visual perception. Houghton Mifflin, Boston, MA

    Google Scholar 

  • Gilbert CD, Wiesel TN (1989) Columnar specificity and intrinsic horizontal and cortico-cortical connections in cat visual cortex. J Neurosci 9:2432–2442

    PubMed  CAS  Google Scholar 

  • Grill-Spector K, Malach R (2004) The human visual cortex. Annu Rev Neurosci 27:649–677

    Article  PubMed  CAS  Google Scholar 

  • Hafner VV, Fend M, König P, Körding KP (2004) Predicting properties of the rat somatosensory system by sparse coding. Neural Inf Process 4:11–18

    Google Scholar 

  • Harnard S (1990) The symbol grounding problem. Physica D 42: 335–346

    Article  Google Scholar 

  • Hershler O, Hochstein S (2005) At first sight: a high-level pop out effect for faces. Vis Res 45:1707–1724

    Article  PubMed  Google Scholar 

  • Hilbert D (1928) Die Grundlagen der Mathematik. Abhandlungen aus dem mathematischen Seminar der Unversität Hamburg 6:65–85

    Google Scholar 

  • Hipp J, Einhäuser W, Conradt J, König P (2005) Unsupervised learning of somatosensory representations for texture discrimination using a temporal coherence principle. Network Comput Neural Syst (in press)

  • Honavar V, Uhr L (1994) Artificial intelligence and neural networks: steps toward principled integration. Academic, New York, NY

    Google Scholar 

  • Hubel DH, Wiesel TN (1959) Receptive fields of single neurones in the cat’s striate cortex. J Physiol 148:574–591

    PubMed  CAS  Google Scholar 

  • Hubel DH, Wiesel TN (1962) Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J Physiol 160: 106–154

    PubMed  CAS  Google Scholar 

  • Hurri J, Hyvärinen A (2003) Simple-cell-like receptive fields maximize temporal coherence in natural video. Neural Comput 15: 663–691

    Article  PubMed  Google Scholar 

  • Hyväarinen A, Hoyer P (2000) Emergence of phase- and shift-invariant features by decomposition of natural images into independent feature subspaces. Neural Comput 12:1705–1720

    Article  PubMed  CAS  Google Scholar 

  • Hyvarinen A, Hurri J, Vayrynen J (2003) Bubbles: a unifying framework for low-level statistical properties of natural image sequences. J Opt Soc Am A Opt Image Sci Vis 20(7):1237–1252

    Article  PubMed  Google Scholar 

  • Jones JP, Palmer LA (1987) An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. J Neurophysiol 58(6):1233–1258

    PubMed  CAS  Google Scholar 

  • Kayser C, Körding KP, König P (2004) Processing of complex stimuli and natural scenes in the visual cortex. Curr Opin Neurobiol 14: 468–473

    Article  PubMed  CAS  Google Scholar 

  • Kjaer TW, Gawne TJ, Hertz JA, Richmond BJ (1997) Insensitivity of V1 complex cell responses to small shifts in the retinal image of complex patterns. J Neurophysiol 78(6):3187–3197

    PubMed  CAS  Google Scholar 

  • Klette R, Schlüns K, Koschan A (1998) Computer vision—three-dimensional data from images. Springer, Berlin Heidelbereg New York

    Google Scholar 

  • Körding KP, Kayser C, Einhäuser W, König P (2004) How are complex cell properties adapted to the statistics of natural stimuli?. J Neurophysiol 91(1):206–212

    Article  PubMed  Google Scholar 

  • Kreiman G, Koch C, Fried I (2000) Imagery neurons in the human brain. Nature 408:357–361

    Article  PubMed  CAS  Google Scholar 

  • Krüger N (1998) Collinearity and parallelism are statistically significant second order relations of complex cell responses. Neural Process Lett 8(2):117–129

    Article  Google Scholar 

  • Krüger N, Ackermann M, Sommer G (2002) Accumulation of object representations utilizing interaction of robot action and perception. Knowl Based Syst 15:111–118

    Article  Google Scholar 

  • Krüger N, Lappe M, Wörgötter F (2004) Biologically motivated multi-modal processing of visual primitives. AISB J 1(5):417–428

    Google Scholar 

  • Krüger N, Wörgötter F (2004) Statistical and deterministic regularities: utilisation of motion and grouping in biological and artificial visual systems. Adv Imaging Electron Phys 131:82–147

    Google Scholar 

  • Krüger N, Wörgötter F (2005) Multi-modal primitives as functional models of hyper-columns and their use for contextual Integration. In: Proceedings of the 1st international symposium on brain, vision and artificial intelligence 2005, LNCS 3704. Springer, Berlin Heidelberg New York, p 157–166

  • Lettvin JY, Maturana HR, McCulloch WS, Pitts WH (1959) What the frog’s eye tells the frog’s brain. Proc IRE 47:1940–1951

    Article  Google Scholar 

  • Linsker R (1988) Self-organization in a perceptual network. Computer 21:105–117

    Article  Google Scholar 

  • Maunsell JHR, Newsome WT (1987) Visual processing in monkey extrastriate cortex. Annu Rev Neurosci 10:363–401

    Article  PubMed  CAS  Google Scholar 

  • Nakahara H, Zhang LI, Merzenich MM (2004) Specialization of primary auditory cortex processing by sound exposure in the “critical period”. Proc Natl Acad Sci USA 101:7170–7174

    Article  PubMed  CAS  Google Scholar 

  • Olshausen BA, Field DJ (1996) Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381(6583):607–609

    Article  PubMed  CAS  Google Scholar 

  • Olshausen BA, Field DJ (2004) Sparse coding of sensory inputs. Curr Opin Neurobiol 14(4):481–487

    Article  PubMed  CAS  Google Scholar 

  • Olshausen BA, Field DJ (2005) How close are we to understanding v1?. Neural Comput 17(8):1665–1699

    Article  PubMed  Google Scholar 

  • Phillips WA, Singer W (1997) In search of common foundations for cortical computation. Behav Brain Sci 20:657–683

    Article  PubMed  CAS  Google Scholar 

  • Orbach J (1998) The neuropsychological theories of lashley and Hebb. University Press of America

  • Quiroga RQ, Reddy L, Kreiman G, Koch C, Fried I (2005) Invariant visual representation by single neurons in the human brain. Nature 435(7045):1102–1107

    Article  PubMed  CAS  Google Scholar 

  • Ringach RL (2002) Spatial structure and symmetry of simple-cell receptive fields in macaque primary visual cortex. J Neurophysiol 88: 455–463

    PubMed  Google Scholar 

  • Ringach DL, Hawken MJ, Shapely R (2002) Receptive field structure of neuons in monkey visual cortex revealed by stimulation with natural image sequences. J Vision 2:12–24

    Article  Google Scholar 

  • Ringach DL (2004) Mapping receptive fields in primary visual cortex. J Physiol 558:717–728

    Article  PubMed  CAS  Google Scholar 

  • Roelfsema PR (2002) Do neurons predict the future?. Science 295(5553):227

    Article  PubMed  Google Scholar 

  • Schiller PH, Finlay BL, Volman SF (1976) Quantitative studies of single-cell properties in monkey striate cortex. III. Spatial frequency. J Neurophysiol 39:1334–1351

    PubMed  CAS  Google Scholar 

  • Schultz W, Dickinson A (2000) Neuronal coding of prediction errors. Annu Rev Neurosci 23:473–500

    Article  PubMed  CAS  Google Scholar 

  • Steels L (2003) Evolving grounded communication for robots. Trends Cog Sci 7(7):308–312

    Article  Google Scholar 

  • Tishby NZ, Pereira F, Bialek W (1999) The information bottleneck method. In: Hajek B, Sreenivas RS (eds) Proceedings of the 37th Allerton Conference on communication, control and computing, Urbana, Illinois, 1999. University of Illinois, Illinois

  • Ullman S (1979) The interpretation of Visual Motion. MIT, Cambridge, MA

    Google Scholar 

  • Vargha-Khadem F, Gadian DG, Copp A, Mishkin M (2005) FOXP2 and the neuroanatomy of speech and language. Nat Rev Neurosci 6:131–138

    Article  PubMed  CAS  Google Scholar 

  • Verschure PFMJ, Pfeifer R (1992) Categorization, representations, and the dynamics of system-environment interaction: a case study in autonomous systems. In: Meyer JA, Roitblat H, Wilson S (eds) From animals to animats: proceedings of the 2nd international conference on simulation of adaptive behavior, Honolulu, Hawaii. MIT, Cambridge, MA pp 210–217

  • Watt RJ, Phillips WA (2000) The function of dynamic grouping in vision. Trends Cog Sci 4(12):447–154

    Article  Google Scholar 

  • Wiskott L, Sejnowski TJ (2002) Slow feature analysis: unsupervised learning of invariances. Neural Comput 14(4):715–770

    Article  PubMed  Google Scholar 

  • Wörgötter F, Porr B (2005) Temporal sequence learning, prediction, and control: a review of different models and their relation to biological mechanisms. Neural Comput 17(2):245–319

    Article  PubMed  Google Scholar 

  • Zhang LI, Bao S, Merzenich MM (2001) Persistent and specific influences of early acoustic environments on primary auditory cortex. Nat Neurosci 4:1123–1130

    Article  PubMed  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Neurobiopsychology, Institute of Cognitive Science, University Osnabrück, Albrechtstr. 28, 49076, Osnabrück, Germany

    Peter König

  2. Cognitive Vision Group, Institut for Medieteknologi og Ingeniørvidenskab, Aalborg University Copenhagen, Ballerup, Denmark

    Norbert Krüger

Authors
  1. Peter König
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Norbert Krüger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter König.

Rights and permissions

Reprints and permissions

About this article

Cite this article

König, P., Krüger, N. Symbols as Self-emergent Entities in an Optimization Process of Feature Extraction and Predictions. Biol Cybern 94, 325–334 (2006). https://doi.org/10.1007/s00422-006-0050-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00422-006-0050-3

Keywords

Search

Navigation