[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Neural correlates of multisensory cue integration in macaque MSTd

Nature Neuroscience volume 11, pages 1201–1210 (2008)Cite this article

Abstract

Human observers combine multiple sensory cues synergistically to achieve greater perceptual sensitivity, but little is known about the underlying neuronal mechanisms. We recorded the activity of neurons in the dorsal medial superior temporal (MSTd) area during a task in which trained monkeys combined visual and vestibular cues near-optimally to discriminate heading. During bimodal stimulation, MSTd neurons combined visual and vestibular inputs linearly with subadditive weights. Neurons with congruent heading preferences for visual and vestibular stimuli showed improvements in sensitivity that parallel behavioral effects. In contrast, neurons with opposite preferences showed diminished sensitivity under cue combination. Responses of congruent cells were more strongly correlated with monkeys' perceptual decisions than were responses of opposite cells, suggesting that the monkey monitored the activity of congruent cells to a greater extent during cue integration. These findings show that perceptual cue integration occurs in nonhuman primates and identify a population of neurons that may form its neural basis.

Access through your institution
Buy or subscribe

This is a preview of subscription content, access via your institution

Access options

Access through your institution

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Heading task and behavioral performance.
Figure 2: Examples of neuronal tuning and neurometric functions for one congruent cell and one opposite cell.
Figure 3: Neuronal sensitivity under cue combination depends on visual and vestibular congruency.
Figure 4: Effect of cue integration on tuning curve slopes and Fano factors.
Figure 5: Combined-condition responses are well approximated by linear weighted summation.
Figure 6: Correlations between MSTd responses and perceptual decisions depend on congruency of tuning.
Figure 7: Summary of effects of congruency on choice probability values across stimulus conditions.
Figure 8: Temporal evolution of population responses, neuronal thresholds and choice probabilities.

Similar content being viewed by others

References

  1. Jacobs, R.A. Optimal integration of texture and motion cues to depth. Vision Res. 39, 3621–3629 (1999).

    Article  CAS  PubMed  Google Scholar 

  2. Hillis, J.M., Watt, S.J., Landy, M.S. & Banks, M.S. Slant from texture and disparity cues: optimal cue combination. J. Vis. 4, 967–992 (2004).

    PubMed  Google Scholar 

  3. Knill, D.C. & Saunders, J.A. Do humans optimally integrate stereo and texture information for judgments of surface slant? Vision Res. 43, 2539–2558 (2003).

    Article  PubMed  Google Scholar 

  4. van Beers, R.J., Sittig, A.C. & Gon, J.J. Integration of proprioceptive and visual position-information: an experimentally supported model. J. Neurophysiol. 81, 1355–1364 (1999).

    Article  CAS  PubMed  Google Scholar 

  5. Ernst, M.O. & Banks, M.S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429–433 (2002).

    Article  CAS  PubMed  Google Scholar 

  6. Alais, D. & Burr, D. The ventriloquist effect results from near-optimal bimodal integration. Curr. Biol. 14, 257–262 (2004).

    Article  CAS  PubMed  Google Scholar 

  7. Molholm, S., Ritter, W., Javitt, D.C. & Foxe, J.J. Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study. Cereb. Cortex 14, 452–465 (2004).

    Article  PubMed  Google Scholar 

  8. Yuille, A.L. & Bulthoff, H.H. Bayesian decision theory and psychophysics. in Perception as Bayesian Inference (eds., Knill, D. & Richards, W.) 123–161 (Cambridge University Press, Cambridge, 1996).

  9. Mamassian, P., Landy, M.S. & Maloney, L.T. Bayesian modeling of visual perception. in Probabilistic Models of the Brain: Perception and Neural Function (eds., Rao, R.P.N., Olshausen, B.A. & Lewicki, M.S.) 13–36 (MIT Press, Cambridge, Massachusetts, 2002).

    Google Scholar 

  10. Knill, D.C. & Pouget, A. The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci. 27, 712–719 (2004).

    Article  CAS  PubMed  Google Scholar 

  11. Stein, B.E. & Meredith, M.A. The Merging of the Senses (MIT Press, Cambridge, Massachusetts, 1993).

    Google Scholar 

  12. Felleman, D.J. & Van Essen, D.C. Distributed hierarchical processing in the primate cerebral cortex. Cereb. Cortex 1, 1–47 (1991).

    Article  CAS  PubMed  Google Scholar 

  13. Ghazanfar, A.A. & Schroeder, C.E. Is neocortex essentially multisensory? Trends Cogn. Sci. 10, 278–285 (2006).

    Article  PubMed  Google Scholar 

  14. Wallace, M.T., Ramachandran, R. & Stein, B.E. A revised view of sensory cortical parcellation. Proc. Natl. Acad. Sci. USA 101, 2167–2172 (2004).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Stanford, T.R., Quessy, S. & Stein, B.E. Evaluating the operations underlying multisensory integration in the cat superior colliculus. J. Neurosci. 25, 6499–6508 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Sugihara, T., Diltz, M.D., Averbeck, B.B. & Romanski, L.M. Integration of auditory and visual communication information in the primate ventrolateral prefrontal cortex. J. Neurosci. 26, 11138–11147 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Avillac, M., Ben Hamed, S. & Duhamel, J.R. Multisensory integration in the ventral intraparietal area of the macaque monkey. J. Neurosci. 27, 1922–1932 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Wuerger, S.M., Hofbauer, M. & Meyer, G.F. The integration of auditory and visual motion signals at threshold. Percept. Psychophys. 65, 1188–1196 (2003).

    Article  CAS  PubMed  Google Scholar 

  19. Tanaka, K. et al. Analysis of local and wide-field movements in the superior temporal visual areas of the macaque monkey. J. Neurosci. 6, 134–144 (1986).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Tanaka, K., Fukada, Y. & Saito, H.A. Underlying mechanisms of the response specificity of expansion/contraction and rotation cells in the dorsal part of the medial superior temporal area of the macaque monkey. J. Neurophysiol. 62, 642–656 (1989).

    Article  CAS  PubMed  Google Scholar 

  21. Duffy, C.J. & Wurtz, R.H. Sensitivity of MST neurons to optic flow stimuli. I. A continuum of response selectivity to large-field stimuli. J. Neurophysiol. 65, 1329–1345 (1991).

    Article  CAS  PubMed  Google Scholar 

  22. Duffy, C.J. & Wurtz, R.H. Response of monkey MST neurons to optic flow stimuli with shifted centers of motion. J. Neurosci. 15, 5192–5208 (1995).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Duffy, C.J. MST neurons respond to optic flow and translational movement. J. Neurophysiol. 80, 1816–1827 (1998).

    Article  CAS  PubMed  Google Scholar 

  24. Page, W.K. & Duffy, C.J. Heading representation in MST: sensory interactions and population encoding. J. Neurophysiol. 89, 1994–2013 (2003).

    Article  PubMed  Google Scholar 

  25. Gu, Y., Watkins, P.V., Angelaki, D.E. & DeAngelis, G.C. Visual and nonvisual contributions to three-dimensional heading selectivity in the medial superior temporal area. J. Neurosci. 26, 73–85 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Takahashi, K. et al. Multimodal coding of three-dimensional rotation and translation in area MSTd: comparison of visual and vestibular selectivity. J. Neurosci. 27, 9742–9756 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Bremmer, F., Kubischik, M., Pekel, M., Lappe, M. & Hoffmann, K.P. Linear vestibular self-motion signals in monkey medial superior temporal area. Ann. NY Acad. Sci. 871, 272–281 (1999).

    Article  CAS  PubMed  Google Scholar 

  28. Ma, W.J., Beck, J.M., Latham, P.E. & Pouget, A. Bayesian inference with probabilistic population codes. Nat. Neurosci. 9, 1432–1438 (2006).

    Article  CAS  PubMed  Google Scholar 

  29. Gu, Y., Deangelis, G.C. & Angelaki, D.E. A functional link between area MSTd and heading perception based on vestibular signals. Nat. Neurosci. 10, 1038–1047 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Wichmann, F.A. & Hill, N.J. The psychometric function. I. Fitting, sampling and goodness of fit. Percept. Psychophys. 63, 1293–1313 (2001).

    Article  CAS  PubMed  Google Scholar 

  31. Britten, K.H., Shadlen, M.N., Newsome, W.T. & Movshon, J.A. The analysis of visual motion: a comparison of neuronal and psychophysical performance. J. Neurosci. 12, 4745–4765 (1992).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Green, D.M. & Swets, J.A. Signal Detection Theory and Psychophysics (Wiley, New York, 1966).

    Google Scholar 

  33. Parker, A.J. & Newsome, W.T. Sense and the single neuron: probing the physiology of perception. Annu. Rev. Neurosci. 21, 227–277 (1998).

    Article  CAS  PubMed  Google Scholar 

  34. Britten, K.H., Newsome, W.T., Shadlen, M.N., Celebrini, S. & Movshon, J.A. A relationship between behavioral choice and the visual responses of neurons in macaque MT. Vis. Neurosci. 13, 87–100 (1996).

    Article  CAS  PubMed  Google Scholar 

  35. Krug, K. A common neuronal code for perceptual processes in visual cortex? Comparing choice and attentional correlates in V5/MT. Phil. Trans. R. Soc. Lond. B 359, 929–941 (2004).

    Article  Google Scholar 

  36. Purushothaman, G. & Bradley, D.C. Neural population code for fine perceptual decisions in area MT. Nat. Neurosci. 8, 99–106 (2005).

    Article  CAS  PubMed  Google Scholar 

  37. Shadlen, M.N., Britten, K.H., Newsome, W.T. & Movshon, J.A. A computational analysis of the relationship between neuronal and behavioral responses to visual motion. J. Neurosci. 16, 1486–1510 (1996).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Nienborg, H. & Cumming, B.G. Psychophysically measured task strategy for disparity discrimination is reflected in V2 neurons. Nat. Neurosci. 10, 1608–1614 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Nienborg, H. & Cumming, B.G. Macaque V2 neurons, but not V1 neurons, show choice-related activity. J. Neurosci. 26, 9567–9578 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Dodd, J.V., Krug, K., Cumming, B.G. & Parker, A.J. Perceptually bistable three-dimensional figures evoke high choice probabilities in cortical area MT. J. Neurosci. 21, 4809–4821 (2001).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. Celebrini, S. & Newsome, W.T. Neuronal and psychophysical sensitivity to motion signals in extrastriate area MST of the macaque monkey. J. Neurosci. 14, 4109–4124 (1994).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Uka, T. & DeAngelis, G.C. Contribution of area MT to stereoscopic depth perception: choice-related response modulations reflect task strategy. Neuron 42, 297–310 (2004).

    Article  CAS  PubMed  Google Scholar 

  43. Parker, A.J., Krug, K. & Cumming, B.G. Neuronal activity and its links with the perception of multi-stable figures. Phil. Trans. R. Soc. Lond. B 357, 1053–1062 (2002).

    Article  Google Scholar 

  44. Tanaka, K., Sugita, Y., Moriya, M. & Saito, H. Analysis of object motion in the ventral part of the medial superior temporal area of the macaque visual cortex. J. Neurophysiol. 69, 128–142 (1993).

    Article  CAS  PubMed  Google Scholar 

  45. Van Essen, D.C., Maunsell, J.H. & Bixby, J.L. The middle temporal visual area in the macaque: myeloarchitecture, connections, functional properties and topographic organization. J. Comp. Neurol. 199, 293–326 (1981).

    Article  CAS  PubMed  Google Scholar 

  46. Komatsu, H. & Wurtz, R.H. Relation of cortical areas MT and MST to pursuit eye movements. III. Interaction with full-field visual stimulation. J. Neurophysiol. 60, 621–644 (1988).

    Article  CAS  PubMed  Google Scholar 

  47. Komatsu, H. & Wurtz, R.H. Relation of cortical areas MT and MST to pursuit eye movements. I. Localization and visual properties of neurons. J. Neurophysiol. 60, 580–603 (1988).

    Article  CAS  PubMed  Google Scholar 

  48. Desimone, R. & Ungerleider, L.G. Multiple visual areas in the caudal superior temporal sulcus of the macaque. J. Comp. Neurol. 248, 164–189 (1986).

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We thank A. Turner and E. White for monkey care and training, M.S. Banks for helpful advice, and L. Snyder, A. Pouget and R. Moreno Bote for comments on an earlier version of the manuscript. This work was supported by US National Institutes of Health grants EY017866 (to D.E.A.) and EY016178 and an EJLB Foundation grant (to G.C.D.).

Author information

Author notes

  1. Dora E Angelaki and Gregory C DeAngelis: These authors contributed equally to this work.

Authors and Affiliations

  1. Department of Anatomy and Neurobiology, Washington University School of Medicine, 660 South Euclid Avenue, St. Louis, 63110, Missouri, USA

    Yong Gu, Dora E Angelaki & Gregory C DeAngelis

  2. Department of Brain and Cognitive Sciences, Center for Visual Science, University of Rochester, 245 Meliora Hall, Rochester, 14627, New York, USA

    Gregory C DeAngelis

Authors
  1. Yong Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dora E Angelaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gregory C DeAngelis
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Y.G., D.E.A. and G.C.D. designed the experiments. Y.G. collected the data and performed data analyses. Y.G., D.E.A. and G.C.D. refined the analyses and wrote the paper.

Corresponding author

Correspondence to Gregory C DeAngelis.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–12 and Supplementary Methods (PDF 4045 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gu, Y., Angelaki, D. & DeAngelis, G. Neural correlates of multisensory cue integration in macaque MSTd. Nat Neurosci 11, 1201–1210 (2008). https://doi.org/10.1038/nn.2191

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.2191

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing