[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to main content

The Corpus of Interactional Data: A Large Multimodal Annotated Resource

  • Chapter
  • First Online:
Handbook of Linguistic Annotation

Abstract

The availability of annotated datasets had been steadily growing for written language and benefited to linguistic studies and natural language processing. The situation for face-to-face spontaneous conversation is more contrasted for several reasons: technicalities in handling raw data (split across several sources and medias), need for a often difficult and time-consuming transcription, large variety of annotation that can be performed. We propose in this chapter a complete annotation workflow, starting from raw data (speech and video) to a richly annotated dataset with many linguistic information (morpho-syntax, prosody, gesture studies, discourse analysis). Our approach consisted in gathering experts from the different domains and work together on the establishment of an abstract schema encoded with types feature structures. We detail how the annotation workflow had been used for developing of a richly annotated version of the Corpus of Interactional Data. The corpus as well as the annotation described here are available through the Speech and Language Data Repository.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
GBP 19.95
Price includes VAT (United Kingdom)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
GBP 279.50
Price includes VAT (United Kingdom)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
GBP 349.99
Price includes VAT (United Kingdom)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
GBP 349.99
Price includes VAT (United Kingdom)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    This is a French-speaking initiative, the partners of the project are ATILF, ICAR, LI, LIMSI, LLL, LPL, SYLED, VALIBEL.

  2. 2.

    A third level of phrasing (the intermediate unit) was proposed but its definition is not yet well established [33] and still requires to be refined more specifically for spontaneous French.

References

  1. Allwood, J., Cerrato, L.: A study of gestural feedback expressions. In: First Nordic Symposium on Multimodal Communication, pp. 7–22 (2003)

    Google Scholar 

  2. Allwood, J., Cerrato, L., Dybkjaer, L., Jokinen, K., Navaretta, C., Paggio, P.: The MUMIN Multimodal Coding Scheme, pp. 129–157. NorFA yearbook 2005 (2005)

    Google Scholar 

  3. Anderson, A.H., Bader, M., Gurman Bard, E., Boyle, E., Doherty, G., Garrod, S., Isard, S., Kowtko, J., McAllister, J., Miller, J., Sotillo, C., Thompson, H.S., Weinert, R.: The hcrc map task corpus. Lang. Speech 34, 351–366 (1991)

    Google Scholar 

  4. Bertrand, R., Portes, C., Sabio, F.: Distribution syntaxique, discursive et interactionnelle des contours intonatifs du français dans un corpus de conversation. Travaux neuchâtelois de linguistique, p. 47 (2007)

    Google Scholar 

  5. Blache, P., Rauzy, S.: Influence de la qualité de l’étiquetage sur le chunking : une corrélation dépendant de la taille des chunks. In: Actes de Traitement Automatique des Langues Naturelles, pp. 290–299. Avignon, France (2008)

    Google Scholar 

  6. Boersma, P., Weenink, D.: Praat, a system for doing phonetics by computer, version 3.4. Technical Report 132, Institute of Phonetic Sciences of the University of Amsterdam (1996)

    Google Scholar 

  7. Bull, P.E.: Posture and gesture. In: International Series in Experimental Social Psychology, vol. 16 (1987)

    Google Scholar 

  8. Carletta, J.: Announcing the ami meeting corpus. The ELRA Newsletter 11(1), (2006)

    Google Scholar 

  9. Carletta, J., Isard, A.: The mate annotation workbench: user requirements. In: Proceedings of the ACL Workshop: Towards Standards and Tools for Discourse Tagging (1999)

    Google Scholar 

  10. Carletta, J., Evert, S., Heid, U., Kilgour, J., Robertson, J., Voormann, H.: The nite xml toolkit: flexible annotation for multi-modal language data. Behav. Res. Methods Instrum. Comput. 35(3), (2003)

    Google Scholar 

  11. Carletta, J.C.: Assessing agreement on classification tasks: the kappa statistic. Comput. Linguist. 22(2), 249–254 (1996)

    Google Scholar 

  12. Carpenter, B.: The Logic of Typed Feature Structures. Cambridge University Press, Cambridge (1992)

    Google Scholar 

  13. Caspers, J.: Local speech melody as a limiting factor in the turn-taking system in dutch. J. Phon. 31, 251–276 (2003)

    Article  Google Scholar 

  14. Clark, H.H., Wasow, T.: Repeating words in spontaneous speech. Cogn. Psychol. 37, 201–242 (1998)

    Google Scholar 

  15. Di Cristo, A.: Vers une modélisation de l’accentuation en français. deuxième partie : le modèle. J. Fr. Lang. Stud. 10, 27–44 (2000)

    Article  Google Scholar 

  16. Dister, A.: La notation subjective de la pause constitue-t-elle un bon indice pour le découpage de corpus oraux ?. In: Constant, M., Dister, A., Emirkanian, L., Piron, S. (eds.) Description linguistique pour le traitement automatique du français, Cahiers du Cental 5, pp. 165–186. Louvain-la-Neuve, Presses universitaires de Louvain (2008)

    Google Scholar 

  17. Fohr, D., Mella, O., Cerisara, C., Illina, I.: The automatic news transcription system: ants, some real time experiments. In: INTERSPEECH-2004, pp. 377–380 (2004)

    Google Scholar 

  18. Fox Tree, J.E.: Listening in on monologues and dialogues. Discourse Process. 27(1), 35–53 (1999)

    Article  Google Scholar 

  19. Gendner, V., Illouz, G., Jardino, M., Monceaux, L., Paroubek, P., Robba, I., Vilnat, A.: PEAS, the first instantiation of a comparative framework for evaluating parsers of french. In: Research Notes of EACL 2003, Budapest, Hongrie (2003)

    Google Scholar 

  20. Guénot, M.L.: Parsing de loral : traiter les disfluences. In: Traitement Automatique des Langues Naturelles, TALN, 6-10 June 2005. Dourdan, France (2005)

    Google Scholar 

  21. Henry, S., Pallaud, B.: Word fragments and repeats in spontaneous spoken French. In: Eklund, R. (ed.) Disfluency in Spontaneous Speech Workshop, Proceedings of DiSS03, 5-8 September 2003, pp. 77–80. Göteborg University, Sweden (2003)

    Google Scholar 

  22. Hirst, D.: A praat plugin for momel and intsint with improved algorithms for modelling and coding intonation. In: ICPhS XVI (2007)

    Google Scholar 

  23. Hirst, D., Di Cristo, A., Espesser, R.: Levels of representation and levels of analysis for intonation. In: Horne, M. (ed.) Prosody: Theory and Experiment, pp. 51–87. Kluwer Academic Publishers (2000)

    Google Scholar 

  24. Ide, N., Bonhomme, P., Romary, L.: Xces: an xml-based standard for linguistic corpora. In: Proceedings of the Second Language Resources and Evaluation Conference, pp. 825–830 (2000)

    Google Scholar 

  25. Jun, S.A., Fougeron, C.: A phonological model of french intonation. In: Intonation: Analysis, modelling and technology, pp. 209–242. Dordrecht (2000)

    Google Scholar 

  26. Kendon, A.: Some functions of gaze direction in social interaction. Acta Psychol. 26, 22–63 (1967)

    Article  Google Scholar 

  27. Kipp, M.: Anvil - a generic annotation tool for multimodal dialogue. In: Proceedings of the 7th European Conference on Speech Communication and Technology (Eurospeech), pp. 1367–1370 (2001)

    Google Scholar 

  28. Kipp, M., Neff, M., Albrecht, I.: An annotation scheme for conversational gestures: how to economically capture timing and form. In: Proceedings of the Workshop on “Multimodal Corpora” at LREC 2007, pp. 325–339 (2007)

    Google Scholar 

  29. Koiso, H., Horiuchi, Y., Ichikawa, A., Den, Y.: An analysis of turn-taking and backchannels based on prosodic and syntactic features in japanese map task dialogs. Language and Speech, 41 (1998)

    Google Scholar 

  30. Kruijff-Korbayova, I., Gerstenberger, C., Rieser, V., Schehl, J.: The sammie multimodal dialogue corpus meets the nite xml toolkit. In: proceedings of LREC06 (2006)

    Google Scholar 

  31. McNeill, D.: Hand and Mind. What Gestures Reveal about Thought. University of Chicago Press (1992)

    Google Scholar 

  32. McNeill, D.: Gesture and Thought. University of Chicago Press (2005)

    Google Scholar 

  33. Michelas, A.: Caractérisation phonétique et phonologique du syntagme intermédiaire en français: de la production à la perception. Ph.D. thesis, Aix-Marseille Université (2011)

    Google Scholar 

  34. Muller, P., Vergez, M., Prévot, L., Asher, N., Benamara, F., Bras, M., Le Draoulec, A., Vieu, L.: Manuel d’annotation en relations de discours du projet annodis. Technical report, CLLE (2012)

    Google Scholar 

  35. Pallaud, B.: Troncations de mots, reprises et interruption syntaxique en français parlé spontané. In: JADT, 8èmes Journées internationales d’Analyse statistique des Données Textuelles, 20–22 avril 2006, pp. 707–715. Besançon (2006)

    Google Scholar 

  36. Pallaud, B., Henry, S.: Amorces de mots et répétitions : des hésitations plus que des erreurs en français parlé. In: Le poids des mots. Actes des 7èmes Journées Internationales d’Analyse statistique des Données Textuelles, Louvain-la-Neuve, 10-12 mars 2004. vol. 2, pp. 848–858. Louvain, PUL(2004)

    Google Scholar 

  37. Paroubek, P., Robba, I., Vilnat, A., Ayache, C.: Data annotations and measures in EASY the evaluation campaign for parsers in french. In: Proceedings of the 5th International Conference on Language Resources and Evaluation, pp. 314–320. Genoa, Italy (2006)

    Google Scholar 

  38. Pineda, L.A., Massé, A., Meza, I., Salas, M., Schwarz, E., Uraga, E., Villaseñor, L.: The dime project. In: Proceedings of MICAI2002, vol. 2313. LNAI (2002)

    Google Scholar 

  39. Portes, C., Bertrand, R.: Some cues about the interactional value of the \(\ll \)continuation\(\gg \) contour in french. In: Discours et Prosodie comme Interface Complexe (2006)

    Google Scholar 

  40. Post, B.: Tonal and Phrasal Structures in French Intonation. Thesus (2000)

    Google Scholar 

  41. Rodriguez, K., Dipper, S., Götze, M., Poesio, M., Riccardi, G., Raymond, C., Rabiega-Wisniewska, J.: Standoff coordination for multi-tool annotation in a dialogue corpus. In: Proceedings of Linguistic Annotation Workshop (2007)

    Google Scholar 

  42. Sacks, H., Schegloff, E.A., Jefferson, G.: A simplest systematics for the organization of turn-taking for conversation. Language 50, 696–735 (1974)

    Article  Google Scholar 

  43. Schegloff, E.: Discourse as an interactional achievement: some uses of “uh huh” and other things that come between sentences. In: Tannen, D. (ed.) Analyzing discourse: Text and talk. Georgetown University Press (1982)

    Google Scholar 

  44. Schmidt, T., Duncan, S., Ehmer, O., Hoyt, J., Kipp, M., Loehr, D., Magnusson, M., Rose, T., Sloetjes, H.: An exchange format for multimodal annotations. In: Kipp, M., Martin, J.-C., Paggio, P., Heylen, D. (eds.) Multimodal Corpora: From Models of Natural Interaction to Systems and Applications, pp. 207–221. Springer, Berlin (2009)

    Google Scholar 

  45. Shriberg, E.E.: Preliminaries to a Theory of Speech Disfluencies. PhD thesis, University of California at Berkeley. (1994)

    Google Scholar 

  46. Shriberg, E.E.: Acoustic properties of disfluent repetitions. In: Proceedings of the International Congress of Phonetic Sciences, vol. 4, pp. 384–387. Stockholm, Sweden (1995)

    Google Scholar 

  47. Shriberg, E.E.: Phonetic consequences of speech disfluency. In: Proceedings of the 14th International Congress on Phonetic Science, pp. 619–622. San Francisco (1999)

    Google Scholar 

  48. Wagner, A.: Unity in diversity: integrating differing linguistic data in tusnelda. In: Interdisciplinary Studies on Information Structure, pp. 1–20 (2005)

    Google Scholar 

  49. Ward, N.: Using prosodic clues to decide when to produce back-channel utterances. In: 4th International Conference on Spoken Language Processing, pp. 1724–1727 (1996)

    Google Scholar 

  50. Ward, N., Tsukahara, W.: Prosodic features which cue back-channel responses in english and japanese. J. Pragmat. 23, 1177–1207 (2000)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Philippe Blache .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Science+Business Media Dordrecht

About this chapter

Cite this chapter

Blache, P., Bertrand, R., Ferré, G., Pallaud, B., Prévot, L., Rauzy, S. (2017). The Corpus of Interactional Data: A Large Multimodal Annotated Resource. In: Ide, N., Pustejovsky, J. (eds) Handbook of Linguistic Annotation. Springer, Dordrecht. https://doi.org/10.1007/978-94-024-0881-2_51

Download citation

  • DOI: https://doi.org/10.1007/978-94-024-0881-2_51

  • Published:

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-94-024-0879-9

  • Online ISBN: 978-94-024-0881-2

  • eBook Packages: Social SciencesSocial Sciences (R0)

Publish with us

Policies and ethics