Abstract
The availability of annotated datasets had been steadily growing for written language and benefited to linguistic studies and natural language processing. The situation for face-to-face spontaneous conversation is more contrasted for several reasons: technicalities in handling raw data (split across several sources and medias), need for a often difficult and time-consuming transcription, large variety of annotation that can be performed. We propose in this chapter a complete annotation workflow, starting from raw data (speech and video) to a richly annotated dataset with many linguistic information (morpho-syntax, prosody, gesture studies, discourse analysis). Our approach consisted in gathering experts from the different domains and work together on the establishment of an abstract schema encoded with types feature structures. We detail how the annotation workflow had been used for developing of a richly annotated version of the Corpus of Interactional Data. The corpus as well as the annotation described here are available through the Speech and Language Data Repository.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
This is a French-speaking initiative, the partners of the project are ATILF, ICAR, LI, LIMSI, LLL, LPL, SYLED, VALIBEL.
- 2.
A third level of phrasing (the intermediate unit) was proposed but its definition is not yet well established [33] and still requires to be refined more specifically for spontaneous French.
References
Allwood, J., Cerrato, L.: A study of gestural feedback expressions. In: First Nordic Symposium on Multimodal Communication, pp. 7–22 (2003)
Allwood, J., Cerrato, L., Dybkjaer, L., Jokinen, K., Navaretta, C., Paggio, P.: The MUMIN Multimodal Coding Scheme, pp. 129–157. NorFA yearbook 2005 (2005)
Anderson, A.H., Bader, M., Gurman Bard, E., Boyle, E., Doherty, G., Garrod, S., Isard, S., Kowtko, J., McAllister, J., Miller, J., Sotillo, C., Thompson, H.S., Weinert, R.: The hcrc map task corpus. Lang. Speech 34, 351–366 (1991)
Bertrand, R., Portes, C., Sabio, F.: Distribution syntaxique, discursive et interactionnelle des contours intonatifs du français dans un corpus de conversation. Travaux neuchâtelois de linguistique, p. 47 (2007)
Blache, P., Rauzy, S.: Influence de la qualité de l’étiquetage sur le chunking : une corrélation dépendant de la taille des chunks. In: Actes de Traitement Automatique des Langues Naturelles, pp. 290–299. Avignon, France (2008)
Boersma, P., Weenink, D.: Praat, a system for doing phonetics by computer, version 3.4. Technical Report 132, Institute of Phonetic Sciences of the University of Amsterdam (1996)
Bull, P.E.: Posture and gesture. In: International Series in Experimental Social Psychology, vol. 16 (1987)
Carletta, J.: Announcing the ami meeting corpus. The ELRA Newsletter 11(1), (2006)
Carletta, J., Isard, A.: The mate annotation workbench: user requirements. In: Proceedings of the ACL Workshop: Towards Standards and Tools for Discourse Tagging (1999)
Carletta, J., Evert, S., Heid, U., Kilgour, J., Robertson, J., Voormann, H.: The nite xml toolkit: flexible annotation for multi-modal language data. Behav. Res. Methods Instrum. Comput. 35(3), (2003)
Carletta, J.C.: Assessing agreement on classification tasks: the kappa statistic. Comput. Linguist. 22(2), 249–254 (1996)
Carpenter, B.: The Logic of Typed Feature Structures. Cambridge University Press, Cambridge (1992)
Caspers, J.: Local speech melody as a limiting factor in the turn-taking system in dutch. J. Phon. 31, 251–276 (2003)
Clark, H.H., Wasow, T.: Repeating words in spontaneous speech. Cogn. Psychol. 37, 201–242 (1998)
Di Cristo, A.: Vers une modélisation de l’accentuation en français. deuxième partie : le modèle. J. Fr. Lang. Stud. 10, 27–44 (2000)
Dister, A.: La notation subjective de la pause constitue-t-elle un bon indice pour le découpage de corpus oraux ?. In: Constant, M., Dister, A., Emirkanian, L., Piron, S. (eds.) Description linguistique pour le traitement automatique du français, Cahiers du Cental 5, pp. 165–186. Louvain-la-Neuve, Presses universitaires de Louvain (2008)
Fohr, D., Mella, O., Cerisara, C., Illina, I.: The automatic news transcription system: ants, some real time experiments. In: INTERSPEECH-2004, pp. 377–380 (2004)
Fox Tree, J.E.: Listening in on monologues and dialogues. Discourse Process. 27(1), 35–53 (1999)
Gendner, V., Illouz, G., Jardino, M., Monceaux, L., Paroubek, P., Robba, I., Vilnat, A.: PEAS, the first instantiation of a comparative framework for evaluating parsers of french. In: Research Notes of EACL 2003, Budapest, Hongrie (2003)
Guénot, M.L.: Parsing de loral : traiter les disfluences. In: Traitement Automatique des Langues Naturelles, TALN, 6-10 June 2005. Dourdan, France (2005)
Henry, S., Pallaud, B.: Word fragments and repeats in spontaneous spoken French. In: Eklund, R. (ed.) Disfluency in Spontaneous Speech Workshop, Proceedings of DiSS03, 5-8 September 2003, pp. 77–80. Göteborg University, Sweden (2003)
Hirst, D.: A praat plugin for momel and intsint with improved algorithms for modelling and coding intonation. In: ICPhS XVI (2007)
Hirst, D., Di Cristo, A., Espesser, R.: Levels of representation and levels of analysis for intonation. In: Horne, M. (ed.) Prosody: Theory and Experiment, pp. 51–87. Kluwer Academic Publishers (2000)
Ide, N., Bonhomme, P., Romary, L.: Xces: an xml-based standard for linguistic corpora. In: Proceedings of the Second Language Resources and Evaluation Conference, pp. 825–830 (2000)
Jun, S.A., Fougeron, C.: A phonological model of french intonation. In: Intonation: Analysis, modelling and technology, pp. 209–242. Dordrecht (2000)
Kendon, A.: Some functions of gaze direction in social interaction. Acta Psychol. 26, 22–63 (1967)
Kipp, M.: Anvil - a generic annotation tool for multimodal dialogue. In: Proceedings of the 7th European Conference on Speech Communication and Technology (Eurospeech), pp. 1367–1370 (2001)
Kipp, M., Neff, M., Albrecht, I.: An annotation scheme for conversational gestures: how to economically capture timing and form. In: Proceedings of the Workshop on “Multimodal Corpora” at LREC 2007, pp. 325–339 (2007)
Koiso, H., Horiuchi, Y., Ichikawa, A., Den, Y.: An analysis of turn-taking and backchannels based on prosodic and syntactic features in japanese map task dialogs. Language and Speech, 41 (1998)
Kruijff-Korbayova, I., Gerstenberger, C., Rieser, V., Schehl, J.: The sammie multimodal dialogue corpus meets the nite xml toolkit. In: proceedings of LREC06 (2006)
McNeill, D.: Hand and Mind. What Gestures Reveal about Thought. University of Chicago Press (1992)
McNeill, D.: Gesture and Thought. University of Chicago Press (2005)
Michelas, A.: Caractérisation phonétique et phonologique du syntagme intermédiaire en français: de la production à la perception. Ph.D. thesis, Aix-Marseille Université (2011)
Muller, P., Vergez, M., Prévot, L., Asher, N., Benamara, F., Bras, M., Le Draoulec, A., Vieu, L.: Manuel d’annotation en relations de discours du projet annodis. Technical report, CLLE (2012)
Pallaud, B.: Troncations de mots, reprises et interruption syntaxique en français parlé spontané. In: JADT, 8èmes Journées internationales d’Analyse statistique des Données Textuelles, 20–22 avril 2006, pp. 707–715. Besançon (2006)
Pallaud, B., Henry, S.: Amorces de mots et répétitions : des hésitations plus que des erreurs en français parlé. In: Le poids des mots. Actes des 7èmes Journées Internationales d’Analyse statistique des Données Textuelles, Louvain-la-Neuve, 10-12 mars 2004. vol. 2, pp. 848–858. Louvain, PUL(2004)
Paroubek, P., Robba, I., Vilnat, A., Ayache, C.: Data annotations and measures in EASY the evaluation campaign for parsers in french. In: Proceedings of the 5th International Conference on Language Resources and Evaluation, pp. 314–320. Genoa, Italy (2006)
Pineda, L.A., Massé, A., Meza, I., Salas, M., Schwarz, E., Uraga, E., Villaseñor, L.: The dime project. In: Proceedings of MICAI2002, vol. 2313. LNAI (2002)
Portes, C., Bertrand, R.: Some cues about the interactional value of the \(\ll \)continuation\(\gg \) contour in french. In: Discours et Prosodie comme Interface Complexe (2006)
Post, B.: Tonal and Phrasal Structures in French Intonation. Thesus (2000)
Rodriguez, K., Dipper, S., Götze, M., Poesio, M., Riccardi, G., Raymond, C., Rabiega-Wisniewska, J.: Standoff coordination for multi-tool annotation in a dialogue corpus. In: Proceedings of Linguistic Annotation Workshop (2007)
Sacks, H., Schegloff, E.A., Jefferson, G.: A simplest systematics for the organization of turn-taking for conversation. Language 50, 696–735 (1974)
Schegloff, E.: Discourse as an interactional achievement: some uses of “uh huh” and other things that come between sentences. In: Tannen, D. (ed.) Analyzing discourse: Text and talk. Georgetown University Press (1982)
Schmidt, T., Duncan, S., Ehmer, O., Hoyt, J., Kipp, M., Loehr, D., Magnusson, M., Rose, T., Sloetjes, H.: An exchange format for multimodal annotations. In: Kipp, M., Martin, J.-C., Paggio, P., Heylen, D. (eds.) Multimodal Corpora: From Models of Natural Interaction to Systems and Applications, pp. 207–221. Springer, Berlin (2009)
Shriberg, E.E.: Preliminaries to a Theory of Speech Disfluencies. PhD thesis, University of California at Berkeley. (1994)
Shriberg, E.E.: Acoustic properties of disfluent repetitions. In: Proceedings of the International Congress of Phonetic Sciences, vol. 4, pp. 384–387. Stockholm, Sweden (1995)
Shriberg, E.E.: Phonetic consequences of speech disfluency. In: Proceedings of the 14th International Congress on Phonetic Science, pp. 619–622. San Francisco (1999)
Wagner, A.: Unity in diversity: integrating differing linguistic data in tusnelda. In: Interdisciplinary Studies on Information Structure, pp. 1–20 (2005)
Ward, N.: Using prosodic clues to decide when to produce back-channel utterances. In: 4th International Conference on Spoken Language Processing, pp. 1724–1727 (1996)
Ward, N., Tsukahara, W.: Prosodic features which cue back-channel responses in english and japanese. J. Pragmat. 23, 1177–1207 (2000)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Blache, P., Bertrand, R., Ferré, G., Pallaud, B., Prévot, L., Rauzy, S. (2017). The Corpus of Interactional Data: A Large Multimodal Annotated Resource. In: Ide, N., Pustejovsky, J. (eds) Handbook of Linguistic Annotation. Springer, Dordrecht. https://doi.org/10.1007/978-94-024-0881-2_51
Download citation
DOI: https://doi.org/10.1007/978-94-024-0881-2_51
Published:
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-024-0879-9
Online ISBN: 978-94-024-0881-2
eBook Packages: Social SciencesSocial Sciences (R0)