Multilingual language models (MLMs), such as XLM-R or BLOOM, are pretrained on data covering many languages and share their parameters across all languages. This modeling approach has several powerful advantages, such as allowing similar languages to exert positive influence on each other, and enabling cross-lingual task transfer (i.e., fine-tuning on some source language(s), then using the model on different target languages). The success of such transfer, however, depends on the model's ability to effectively share information between different languages in its parameter space. Yet, the cross-lingual information sharing mechanisms within MLMs are still not fully understood. In this talk, I will present our recent research that investigates this question from three different perspectives: encoding of typological relationships between languages within MLMs, language-wise modularity of MLMs and the influence of training examples in specific languages on predictions made in others.

Ekaterina Shutova is an Associate Professor at the ILLC, University of Amsterdam, where she leads the Amsterdam Natural Language Understanding Lab and the Natural Language Processing & Digital Humanities research unit. She received her PhD from the University of Cambridge, and then worked as a research scientist at the University of California, Berkeley. Ekaterina’s current research focuses on few-shot learning for language interpretation tasks, multilingual NLP, generalisability and robustness of NLP models and interpretability in deep learning. Her prominent service roles include Program Chair of ACL 2025, Senior Action Editor of ACL Rolling Review, Action Editor of Computational Linguistics and Demonstrations chair at EMNLP 2022. She is also an ELLIS scholar.