<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="hrinchuk-etal-2020-tensorized">
    <titleInfo>
        <title>Tensorized Embedding Layers</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Oleksii</namePart>
        <namePart type="family">Hrinchuk</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Valentin</namePart>
        <namePart type="family">Khrulkov</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Leyla</namePart>
        <namePart type="family">Mirvakhabova</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Elena</namePart>
        <namePart type="family">Orlova</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ivan</namePart>
        <namePart type="family">Oseledets</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2020</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Trevor</namePart>
            <namePart type="family">Cohn</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yulan</namePart>
            <namePart type="family">He</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yang</namePart>
            <namePart type="family">Liu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The embedding layers transforming input words into real vectors are the key components of deep neural networks used in natural language processing. However, when the vocabulary is large, the corresponding weight matrices can be enormous, which precludes their deployment in a limited resource setting. We introduce a novel way of parameterizing embedding layers based on the Tensor Train decomposition, which allows compressing the model significantly at the cost of a negligible drop or even a slight gain in performance. We evaluate our method on a wide range of benchmarks in natural language processing and analyze the trade-off between performance and compression ratios for a wide range of architectures, from MLPs to LSTMs and Transformers.</abstract>
    <identifier type="citekey">hrinchuk-etal-2020-tensorized</identifier>
    <identifier type="doi">10.18653/v1/2020.findings-emnlp.436</identifier>
    <location>
        <url>https://aclanthology.org/2020.findings-emnlp.436</url>
    </location>
    <part>
        <date>2020-11</date>
        <extent unit="page">
            <start>4847</start>
            <end>4860</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Tensorized Embedding Layers
%A Hrinchuk, Oleksii
%A Khrulkov, Valentin
%A Mirvakhabova, Leyla
%A Orlova, Elena
%A Oseledets, Ivan
%Y Cohn, Trevor
%Y He, Yulan
%Y Liu, Yang
%S Findings of the Association for Computational Linguistics: EMNLP 2020
%D 2020
%8 November
%I Association for Computational Linguistics
%C Online
%F hrinchuk-etal-2020-tensorized
%X The embedding layers transforming input words into real vectors are the key components of deep neural networks used in natural language processing. However, when the vocabulary is large, the corresponding weight matrices can be enormous, which precludes their deployment in a limited resource setting. We introduce a novel way of parameterizing embedding layers based on the Tensor Train decomposition, which allows compressing the model significantly at the cost of a negligible drop or even a slight gain in performance. We evaluate our method on a wide range of benchmarks in natural language processing and analyze the trade-off between performance and compression ratios for a wide range of architectures, from MLPs to LSTMs and Transformers.
%R 10.18653/v1/2020.findings-emnlp.436
%U https://aclanthology.org/2020.findings-emnlp.436
%U https://doi.org/10.18653/v1/2020.findings-emnlp.436
%P 4847-4860

Markdown (Informal)

[Tensorized Embedding Layers](https://aclanthology.org/2020.findings-emnlp.436) (Hrinchuk et al., Findings 2020)

• Tensorized Embedding Layers (Hrinchuk et al., Findings 2020)

ACL

• Oleksii Hrinchuk, Valentin Khrulkov, Leyla Mirvakhabova, Elena Orlova, and Ivan Oseledets. 2020. Tensorized Embedding Layers. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4847–4860, Online. Association for Computational Linguistics.