<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="burdick-etal-2022-using">
    <titleInfo>
        <title>Using Paraphrases to Study Properties of Contextual Embeddings</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Laura</namePart>
        <namePart type="family">Burdick</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jonathan</namePart>
        <namePart type="given">K</namePart>
        <namePart type="family">Kummerfeld</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Rada</namePart>
        <namePart type="family">Mihalcea</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Marine</namePart>
            <namePart type="family">Carpuat</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Marie-Catherine</namePart>
            <namePart type="family">de Marneffe</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ivan</namePart>
            <namePart type="given">Vladimir</namePart>
            <namePart type="family">Meza Ruiz</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Seattle, United States</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We use paraphrases as a unique source of data to analyze contextualized embeddings, with a particular focus on BERT. Because paraphrases naturally encode consistent word and phrase semantics, they provide a unique lens for investigating properties of embeddings. Using the Paraphrase Database‘s alignments, we study words within paraphrases as well as phrase representations. We find that contextual embeddings effectively handle polysemous words, but give synonyms surprisingly different representations in many cases. We confirm previous findings that BERT is sensitive to word order, but find slightly different patterns than prior work in terms of the level of contextualization across BERT‘s layers.</abstract>
    <identifier type="citekey">burdick-etal-2022-using</identifier>
    <identifier type="doi">10.18653/v1/2022.naacl-main.338</identifier>
    <location>
        <url>https://aclanthology.org/2022.naacl-main.338/</url>
    </location>
    <part>
        <date>2022-07</date>
        <extent unit="page">
            <start>4558</start>
            <end>4568</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Using Paraphrases to Study Properties of Contextual Embeddings
%A Burdick, Laura
%A Kummerfeld, Jonathan K.
%A Mihalcea, Rada
%Y Carpuat, Marine
%Y de Marneffe, Marie-Catherine
%Y Meza Ruiz, Ivan Vladimir
%S Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
%D 2022
%8 July
%I Association for Computational Linguistics
%C Seattle, United States
%F burdick-etal-2022-using
%X We use paraphrases as a unique source of data to analyze contextualized embeddings, with a particular focus on BERT. Because paraphrases naturally encode consistent word and phrase semantics, they provide a unique lens for investigating properties of embeddings. Using the Paraphrase Database‘s alignments, we study words within paraphrases as well as phrase representations. We find that contextual embeddings effectively handle polysemous words, but give synonyms surprisingly different representations in many cases. We confirm previous findings that BERT is sensitive to word order, but find slightly different patterns than prior work in terms of the level of contextualization across BERT‘s layers.
%R 10.18653/v1/2022.naacl-main.338
%U https://aclanthology.org/2022.naacl-main.338/
%U https://doi.org/10.18653/v1/2022.naacl-main.338
%P 4558-4568

Markdown (Informal)

[Using Paraphrases to Study Properties of Contextual Embeddings](https://aclanthology.org/2022.naacl-main.338/) (Burdick et al., NAACL 2022)

• Using Paraphrases to Study Properties of Contextual Embeddings (Burdick et al., NAACL 2022)

ACL

• Laura Burdick, Jonathan K. Kummerfeld, and Rada Mihalcea. 2022. Using Paraphrases to Study Properties of Contextual Embeddings. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4558–4568, Seattle, United States. Association for Computational Linguistics.