2026-09-20T19:40:52Zhttps://repository.rice.edu/server/oai/request

oai:repository.rice.edu:1911/1093092024-01-11T20:57:42Zcom_1911_8299col_1911_13110

Baraniuk, Richard G. Sonkar, Shashank 2020-09-03T14:00:35Z 2020-09-03T14:00:35Z 2020-12 2020-09-02 December 2020 2020-09-03T14:00:35Z Sonkar, Shashank. "AWE: Attention Word Embedding." (2020) Master’s Thesis, Rice University. <a href="https://hdl.handle.net/1911/109309">https://hdl.handle.net/1911/109309</a>. https://hdl.handle.net/1911/109309 Word embedding models learn semantically rich vector representations of words and are widely used to initialize natural processing language (NLP) models. The popular continuous bag-of-words (CBOW) model of word2vec learns a vector embedding by masking a given word in a sentence and then using the other words as a context to predict it. A limitation of CBOW is that it equally weights the context words when making a prediction, which is inefficient, since some words have higher predictive value than others. We tackle this inefficiency by introducing the Attention Word Embedding (AWE) model, which integrates the attention mechanism into the CBOW model. We also propose AWE-S, which incorporates subword information. We demonstrate that AWE and AWE-S outperform the state-of-the-art word embedding models both on a variety of word similarity datasets and when used for initialization of NLP models. application/pdf eng Copyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder. Natural Language Processing Machine Learning Word Embeddings AWE: Attention Word Embedding Thesis Text Electrical and Computer Engineering Engineering Rice University Masters Master of Science open.access