PME: pruning-based multi-size embedding for recommender systems

Simple item page
dc.citation.articleNumber1195742en_US
dc.citation.journalTitleFrontiers in Big Dataen_US
dc.citation.volumeNumber6en_US
dc.contributor.authorLiu, Ziruien_US
dc.contributor.authorSong, Qingquanen_US
dc.contributor.authorLi, Lien_US
dc.contributor.authorChoi, Soo-Hyunen_US
dc.contributor.authorChen, Ruien_US
dc.contributor.authorHu, Xiaen_US
dc.date.accessioned2023-08-01T17:29:49Zen_US
dc.date.available2023-08-01T17:29:49Zen_US
dc.date.issued2023en_US
dc.description.abstractEmbedding is widely used in recommendation models to learn feature representations. However, the traditional embedding technique that assigns a fixed size to all categorical features may be suboptimal due to the following reasons. In recommendation domain, the majority of categorical features' embeddings can be trained with less capacity without impacting model performance, thereby storing embeddings with equal length may incur unnecessary memory usage. Existing work that tries to allocate customized sizes for each feature usually either simply scales the embedding size with feature's popularity or formulates this size allocation problem as an architecture selection problem. Unfortunately, most of these methods either have large performance drop or incur significant extra time cost for searching proper embedding sizes. In this article, instead of formulating the size allocation problem as an architecture selection problem, we approach the problem from a pruning perspective and propose Pruning-based Multi-size Embedding (PME) framework. During the search phase, we prune the dimensions that have the least impact on model performance in the embedding to reduce its capacity. Then, we show that the customized size of each token can be obtained by transferring the capacity of its pruned embedding with significant less search cost. Experimental results validate that PME can efficiently find proper sizes and hence achieve strong performance while significantly reducing the number of parameters in the embedding layer.en_US
dc.identifier.citationLiu, Zirui, Song, Qingquan, Li, Li, et al.. "PME: pruning-based multi-size embedding for recommender systems." <i>Frontiers in Big Data,</i> 6, (2023) Frontiers Media S.A.: https://doi.org/10.3389/fdata.2023.1195742.en_US
dc.identifier.digitalfdata-06-1195742en_US
dc.identifier.doihttps://doi.org/10.3389/fdata.2023.1195742en_US
dc.identifier.urihttps://hdl.handle.net/1911/115043en_US
dc.language.isoengen_US
dc.publisherFrontiers Media S.A.en_US
dc.rightsExcept where otherwise noted, this work is licensed under a Creative Commons Attribution (CC BY) license.  Permission to reuse, publish, or reproduce the work beyond the terms of the license or beyond the bounds of Fair Use or other exemptions to copyright law must be obtained from the copyright holder.en_US
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/en_US
dc.titlePME: pruning-based multi-size embedding for recommender systemsen_US
dc.typeJournal articleen_US
dc.type.dcmiTexten_US
dc.type.publicationpublisher versionen_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
fdata-06-1195742.pdf
Size:
1.97 MB
Format:
Adobe Portable Document Format
Download
Collections
Faculty Publications
Computer Science Publications