Making language models robust to unknown words

First paper with the soda affiliation: Imputing out-of-vocabulary embeddings with LOVE makes language models robust with little cost (https://arxiv.org/abs/2203.07860) Making language models robust to unknown words (eg typos): a bit of contrastive learning can extend language models without retraining them! The idea of LOVE (Learning Out-of-Vocabulary Embeddings) is to map unknown…

Continue reading