Speaker: Georgios Zervakis
Data and place: January 20, 2022, at 10:30 – Videoconference
Abstract: Contextualized language models are emerging as a de facto standard in natural language processing due to the vast amount of knowledge they acquire during pretraining. Nonetheless, their ability to solve tasks that require reasoning over this knowledge is limited. Certain tasks can be improved by analogical reasoning over concepts, e.g., understanding the underlying relations in “Man is to woman as King is to Queen”. In this work, we propose a way to formulate target sense verification as an analogy detection task, by transforming the input data into quadruples. We present AB4TSV (Analogy and BERT for TSV), a model that uses BERT to represent the objects in these quadruples combined with a convolutional neural network to decide whether they constitute valid analogies. We test our system on the WiC-TSV evaluation benchmark, showing that it outperforms existing approaches. Our empirical study shows the importance of the input encoding for BERT. This dependence gets alleviated by enforcing invariance to permutations of analogical proportions during training while preserving performance and improving interpretability.
