An Efficient Framework for Constructing Generalized Locally-Induced Text Metrics
Saeed Amizadeh, Shuguang Wang and Milos Hauskrecht
In this paper, we propose a new framework for constructing text similarity metrics which can be used to compare and support inferences among terms and sets of terms representing text components such as sentences, paragraphs, abstracts or whole documents. Our metric is derived from data-driven kernels on graphs that let us capture global relations among terms and sets of terms, regardless of their complexity and size. To compute the metric efficiently for any two subsets of terms, we develop an approximation technique that relies on a precompiled term-term similarities. To scale-up the approach to problems with huge number of terms, we develop and experiment with a solution that subsamples the term space. We demonstrate the benefits of the whole framework on two text inference tasks: prediction of terms in the article from its abstract and query expansion in information retrieval.