Making Better Informed Trust Decisions with Generalized Fact-Finding
Jeff Pasternack, Dan Roth
Information retrieval may suggest a document, and information extraction may tell us what it says, but which information sources do we trust and which assertions do we believe when different authors make conflicting claims? Trust algorithms known as fact-finders attempt to answer these questions, but consider only which source makes which claim, ignoring a wealth of background knowledge and contextual detail such as the uncertainty in the information extraction of claims from documents, attributes of the sources, the degree of similarity among claims, and the degree of certainty expressed by the sources. We introduce a new, lifted (generalized) fact-finding framework able to incorporate this additional information into the fact-finding process. Experiments using numerous state-of-the-art fact-finding algorithms demonstrate that these lifted fact-finders achieve significantly better performance than their unlifted variants on both semi-synthetic and real-world problems.