60 - FEVER: a large-scale dataset for Fact Extraction and VERification, with James Thorne

NAACL 2018 paper by James Thorne, Andreas Vlachos, Christos
Christodoulopoulos, and Arpit Mittal James tells us about his
paper, where they created a dataset for fact checking. We talk
about how this dataset relates to other datasets, why a new one was
needed, how it was built, and how well the initial baseline does on
this task. There are some interesting side notes on bias in dataset
construction, and on how "fact checking" relates to "fake news"
("fake news" could mean that an article is actively trying to
deceive or mislead you; "fact checking" here is just determining if
a single claim is true or false given a corpus of assumed-correct
reference material). The baseline system does quite poorly, and the
lowest-hanging fruit seems to be in improving the retrieval
component that finds relevant supporting evidence for claims.
There's a workshop and shared task coming up on this dataset:
http://fever.ai/. The shared task test period starts on July 24th -
get your systems ready!
https://www.semanticscholar.org/paper/FEVER%3A-a-Large-scale-Dataset-for-Fact-Extraction-Thorne-Vlachos/7b1f840ecfafb94d2d9e6e926696dba7fad0bb88