109 - What Does Your Model Know About Language, with Ellie Pavlick

How do we know, in a concrete quantitative sense, what a deep
learning model knows about language? In this episode, Ellie Pavlick
talks about two broad directions to address this question:
structural and behavioral analysis of models. In structural
analysis, we often train a linear classifier for some linguistic
phenomenon we'd like to probe (e.g., syntactic dependencies) while
using the (frozen) weights of a model pre-trained on some tasks
(e.g., masked language models). What can we conclude from the
results of probing experiments? What does probing tell us about the
linguistic abstractions encoded in each layer of an end-to-end
pre-trained model? How well does it match classical NLP pipelines?
How important is it to freeze the pre-trained weights in probing
experiments? In contrast, behavioral analysis evaluates a model's
ability to distinguish between inputs which respect vs. violate a
linguistic phenomenon using acceptability or entailment tasks,
e.g., can the model predict which is more likely: "dog bites man"
vs. "man bites dog"? We discuss the significance of which format to
use for behavioral tasks, and how easy it is for humans to perform
such tasks. Ellie Pavlick's homepage:
https://cs.brown.edu/people/epavlick/ BERT rediscovers the
classical nlp pipeline , by Ian Tenney, Dipanjan Das, Ellie Pavlick
https://arxiv.org/pdf/1905.05950.pdf?fbclid=IwAR3gzFibSBoDGdjqVu9Gq0mh1lDdRZa7dm42JuXXUfjG6rKZ44iHIOdV6jg
Inherent Disagreements in Human Textual Inferences by Ellie Pavlick
and Tom Kwiatkowski
https://www.mitpressjournals.org/doi/full/10.1162/tacl_a_00293