58 - Learning What’s Easy: Fully Differentiable Neural Easy-First Taggers, with André Martins

EMNLP 2017 paper by André F. T. Martins and Julia Kreutzer André
comes on the podcast to talk to us the paper. We spend the bulk of
the time talking about the two main contributions of the paper: how
they applied the notion of "easy first" decoding to neural taggers,
and the details of the constrained softmax that they introduced to
accomplish this. We conclude that "easy first" might not be the
right name for this - it's doing something that in the end is very
similar to stacked self-attention, with standard independent
decoding at the end. The particulars of the self-attention are
inspired by "easy first", however, using a constrained softmax to
enforce some novel constraints on the self-attention.
https://www.semanticscholar.org/paper/Learning-What's-Easy%3A-Fully-Differentiable-Neural-Martins-Kreutzer/252571243aa4c0b533aa7fc63f88d07fd844e7bb