Tweeted By @AlecRad
We *are* as a field developing and training models that *are* using more context but where exactly where we are on that trend-line is a great question.
— Alec Radford (@AlecRad) February 11, 2019
Keep in mind nets are lazy and if you can "solve" a task by doing something "basic" you'll only learn "basic" things.