Tweeted By @GuggerSylvain

I just finished adding the Funnel Transformer model to 🤗 Transformers, a transformer model leveraging ideas from regular CNNs (pooling the hidden states after a block of n layers), ELECTRA pretraining and Transformer-XL positional attention. https://t.co/EG6wF1GWXm

— Sylvain Gugger (@GuggerSylvain) September 15, 2020