Here is the sequel to "Just ask for Generalization" - in this blog post I argue that Generalization *is* Language, and suggest how we might be able to re-use Language Models as "generalization modules" for non-NLP domains. Check it out!https://t.co/TI5FrbAU2R
— Eric Jang 🇺🇸🇹🇼 (@ericjang11) December 17, 2021
We trained a research version of GPT-3 that can search the web, synthesize information, and cite its sources to provide more accurate answers to questions. https://t.co/YEEazt0oLZ
— OpenAI (@OpenAI) December 16, 2021
🚨New paper🚨 SOTA dialogue models are not winning Oscars anytime soon, as they cannot effectively stay in character.
— Jason Weston (@jaseweston) December 14, 2021
We analyze and propose methods to measure & mitigate -- but it's still an open problem.https://t.co/C9SjBPcT4S@shtruk @JackUrbs Arthur Szlam @jaseweston pic.twitter.com/i4V5WkNePT
Self-attention Does Not Need O(n²) Memory
— hardmaru (@hardmaru) December 13, 2021
“We provide a practical implementation for accelerators that requires O(√n) memory, is numerically stable, and is within a few percent of the runtime of the standard implementation of attention.”https://t.co/iMQvCMnWgi
Tax 101. This was sent to me and I think it’s worth sharing around. Basic knowledge but worth remembering. pic.twitter.com/17aAD8SR6h
— Simon Kuestenmacher (@simongerman600) December 11, 2021
I think we may need to recalibrate our idea of typical case numbers as Omicron takes off.
— John Burn-Murdoch (@jburnmurdoch) December 10, 2021
Here’s what UK cases could look like *in the next week or two alone* if Omicron continues to double every 3 days (some actually estimate faster growth)
Story: https://t.co/mdY7GFiNsD pic.twitter.com/GbXpEFkR6n
This is a terrific comparison of the main free GPU @ProjectJupyter providers available. I agree with the conclusion - the new @awscloud SageMaker Studio Lab is a fantastic option.
— Jeremy Howard (@jeremyphoward) December 9, 2021
It's my 1st choice now for training models where I don't need much disk space. https://t.co/M1XsEoMBH6
The three studies explore: Gopher - a SOTA 280B parameter transformer, ethical and social risks, & a new retrieval architecture with better training efficiency.
— DeepMind (@DeepMind) December 8, 2021
1: https://t.co/WDUeFd5DiF
2: https://t.co/cZcWHCg128
3: https://t.co/h9fdMP6C5W (more https://t.co/4QiVDqntTS) 2/
The ongoing consolidation in AI is incredible. Thread: ➡️ When I started ~decade ago vision, speech, natural language, reinforcement learning, etc. were completely separate; You couldn't read papers across areas - the approaches were completely different, often not even ML based.
— Andrej Karpathy (@karpathy) December 8, 2021
Just came across this eminently readable and beginner friendly introduction to Transformers. Probably the best such text I've seen.
— Bojan Tunguz (@tunguz) December 7, 2021
"Transformers from Scratch"https://t.co/qwI75Ezu9O#AI #ML #DS #NLP #artificialintelligence #machinelearning #datascience 1/2 pic.twitter.com/vvhFuzRdC9
🎉Part 2- Summary of 10 summaries on:
— AI Fast Track (60/60) (@ai_fast_track) December 6, 2021
Tips & Trick & Best Practices in training (not only) object detection models.
Don't miss any of those posts, follow @ai_fast_track to catch them in your feed.
🎁 Summary of summaries: ... pic.twitter.com/VLcWNkMaph
Minitorch🔥(https://t.co/41DIF6vAUl, v2021) Build-it-yourself deep learning.
— Sasha Rush (@srush_nlp) December 6, 2021
Learn about auto-diff, tensors, gpu's, and advanced nn models. Build everything from scratch in pure python with full unit test coverage and interactive visualizations. pic.twitter.com/DmSLwuzoZQ
