Transformers; transforming; transformed!
Improve your practice of data science and level up your understanding of Transformer models!
Hello fellow datanistas! In this edition of the Data Science Programming Newsletter, I wanted to share two collections of resources that I believe are awesome:
Stuff on Transformers and NLP
Stuff on great data science practice
My hope is that you find them valuable for your learning journey as an ever-improving data scientist!
This page is an absolute gem by Brandon Rohrer. Here, he explains transformers right from the beginning, along the way disambiguating many terms in the NLP and Transformer literature that were never explained clearly by the original authors. If you’re looking for a comprehensive starter guide to understanding how NLP methods came about and where terms originated from, then his tutorial is a wonderful place to begin.
One of the key resources I constantly refer to when learning about Transformer models is Jay Alammar’s “The Illustrated Transformer”. I’m not sure what more to say except that if you’re interested in Transformers and want a detailed understanding, this is an invaluable resource to look at!
I love this thread! It’s a great example of careful research that attempts to disentangle whether large language models (LLMs) are capable of “reasoning” or not. Are we being confounded by not knowing what’s in the training data?
This blog post by Ethan Rosenthal is one that has resonated with me for a long, long time. I enjoyed reading it because it wrote, in very concise terms, the advantages of treating our data science projects in a portable fashion, leveraging software development practices. In many ways, Ethan’s blog post is like a miniature version of my Data Science Bootstrap Notes knowledge base. His blog post is a highly recommended read!
Rodrigo (Twitter handle
@mathsppblog) uses this Twitter thread to highlight why virtual environments are such a wonderful idea. I can relate - because once upon a time (in the year 2015, I believe), I had to nuke my macOS system Python because of conflicting package requirements for my projects, and I broke iPhoto along the way :). So heed Rodrigo’s call: use virtual environments, my friend! (I also have notes on using
conda environments in my DS bootstrap notes that you can check out too!)
Once you’re ready to ship your data science project, at some point you may end up looking at the use of Docker containers. Uwe Korn has a great blog post on how to dockerize your project the right way - i.e. minimizing disk usage and maximizing performance. Check it out!