Discover more from Data Science Programming Newsletter
Hello, fellow datanistas!
It's been about a year-ish since Large Language Models burst onto the scene. In this edition, I want to share thoughtful opinions on what will be a new era of computation.
On capitalizing on the opportunity ahead of us
Chris Albon - of ML Flashcards fame - has a thoughtful bird site thread on the window of opportunities. His argument is that there are no experts with decades of experience in LLMs and generative AI, so if one wants to jump into the field, this is the moment. Check out the thread in more depth here.
Where did the notion of AGI come from?
Timnit Gebru (Xoogler) has her answer in this YouTube video. Whether or not you agree with her views, it is a thought-provoking talk - and an exercise in tracing logical connections between historical eras of thinking and seeing their evolution.
LLM Speak
Andrej Karpathy (Tesla, OpenAI) presents to us LLM speak: how to say things you wouldn't want to say otherwise, expressed in the vernacular of large language models. It's a hilarious take!
What’s different about this AI revolution?
Cassie Kozyrkov has her take: it’s no longer about fancy math. It’s going to be all about user experience and product philosophy. Here’s her take on Medium!
LLM tooling: vector databases
Because of LLMs' context length limit, vector databases are surging in popularity. Postgres already supports vector stores, pgvector. Still, there are other players, too: chroma, pinecone, and even the simple vector stores in llama_index. As the technology stack evolves, we'll likely see the following:
Head-to-head battles between these databases, followed by differentiation and specialization,
Fundamental research into LLMs to scale context length, possibly to infinity, which may fundamentally shift how vector databases are positioned, and
Shaking out of unit economics - will we move beyond LLM tokens to some other unit?
I can't claim to know how things will play out, but that is the mental framework I'm working with as I observe the field.
From my collection
A developer-first guide to LLM APIs
I wrote this blog post in March while mucking around with LLM APIs. I was confused about whether we should use the OpenAI official Python API, LangChain, or LlamaIndex. As such, I decided to compare the three packages head-to-head formally. I document what I've learned in this blog post and am thrilled to share it with you!
LlamaBot
After that experimentation with the three APIs, I soon realized that there was an 80/20 situation with LLMs - the vast majority of use cases that I could plausibly see going into production in the next year fell into one of three patterns:
Simple queries of a non-fine-tuned LLM
Chat-based queries of the same
Document-enhanced queries
This knowledge led me to create a new package called llamabot that makes building out those three use cases more effortless than ever! You can find the package here on GitHub; I also wrote a blog post about it. Please try it out; I'd love to receive feedback on where I could make it better!