122. Sadie St. Lawrence - Trends in data science
Towards Data Science
05/04/22
•43m
About
Comments
Featured In
As you might know if you follow the podcast, we usually talk about the world of cutting-edge AI capabilities, and some of the emerging safety risks and other challenges that the future of AI might bring. But I thought that for today’s episode, it would be fun to change things up a bit and talk about the applied side of data science, and how the field has evolved over the last year or two.
And I found the perfect guest to do that with: her name is Sadie St. Lawrence, and among other things, she’s the founder of Women in Data — a community that helps women enter the field of data and advance throughout their careers — and she’s also the host of the Data Bytes podcast, a seasoned data scientist and a community builder extraordinaire. Sadie joined me to talk about her founder’s journey, what data science looks like today, and even the possibilities that blockchains introduce for data science on this episode of the towards data science podcast.
***
Intro music:
Artist: Ron Gelinas
Track Title: Daybreak Chill Blend (original mix)
Link to Track: https://youtu.be/d8Y2sKIgFWc
***
Chapters:
- 2:00 Founding Women in Data
- 6:30 Having gendered conversations
- 11:00 The cultural aspect
- 16:45 Opportunities in blockchain
- 22:00 The blockchain database
- 32:30 Data science education
- 37:00 GPT-3 and unstructured data
- 39:30 Data science as a career
- 42:50 Wrap-up
Previous Episode
121. Alexei Baevski - data2vec and the future of multimodal learning
April 27, 2022
•49m
If the name data2vec sounds familiar, that’s probably because it made quite a splash on social and even traditional media when it came out, about two months ago. It’s an important entry in what is now a growing list of strategies that are focused on creating individual machine learning architectures that handle many different data types, like text, image and speech.
Most self-supervised learning techniques involve getting a model to take some input data (say, an image or a piece of text) and mask out certain components of those inputs (say by blacking out pixels or words) in order to get the models to predict those masked out components.
That “filling in the blanks” task is hard enough to force AIs to learn facts about their data that generalize well, but it also means training models to perform tasks that are very different depending on the input data type. Filling in blacked out pixels is quite different from filling in blanks in a sentence, for example.
So what if there was a way to come up with one task that we could use to train machine learning models on any kind of data? That’s where data2vec comes in.
For this episode of the podcast, I’m joined by Alexei Baevski, a researcher at Meta AI one of the creators of data2vec. In addition to data2vec, Alexei has been involved in quite a bit of pioneering work on text and speech models, including wav2vec, Facebook’s widely publicized unsupervised speech model. Alexei joined me to talk about how data2vec works and what’s next for that research direction, as well as the future of multi-modal learning.
***
Intro music:
Artist: Ron Gelinas
Track Title: Daybreak Chill Blend (original mix)
Link to Track: https://youtu.be/d8Y2sKIgFWc
***
Chapters:- 2:00 Alexei’s background
- 10:00 Software engineering knowledge
- 14:10 Role of data2vec in progression
- 30:00 Delta between student and teacher
- 38:30 Losing interpreting ability
- 41:45 Influence of greater abilities
- 49:15 Wrap-up
Next Episode
Two ML researchers with world-class pedigrees who decided to build a company that puts AI on the blockchain. Now to most people — myself included — “AI on the blockchain” sounds like a winning entry in some kind of startup buzzword bingo. But what I discovered talking to Jacob and Ala was that they actually have good reasons to combine those two ingredients together.
At a high level, doing AI on a blockchain allows you to decentralize AI research and reward labs for building better models, and not for publishing papers in flashy journals with often biased reviewers.
And that’s not all — as we’ll see, Ala and Jacob are taking on some of the thorniest current problems in AI with their decentralized approach to machine learning. Everything from the problem of designing robust benchmarks to rewarding good AI research and even the centralization of power in the hands of a few large companies building powerful AI systems — these problems are all in their sights as they build out Bittensor, their AI-on-the-blockchain-startup.
Ala and Jacob joined me to talk about all those things and more on this episode of the TDS podcast.
---
Intro music:
Artist: Ron Gelinas
Track Title: Daybreak Chill Blend (original mix)
Link to Track: https://youtu.be/d8Y2sKIgFWc
---
Chapters:
- 2:40 Ala and Jacob’s backgrounds
- 4:00 The basics of AI on the blockchain
- 11:30 Generating human value
- 17:00 Who sees the benefit? 22:00 Use of GPUs
- 28:00 Models learning from each other
- 37:30 The size of the network
- 45:30 The alignment of these systems
- 51:00 Buying into a system
- 54:00 Wrap-up
If you like this episode you’ll love
Promoted