58. David Duvenaud - Using generative models for explainable AI
Towards Data Science
11/18/20
•36m
About
Comments
Featured In
In the early 1900s, all of our predictions were the direct product of human brains. Scientists, analysts, climatologists, mathematicians, bankers, lawyers and politicians did their best to anticipate future events, and plan accordingly.
Take physics, for example, where every task we think of as part of the learning process, from data collection to cleaning to feature selection to modeling, all had to happen inside a physicist’s head. When Einstein introduced gravitational fields, what he was really doing was proposing a new feature to be added to our model of the universe. And the gravitational field equations that he put forward at the same time were an update to that very model.
Einstein didn’t come up with his new model (or “theory” as physicists call it) of gravity by running model.fit() in a jupyter notebook. In fact, he never outsourced any of the computations that were needed to develop it to machines.
Today, that’s somewhat unusual, and most of the predictions that the world runs on are generated in part by computers. But only in part — until we have fully general artificial intelligence, machine learning will always be a mix of two things: first, the constraints that human developers impose on their models, and second, the calculations that go into optimizing those models, which we outsource to machines.
The human touch is still a necessary and ubiquitous component of every machine learning pipeline, but it’s ultimately limiting: the more of the learning pipeline that can be outsourced to machines, the more we can take advantage of computers’ ability to learn faster and from far more data than human beings. But designing algorithms that are flexible enough to do that requires serious outside-of-the-box thinking — exactly the kind of thinking that University of Toronto professor and researcher David Duvenaud specializes in. I asked David to join me for the latest episode of the podcast to talk about his research on more flexible and robust machine learning strategies.
Previous Episode
57. Dylan Hadfield-Menell - Humans in the loop
November 11, 2020
•64m
Human beings are collaborating with artificial intelligences on an increasing number of high-stakes tasks. I’m not just talking about robot-assisted surgery or self-driving cars here — every day, social media apps recommend content to us that quite literally shapes our worldviews and our cultures. And very few of us even have a basic idea of how these all-important recommendations are generated.
As time goes on, we’re likely going to become increasingly dependent on our machines, outsourcing more and more of our thinking to them. If we aren’t thoughtful about the way we do this, we risk creating a world that doesn’t reflect our current values or objectives. That’s why the domain of human/AI collaboration and interaction is so important — and it’s the reason I wanted to speak to Berkeley AI researcher Dylan Hadfield-Menell for this episode of the Towards Data Science podcast. Dylan’s work is focused on designing algorithms that could allow humans and robots to collaborate more constructively, and he’s one of a small but growing cohort of AI researchers focused on the area of AI ethics and AI alignment.
Next Episode
59. Matthew Stewart - Tiny ML and the future of on-device AI
November 25, 2020
•43m
When it comes to machine learning, we’re often led to believe that bigger is better. It’s now pretty clear that all else being equal, more data, more compute, and larger models add up to give more performance and more generalization power. And cutting edge language models have been growing at an alarming rate — by up to 10X each year.
But size isn’t everything. While larger models are certainly more capable, they can’t be used in all contexts: take, for example, the case of a cell phone or a small drone, where on-device memory and processing power just isn’t enough to accommodate giant neural networks or huge amounts of data. The art of doing machine learning on small devices with significant power and memory constraints is pretty new, and it’s now known as “tiny ML”. Tiny ML unlocks an awful lot of exciting applications, but also raises a number of safety and ethical questions.
And that’s why I wanted to sit down with Matthew Stewart, a Harvard PhD researcher focused on applying tiny ML to environmental monitoring. Matthew has worked with many of the world’s top tiny ML researchers, and our conversation focused on the possibilities and potential risks associated with this promising new field.
If you like this episode you’ll love
Promoted