ai.bythebay.io: David Silver, Becoming a Self-Driving Car Engineer
Recording: ai.bythebay.io: David Silver, Becoming a Self-Driving Car Engineer
you so I am here this afternoon for calisthenics and to talk about training self-driving car engineers because that's my job at Udacity before we talk about how we train self-driving car engineers I thought you might want to know a little bit about how I got into this I was for many years a web software engineer that red gem you can see up on the slide there that's actually the logo for Ruby of Ruby on Rails fame I was a Ruby on Rails engineer for many years I was working on recruiting software and I decided I want to change things up to something a little different I got really excited about self-driving cars but I didn't have any background in automotive software and system software and I wasn't sure how I would get anybody to hire me to work on self-driving cars and I started searching the internet and I stumbled upon a class at Udacity called AI for robotics it's a class taught by our founder Sebastian Thrun and it goes through some of the basics of how self-driving cars work and I took that class and then I took other classes at Udacity and then I took classes at Coursera and classes at EDX and I started doing projects and then I started knocking on a lot of doors and eventually I clawed my way onto the autonomous driving team at Ford Motor Company with their research and innovation Center in Palo Alto which is just an amazing place to work I really really loved working at Ford a great team my my old boss sho'nac is here he's great the whole team at Ford is terrific and I was really privileged to work there while I was working at Ford the team at Udacity was reading my blog about how I had become a self-driving car engineer and my excitement about autonomous vehicles and trying to teach people through my blog about what it meant to be an autonomous vehicle engineer and they contacted me and said David what would it be like if you came here to um to Udacity and helped build out of course to to teach other people how to do what I had done to become a self-driving car engineer just make it a lot easier and less painful and so we talked for a while and and eventually I came over to join Sebastian Thrun the the founder of Udacity to build the self-driving car engineer nano degree we launched the nano degree in October it's a nine month course that takes people through the entire technology stack for autonomous vehicles the goal is to prepare students to work in the autonomous vehicle field and I'll take you through a little bit of what we cover and what students learn and then show you a little bit of work that that students have done so we'll start by talking about my partner in this Sebastian Thrun Sebastian is kind of a luminary in Silicon Valley he was a professor at Stanford where he won the 2005 DARPA Grand Challenge which was a really famous self-driving car race to the Mojave Desert that kicked off the the self-driving car boom he left Stanford to go start Google self-driving car project which is now known as way mo and Sebastian built that out and he demonstrated Udacity um to help teach people all around the world how to develop software and to democratize education so that anybody anywhere can get the world's best education this past summer I joined the team and we kind of merged Sebastian's interest in both education and autonomous vehicles and we've been building the Udacity self-driving car engineer and a degree program it's a real privilege to work with him so I'd like to take you through a little bit of of what we do in the self-driving car nano degree I'll I'll walk you through what we cover in the nine months and then we'll talk a little bit about what some of the students have done because I think it's really exciting to see some of the work that that our students have put together and in the middle you'll actually learn something hopefully so the nine-month program starts out with an introduction and then we dive into deep learning deep learning is one of the hottest fields in in Silicon Valley is certainly a super hot field for autonomous vehicles George Hotz who spoke this morning is a big proponent of you things like deep learning the behavioral cloning to help train cars to drive themselves he talked in more depth than I possibly could about all of the challenges and advantages of of trying to use be wanting to to to train vehicles we do that in about a month we cover convolutional neural networks and deep neural networks and the the month ends with students building a network to drive a car in a simulator so it's a really fun project to actually implement behavioral cloning in a simulated environment the next month the third months of the program after the introduction and deep learning is computer vision where we go over more standard computer vision techniques for doing things like finding lane lines and detecting vehicles on the road and students in that month by building a computer vision pipeline to draw bounding boxes around vehicles and it's really cool by the time you get to the end that month you can kind of see something that looks a little bit like what you might imagine in an Tesla auto pilot where you've got the lane lines and and the bounding boxes around the vehicles and I'll show you one of our students who who completed that project in their approach to it the next term is another three months in this term is pretty intensely focused on robotics so we start out with a month on sensor fusion which is the science of taking lidar data like the Vela Dyne folks create and radar data like our friends at Bosch and other automotive suppliers field and fusing that data together to help the vehicle understand what the environment looks like because if the lidar tells you that a pedestrian is there and the radar tells you no the pedestrian is over there you have to figure out how to use that data together and understand what what does the world really look like out there in the next month is localization which helps students understand how to figure out where the vehicle is so how to take a high-resolution lidar map and then figure out where are we actually on that map that seems like a really easy problem in the world of GPS right you just kind of pull out your phone and your phone says oh you're in San Francisco self-driving cars need much higher resolution than that you kind of think about driving down the street if you're off by you know a hundred centimeters which I guess is a meter by definition then you might wind up on the sidewalk or hitting something so you need to know what kind of single digits centimeter level accuracy where are you in the world and so we teach students how to build particle filters that's the project at the end of that month the the six month is controlled and that's building some of the classical mechanical engineering control algorithms PID control when your quadratic regulators and for this project students go back into the simulator that we use for behavioral cloning I mean deep learning and they try a different approach to drive the car in the simulator using mechanical control algorithms so all of this is is kind of project-based learning where there's a project at the end that students are building up to and we're teaching students what do you need to know in order to complete that project and then the gold the project is to have something really cool to to take to employers and say you know I know how to build parts of the autonomous vehicle stack and I would be a really valuable contributor to your team our entire guiding star with this program really is jobs how do we help students get jobs in the autonomous vehicle industry the last three months of the program are termed three well we have a month on path planning this is taught in conjunction with our partners at mercedes-benz almost all of our modules actually are taught in conjunction with various partners we work with uber on Controls path planning and localization sensor fusion with mercedes-benz deep learning with our partners at Nvidia as well as system integration Nvidia is helping us with we're very heavily tied into industry because you know what we do is we go out to employers and say what what do students need to know what the potential candidates need to know in order to walk in the door at Nvidia or uber or Mercedes Benz or Bosch or Polly Sync or autonomous stuff and be a credible candidate via candidates that you would want to hire so for past planning will be covering trajectory planning and decision-making leading up to a highway driving project in the simulator with traffic and and kind of managing a more complex environment there'll be an elective month in the middle and then the final month will be a month on system integration and the big hook at the end of the whole program at the end of the system integration month is that students will put their code on a real self-driving car so Wolfgang from autonomous stuff mentioned earlier that we purchased a car from them a Lincoln MKZ and it's kitted out with with all the drive-by-wire hardware necessary and we've been working on this MKZ to set it up so that at the end of the course the first students will hit this in the summer students will be able to form teams and put their code on the actual vehicle and see if they can get a real vehicle running real software like Ross to drive around a test track so we we think of this as what we call an only at Udacity experience it's the type of a program and project that you only really get at a company like Udacity that has the resources to help students get real-world experience that they can take to employers and say you know not only have I build projects you know with with simulated data but I've actually put my coat on the cars so we're very excited about this we have several thousand students in the program now we start a new cohort every couple of months and if you're interested we encourage you to come study and learn about self-driving cars with us so that's kind of the high-level overview of the program because I worked in education I thought it might be fun to teach folks about a little bit of of the self-driving car technology stack this this video clip actually comes from Sebastian's free AI for robotics course so if you're interested you can just go to Udacity calm this particular course is available for free and it covers a little bit about how do we actually control a vehicle so let's get started and see what Sebastian has to say he is in control and many many classes can be taught about this one subject matter so what I do is they give you the very basics and let you implement the very basics and I promise it will be fun you'll be able to drive a car around and the Google car to the present date uses a version of this exact same controller that's of course much more tuned to the specifics of our car but you get to see some of the essence of what it means to control the car so here's the problem consider the following car with the steerable front axle and two non-sterile wheels in the back and say we wish this car drive along this line which is the output of our smoothie that we just discussed let's assume the car has a fixed forward velocity but you have an ability to set the steering angle of the car how would you do this you would keep the steering constant it will be random steering commands or you could set the steering angle in proportion to what's known as the crosstrack error which is the lateral distance between the vehicle and the so-called reference trajectory so the third possibility is steer in proportion to this cross track error CTE choose one of those that you think are best suited to control the car okay so this is a little sample of how we teach at Udacity and it captures one of the core elements of online learning Udacity which is quizzes we have lots and lots of quizzes because it's easy to kind of watch a video and have it kind of wash over you and not really retain it having these frequent quizzes helps you kind of stop and and think about what you just learned and how to apply it so I'll ask folks you know if what you want to do is take that car there and you want to get it down to the line and steer it along the line you want to hold the steering constant do you want random steering or do you want to steer in proportion to the crosstrack error so raise your hand if you want to hold the steering constant all right we got we got we got a taker raise your hand if you want to steer randomly oh we got a taker for random steering alright raise your hand if you want to steer in proportion the crosstrack error alright let's do it answers and yes use steer in proportion to the crosstrack error which means the larger the error the more you're willing to turn towards the target trajectory and you can see that this works as you get close the trajectory your steering they'll be slower and slower and you will reach this directory can do the other two answers I really bad a constant steering we put you in a circle and not on a straight line and random steering if you ever implement this is a really bad idea and believe me we accidentally did this once it's a really bad idea alright okay so so now we know we want to steer in proportional crosstrack error that one seems like a little bit of a gimme right I mean it wasn't the toughest question anybody's ever faced let's try something that's maybe a little bit tougher so what you just learn is called a P controller where P stands for proportional and here is a really tricky question which I want to test your intuition one that doesn't have a unique answer but it has a best answer suppose you do what I just said you steer in proportion to the cross leg error that is history angle is proportional by a some facts of town to the crosstrack error what will happen with the car it never quite reaches the reference trajectory it overshoots or either can happen all right so we've decided we're going to stay on proportion of the crosstrack error and if we do that do we think it's never going to quite reach that that line that reference trajectory so it's going to overshoot the reference trajectory or it can go either way so raise your hand if you think it's never going to get there to be like asymptotic okay so we have some takers on that raise your hand if you can get overshoots all right a couple people raise your hand if you thing you can go either way all right interesting lots of people can go either way so let's see what the right answer is in actually overshoots the problem is no matter how small this constant is over here they would eventually return as wheels quite a bit towards this trajectory then it will move towards the directory more and more and when it hits it it will also be straight but the robot itself will still be oriented a little bit downwards so it's forced to overshoot what this means that applies to a car ap controller will act like this it'll slightly overshoot that could be oke we're shooting is very small but it never really converge it'll be what's called marginally stable or often just stable in the literature okay so that was maybe a little bit counterintuitive but that's actually the the fundamental concept behind what's called a PID controller which is the most common type of mechanical controller the P is for proportional so we're steering in proportion to the crosstrack error and then how we deal with this this oscillation here it's called the derivative part so that's the D in the PID controller there's actually a third part di which is the integral part and you learn all about all of those if you take Sebastian's free app or a lot of course and you'll learn about them a lot more along with a lot of other mechanical controllers if you take the Udacity self-driving car nano degree so this is all great we can you know watch videos kind of think about some of the concepts answer quizzes but we still really know like how do we build this PID controller and this is a big focus of Sebastian Sebastian has this saying that he he loves to repeat I've heard it a thousand times you don't lose weight by watching other people exercise so the idea behind that and how we apply to NASA T is that in order to learn how to build autonomous vehicles you have to write code and so the next video will show you a little bit about how we do that so I want you to implement such a controller so here's the Cortez procreate from you there's a class robot with which you're familiar it has an init you could set the position using the function set as before there's steering noise and distance noise you familiar with it there's also something called drift which we won't use right now but later on it will become handy and there's your move command all the way of amusement is before and have improved little bit the printout of the coordinates using floats I want you to implement the run command which takes as input the control parameter that governs the proportional response of the steering angle to the crosstrack error the Vogel has an initial position of 0 1 and 0 at speed of 1 and I wanted to simulate this for 100 steps so here's what I envisioned to happen your robot is initially off the x-axis by 1 I wanted to drive along the accesses so the Y value is the same as the crosstrack error by turning University proportional to the Y value using a parameter tau that sets the response strength of the proportional controller I wonder what were to turn towards the x-axis drive in that direction overshoot turn around and drive back to do this simulate the word for a hundred steps and use the proportionality term that sets my steering angle alpha in proportion to the crosstrack error Y so enter your code here and when you are done with it and you violence with the coefficient 0.1 here is the output I want you to produce it's a hundred lines you can see the bollard position starting one off and Y it then reduces Y over time to go on to negative certainly on the right side you see the store corresponding steering orientation and you can see as you move on the Y coming back into positive territory and you can see in other words overshoot slowly around the reference trajectory of the x axis so please go implement this cool so I'll stop there I won't make everybody get out their computers and actually implement that yet but that gives you kind of example for how we do education at Udacity a lot of quizzes a lot of programming exercises on big projects at the end that helped you put all your ideas into practice and build a portfolio of of a terrific work that you can take to automotive employers who are looking for candidates to hire who can do these things so the next thing I'd like to do is actually jump out of the slide presentation and show you some of the work that our students have done because I find it really exciting when students write up the work that they've done and talked about the interesting ways they've approached their projects and I think you all might be interested in what some of the students have done too and if some of these projects interest you if you move the nanodegrees out there feel free to sign up and you can learn how to do these things too so I'll start with a project by a guy named Joshua ome Joshua is a Nigerian student who is a PhD candidate in at a university in Japan it kind of illustrates what we talked about at Udacity as far as democratizing education we want to make the world's best education available to people all over the world and we find it really neat that someone you know on the other side of the world and from a whole other side of the world it's the opportunity to work on these projects with us so I'm going to pop out here and we'll we'll look a little bit at Joshua's project so Joshua's project has to do with behavioral cloning George Hotz talked about because the high-level concept of behavioral cloning earlier in his talk it's the idea of taking human driving behavior and feeding that data into a deep neural network and training the network using that to train the network to drive a car so we do this in the simulator we built this simulator that you see on the screen right now it's built kind of like a video game I'm going to demonstrate a little bit of a sample of how it works and then we'll look at Joshua's submission so this is our simulator you drive it just like a videogame there are a few different tracks you can drive on if we press the record button up here it will record our driving data and then we can feed these camera images the driving data basically consists of simulating cameras that sitting on the front of this car we can feed those camera images and steering angles and other data into a deep neural network and train that network to drive the car economist Lee so what I'd like to do is pull up here really quickly Joshua's project so Joshua wrote a really cool medium post about doing behavioral cloning on a simulated track and so as I said just walk through with you a little bit of kind of how he approached this project there are a lot of different ways to approach it but he talks about first taking a bunch of different networks so he talks about this Nvidia paper you saw Danny Shapiro talk this morning about their self-driving car they have a paper they published on the neural network they use to drive their car , AI so that's george hotz this company they've published a paper about a neural network that they've used and then there are other people out there the vectors another one of our students who's published some interesting models so what Joshua is doing is he's taking all these different networks and he's running his data through these different networks to see how the different networks perform so first how does he get the data well he goes through and he he drives around the track he collects a ton of data he visualizes it and so this is this histogram is what the data looks like so what you have is this big spike here that that is the car driving straight right there's zero steering angle the car is just going straight and then you have like a little bit of the car turning to the left and why is this it's because it's a counterclockwise track so the car is driving around the track it turns left a little bit but most of the time it's going straight so the problem is you feed this data into the network the network basically just learned to drive straight all the time then you turn the network on the car just drives straight off the road no good so Joshua does some really cool things as far as choosing images from different cameras the car access three cameras mounted on it he jitters the steering angles a little bit he randomly flips them and so he says if you have a camera shot that that says turn west if you see a camera shot like this he just flips the camera shot and says if you see a flipped version of this trend right and that helps the model generalize better so that in the situations where the model does need to turn right where the car does need to turn right the car can figure out how to do that Joshua does some really neat things with color spaces and adding brightness and shadows to the images and you can see the histogram he winds up with at the end is a much more balanced histogram so this is a lot of data the car driving in a lot of different directions and this means that the models going to generalize really well it's not going to learn to just drive straight it's going to learn to drive straight and less and right depending on what the appropriate behavior is then Joshua talks about diving into the Nvidia model and this this image right here is a screen shot straight out of Danny Shapiro's Nvidia model this is the model that Nvidia used to drive their bb-8 vehicle I mean he implemented this in code we teach you know how to do this as part of the course Joshua implemented it and and this is I should go back and explain this is what's called a convolutional neural network you feed data in at the bottom here these are camera images you perform some what's called normalization on the under images and then you you run filters over it and over the course of seven or eight layers here the car figures out or learns whether to steer right or left or straight depending on what the camera image shows so then Joshua tests his model so this is Joshua's neural network driving the car autonomously in the simulator he has some pretty neat music kind of underlying it I'll turn that down for a second and you can see this is not Joshua driving the car this is the car driving itself you can actually tell us if there's this little white dot up here that's when the cars own autonomous mode and this is pretty cool the car looks pretty good he's built and trained in neural network that that drives a car in a simulator and then the really cool thing he does is he takes the second track so he's trained this car to drive itself purely on that first test track that you saw so there's a second test track that looks a little scarier and he puts his model on this test track and lo and behold the car drives itself and that's really cool that you could train a car in one environment are a network to drive a car in one environment and then you put the network in another environment you've done such a great job generalizing the model and showing it how to handle different scenarios that the car drives itself in a different environment so that's just a sample of one of the projects what one student did for one project in the Udacity self-driving car manner degree program it's a pretty cool project we're very proud of it but we're really really proud that our students have done such amazing work and tackled it in so many different ways that's the show if if you're interested we're always happy to talk to folks come up and say hello later and if you're hiring autonomous vehicle engineers we have lots and lots of students who'd love to work for you so please come talk to me about that as well thanks so much all right thank you David hi we apply do one question so do right here and then not the move online I'm actually baking this cook now very good books and lands nation's biggest home book on traffic sign recognition and you use German and database do you know exactly the u.s. tactics and database yeah that's a good question so this is a kind of specific question of the course one of the projects that we have in the course is to take a traffic sign database and build a neural network to classify the traffic sign so your network should be able to say this is a stop sign this is a yield sign this is a 60 mile an hour sign we use the German traffic sign recognition benchmark data set because that is the widest publicly available data set there is not an easily accessible US version of that data set there are a lot of proprietary versions that different companies have I think if I remember right it's been a while since I look if you go online you can I think find some that you can use yourself to to train networks on US traffic times but the licensing was such that we couldn't pull them into the course and distribute them to students so we used the German traffic sign data set because they had really nice licensing and we could share it with students and everything was was on the up-and-up cool we can do one more question I enter some Udacity classes before and sometimes they require like if you're working is two hours a day of study that they recommend is that the same for this class and also what are their prerequisites yeah great question so the the prerequisites are intermediate level software programming ability ideally in Python or C++ because those are the two languages of the course but if you're familiar with another language you can probably make that work as well as basic linear algebra statistics and calculus so basically matrix multiplication integrals derivatives things like meme mode standard deviation you know the goal is not to take people from nothing to become an autonomous vehicle engineer but the goal is to take people from being like what I was a web software engineer and helping them become autonomous vehicle engineers as far as how much time to spend in the course we have targeted about 10 hours a week what we hear from students as students are spending about twice that much time in the classroom and so where well you know they're very dedicated is really amazing how excited students are about this and how much effort they're putting into it we don't want to burn people out so we're doing things to to to make the course a little bit less intense while still retaining a lot of the knowledge that students need to learn I think you can benchmark somewhere between you know some around fifteen hours a week is kind of I think where we're probably going to wind up as far as working on the course [Applause] you