ai.bythebay.io: Jur Van Der Berg, Technical Challenges of Self-Driving Trucks
Recording: ai.bythebay.io: Jur Van Der Berg, Technical Challenges of Self-Driving Trucks
So my name is Jure van der Berg, this is Alexi Sepp. I am now a senior staff engineer at Uber, but I started at Otto. was a start-up in self-driving trucks, as you may have heard, and we were acquired by Uber. And I'm going to present some technical challenges that are related to self-driving trucks. And I do that with a short history of our start-up Otto, and I'll really give you some insight into what it is like to be working for a start-up like Otto on self-driving trucks, which is super cool. And I was there from the get-go, and it's been an amazing experience. So just to start where we started, this was last year, we were just with, well, you only see 10 people here or something, but we were actually with 20. Regrettably, you only see men in this picture
We also had definitely also a couple of women, but they were unfortunately not present at the moment the picture was taken. But this is when we started with a very small group of people, but with a very good group of people. Like all the elements of what makes a truck drive itself were covered. Like somebody was working on motion planning, somebody was working on perception, people were working on the hardware, etc. And we had experts in all of these areas, and that made us a very good group to start with. We were really literally starting from a house in Palo Alto, as you can see here. This was a little bit later, as you can see from the weather. And we had one truck at the time, and we started working
And we made super, super rapid progress, and that was fun. So just one month after we started, we were doing a figure eight on a parking lot. And that was really, really a cool moment. That was for the first time, we had all been working on our separate aspects of a motion planning, oh sorry, of a self-driving system. I work on motion planning, so that was my responsibility. And then that very day, it came together. We for the first time tried all our software pieces together. And of course, we were on that parking lot and nothing worked, and we had to subtract pi over two from some number, because we misinterpreted the angle and the sign, and how it was actually related to the north and all that stuff
But then we fixed all those little things, and then all of a sudden, it started working. And that was really a phenomenal moment. And that was just one month after we started a video of this. So what we do here, just to give you some context, is we have a figure eight on a parking lot, and that was just pre-drawn figure eight by hand in terms of GPS waypoints. And the only thing that we were doing is following that figure eight with a truck, self-driving with steering and gas. So of course, in the beginning, as you see here, this is where we started driving the figure eight. We were very cautious. We were able to set the speed of the truck and control, of course, the speed from the inside
So very cautious. Let's see if it works. And it worked. And yeah, that was for everybody, every one of us, a very exciting, exciting moment. So this video is a few minutes, because what you will see happening here is that, of course, it's fine that it works very slow, but of course you immediately want to start stress testing the system, right? You want to see what happens if you want to go faster and faster and faster. And hopefully the video will be fast. Now let me actually fast forward a little bit, otherwise it takes a long time. So by this time we have actually said, like, okay, let's speed it up a little bit
And you can really see that the cab is going to sway a little bit in the curves. And if you can listen carefully, you can hear the commentary of my colleagues right there. Yeah, so there we nearly lost contact with the floor, so we really stressed it. But it worked. And that was, of course, a really, really cool moment. Yeah, and then we didn't take it much further, of course, because it was getting a little bit risky. But so that's just from the outside, of course, looks fine, looks cool. But what is it actually from the inside? Oh, there we are
Those are my colleagues. This is what it looks from the inside. So this is maybe not as interesting in some ways, but in other ways maybe it is, because you can kind of see how our system works. Here you can see that figure eight, that set of GPS waypoints that we just drew by hand that actually correspond to some figure eight on the parking lot. And here, this is my colleague also experiencing that it's working for the first time. So what you see there is a green line in the picture. It's working. Oh, it's going to turn left
It never had turned left at that time. It never turned left. It never turned right. And then it gets a little bit annoying because you have these parameters that are very hard to observe, very hard to estimate. They have very unintuitive numbers and meanings to them. Right. And then then such a model becomes already less useful, but more precise. But but but you have to know more parameters to be able to use them for planning and for exact control of the of the truck
And this is something specifically you see with trucks because of the mass. With cars, you can get away with simple approximations of the dynamics. And here is an here's an example of of of of the effect of dynamics and the mass distribution that you have in the car that has a trailer. And this is a video of new hole. And this is actually very educational. So even if you ever haul yourself sometimes if you move or so between places, then you can see what happens in this case. Hey, is the video not playing? Oh, that's too bad. It's a fun video
Oh, there we go. So the weight of the of the trailer is now placed on the front and now it's placed on the back. And you can see the difference by entering a disturbance in the system. It goes completely crazy. Right. So that is really the effect of the dynamics there. So if you ever move something with the trailer, always put the mass as much as possible in the front of your trailer so that you have like a nice stable system. And and if you put it in the back, it's much less stable or unstable even
And that is really a dynamics that you can only model with forces that are still finding trying to find their equilibrium. And you can see the time scales. It takes a while to reach that equilibrium. But so yeah, so these equations are stiff. If you see it from a mathematical point of view, makes it also harder again to model. So these are definitely challenges that you see with with dynamics with trailers that you don't really have to worry about with with cars. And again, just a reminder, like we don't want to put any sensors on the trailer, right? Because that that makes it basically makes it less usable because you want to quickly shift trailers in and out. So you don't really have direct sensing of what happens to your trailer
So you have to all model that to have an idea what's going on. Yeah. So this is what you saw in the first video. This is snapped exactly at that moment where we yeah, we're pretty in a nice angle there on the parking lot. But one of the technical challenges interesting also you don't see with cars is that a truck is not one rigid body is actually two. You have the chassis that is with the part of the truck where the wheels are attached. And then you have a cap and the cap is suspended separately from the chassis. is also suspended to the with the axles
And then you have the cap that is suspended separately to the chassis. So your camera and all your sensors are on top of the cap because that's where you have the right vantage points. But that is on the second rigid body. And so you want to but you want to kind of know what the what the orientation and of the chassis is. That's what you what you want to control. So there is always this transformation between them and it's hard to estimate what actually the transformation is of your cap in the car you don't have it like there's only one rigid body in a truck you have two. So that makes it extra complicated to control and steer very very precisely. Because the sensors are are mounted
And you can also some shaking of the cap. So there is some feedback possibility there that if you start steering to correct for it will only get worse and right then you get into some loop that is that is that is unstable. So an interesting intricate detail that you will definitely that you definitely find the trucks and you don't have to worry about it with cars at all. And to conclude I have a video. This is like this very nicely produced video. This was about our Budweiser beer delivery. Is this playing? Sorry. Anheuser Busch is the largest brewer in the United States
We ship over 1.2 million truckloads each year. We are always looking for new innovations in technology. Autos trucks are the next area of transportation innovation. The driver would still be involved with the pickup loading the freight making sure it's secure in the back of the vehicle. And then once you are on the interstate one switch and it's driving itself down the road. Well, auto technology is all about making a road safer. It's like a train on software rails. And so when you will see a vehicle driving with nobody in it, you'll know that it's very unlikely to get an inclusion
I proclaimed to one of the technicians. I said, I don't think I could have done that better myself. I said, I'm not sure if I could have done that. Thank you very much.