ai.bythebay.io: Wolfgang Juchmann ,Autonomous Vehicles 101
Recording: ai.bythebay.io: Wolfgang Juchmann ,Autonomous Vehicles 101
Thank you very much. Oops, looks like the speaker is on. Now we need the slide there. That's great. Obviously, I'm honored to be here speaking after George Hotz. It's a little tricky, but I'm sure I can. still interest you. George made it sound like everything is almost solved already
Well, for him, not for everybody, and that's where we come into the game. What I'm going to speak about is essentially a summary of all the things that you have heard today, all the different elements that make a self-driving car, and I want to give you an overview. I'm not trying to sell your car. Autonomous stuff is there to help companies to get to autonomy faster, and that's what we're about. So I'm going to speak about different kinds of autonomous applications, different vehicle platforms, like the Udacity vehicle that Kevin Devin spoke about, also the NVIDIA car that you saw earlier this morning. We built that car for NVIDIA for Danny. I'm also going to speak about perception and positioning hardware that most people use on the car. And then about data fusion, how to get all this data come together and make sure that that works together
And then, of course, the fun part is the control algorithms. That's where the artificial intelligence comes in to actually drive a car autonomously. But before I start, I want to give a quick introduction about myself and then also about the company, and then we get to the real fun part. I have some videos at the end so you don't rush off to lunch early. I'm actually born in Germany. I got a PhD in physics, but somehow I ended up in technical sales. I moved to the Silicon Valley in 2001, so 16 years ago. And the last four years, I was at Velodyne Leida, which a lot of you know from that spinning thing that's on the Google self-driving car and a lot of other self-driving cars
And since January 2016, a little bit longer than a year, I'm with Autonomous Staff as the Vice President for Sales and Business Development. I live in Santa Cruz with my wife, and it's fun to be here in the Silicon Valley. The company itself, Autonomous Staff, was founded in 2010, seven years ago. We have about 2,000 customers worldwide. Obviously, they don't all have an MKZ or a car from us. Some of them have LiDAR sensors. Some of them have radar sensors. Some of them have other equipment
But we are aggressively growing. We tripled revenue and size of the company in the last year. And we have a headquarter. Our headquarter is close to Chicago in Peoria, Illinois. And we have two guys out here in Silicon Valley. And we have a representative in Detroit, our business development person there. And then most of our actual hardware stuff is done in Peoria. But we are going to open an office this year here, so we can also provide technical support and show our vehicles here in the Silicon Valley
There's about 25 of these vehicles actually out here in Silicon Valley. And we consider ourselves autonomy and hardware experts. So I mentioned that I came from Germany over here. And people ask me, so why did you come? What made you come to America? And I had this kind of thing in mind, Highway 1. I live in Santa Cruz, where you drove along Big Sur and you have an open, like an open-air convertible kind of car. And then I actually had to drive into the Bay Area the first time and reality hit me. And this is kind of what I'm experiencing every day. I tried to drive past the rush hour to avoid this, but sometimes obviously that's not avoidable
But actually, because of this kind of traffic, this of course is not all from the Silicon Valley. The internet is beautiful. But because of all of this traffic, people actually start doing other things while they're driving because they're bored to death. And they want to do other things. They want to use their time. And they do a lot of different things, which comes from the reality of driving. It goes into the dangers of driving because there are some things they shouldn't be doing, at least not until we have a self-driving car. So I think it's very obvious we need autonomous cars and everybody wants autonomous cars
And that doesn't mean in all cases. It doesn't mean you have to be driven all the time. You can choose to drive. If you're on the slide before on Highway 1, not that one. If you're on Highway 1 and it's beautiful, yes, you want to drive. You want to have the ultimate driving machine or Fahrvergnügen or whatever the slogans are. But if you're stuck in traffic, you want to be driven. You want to do other things
So that's why we need vehicles. So autonomous staff is here to help you. We help companies to get to autonomy faster. As I said, we don't build a full self-driving car. We provide components that you can use to make a self-driving car. And we are very well established in the academia. So a lot of the mainstream universities like Texas AM, Berkeley University, Udacity we mentioned, University of Waterloo, Michigan University, M-City. Sorry, University of Michigan, M-City and also of course Michigan State University
They all have one of our vehicles. And even last year when Secretary Fox made the announcement about the self-driving car rules, there were four cars in the background that you can actually see. Where's my mouse? See here. One of them we actually built. And all the other ones also had components from us that we provided to these self-driving car companies. So we're actually in a very unique space where we not just interact with OEMs and Tier 1s or on the other side with startup companies like some of them that speak here as well later today. We actually interact with all of them because all of them need automotive grade sensors. All of them need GPS solutions
And some of them even want to have a platform that they can use for self-driving. And there's actually a funny story about the Sebastian Troon one for Udacity. I learned and David Silver is here. He's going to speak a little bit later. I gave to about 25 people at Udacity, I gave a demo of our self-driving platform, a research platform. And they were all interested. And then Sebastian Troon himself showed up. I had never met him
So I was a little bit in awe because he's somewhat of the godfather of self-driving cars. And I knew he was going to start this nano degree program and was thinking about buying some sensors from us and putting that car together. And I said, well, why don't you get our demo car that we already have here? You can actually drive it home. I take a taxi home. I just need your credit card. And seriously, he pulled out of his pocket his credit card and gave it to me. Obviously, that wasn't the transaction. We had to do some paperwork
And he is a good negotiator. But after a week, he had his car. And that's the car you see right now. It used to be white. Now it actually looks blue. They put a nice wrap around it with the Udacity logo. And I used to drive that car for six months before Sebastian Troon is driving it. We also work with the commercial world
I mean, as I said, NVIDIA and Udacity. And also, if you went to CES, there was at least five of ours cars driving around. I can't mention them all publicly because some of them are confidential. But essentially, we work with all of them. And if you went to CES, there was Renaissance, NVIDIA, Hitachi Automotive, QNX, our own car. So they were all driving around using a platform from us that they used to develop their own algorithms. Whatever they do to add value to what they do, that's what they use our components for. So that was the quick introduction
So the different applications, let me start with step zero. When people say they want to do self-driving cars, you actually really have to think about what do you really want to do. I mean, there's different levels of self-driving. You can offer a shuttle service like Aura Robotics or Vaden Labs, which actually have changed the model now to Embark. There's Navia, there's Zoox, there's Utonomy, there's Uber. There's many of these shuttle services that want to get into that space. But that's, of course, very different than what Comma.ai is doing or highway autopilot that Tesla is doing. Because this would be in a city environment and the other one would be highway driving
There's, of course, the passenger vehicles. So all the OEMs in Tier 1s are working on self-driving vehicles. Or most of the time they're taking an incremental approach. They go from low-level autonomy to more-level autonomy to full-level autonomy. And then there's, of course, all the electric vehicle companies that all have an office here in the Silicon Valley. Like Faraday Future, Next EV, now called NIO, or Ativa, now called Lucid, Future Mobility. So all of these companies also know if they want to compete with Tesla, they have to be at least as good as Tesla and the autopilot. So they're all working on autonomous driving in parallel to making an awesome electric vehicle
There's the racing guys. So Roborace, people have heard about. They had their first race. Unfortunately, they also had a crash. But that's part of learning. They have to do these things to improve and get better. And I think that's actually a reason why people are in racing. It's not, well, one part it's fun
Second part it's advertisement. There's sponsorships, obviously. It's very public. But the other part is you're testing the limits. You're driving at a really, really high speed that normally people won't do. And if your car survives that kind of environment, you bring that one notch down. And you know that in the normal kind of environment, the car might also work. Then there's trucking companies
Otto, Peloton, and I have to move my Vardhan Labs is now in Bach. That's also part of the trucking company that just recently became public. And then there's special purpose cars. I mean, a lot of people ask me, when will self-driving cars be coming? Well, actually, self-driving cars are here. Well, not here, but in Australia in the mines. There's hundreds of mining trucks driving around completely autonomously, holding dirt from place A to place B. And there's no driver inside. And it's all happening autonomous
So they are here. But usually in a very limited use case or in a limited geofenced area, they are already being deployed. So I guess the question about when will self-driving cars come for everything, for every application, for every use case, I don't know exactly. But they are here for certain use cases. And the number of applications expands day by day where this is being introduced. And then there is, of course, also the Internet of Things, the Googles, Baidu, Alibaba, Microsoft that want to play in this space. Most likely for the Internet purpose of it, for the operating system, not necessary to sell cars. However, in the Google case now, they have founded YMO
So they're kind of doing both. Based on your application, you now need to make a decision on the platform what kind of vehicle you want to base your vehicle on. And it needs to be a car that you can control by sending computer commands. And that's really the key. And that's, of course, something that most of the OEMs don't want to give out lightly. Because when you open up a platform for research, there's always the danger that somebody misuses it or accidentally does a mistake. So you open it up for research. But on the other side, there's also a certain responsibility that goes along with that
You need to make sure that it's fail safe. And the different kinds of platforms that are out there, there's the golf carts. We offer a solution based on the Polaris Gem, which is an electric vehicle. It goes 25 miles per hour. It's street legal on California streets. In some other states, you have to register it with a DMV. Of course, you can't do highway automation with it. But you can test a lot of algorithms with it, driving at 25 miles per hour
And if you do get in an accident, the consequences are a little bit less. And also the hurdle of entry, the cost point, is less than the real commercial vehicle. Then there's the traditional vehicles. And I'm going to show some images of that shortly. And then there's the special purpose electric vehicles, of course. The advantage on electric vehicles is that a lot of times the controls that are already in an electric vehicle, they're already by wire enabled. You just need to be able to control them. But the steering, the braking, the acceleration on electric vehicles, a lot of times is designed that it's easy to control them by computer if you have access to it
While some of the traditional cars don't really have that. I mean, if you have a gear shifter, that's still a stick that moves back and forth from parked to neutral to drive. That's a little bit harder to control. You would have to put a motor in there for that. The special purpose vehicles I talked about, the Caterpillar mining trucks, things like that. And then, of course, if you're into racing, you need to develop racing cars. But I think we don't go into that right now. So two platforms that we offer
One of them is essentially what you saw, the NVIDIA car or the Udacity car. This is one from Hitachi. This is a Lincoln MKZ or Ford Fusion. What's really beautiful about this platform is that through a partnership, we have full access to steering, acceleration, braking, and shifting through the gears. Why computer commands? And it is fail safe in a manner similar to what George Hart said earlier. When you press the brake, it disengages. If the backup driver presses the acceleration, it disengages. If you touch the steering wheel and you take control of the steering, you're in control again and the automated system is disabled
You can activate it, of course, again. But it's very intuitively, if there's a danger situation, you have your hands close to the wheel, you just grab it and steer and you're in control. You don't have to press an emergency button. It's there for regulation purposes. But the much more natural way to take over as a driver is actually touching the steering wheel. And I told you the story about Sebastian Trun earlier. So, of course, before he bought the car, he wanted a test drive. So he drove around and I let him do that
So we were in automated mode and he touched the steering wheel and took over. And you could just see the smile on his face. Because when he started the whole thing at Google X 10 years ago, all of that needed to be developed themselves. So they had to hack into the Prius, that's the first vehicle. They had to drive algorithms. They had to drive the algorithms for the manual takeover from automated mode. And it wasn't there. Now we're about 10 years later
And he essentially can buy a car off the shelf. It's not completely off the shelf. But off the shelf within one week that does all of that that he used to develop. So, I mean, he could really see that this industry from the DARPA Grand Challenge days to now has really improved and things have happened that this is now available. So on the left side is the Lincoln MKZ or Ford Fusion. And on the right side is that Polaris gem vehicle. In both of these cases, you see there's a lot of racks on the vehicle. And that's on purpose
In these cases, these are for research purposes. So that's what we call our R&D style. We also make very nice looking ones like the NVIDIA one that wants to show it off at the GTC conference and at CES and events like that. So it depends on what your purpose is. I mean, if it's for research and development, you want to have the flexibility of the racks. If it's for showroom style, you want to show it off. You want it to look as much as possible like a normal car. As I said, I mentioned already the Polaris platform there
It also has racks. It has a Valadine HDL32. And we don't really care which sensors you put on there. It all depends on what you want to do. If you're like into artificial intelligence, you used to drive PX, you're probably very heavily camera-centric. So we can put cameras all around. You can bring your own cameras. We put them on for you
Or we can help you to put them on. Or we can just give you the vehicle and you put them on yourself if you think you have a special method of putting them on. And really the beautiful part is you don't have to develop this vehicle. You don't have to go and run these cables through the vehicle. You call us and we can give you a vehicle like this in four to six weeks depending a little bit on how many sensors go on there. And that's true. Four to six weeks. If you would do this yourself, it probably would take you half a year to a year
If you have never done it, probably even longer than a year. And you don't know what comes out of it. In our case, you can just call us and make sure that everything works. This is the overview. In 2017, we're also going to come out with a Polaris Ranger, which is an off-road vehicle. It's very similar to the JEM. And we need to make a few adjustments from the JEM to the Polaris. But about the middle of the year, that's also available
And this shows you a little bit that comparison that I spoke about between showroom style and D style. Left one, this actually does have Velodyne LiDAR sensors here, the VLP-16, nicely hidden in the bumper. But you don't see a roof rack. There's a mobile eye camera behind the windshield. There's a radar sensor in the front. And in this case, you actually see the sensors on the roof. But as you can imagine, it's much more flexible. You can put another sensor on there even after you purchase the car
Or you can move it around depending on your liking and really find the optimum position. I mean, in this car, if you made the decision, this is where the sensor goes, it's fixed. It's very difficult to move it. You have to cut another hole into the bumper. And then it almost looks like a research vehicle. So, yeah. This is just some more pictures of these racks for flexibility. So here's a radar sensor
We typically work with Delphi radar sensors. These are automotive-grade sensors that you get a manual and the DBC file with. We work with Continental. We also are talking to Bosch. There's Ibeo LiDAR sensors, Velodyne LiDAR sensor, Quantigi LiDAR sensor. I mean, that's what the name says. Autonomous stuff. Anything you need, any stuff you need to make your own self-driving car, you call us
This is a showroom-style vehicle where we integrated the Ibeo LiDAR sensor in the bumper here. I mean, we made a nice bezel. There's one in the back here and there's one in the front. And this is actually the NVIDIA car before it had its wrap. So if you look at this car, except for NVIDIA made it bright green, you would not have really seen that this could be a self-driving car. You really have to look twice to see all of that. And this is the car that drove around, actually without a driver, at CES. I think they had a remote e-stop that they could use if something goes wrong
But, yeah. To show you how this all works, I'm going to show a little video. Let me see where that is. Yep. Oops, that's not it. There we go. And this shows you a little bit how easy is the handover and how easy the control is, that we actually really have full control over it. Turn off the sound and speak to it
So, basically, one way to demonstrate us is we use an Xbox game controller standard. We press some secret buttons on the steering wheel and I can steer the car. This is a demo that comes with the car. I can turn on and off the indicator. I get a lot of the information from the canvas, like George said. There's a lot of information available. I can shift through the gears. So, I have to press the brake
There's the gears. It's in park right now. And now I press into drive. You see it shifts into drive. And now if I let go of the brake, the car actually starts driving. So, we tested that out with one of our friends from Renaissance. And he's sitting on the passenger seat. We did this on a very remote street where we didn't expect any other traffic
So, this is actually not something you should do at home. You should go to your neighbor's place. No, you should go somewhere where you can't hit anybody. And now you can actually demonstrate driving the car with a remote control. And this shows you really the full access. In this case, of course, it would be harder to take over because there's nobody on the driver's seat. But normally, we have somebody sitting there. This is more for show and tell
And somebody can take over the steering wheel very easily or press the brake or press the acceleration. Yeah. But the goal is not to drive the car with a remote control. That's really just to show you we can program it. We have access to all of these commands. The goal is, of course, to send computer commands to drive it. It's terribly, if you see these little buttons there for steering, if you drive a little bit faster, your car goes really jerky to the right and the left. It's very tricky to drive it with that
So, it's more of a demonstration mode. And you won't hear it right now because we don't have hooked up the audio. But at the end, when we stopped the car, I asked the guy, how was it? And it's actually on the video, but the sound isn't on. And he says, that was awesome. I mean, anybody who sits in that car comes out with a big, big smile around their face. Okay. Let me stop this. So, I think we have talked enough about the platform
So, the next step is, after you have decided what kind of vehicle you want, what vehicle platform you're going to use, is what are you going to put on that vehicle? Yes, you can do it with a camera and hack into the radar sensor, but there will be limited use cases, or you have to do a lot of learning to actually get somewhere. Typically, most customers actually rely on more sensors. And I want to give you an overview of what kind of sensors those are. Yeah. I obviously can't go into every sensor here, but on the sensor side, there is radar sensors, which almost every car already has. They are relatively cheap because they're made in automotive quantities. However, so far, it has been slightly difficult to get access to those radar sensors because the tier ones don't really want to sell them to one-off, to our startup companies. They actually want to sell them to the OEMs in million quantity
And that's where autonomous stuff comes in. We have an agreement, for example, with Delphi or Bio. We sell to the low-volume customers, enabling autonomy, so that when you go into full production, Delphi is already integrated, or Bosch is already integrated. And then, of course, they might take over the thing. But we enable the early steps. So we were already working with Otto, for example, back in February last year, when Otto wasn't even public yet. There were only 15 people in a house in Palo Alto. And when they made their first video of the truck driving around, if you look at all of the sensors, we helped integrating them onto that truck in those very, very early stages
Radar, LIDAR. is not at the price point yet where it needs to be, but people are working on it. The traditional LIDAR companies, the one I used to work for, they promise it's going to come down if you buy in automotive quantities, meaning a million a year. So, well, since nobody's buying a million right now, that's an easy thing to say. So we'll see. I think we have Velodyne's John Eckert speak later, see where they are on that curve to $250. Cameras, of course. Cameras are very heavily used for the artificial intelligence, together with the DrivePX
We have an agreement with NVIDIA. We also sell the DrivePX. Together with our vehicles, together with our perception kits. And, I mean, it does make sense. You get the DrivePX from somebody else, and then you have to send it to us. So we might as well get it from NVIDIA and integrate it into the car, if that's something you want to work with. We offer, we have access to the ultrasonics on the vehicle, and we also offer external ultrasonics that you can mount on a car. We offer GPS, IMU, and RTK solutions for exact positioning
There's different computing solutions. And one thing that a lot of people actually don't really think about, it was touched on the presentation before a little bit, what happens to all of the data that all of these sensors are recording. And people think, oh, yeah, I buy a 4-terabyte hard drive from Costco or something, and then I buy another 4-terabyte one. But eventually, I mean, if you're generating 4 terabytes a day, you're going to have a whole bunch of library of those hard drives. And if you want to access something again, you need to have a little bit more of a management system. And so we have partnered with Quantum Data Storage, which has an onboard solution, but they also have an easy solution on site where the data is being ingested. They have a very smart management of the data because the first 10 days after the data comes in, it's very hot. Everybody wants to do something with it and calculate and do algorithms
But then eventually, it becomes stale. People don't really look at it anymore. And it's a waste of your expensive real estate on a computer to keep it on cash where you can really access it fast. It's much better if you can outsource it to a space where it's not as expensive. And then if you do need it again, maybe a year later, you want to look at that one edge case, then you pull the data in again. So they have an automated system where you set up rules and say, after 10 days or after 20 days, the data is being moved over to that cheaper real estate. And if I want to, then I can pull it in again at a later time. Yeah, I mean, the other part of it is actually last week, there was a big outage on the Amazon cloud
I mean, people think they can use the cloud. Well, there's two downsides to that. First of all, the cloud is not free if you use it for commercial purposes. You actually pay a lot of money for the cloud and uploading and downloading it when you talk about terabytes and petabytes. And the other part is, well, what happens if the internet is down, if the cloud is down? Your data isn't accessible. So there is actually value of having a server rack in your facility and not having to send data back and forth, but actually have it in your facility. And that's part of the solution. This is what these different perception kits that we put together kind of see
This is just one example. To be honest, it really depends on your application, what kind of sensors you want to use. Highway automation, highway driving, forward-looking radar, a mobile-like camera, something like that is probably fine. If you want to do a shuttle service in an inner city where you might have a child at any corner of your car, you definitely need more sensors. So when somebody says, what sensor should I be using? I typically immediately ask, what do you want to do? Because it depends on what you want to do is which sensors you want to use. And this is just an overview. And to be honest, we have put these kits together. You can find them on our website, www.autonomousstuff.com
But nobody actually buys them in exactly this configuration. Typically people say, I don't want the two LIDAR. I want more radar. I want more cameras. I don't want this. And that's fine. We are there to help you to bring you to autonomy faster. So you tell us what you want and we put it together
As I said, positioning kits. On the high-end side, we work with Novotel mostly. They have a thousand different options. So we put together these kits that are called good, better, and best based on what other customers have bought before. And that, of course, means how much accuracy you want. And actually, the big differentiator between the different ones is not so much on the GPS side. The big differentiator is on the IMU, the inertial motion system, that tells you when you don't have GPS, how long can you still predict reliably where you are. And if you have a fiber optical gyroscope, a fog, you can predict that much longer than if you have a MEMS base for a few thousand dollars
So these solutions range from ten, twenty thousand dollars up to eighty thousand dollars. Everything we sell is not ITA restricted, so you can even export it to Korea or China. That's not a problem. But that's the main difference between these solutions. We also offer a solution from XSense that's in the thousand dollars range. That's MEMS based, but it doesn't have centimeter accuracy. It has about meter accuracy. So that's kind of the differences
I talked about different computing platforms. We had Danny Shapiro from NVIDIA here this morning. The DrivePX2, DriveWorks, clearly one of the leading ones. However, we also have worked with Renaissance AKH3 system. We have an Intel i7 quad core that we can put in our vehicles. And the NXP Blue Box, we have experience with that one as well. So whichever one you prefer, you can let us know. And we probably have experience with one of them
And we're not fixed on one of those platforms. However, the DrivePX is awesome. On this data storage, already mentioned, we work with Quantum. What you see here is the trunk of my car, which is actually outside, but unfortunately on a parking lot three blocks away, because this is not the perfect area for demonstrations on vehicles. But what you see here in the trunk is a typical rack. Maybe not as messy as here. This was right after we installed it. But normally all these cables are hidden
But what you see here is a compute platform, a power distribution. And then this rack, this 19-inch rack is 32 terabytes of data storage. And this could be, this is only half populated. This could be up to 80 terabytes of data storage that over a fiber-optical cable go into here while you drive and while you record things. And then when you come back to your garage, you connect another fiber-optical cable to your server in your garage or in your network center. And then you upload all of that data into your network center. We also offer engineering services, even if you don't buy our platform. If you have your own car, no problem
We can come out or we can bring the car to us. We install tons of cables into these cars, Ethernet, CAN, and USB to go to all these sensors. As I said, we make nice integrations where the sensor is hidden in the bumper. Here's one in the side view mirror. Once it's done, everything looks clean. We even protect it while we transport it. So if there would be some rocks that it still stays nice, that's also available. Next step is what do we do with all the data that comes from these different sensors? We have to fuse the data together
And clearly every sensor comes with its own software. But what I mean by data fusion middleware is there needs to be a way of fusing this data into the same time space, into the same 3D domain to be able to do something with it and make real-time decisions. And there's a few different options for that. I mean, most of the OEMs or university labs have their own custom data acquisition system that's kind of homegrown, organically grown over time. Maybe somebody wrote a data logging program for a radar. Somebody else wrote something for a LiDAR sensor. But they might not be there at the same time. And the challenge then is, of course, combining those
Because now it's not just an AIDAS radar feature anymore or a LiDAR feature anymore. Now you need both of these things together at the same time. And a lot of companies are actually thinking about rethinking what they have grown organically and actually going over to a middleware platform. So the next most common, I would say, is ROS, the Robotic Operating System. There's a ton of drivers around. It's open source. It's supported by autonomous staff. We actually started maintaining drivers, take responsibility for the maintenance of the relevant drivers that are used for ROS
And when you get the car, when you boot it up, you immediately see an Avis model of all the data from the sensors that's on your car in real time. And if you want, we can configure that for you so that when you pick up the car, all of that is immediately available and you don't have to start programming ROS or installing ROS and measuring where the sensor is located. We can take that for you. Again, just to make things easier for you. There's also PolySync, which is not an open source platform. It's a for sale platform. We work very closely with the PolySync folks. They, as I said, they are useful specifically if you can't use open source when you want to go into the commercial space
There's also a path towards production environment, but I believe the speaker after lunch is Josh Hartung or somebody from the PolySync team. And I'm not going to go into too much detail there. I let him introduce what PolySync is up to nowadays. And then just for completeness, there's also a French startup company called RT Maps. They do something similar. And then there is ADTF, which is mainly used by the German European automotive companies. It's actually very well established. I think it's about 10 years old
The challenge is it's a little bit old. It's 10 years old. So people really like the more modern concept of ROS or PolySync. In that respect, eventually, it will probably be replaced. However, right now, everybody uses it. So it will take a little while to get it replaced. This is a screenshot of PolySync. ARVIS on ROS looks very similar
So what you see here is essentially several sensors fused together. The circles there is the Valerian LiDAR sensor. You see the yellow line is from the mobile eye recognized, the lanes on the street. And then you see these little writings here, which say something like car one at 50 meters. And that's recognized objects also by the mobile eye camera or by an IB or LiDAR sensor. So all of this data is fused together in this middleware. As I said, same on ROS or PolySync. It just depends on is your company, does it allow you to work with free but open source, or do you need to be on a commercial site and you can't really work with open source
And then the last part is obviously the fun part that the automation algorithms, we currently don't really sell on these algorithms. We have developed them. We are looking right now for the right partner. And again, the goal is not to solve the world's problems. The goal is actually to help you to get there. I mean, we developed some of these underlying algorithms, for example, smooth steering, smooth acceleration and braking. So why should you, if you get one of these vehicles, start completely from zero? You could partner with us. Find a commercial agreement and then you can start at 60%
You don't have to start at 0% and build up on what's already there. So let me show you some of those software algorithms. So what we did is a modular approach, meaning we kind of framed a certain problem, for example, what we call shuttle automation, going from A to B on a parking lot or something, and have developed a software for that. We have lane centering based on the Mobileye camera. And I guess I should say the Mobileye camera that we have is an aftermarket product, but it does come with an extended feature set, so you get access to all the objects that the Mobileye recognize. So via CAN, you can read out all the objects, the object classification, the lane modeling, and also the headway display indicator and the traffic signs on a highway. All of that you have access to reading that out. And that's nothing we have hacked
This is actually in partnership with Mobileye for the aftermarket, not for OEMs, but for the aftermarket world. We have a traffic jam assist based on radar, looking forward, slowing down if the car in front of you slows down, accelerating again. And then on the sensing side, we're also developing, that's not completely done, but we're developing LiDAR-based algorithms for classification. And that's basically, and also vision-based algorithms. So I think I will show you some of these algorithms. So this is a screenshot of the shuttle automation where we basically use GPS to drive around one time, record where we go. We record the breadcrumbs like Hansel and Gretel, and then the car afterwards drives itself and follows that path. We can put stop signs in there, speed bumps, those kind of things
And then the car will automatically stop, wait for the input from the driver to continue driving. And again, there's a limited use case for this. At this point, there's no object detection part of it. So if a pedestrian steps in our way, the backup driver has to take over. But that's our way. That's why we call it modular. That's one module. By the end of this year, we're going to take all of these modules and put them together
And then we have an automated vehicle. But at this point, we're working on these different modules before we combine it all. And again, even at the end of the year, it's not going to be a perfect solution that everybody can drive on every street with all these modules. It's again, it's probably at 60, 70% level, not at the 100% level, because that last 20%, that's really, really hard. And we're there to start you up, to help you to get there. And then you can add your value to actually finish it with whatever tricks you're using. I'm not going to show you the one actually on the street. I'm going to show you the one on a racetrack
And that was a lot of fun. So basically, we went... Let me get out of here. Sorry. These are not actual movies. These are pictures. So this is an event that was partially sponsored by NVIDIA and some other companies. It was last year Memorial Day weekend at Thunder Hill Racetrack, which is a little bit north of Sacramento
And you can see here the car driving around. As I said, first time we drove around just recording the GPS waypoints. And now this time, it's driving around by itself. Because of all the GoPro cameras, you can see there was a lot of media attention. You don't really see very well out of this view. So I'm going to switch over to an overhead view from the top. And that's this one where you can actually see one of the webcams and you see where we have been driving. Let me make this a little bit larger
So there's the path we're driving. This is autonomous driving. No hands, no feet, mama. This is self-driving. The professional driver that was on the driver's seat as a backup with the helmet, he was a little bit nervous at the beginning. But after 30 seconds, he actually started chatting with the engineer that was in the car. And he didn't almost notice that the car drove itself. And you can see the speeds here
I didn't stop it at the right moment. But on the straight, we actually went up more than 60 miles per hour, 100 km. And that was full autonomous driving. And what's most noticeable is how human-like the driving is. The steering is very smooth. It's not like if you have a 15-year-old that learns to drive and he goes right and left or forward and backwards. It's none of that. It's very human-like driving
And when we come to a very tight curve, because we know the waypoints, it knows what the lateral forces will be and it's going to slow down not to exceed certain lateral forces. Like here in this case, it went down to 20 miles per hour. And on the other side, it's slowing down again and then it's accelerating again. I have one more video. And then that's the one... Hang on. Oops. I kind of all went everywhere
So this is one that's more related to highway driving. So this is using the mobile eye camera. As I said, very limited module using the mobile eye camera, looking at the white lines on the street and making a smooth algorithm to stay between the white lines. At this point, if there's no white lines, it's not going to work. You need a different sensor to compensate for no white lines. Very clear. But there is algorithms that are behind this that you need to program that we could make available already for somebody to use. So let's see this
So we're driving. This is at about 50, 55 miles per hour. It's detecting the lines. There's no steering involved. As I said, we could take over the steering wheel at any moment. But we're going to see this one curve. And again, if you ever have been in a lane keeping assist, the early versions of that is a little bit like playing ping pong. You go to the right lane, you go to the left lane, You have none of that here
It actually stays very smoothly between the lanes. And as I said, those underlying algorithms could be made available for the right terms that we have. And I think my time is up. That brings me to my very last slide, which is, that's what we're here for, fast tracking autonomy. And that's what we all get, to get the hands off the wheel and start doing something else, like playing the trumpet or eating hot dogs like in one of the first slides, because that's what we really have been waiting for. Thank you.