Scale By The Bay 2018: Roman Shaposhnik, Edge ain't your gramp's IoT...
Recording: Scale By The Bay 2018: Roman Shaposhnik, Edge ain't your gramp's IoT...
you so thank you much for coming I know it's the last talk of the entire conference so I really appreciate you being here I also have absolutely no clue why and I ended up in the data track because this talk has absolutely nothing to do with data big data analytics or anything like that and in fact I think you know the fact that people associate iut with sort of data is a bit of a you know throwback to like mm I don't know 14 or 15 I think there's much more interesting things are playing out on the edge you know when it comes to operating systems and that's actually what this talk is all about but before before I actually explained to you you know what we've come up with let's try to take a look how we actually get here right so in the beginning you know computers were really big right and awkward to use you know cause really a lot of money and then you know they sort of got smaller and you know more affordable but they also split right so the thing on the left is IBM 360 you know something that you would use for enterprise computing and the thing on the right is basically a panel that is actually a computer you know part of it is analog so it's an analog computer you know in part that it's actually a computer that was installed in the Soyuz space you know spacecraft right to basically calculate trajectories and make sure that you know the rocket doesn't fall back to earth still a computer it's a very different computer compared to this IBM 360 system on the left and then you know came the PCs you know the PC revolution so again on the left for the business user we had a spark station and on the right we started to see PLC's you know programmable logic controllers being deployed you know everywhere factories you know trains you know buses you know you name it right you know basically again we sort of still have that split now finally came the cloud right so now if you do any kind of business computing you probably go back to Amazon or Google or you know Microsoft cloud and you do all of your cloud computing there which is kind of like what it used to be in the mainframe era but on the right we sort of a still stuck with embedded system and there's actually not really that big of a difference between a PLC controller and an embedded system you know let me tell you and even today and in most of the wearable devices you know most of the stuff that's around us you know factories again you know all these things they still designed as though they're in a minute-- system so is there actually a problem with that well I submit to you that today is embedded system architecture is basically predicated on the fact that those small computers are supposed to be really small right if you know we designed for basically machines that have you know just a few kilobytes of RAM you know it didn't have really much of a persistent storage to speak of you know had really slow CPUs and that was you know that used to be very true the problem is that because of the economy of scales you know because of the mobile devices a lot of that has changed so now it's actually cheaper to get a really powerful you know arm 64 CPU then try to go back and do some kind of a z80 you know microcontroller units right just because of you know the economy of scales and you know how much of these things are produced for the cell phones you know the cost factor definitely drop down so the only thing that should stop you from actually using pretty much you know general purpose CPU for the embedded side of things is probably power consumption so it still consumes more power than you know a typical sort of wearable CPU would consume but again that is a very very narrow domain of what embedded systems are used for so like all that stuff you know that these controlling robots at the Tesla plant and it happily no reason for them to be under power it you know the way they are so the question then becomes okay so if we can afford you know smarter and bigger CPUs what can we do with them well the problem is that given the today's architecture of embedded systems not much because all of the embedded system design again has been predicated on essentially a very purpose-built system where there was no really reusable components you know that much or even if they have the reusable components when you built an image it's not like within that image you can basically upgrade components independently like we used to you know we buy now used to do in the cloud so it's not like you can just upgrade you know the operating system or the middleware or the application so basically with embed it you always building the stacks you know this fully integrated stacks and the trouble with that of course is that in the terminology of cloud people you know all of these things have pets right you know you know how in the cloud there is difference between passing cattle you want everything to be careful because it's just much easier to manage and in the cloud we used to have pets and unfortunately in the embedded side you know they still stuck with pets some of them are nice pets you know some of them are you know weird pets and some of them I actually downright evil pets because you know once that pet gets compromised with you know something like important how the hell are you you know it but supposed to patch it right it you know it's like it's not like you can upgrade your OpenSSL to a different version you actually have to upgrade the entire system so that's you know that's a big deal but maybe the systems that we use for embedded you know again all of these you know things controlling you know industrial equipment maybe they are fine like that right you know maybe that's just a single purpose-built system and you don't feel is supposed to even connect it to the Internet so if you don't connect it to the Internet maybe we can still survive with them well I submit to you that we cannot and the reason we cannot is this trend that actually started in Germany towards industry 4.0 I hate to say but I think you know Germany in China is way ahead of everybody else on the planet in that but basically these guys are talking about you know how can you go from essentially sort of this type of a factory that was given to us by a third Industrial Revolution to more of an autonomous factory that is fully cyber-physical enabled right where you basically have systems that can adapt to a product that you're building right so the factory itself becomes you know software-defined and the only way you can do that is well you actually have to do two things first of all you have to connect all of these systems you know that are currently in data systems to the internet because you need to gather as much data as possible from the factory and that's by the way the only place where the data will be featured in this presentation right you still care about you know collecting all of the data points right and second of all because all of your factories are now becoming software-defined you actually have to have an ability to rev up the software that's running on these control systems everything time so let me give you an example I was actually talking to a customer who is basically into oil and gas and you know we were discussing in a sort of price of crude and you know all that and at some point he looks at me and says like I think you misunderstand what kind of business wearing and I'm like what kind of business are you getting both getting stuff from the ground and silicon on the market and he goes no that used to be true ten years ago now we're basically in business of reconfiguring carbon molecules and yes most of these raw source of carbon molecules still comes from the ground that is absolutely true but we in business of figuring out what configuration of carbon molecules can fetch the highest price on the market as of this given moment right so if it's you know maybe aviation fuel we will do that if it's plastics of some sort we will do that again we're still extracting these carbon molecules from the crude oil but we are no longer treating our refineries as a fixed one-time build product we want to have software-defined refineries and of course if you want to do that you know then you basically have to have configurable software to manage those things Tesla is basically building a conveyor belt you know sort of the factory pretty much the same way because they're trying to anticipate what kind of products their customers would want so they don't want to end up with the same style of factory that for them you know more classical automotive companies have where they basically predict that for next you know 10 15 years they will be building a certain model and then they just like go and build out those factories you know in different places you know depending on how they project the market to evolve and then they kind of stuck with them right you know it's like if that model if you know all of a sudden pickup trucks are not selling anymore well they're still stuck with the factories right so we have to transition to this new model where the factory becomes really software-defined in the only way as we know is to make sure that we as software engineers can actually manage software on all of the systems that are running in the industrial setting I mean there's different other applications I mean I can talk about your home age but I think you know this will be the biggest driver for the flexibility on the edge so to put it you know in a concise slide I think how industry typically goes through waves right you know we basically go between centralization and decentralization so enterprise computing in the 80s like I said was all about workstations which actually replace you know mainframes then came cloud computing we is all centralized on our everybody that's you know computing on this centralized resource in the big cloud and I submit it to you that by 2020 we will basically see the emergence of the edge computing and edge computing today to me means three things so it means autonomous so basically the system that runs on the edge doesn't actually have to talk to the big cloud in the sky all the time I mean it may but it doesn't have to second of all it means real-time so you basically have to be really close to the physical objects in a physical world and it has to be cyber physical in a sense that the behavior of this thing has to be software-defined so more specifically I think you know this is a helpful analogy so today all of the things you know people talk still about Internet of Things you know all of the things on the edge they basically have a very simple routing of the traffic and sort of communication right you know pretty much the game today is how can we get data out of those things into the cloud as quickly as possible you know MQTT and all the other protocols basically are aimed at like okay so we don't trust the edge edge is really inflexible because of all the reasons I told you about you know in terms of the how the embedded industry is putting things together so the best we can do is just fetch out you know all of the data point and just like do useful stuff with them in the cloud again I really believe that this is very inflexible to the kinds of applications that we will build a building on the edge you know especially for the industrial use cases so I submit to you that again maybe it's by 2020 maybe it's like 2025 you know we as a startup company really feel that it will be by 2020 but who knows a little transition to this picture at the bottom where you will see a lot of the meshing actually happening on the edge itself and yeah sure there will be some still communication with the big cloud in the sky the cloud is not going away you know don't make any mistake about that but you will see a lot of the data processing actually happening on the edge itself moreover you will see a lot of the AI happening at the edge itself you will see not just a I as an recognition you know sort of the training side of the AI but you also see the actuation side of the AI coming to the edge you know in a very very big way to give you an example so again the customer and other customer of ours you know they're basically managing traffic lights you know and pretty much you know all major cities in the United and you know other countries so today they basically have an ability for the traffic lights to communicate with the cloud right and that's that's cool but what they want to do they want to basically try different machine learning models that are local to the given chunk of the city right so they would like traffic lights to actually communicate between each other and based on the localized information figure out how they can behave as a cluster not as an individual traffic light so again for that you would actually definitely have to have something like at the bottom so the question then becomes can we actually fix the problem of embedded computing in incremental and let's let's let's run with this argument so maybe we can when you do embedded computing today you know when you connect things to the cloud you actually have to care about three things you have to care about security you have to care about management and you have to care about networking like that is literally what it means to actually be our UT internet-of-things right without those three you know you're not really either internet or another thing or I guess you're not off I don't know and in security I mean there are different things that you have to care about like runtime you know trust the compute PKI I mean management is sort of the same deal you have to care about management of the device itself and the application that's running on the device you have to care about overlays and VPNs and monitoring networking wise and honestly today you have startup companies in every single segment right if you know all of them are trying to integrate with some embedded Linux distribution or maybe you know something like you know our toss systems you know real-time operating systems there is no shortage of these companies right you know some some of them actually pretty big Cisco for example is a big player on the internet side of the of the edge but the problem is the problem with all of this is that it is actually to the customer very difficult to integrate it exponentially explodes the cost of you know actually acquiring the entire stack and it leads to this proliferation of really ugly architectures where something was cobbled together and it kind of sort of works but you know next thing you know it's actually used you know to mind Bitcoin and you know all of your sort of industrial infrastructure is now owned by you know somebody in a in a completely different country so we decided to do as you know whole bunch of sort of Exploud people we decided to basically fix it you know the way we fix it in the data center because if you think about it I mean Dana sentry used to be as ugly right you know in the data center everything used to be a snowflake you know we used to manage servers as individual entity it's like it was just like really really really painful and then we kind of decided that you know containers and virtualization and things like kubernetes and docker actually a much better way to do it and now we just treat our computer infrastructure it's just substrate right we don't feel it like honestly today I just care about you know producing my docker container and if the operating system that runs the docker container is made on Mars I wouldn't know right I don't care like the question of what is it compatible with like easier application compatible with a given up you know operating system that question stops being interesting to me the only question I'm asking is can that operating system run the docker container and then I'm off off to the races so if you think about it that way then what we are trying to do sort of our way of trying to fix this problem is we're basically trying to find what is the intersection between the cloud technologies and the embedded technologies and we're trying to operate in that space and if you look at it more from a business angle you know we're basically trying to see how we can make sure that ot which stands for operational technology and IT people can actually talk to each other in a meaningful way right I would say ot in general I mean people who basically care about HVAC systems in the building or people who care about you know physical security of this building rate this is ot or you know on the factory in a factory setting that would be like people who actually manage robots and you know industrial ethernet network right you know these are all tea people so I woulds I would say to you and again I actually say it lovingly I mean I don't diss them it just so happened right you know but it so happened that ot is now stuck behind IT by I would say about you know 10 to 15 years right so we actually anticipate that all of the cool stuff that happened to IT 10-15 years ago which was Amazon AWS you know cloud computing all of that is actually about to hit us in OT because that's the only way to manage you know the complexity that's about to hit those people so what wait to offer instead of the traditional embedded sort of world view of how to build your systems well obviously I mean there still has to be hardware and you know we have very cognizant of the fact that that hardware will not be as powerful as a you know data center machine you know that you buy from Dell or you know HP or any of those guys but it still won't be as you know low powered as what an embedded assumes today so it typically would be like at something again think about that same idea of like booty is stuck behind IT buy 10 15 years 10 15 years we actually used to have computers and data centers that already did cloud computing right and if you look at the specs of those computers it's kind of what you're actually about to find on the edge today so it's like about you know one gigabyte of RAM about one gigahertz of CPU power you know again you can basically divide this number you know by by half like that still kind of like the same ballpark right you know like terms of you know gigabytes of flash the cost of that system today is 5 bucks right and I'm talking actually a fully enclosed system if you're just buying an SOC it's a couple bucks so again just economically feasible to build the system that we used to have in the data center you know 10 15 years ago so what do we do with the system well we submitted to you that you know you basically abstract the hardware complexity away the same way we do in the data center so you actually put a hypervisor in and the hypervisor you know the virtualization is actually now possible because all of the major CPUs you know Intel obviously but also arm and soon risk 5 they actually support virtualization extensions and again think about economy of scale it's actually much more expensive for the CPU designers to take them out than to keep them in right if they develop them for a use case that is a data center use case it is now much more economically viable for them to just keep producing the most common denominator and not really looking to you know tweaking it to save you know a couple of cents right so pretty much all of the system that you see on the market today can support virtualization so you put a hypervisor in and then you're basically off to the races in the very same way that you know a cloud sort of computing platform would be built today so you basically put you know a few micro services on top of that and then you put applications right and one cool thing that you can do with the application is if today your Industrial PC just runs Windows you could basically do the same trick that VMware did you know back in 2002 thousand up to 2005 you can show up and you can say like well if you run you know this system we can just lift and shave that you know windows you know based SCADA application and it would run on the hypervisor as nothing has happened in fact it actually will be better off because I can guarantee you that it's probably running on something as old as Windows XP sometimes Windows 95 you know in most of the industrial settings so I can at least you know protect it from the firewall perspective right you know because on the hypervisor I can fully control what it can and cannot talk to I can fully control how it use resources in fact you can actually upgrade your hardware because I can bet my you know ten bucks that if you buy the new hardware the old Windows XP or Windows 95 won't have drivers for it you know but with my hypervisor it's just the back end of my virtualization system so I can still mimic you know whatever the system used to be the you know when you install it but of course again we all know that in the cloud you know it started to be that way but then we very quickly transition to containers and you know there's a very quick transition to lump there so we don't want to spend as much time as cloud spent on sort of figuring it all out we just want to bring all the same technologies to the edge and you know then of course because it's all virtualized you can now move applications between devices so they are no longer coupled you know siloed stacks you know the application is truly detached from the middleware and from the hardware as well so you may be wondering I mean what systems are we targeting right so again what I'm talking about is not some kind of a pie in the sky we actually have a startup company with real customers so the majority of our customers actually run a box like that I mean not necessarily exactly the same a box but pretty much the same sort of cost you know point so this is known either as industrial PC or IOT gateway it's typically based on Intel I mean people can you know like to assume that there is a lot of arm out there that is actually not the case arm is not really present as much most of these machines in the industrial setting is still x86 so the cost point is about five hundred bucks it's typically like I said is running you know some windows-based SCADA application and which is showing up and we are making in fertilizers right now because a lot of our customers are King as like well okay but you know parm is going to show up really soon and we also would like to transition to a much you know lower sort of cost point so the smallest system on which this quadrant is a seven bucks orange pile so it's basically a CPU from Huawei you know four-way subsidiary called high key you know high silicon so it's kind of like the same as within high keyboard and you know basically it's very very similar to what you have in your Samsung phone right you know definitely the same as you know you have a new hallway phone although I don't think anybody in the United States actually has far away phones I do but not a lot of other people's do so you can basically go as low as seven bucks and that's that still runs if you look into how these we call it cloud native IOT edge computing platform is constructed we basically uh just to summarize you know what I kind of talked about but just to kind of like make it a little bit more real for you so we're definitely looking into decoupling device lifecycle management from the application lifecycle management so in the marketplace you now have the same story as you have with servers in the datacenter right you know basically you no longer have to give whatever it is that you will be running on the system to the actual manufacturer of the system because imagine how stupid it would be if you as a data center guy would have to talk to Dell people to make sure that they pre install all of the software that you would be using in your data center that is literally what is happening today with edge and are UT people you know in the industrial setting they actually have to go to manufacturers like you know Super Micro and advantech and you know Avnet and they have to tell them what is the exact stack stack of software that needs to be pre-installed on those systems so what we are doing we're basically decoupling that so you can actually ship blank boxes you know just pre-installed with the virtualization layer that we are working on and then you can decide what applications you would be running on that the applications of course could be either VMs or containers or you know even lambdas function of the service we pay a lot of attention to security and robustness so basically our goal is to make sure that nobody ever has to roll a bucket truck and you know physically touch the device because again a lot of times today if this device but if the device breaks people basically have to physically walk to the device and you know they have to stick a USB stick in or you know some kind of like serial console into it and actually try to figure out what's wrong what's going on right again we're trying to make sure that you always have the control point you know on the sort of other side of the hypervisor and you always have a networking way of exercising that control point so an analogy here would be how many of you do know about whisper net from Amazon do you do you guys know about whisper net okay one guy so with pronounce is basically what's built into your Kindle and you don't even know that it's there right you know you just know that when you turn it on it's somehow magically connects and it actually connects in most of the countries on earth well what happening behind the scenes of course is you know there's basically a small LTE modem right you know there's a SIM card that Amazon is more than willing to pay for because they're you know theory is that if they give you that connectivity that you don't have to pay for yourself most of the time you will be downloading ebooks and they will more than recoup their costs of you know paying to these third-party providers of the LTE connectivity so same idea here so all of these devices will basically be fitted with essential and networking capabilities you know sometimes LT sometimes you know we're working with some of the 5g manufacturers and that control point is something that we would basically always have so we can always you know make sure that the device is operational and the customers don't have to pay for it we are also trying to basically spend a lot of time on making sure that we are adding value for the brownfield applications because a lot of the edge computing platforms today IOT computing platforms today they kind of start with this you know presumption that well all of that you know stop that I talked about that is true and thus you know people will just start writing new applications while that is you know true to some extent I don't think you can underestimate the power of legacy and I mean every successful computing platform you know starting from Windows actually showed us that if you pay a lot of attention to legacy you make a lot of money because you know that's what people are willing to pay for so again we're very focused on making sure that you know that Windows applications still runs right so in terms of the architecture just you know going a little bit deeper into the stack we obviously have the devices on the device he would have you know virtual machines sometimes you would have virtual machines that are tiny so the way to build that type of a virtual machine is called a unicorn or a unicorn all is basically stripping down the operating system to just bare essentials now again it still runs on the hypervisor but it's basically a single process you know single address space that doesn't really have any notions of users or multiplexing or anything right you know it's just basically a way to construct an image that looks like a virtual machine but inside of it just like a single application and then you know we would have to have a whole bunch of this unicorn so we're actually using unicorns ourselves as our services that would provide networking and some of the other capabilities now all of these devices will basically connect to you know some of the enterprise network you know some of the shopfloor networks you know and that is something that we have no interest in sort of helping the customer with I mean those networks they would have still to you know maintain somehow but we can help them virtual eyes access all the applications to these networks right and same deal with i/o so but then of course you know this control point would also connect to the big cloud in the sky so depth cloud you know we are operating one version of that cloud is a SAS offering but we also have customers who would like to deploy those clouds on-site so that control point basically remains let's say on an oil rig because an oil rig you know has tons of these edge devices but unfortunately you know it cannot really count on being able to connect even you know through the satellite you know to the big cloud in the sky all the time so we also selling it as a software offering as well so like I said I mean we're decoupling device an application lifecycle management so there is you know it's um some of the pre-installed runtimes but we can basically depending on how the communication with the cloud goes we can add additional building blocks and that would be just fine in terms of the hypervisor today we're using type 1 embedded hypervisor which is them today we're also looking into this project from Intel called acorn that recently been donated to the Linux Foundation interestingly enough and in Zaman arm actually has very little to do with xenon Intel and acorn is sort of what xenon Intel would have been if it was designed by the same guy who designed xenon arm right so it's like much more appropriate for the embedded world view you know we still like them I mean then it's a little bit bloated I would say on Intel but I think you know those guys are doing a nice job of actually cutting down on some of that bloat and we're still actually not really hypervisor you know opinionated right then work for us today we actually have to have a type 1 hypervisor so key VM would not work the reason for that is because we actually like I said we have to support a lot of legacy so that includes real-time operating systems we cannot count on Linux kernel scheduler to basically deliver on the real-time operating systems guarantees type 1 hypervisor allows us to statically partition the hardware and basically you still get a little bit of an overhead right you know but you can predict it in a much more meaningful way so again very quickly going through the architecture like I said I mean we're basically supporting you know three types of workloads so we're basically supporting sort of Linux containers you know any different fashions and we're supporting unique kernels running directly on the hypervisor there's actually a fascinating amount of unique kernels research you know happening right now so for example what we're doing ourselves because almost all of our stack has written go we're actually using this project called app man OS that allows go to compile directly to an executable that can run on them so and from the go standpoint it's actually really cool because all you have to do is specify because go kind of like builds the cross compilation right into the tool chain all you have to specify is go oh s equals admin and out pops a binary that can actually run directly on then that's actually like super cool so some of the hardware partitioning that we do is you know we sort of experimenting mostly with you know real-time scheduler sort of from these boxes right now and we can basically realize you know different types of topologies you know for that in terms of the micro services that we've developed ourselves to enable the platform so think of it this way right so basically there is a few a few micro-services right here that are essentially making sure that the hardware can be trusted so we are basing all of the trust in the system essentially in the hardware itself the root of trust is either in the TPM or you know trans ohms on arm we're basically virtualizing it back to the applications but all of the crypto material never leaves the box so I'll promise to the customers is that everything that would build into our system you know including the hardware itself basically has a crypto identity associated with it which you can then use to route as well because the VPN that we're building right into the system where is it right here that router is based on this protocol called Lisp nothing to do with the language everything to do with the one of the RFC's and Lise basically allows you to define ipv6 you know addresses you know however you want because it's an overlay so what we're doing we're basically taking a half of the private key of a public key you know of essentially key pair that was generated by the TPM itself and we're using that hash as a actual IP address right you know because ipv6 gives you you know hundreds maybe eight you know size we can actually do quite a bit with that so it's actually very interesting you know I'm kind of like highlighting a lot of things but you know if you're curious you can talk to me about the about all of that you know after the talk there's actually a lot of m2 emptiness in what we're trying to do because again remember we're actually building the system from scratch we're not trying to fit into any kind of real-time operating system or any kind of you know existing Linux you run our software as a baseline as a foster s and then everything else gets expressed as a VM so we don't really have to integrate with anything but the hardware itself so it's basically like a few micro services that have to do with essential and managing the secrets and managing the crypto identity of the device then there is the orchestrator that basically just make sure that we run containers and VM so think of it as a kubernetes you know agent orb you know like a pod agent and then there is basically a few micro services that have to do with IO because one very interesting difference of the edge universes datacenter is that in the data center you never really have to think about virtualizing IO too much because there is not feeling much of it I mean there is not working and that's about it on the edge you basically deal with the zoo over you I mean there is industrial either an ad there is Modbus as you like tons of stuff so we're trying to virtualize a lot of that and present to the software sort of this coherent view of what the IO is so again you know that's kind of how we view ourselves there is also a bit of a security elements like I said I mean there's device security software updates network security and cloud security and again this is mostly just to kinda like capture the high-level points but we're using tons of standards you know like TLS and you know TPM and you know stuff like that so it is pretty standard compliant but again because we don't have to integrate with anything we can actually do a much better job of what you know compared to what like a traditional Linux distribution would have to struggle with so there's a lot of interesting you know ideas on the TPM so again talk to me about the TPM after this presentation and with also doing essentially key management you know in the same way that hacker Corp is doing it with you know some of the things that they are trying to build but we actually again building it right into the platform so you don't have to worry about where you store your keys even if the image you know let's say a docker image actually needs to have an access to a private key so that gets handled by the platform itself finally a really cool thing that we are also working on is protecting the application from the operating system not just they have operating system from the application because typically so far it's been one way because it's like well I mean your hardware is trusted like in an Amazon datacenter you don't expect people to hack Hardware on the edge it happens all the time so if you have an application you actually have to protect from the host operating system being hacked so we're looking into Intel SGX and aren't razones for that there is a very interesting project going on in a sort of on one hand very unrelated on there they can very related space called Gollum you know it's sort of this blockchain you know a computing project and we're kind of like trying to sort of work with those guys to figure out what we can do there so the name of the project that we about to open source because hey it's actually all going to be open source pretty soon it's called project Eve edge virtualization engine and we're trying to do it within Linux Foundation umbrella so with that if you forget every single slide and every single line that I told you about today remember this so Eve is to the edge computing what Android has become for mobile computing that's our true belief and hopefully 2019 will be the year of Eve with that I think we're out of time thank you so much do we have time for questions so are we really out of time inside this okay okay any questions cool awesome all right thank you so much