Scale By The Bay 2019: Charles Pretzer, Linkerd and the Service Mesh

Scale By The Bay 2019: Charles Pretzer, Linkerd and the Service Mesh

Recording: Scale By The Bay 2019: Charles Pretzer, Linkerd and the Service Mesh

[Music] thanks everybody for first of all things to scale by the way for having me here and thank you all for sticking around on a Friday it really speaks to the quality of the conference that people are willing to stick around when it's happy hour time I will quickly be running to a nearby bar after this so I hope you all do something similar my name is Charles I'm a field engineer at a company called buoyant which is the main sponsor for the open source project linker D which is a service mission who here is familiar with the term service mesh alright anybody running a service mesh I like it in production yeah well I'll buy you a beer after this ok today I am here to talk to you about the history of linker D how we migrated from linker day one to linker day two and accomplished a pretty significant feat of basically complete rewrite using new architecture and new technology in the face of lessons that we learned from linker d1 we'll talk first about the history of the service mesh to reefa Miller eyes folks with it and also introduce other folks to the service mesh the concepts that are important and how linker D implements those from there we'll talk about the history of linker D we'll look at the architecture of linker D 1 compared to the architecture of liquor T 2 and hopefully my voice holds up and we still have time for questions afterwards so anyway let's get started so linker D is an open source service mesh project it's the only service mesh that is fully donated to the CNC F we have a large number of users in production and more importantly we have a large community the CNC f pork part is really important to us because it ensures our community and our users that we are bound by rules of open governance so take that for whatever you like we are we're pretty happy with it there are high-level concepts to a service mesh and what we believe is important that a service mesh is responsible for is the exact functionality that has been built into Lincoln II from since day one those concepts are observability to be able to capture the metrics for your services that are running in your application success rates Layton sees and other pieces of information that are considered useful and making sure that your application is healthy we also focus on reliability and we accomplish that through allowing folks to build in retries and timeouts load balancing is a really big piece of what Lincoln D does what we think a service mess should do and I'll talk about how we accomplish that with different protocols and finally security is a an extremely important topic for us we talk about a lot of our users are banks or telcos folks who need security of their east-to-west traffic either by the by regulation or or by internal requirements and so we'll talk about how each of these factors into a service mission in order to do that let's do just a quick background and go over what a service measure looks like what we have here is a simple distributed application there's no service measure we've got in fact I would call this a trivial application we get traffic from the outside that Northwest traffic which is then passed to services on the inside of the of the application which is then the east-west traffic the service mesh for us operates solely on east-west traffic so let's say that you're working at this company it's a start-up you've come up with a brilliant idea and we now have an application that is serving hundreds maybe thousands of users every day now you're excited some suit up in a corner office comes down and says you know what we need to do more of a B and C and I'd also like us to add XY and Z so some of us as engineers or infrastructure operators we go and we run off and we take care of these things we build the infrastructure and I'm glossing over this part intentionally we build the infrastructure we write the code and we end up with our amazing application now we are a start we have multiple services multiple instances of services serving hundreds of thousands of users every day and we have added these applications or these services to our application X Y & Z now there's a difference here that I think some of the smiles on your faces can see where I'm going with this X now has a dependency on an external data store that's interesting why has an external dependency on a an external API or an API also interesting Z is less interesting but it's still something that we have to make sure that we're aware of it's part of our application it's part of our service we want to know how it's performing at all times we need to know its metrics but we've got our application now we are printing money right and this is this is the dream and the reality of this is we we don't know what could go wrong at any given time who's had something go terrible in production in the last year yeah everybody does we can't plan for these things so hypothetically let's say your database just decides to go on fire okay well we can probably track that down fairly quickly or your external API stops responding we'll figure that out eventually as well or some new version of your service that's been deployed has some really terrible performance in it that your test didn't pass so it was still released well we can manage that as well or we'll find it we'll figure it out and we can solve for it could this all ever happen at once who knows we've seen production instances but certainly things don't go wrong in a convenient way for us so with this in mind let's step back to our trivial application and add a service mesh into it there are two important aspects to a service mesh from our from serve or service mesh the first is the data plane and our architecture the data plane consists of multiple proxies that run alongside the service services themselves one thing to take note of here is that there's actual network traffic that's happening between the proxies and the services that they're adjacent to I just failed to put an arrow in there to fully indicate that so imagine if you will there's network traffic going on there what we see is that for every instance of every service we have a proxy that's deployed so there are potential issues that could happen with that but we'll talk about those in a little bit the next piece of service mesh is the control plane so the control plane and the data plane are are not new concepts my history serves me correctly which it doesn't always do the original service our control plane and data plane come from telcos where they were creating routers for packets and on the towers themselves so embedded at hardware so they had an even tougher challenge than we do with our software proxies the control plane is used simply to configure the proxies so that the proxies themselves can remain lightweight and perform with liquor D we accomplish this through this particular architecture what you see here is all the components of our control plane on top and on the bottom is an example of a service that is part part of the mesh that has been injected with the proxy and here the arrows are correct we have inside of a container and a lot of the work that we're doing is container and I'll tell you that liqueur t2 is kubernetes first so we run inside a pod alongside the same same container as the service so that's localhost traffic that's happening between those two so how did we get from liquidy one out over delinquent e to the pedigree of liquor do you itself comes from our CEO CTO several engineers and maintain errs who previously worked at twitter during a time where they were going through exponential growth and through through their experiences they recognized the pieces that needed to be built into what we consider the service message today so we have linker D one I'm sorry the Twitter experiences which is an actual album name just kidding Twitter experiences from 2010 to 2015 Linda D one written in 2016 donated to the cloud native computing Foundation in 2017 and Liberty 2 released in 2018 just to give you another example of what happened at Twitter they had a gigantic monolith application built on Rails and I'll preface all this by saying I wasn't there for any of this and this is all things that I heard so their architecture doesn't or never actually look like this so they had a giant gigantic monolithic application built on Rails and they started getting tremendous exponential growth in the late 2000s I'm sure some of you experienced the fail wail when that happened and were frustrated by the fact that people couldn't read your tweets I certainly was so they their team their infrastructure team began to look at what they could do using virtualization to break out the monolith into separate pieces so that they could begin to manage different parts of the application that's at smaller pieces of the application at larger scale and the result of this is that you have more moving parts more complexity but better control over each of those pieces and I think it was somewhere in 2015 that they officially retired the fail wail so Twitter did a good job of doing that and the lessons learned by the folks who work in our at our company and who are maintainer for our open source project we brought all of those over into what is now linker do you want so what is linka d1 it should come as no surprise based off of that that we built it off of the twitter stack that means that we use the JVM with Scala Neddie and finagle fagles the library built on top of MIDI in Scala that the twitter team has open sourced and is widely available anybody familiar with finagle you'd have you used it in production okay good so we like to call it largely 31 was finagle in a box we leveraged all of the functionality with the Senate we didn't have to rewrite new code we could get this product an open source product out to people who needed it who were looking to migrate do a similar migration from a bottle with over two microservices the result of this was that we exposed aspects of the finagle in a proxy forum so this is a process that you can run that would intercept requests and send them wherever you want it to go want them to go how do you configure them where you want them to go well you use a Yama file of course I'd like to point out that we were way ahead of the entrant so you're welcome and so I say that tongue-in-cheek because we learned some lessons as a result of that anybody tried to run the JVM on every request that comes in into a system multiple services like he gets it this guy gets it so yeah it has a high resource cost it was tremendous - it was hurt a hard pill to swallow sometimes the trade-off is that the functionality was there people could use the lincolni proxy to get these metrics to make sure that their application was running running well there was just high overhead the high cost to doing that it's kind of like a Ferrari right I tried another issue that we ran into was cultural resistance you had operators infrastructure folks who just didn't want to were hesitant to provision this much resource to the JVM and a good example of this here is 150 megabytes that's not great for a Java app actually it's pretty good for a java application but when you're doing that across thousands of proxies all of a sudden most of your infrastructure hardware is going towards your JVM management and not enough is going towards the actual services that are providing the value that your company is offering so we saw that and said maybe we can change that and the last piece is the complexity of finagle as you might know has a lot of options are a lot of environment variables that you can configure and that yamo file that ended up generated to configure blinkered e-especially are you familiar with d taps d tabs are fun to configure as well so you know these are not unlike routing mechanisms so if you want to route to multiple services this list of D tabs can be very very long so you end up with a very pretty large Yama file and the complexity means that there's a lot to learn to find to not only the JVM if you haven't done any JVM tuning in the past you should spend your entire weekend doing it this weekend because it's it's going to be on the test it's a lot of fun I'm just kidding I I did a lot of JBoss and e-commerce work in my career and there's a black black art to fine-tuning a JVM especially for j2e application so these are the lessons that we learned things that we took away that when we got to an inflection point we had to figure out do we continue to use this finagle architecture or do we look at something else and surprise surprise we did a rewrite the goal was to do less not more so we sacrificed some functionality but gained a tremendous tremendous amount of performance some of the because from those lessons that we learned what we wanted to put into this newer version of linker D was that it just works take away that huge Hamill file turn it into a zero config out of the box process that that can be run within minutes I would do a demo for you now but I don't feel like the demo gods are on my side today just getting the there also we wanted it to be ultra light so what that means is not consuming all the resources that the JVM did and we want it to be simple people who manage kubernetes clusters have a lot to think about is anybody here using kubernetes yeah good whose kind of cube god am I gonna see you there yes okay so we wanted to reduce that computational complexity because again as kubernetes operators you have a ton to think about there's a lot to be done and the service mesh should be transparent and should give you metrics and not give you headaches should give you security and not nightmares so by doing this we reduced our control plane size well first of all we never had a control plane with linker do one every every proxy ran in its own space it was configured on its own you could potentially have two proxies that were configured differently which is problematic the control plane solved that single configuration issue and we wrote it and go because as a kubernetes first application we wanted to very seamlessly integrate with kubernetes primitives and the kubernetes api s and and also reuse as much of that code as possible and so that's how we managed to run a control plane which is all those components at less than or at around 200 megabytes of memory the data plane is even smaller so when we talk about running the process for the data plane by using rust we were able to on average run a proxy at 10 megabytes with a p99 latency of lesson a millisecond so I had to it took me a minute to wrap my head around that when I first started working with lickity working on the project it was really impressive how the team has managed to really I want to say they dominate rust and I'll talk about why we made that particular choice to rust let's start with talking about why we used to go for the control plane I mentioned some of this already it's very common to the cloud native ecosystem has been growing just had its 10-year anniversary birthday it's been growing like crazy the a lot of the other open source projects kubernetes prometheus talker all written and go it meant we could use kubernetes libraries directly as I mentioned and it's easy to learn which if you've never for me I struggled because I came from Java world and so I tried to apply Java techniques to go programming and that that's just that's so I learned that I work with some great people who taught me how to do it the right way aside from that the go community is huge that's wide adoption made it really really the right choice for us to use for for control plane why did we choose rust for the data plane well first of all we wanted the proxies to be small fast and as safe as possible the safety part there is really important to us when it comes to memory to give you the alternative we we could have considered C++ and at that time around 2015 there were still memory issues no memory issues and C++ heartbleed was one of them and I think was around about that time so we wanted we looked this is what the CTO told me we look to where the puck was going to be not to where it was so we took the opportunity to to see what was new what languages were going to be highly performant and where we could contribute to those communities as well so much of the rust development has been prompted by the the work that we've done in lis Rd specifically with regard to the HTTP networking crates at the libraries that belong to so Russ is rust is an ideal language for being small fast and safe the native code has the memory safety guarantees zero cost abstractions and again direct control over the memory allocation the additional piece of that is coming from Scala the engineers at the time felt that rust was a good fit a good transition without having to learn too many new idioms finally the rust motto which I actually learned today is fast reliable productive pick three and that's kind of the mentality of the lincolni community is let's just do something and do it really really well so that's why that's how we landed on rust coming from Scala as a result of making those choices we had some scary questions to answer yeah so do we really want to write finagle no that actually led to it answering those questions we're all the networking libraries there weren't any in rust at the time or there were very few will the rust community keep drawing who knows it could be a language that just doesn't continue the JVM is getting better we actually tinkered with the growl VM for a little bit it didn't quite get us down to that smaller memory from put footprint where we wanted it to be and then the question that nobody would even would have even thought to ask now is is kubernetes going to take off and you know it has so we are fully on board and these are the questions that we had to answer at the time in retrospect it seems like they're very easy questions to answer at the time they were challenging we weren't sure what what there were many conversations that had to be had in order to properly answer these questions so fast forward today to today the gamble paid off there are a lot of great rust networking libraries tower Tokyo and hyper and again our community has contributed to building those working with the rust community a lot of people are picking up on rust the community itself is growing like we like a weed contributors salinity we've got we're continuing to grow our liquor the community that's one of the parts that I enjoy about the work that I do is coming out meeting folks like with you folks like you chatting with you telling you about the work that we're doing how cool it is uncle I think it is and so as a result of the work that we've done as a whole word son I didn't get all these hundred contributors I wish I could send it but as a work as a result of the work that we've done as a whole the communities thriving it's very healthy it's very friendly we have people from all over the world so to me that's one of the largest successes and there are many hundreds of kubernetes cluster is running linker team we have metrics that we can use to verify this and now we see that as we need to adoption has taken off linker t1 adoption as we expect as tail off this is because we have actively put linker t1 into maintenance mode there are a few companies that continue to use it and we work with them on a weekly basis making sure that they're happy they're continuing to be successful our focus is largely on growing lickity to adoption so if you want to give us a star we'll get there we need your help and with that I'll say more than just give us a star come check us out on slack I'm always on there that's probably the best way to get ahold of me our give account we're active there as well the community like I said it's great very open and warm and welcome I think every community person says that about their community they're also brilliant some of the smartest people I've ever worked with and so I learned something new every day if you all are interested in doing that as well come check us out so that gives us about five minutes for five minutes for questions what can I tell you further control paint these yeah okay so question is why did we pick go or this top part here and rust for the the bottom part and the reason for that was because it's become a common language in the cloud native compute ecosystem kubernetes prometheus duck are all run on it and most importantly we wanted to work with working to the go I'm sorry the kubernetes api without having to go to write new libraries to interface anybody else yeah we definitely did there was some it was very attractive to to do everything all in one language at the end of the day we're kind of tinker's and it's like why not try something new and build it the way that we wanted to so go had some strong opinions that were already had been made who where as rust there were some some specs that were still open that we could help to define as part of the community not forced our way on there we we worked with them yeah am I the only one who noticed that this is I'm asking for a friend question I'm just gonna I'm a little punchy late on a Friday so I'm just making a joke yeah so for me personally I enjoy writing go more than Java because or Scala right or Scala and to be clear I haven't written a whole lot of Scala and pretty much jumped straight from Java to go with a little Python and JavaScript in between the reason that I like it is it's much more readable to me the idioms I think are are more practical the best example I have of this I was at the go SF meeting the other night and you might send some bias on my side is that exception air handling or exception handling when you do this and go it's very much a collar function if it returns an error do something get out whereas with reading Scala or Java probably more so Java than Scala you get your try and then you have a bunch of stuff exception exception exception so that's that's I think probably be probably more of a preference thing where it's easier for me to read let's see other practical pieces I think the garbage collection is better it's certainly faster in all cases of uses the lower memory footprint lower profile and there are is a very healthy number of third-party libraries that you can do to get started in public application tonight if you want to help okay good yep two questions there so the I'll answer this what was the first one again is there configuration so what we mean by zero configuration is that you install it and it will read your kubernetes manifest it will discover your services and you can begin injecting proxies without doing anything else that being said there are configuration options one of the most popular is the - - AJ option which computers link ad the control plane for high availability mode there are other things in there like we can configure resource memory sorry resource requests if you're familiar with resources and kubernetes we can configure those the when we say zero configuration we minimize the amount of things that you have to configure - we contrast that with Lincoln d1 there is a gigantic Yama file for a mo file which could grow to be very large that you need it at a bare minimum to get started with linker d2 there's no you can there's no configuration file that you have to write there's no keys or anything that you have to put into some file somewhere in order to get it up running that makes sense okay and the second question was yeah so yeah the the main feature that we threw away was plugins and what I mean by that is there was a place for in liquor t1 where or a mechanism and liquor t1 where you could write your own functionality almost like middleware right so request comes in you could write your code to do something with that request and then pass it on to the rest of the proxy we got rid of that so right now there's no plug ability you can't customize but we do that intentionally because one of the first things that we saw is that people would come in they'd write their plugins and say your service mesh is just way too slow let's look and see how many plugins you've got running there let's see what plugins you've got running so but we are looking to solve that in the future if anybody's curious with about azzam we're thinking about using that to solve that problem to be able to enable plugins again that's that's probably the main one there are others there too which I'm happy to discuss after anybody else I'm officially out of time even the computer knows it this is a screen saver of Keshe it's where Chewbacca lives okay all right thanks Charles [Applause] [Music]