SBTB 2023: Alex Leong, Five Years of Cloud Native Rust.

SBTB 2023: Alex Leong, Five Years of Cloud Native Rust.

Recording: SBTB 2023: Alex Leong, Five Years of Cloud Native Rust.

thank you uh yeah this is five years of cloud native rest uh so my name is Alex Leong uh I am one of the maintainers of a project called linkerd has anybody heard of linker is that name familiar to anyone maybe just a few people Okay cool so Linker is a uh service mesh it's an open source project uh so it's a service mesh for kubernetes and I'll talk a little bit more later about exactly what that means um it's a graduated project of the cloud native Computing foundation and we made this big decision about five years ago to adopt rust as the programming language that we would write Linker D in and so this talk is basically about that journey and why we made that decision and kind of what the the outcomes of that have been so I'm going to talk about why a little bit closer yeah I'm going to talk about why we decided to use rust in linard kind of what what went into that decision and how we decided that that was kind of what we needed to do and why we made that bet uh and I'm going to talk about how that went and uh kind of what the outcomes of that were for that project and then I want to talk a little bit about what that means for the ecosystem for the greater ecosystem and whether that means uh that Russ should be adopted more broadly so number one why why did we choose to use rust well before we can even answer that what what even is linkerd uh so linkerd is a service mesh um and so what that means is that we run a small sidecar proxy alongside every instance of your application and all of your applications traffic is redirected so it goes through that proxy so all the traffic going in uh to your application first goes through the proxy and all of the traffic going from your application out goes through the proxy and what that means is that proxy since it's uh intelligent and it's aware of level four and level seven it's able to do a lot of really cool things with that traffic like it's able to add um mutually authenticated TLS automatically on all those connections it's able to detect uh things like success rate latency and request rate and emit those as metrics that you can consume in Prometheus it can use those latency data that it observed to do intelligent request load balancing so it can load balance HTTP requests and wait that by the latency so you get more traffic going to the faster uh replicas and less traffic going to the slower replicas um and all kinds of other really really cool stuff circuit breakers observability reliability retries timeouts all kinds of stuff it's super cool but kind of the consequence of this architecture is that all of your application's traffic is through flowing through this proxy in fact because your application probably is made up of multiple uh Services uh the traffic is going through that proxy or those proxies multiple times times one at each hop or two at each hop if you consider the traffic going in and then coming out so Linker is infrastructure and there's kind of two different things that I mean by this uh number one Linker is infrastructure for your application it's Network infrastructure whenever your application is going to make a HTTP call or a TCP connection or any kind of network call uh it uses linkerd under the hood kind of transparently and linkerd is responsible for getting those packets where they need to go so just like your operating system is infrastructure or your network card is infrastructure Linker is infrastructure to your application but Linker is also used by infrastructure for humans so for example 911 call centers used linkerd Healthcare organizations and financial institutions all use Linker as part of their um part of their applications and these are very critical Services these are things that people rely on these are not things that you know if they work that that's great if they don't who cares these are like things that cannot fail and so infrastructure needs to be secure reliable and fast in that order and it's not negotiable you can't compromise on any of these so when we talk about these things being secure well what do what do we mean what are we protecting against what types of things cause security vulnerabilities so one example of something that can cause security vulnerability is a buffer underrun right you can be trying to access uh a piece of memory outside the the legal bounds for that memory and access you know some invalid memory and and cause a buffer underrun buffer overrun uh you can have an issue with dangling pointers or used after free so trying to use a pointer after that pointer is no longer valid either because the the data that that pointer has pointed to has moved or has been deleted uh or for whatever reason that pointer is no longer valid but you try to use it anyway or kind of the flip side of that reading uninitialized memory so reading a piece of memory before it's valid so these are all different examples of what we call memory unsafety basically accessing a piece of memory assuming that it's one thing when it's actually something else and it turns out that these memory on safety issues are responsible for a surprising majority of security vulnerabilities 90% of Android vulnerabilities are caused by memory on safety issues 60 to 70% of iOS and Mac OS cves 70% of cves and chromium all due to memory and safety in one form or another so one way to get around this is to use a managed language right you can use Scola you can use Java go C Ruby python any language that has kind of this runtime that checks these memory safety things uh as the program's running at runtime and do things like bounds checking and and making sure that that uh the memory you're accessing is valid but doing those kinds of memory safety checks at runtime have a performance cost right there's always some overhead associated with that runtime so on the other hand there's languages like C and C++ which are very fast you know you don't have this overhead from doing those runtime checks to make sure that the memory you're accessing is valid uh but they're also unsafe they're vulnerable to these types of issues so when we were thinking about what we needed to do for linker's data plan and when I say data plan I'm talking about those proxies those proxies which all the data in your application is flowing through uh we decided memory safety is unacceptable there's too many uh too large of a class of security issues that are caused by memory safety issues that we didn't want to be susceptible to but on the other other hand using a managed language like for example something that was built on the jvm uh means you're susceptible to things like garbage collection pauses or other pieces of overhead um caused by the runtime you know when you're handling traffic for a critical application like a 9911 call center you don't want to have a request paused because your application is going through GC right that's not not really acceptable so we thought well what if there was something else that we could use here that would give us kind of the best of both worlds something where we can get both memory safety but also uh kind of these fast uh speeds that you get from a language like C C++ so that's where rust comes in uh so rust is a system's programming language and it's very cool because it ensures memory safety at compile time uh the rest compiler does all of these uh checks when the program is compiled to make sure and to guarantee that the coded outputs will perform uh will not perform any illegal memory operations and because of that you don't need to do these checks at runtime anymore because they're statically guaranteed so how does Russ do this how does this work how does Russ help us write safe code uh so one idea in Russ is that there is this idea of unique ownership which means that any piece of data in your program is owned by exactly one piece of code and so you don't have two different pieces of code which are both trying to access the same data both thinking they own it both trying to manage it um and kind of leading to to issues where maybe one deletes it while the other's accessing it or something like that um if you want to access a piece of data in your program that you don't own there's this concept of checked borrows where you can kind of borrow it but there are a lot of rules uh in place to make sure that if you're borrowing data uh it's not going to change out from under you and this is kind of accomplished with this idea of exclusive mutability which is that if you want to change a piece of data in Rust uh you need to kind of be the sole person or the sole piece of code uh that has access to that data at that time so you're never going to change data out from someone else while they're reading it so let's look at some code examples because this is a little bit abstract uh so this is a C++ program it uh is pretty simple so we start out by creating a vector of strings called V so that's a growable array uh we're going to push a string into that Vector called hello world we're going to create a pointer to the first element in that Vector called PTR then we're going to print out the contents of that pointer or what that pointer references uh so hopefully that's hello world uh then we're going to push a whole bunch of other strings into there and then finally we're going to print out that first pointer again so that should print hello world again so we would expect if this program work correct it would just print hello world twice so if we try to run it though uh it doesn't do that it prints hello world once and then seg fults so what happened like this is such a simple program what could have gone wrong uh well it turns out that V remember I said is a vector is a growable array and the way that works is that it's backed by a fixed length array somewhere in memory and so we create a pointer to that first element of that fixed length array and then we add a bunch of things to the vector and that causes it to grow and if it exceeds its capacity in that fixed length array it needs to create a new array somewhere else in memory that's larger and move all of its contents over to that new array and when it does that our pointer to that first element of that Vector is no longer valid we're now pointing to where that Vector used to be and uh and so that causes the the segmentation fault so we're accessing a pointer that is no longer valid because it has changed out from under us without our knowledge it's pretty sneaky so let's see how rust can help us here so this is the same program written in Rust so this should look pretty much identical we've got a vector called V we push a string into it hello world we take a pointer which is points to the first element of that Vector uh we print it out we push a whole bunch more strings into it and then we print it out again right so basically the exact same program anyone have any predictions I think this is going to work when I run it think it's not going to work it's not even going to compile we get a compiler error so this is really cool uh so what does this compiler error tell us it says cannot borrow v as mutable because it's also borrowed as immutable and so what that means is that we have this pointer to this vector and while that pointer is still valid while that reference is outstanding we tried to edit V we tried to mutate it and that is not allowed that violates exclusive mutability and so the r compiler is able to tell us hey this is you know this is going to be a problem you can't do this this violates the rules of of exclusive mutability so how do we fix it well it's pretty simple uh if we just recreate that pointer down at the end the compiler is smart enough to know that that first instance of the pointer is shortlived it only lives as long as that first print line and then it's never used again so they can be discarded at that point and then once we try to do the v. pushes that mutation it's allowed because we don't have any outstanding references to v no one else is looking at that data so it's safe to mutate then once those mutations are complete we can recreate that pointer and everything should work Tada hello world twice pretty sneaky uh let's try another one okay so here's another C++ program uh we have a class called greeter that contains a reference to a string called who uh there's a Constructor for it and then there's a greet function which prints out hello and then whatever that who string references we have a function called make Hello World which creates a string called world and then passes a reference to that string into a greeter and returns it and we have a main method that uh calls make hello world and then calls greet or. greet uh there's a bug in this program too this one's maybe a little bit easier to see but let's try running it and see what happens whoops here we go so it does print hello uh but it doesn't print hello world it prints hello and then a whole bunch of nonsense um so why did this happen well it's a little bit similar to the previous one when we create that string world in the make Hello World function that is a stack local variable and so we return a or we take a pointer a reference to that local variable pass it into greeter and then we return and when we return from that make Hello World function all of those stack local variables go away but that pointer to that stack local variable kind of escapes that scope because we've we've given it to the greeter and and returned it and so now we have this reference that is no longer valid it's a reference that has outlived the lifetime of the data that it references and so when we try to print it out we're pointing to who knows what and we get a whole bunch of garbage so again we can kind of rewrite this program in R and see how how is that different so this is the same program in R we have a struct called greeter it holds a reference to a string called who we have a greet function which prints hello and then the contents of who and then we have a make Hello World function which creates a string called World passes into greeter and then we have a main method that calls make hello world and greeter dog greet so again we can try to run this and see if rust gives us any more protections against these types of bugs uh and we get a compiler error again so this is great this is the rust compiler protecting us telling us that these types of things are invalid and so what is this error this says missing lifetime specifier expected named lifetime parameter and so what this is saying is that in Rust there's this concept of reference lifetimes where a reference might be only valid for a certain amount of time um and it's telling us here we need to annotate these uh this code to to talk about those lifetimes to say hey this is a reference but it's not just any reference it's a reference that's going to live from here to here and if you try to access it outside that valid lifetime that's not going to work the other cool thing here is that not only does the compiler tell us that there's an error but it also gives us a suggestion about how to fix it so you can see there it says try adding these you know tick a lifetime indicators uh and uh that's kind of what you need to do so we can just do what the compiler is suggesting here and add those lifetime indicators to our program and now we've kind of got things parameterized on a uh which is to say when we talk about a greeter we're not just talking about any greeter we're talking about a greeter that holds a reference whose lifetime is blah and if we try to run this it's actually just going to work uh and the reason for that is that the compiler is able to determine that that world string is just a static string it's you know it's not created dynamically so it can be allocated statically and therefore it's lifetime is the entire lifetime of the program and so it's able to kind of infer that all of those lifetimes are going to work out you're always going to access those that reference at a time that's valid for it and everything's great so this is a little bit of a trivial example because the the lifetime in question is just the static lifetime which lives for the whole program but there's definitely a lot more sophisticated things you can do with lifetimes that you know only live for certain amounts of time uh and the rust compiler will check you at every step of the way to make sure you're not doing anything invalid valid so those are kind of the reasons why we adopted rust for linkr that's what what motivated uh it for us we wanted to avoid these memory safety issues we wanted to have confidence that we were kind of free of this category of bug um and we also wanted that kind of Blazing fast performance and so we had these design principles uh when we were starting out with Linker uh specifically Linker 2 which was the rewrite and rust uh so the principles were keep it simple minimize resource requirements and have it just work out of the box and I want to especially talk about number two minimize resource requirements because with a language like rust it's very easy to minimize memory allocations because you're very aware of every time you allocate memory you're very aware of how memory is created and shared um that's kind of always at the Forefront when you're writing a program in Rust so it's very easy to to keep that in mind and not have memory kind of run away from you and allocate a bunch of stuff unintentionally uh so in link 2 we were definitely able to to minimize uh memory and similarly CPU because we didn't have that that runtime overhead but we're also thinking a lot about resources like latency you know if your application has a latency budget of 100 milliseconds before it starts to feel slow if your proxy at every step is eating a significant portion of that latency budget you know that's not good and that's so latency is kind of a resource too that we wanted to make sure that um the proxy was fast not just efficient but also fast and the other type of resources is human resources if you need a whole team of people to manage this piece of software you that's very expensive in terms of you know Engineers or or uh operations people or you know just human resources that are very expensive um and similarly if you have a security team that has to fight fires and respond to vulnerabilities uh all the time that's also very expensive so security and operability were were huge huge uh things of huge importance um Now link is actually made of two different pieces I've been talking a lot about the data plane which is the proxy where all the traffic is flowing through there's also the control plane which is a piece that sits separately to that um and controls those proxies so it pushes configuration to them it reads metrics from them it gives them things like TLS certificates and and manages revocation and rotation um and so the control plane and the data plane have very different requirements uh because the data plane is is proxying all that traffic that's the hot path uh those need to be secure fast and efficient right no compromises allowed there and because of this rust was kind of the only choice for us it was the only language that really gave us uh the best of the Both Worlds in terms of speed and safety but when we were making this decision back in 2018 the state of R was was not as mature as it is today so for example there were no production ready htb2 or grpc implementations at that time uh so we helped to build them uh it's hyper H2 and hypertonic and async rust just in general was still in its infancy so we invested a lot in a lot of the different projects in that ecosystem to kind of help get it off the ground so for example we invested a lot in Tokyo which is rust's async runtime for doing async iio uh like I said we've invested a lot in hyper the HTP and http2 implementation and in a project called tracing which is an async aware logging library that was created for linkerd but has since been adopted kind of widely throughout the ecosystem and this choice for us to adopt Linker or for Linker to adopt rust has really paid off um so this is a latency comparison between linkerd and another service mesh called ISO which uses a different Pro uh different proxy called Envoy uh and you can see that the the latencies here in Linker are just uh they stay nice and low and fairly close to the Baseline of just having uh the application by itself it's kind of paid dividends in resource efficiency so that's the same comparison again uh but in terms of CPU usage and memory consumption you can see that Linker is just using a tiny fraction of the resources and maybe most importantly it's paid dividends in security so this is Envoy the the proxy uh the other proxy that I was talking about uh written in C++ uh and you can see that the the majority of these vulnerabilities are in that column there memory corruption so these are exactly the issues that rust protects us against these are the exactly the things that we don't need to worry about because rust guarantees that that these types of things won't happen so this is all for the the data plane the proxies but what about the control plane the control plan is not actually in the application's data path you know it's just pushing down configuration and managing these proxies so if a piece of configuration takes you know 100 milliseconds longer to propagate it's maybe not the end of the world so because it doesn't have that latency requirement that the data plane does uh we have a little bit more freedom in our choice of language here you know rust isn't maybe as necessary so a managed language like go is actually a great choice and that's what we decided to do uh we used go for the control plane in the CLI because this led us very easily integrate with the rest of the kubernetes ecosystem which is uh very go heavy and we were able to get access to a lot of good libraries for interacting with the kubernetes API which is what the control plane primarily does um but with that said in 2022 we introduced a new policy controller into the control plane and we wanted to see what it would be like to write a piece of the control plane in Rust uh so in order to do this we used a library called cubars which is for interacting with the kubernetes API uh it's a rust Library it's kind of the equivalent of client go from go uh and it we did it it worked pretty well but there were a few things that were still a little bit difficult um especially around writing controllers in Rust uh there's a lot of stuff in client go around um watching resources for State changes in kubernetes and doing reconciliation Loops that just kind of wasn't there there wasn't a lot of infrastructure in place for that uh so we wrote some uh we created a library called kubert which is a batteries included runtime for writing controllers using cubars and it kind of handled all of those those shortcomings uh that I was talking about in terms of uh what you need in order to write a controller for kubernetes and rust so if that's something you want to do check out that Library for sure okay so why does this matter to you well I think if you're doing anything with financial data Health Data any sensitive data whatsoever you really owe it to your users to avoid these security vulnerabilities that are you're susceptible to if you're using a memory unsafe language and if you also can't afford the runtime overhead of of a managed runtime rust may be one of your best options so if that sounds like you if those sounds like your requirements try rust like give it a go it's pretty cool uh this is a quote from Alejandro who's another one of the Linker maintainers who didn't have any rust experience prior to working with it uh on Linker D and he says it doesn't cease to amaze me that when something finally compiles in Rust there's a high chance the program will simply work as intended which is like kind of the best thing you can say about a compiled language so we believe that the future of the cloud will be built in Rust future of linkr certainly will be thank you very [Applause] much